From 481c11ddf2e542223b5aff4fb762d5e73cd08ef4 Mon Sep 17 00:00:00 2001 From: Simon Brooke Date: Mon, 3 Apr 2023 10:22:06 +0100 Subject: [PATCH] Prevaricating. But you get (most of) a good new essay out of it. --- doc/lisp1.5.md | 46 ++-- doc/mexpr.md | 13 +- doc/values.md | 243 ++++++++++++++++++++- docs/codox/beowulf.bootstrap.html | 2 +- docs/codox/beowulf.cons-cell.html | 2 +- docs/codox/beowulf.core.html | 2 +- docs/codox/beowulf.gendoc.html | 2 +- docs/codox/beowulf.host.html | 2 +- docs/codox/beowulf.io.html | 2 +- docs/codox/beowulf.manual.html | 2 +- docs/codox/beowulf.oblist.html | 2 +- docs/codox/beowulf.read.html | 2 +- docs/codox/beowulf.reader.char-reader.html | 2 +- docs/codox/beowulf.reader.generate.html | 2 +- docs/codox/beowulf.reader.macros.html | 2 +- docs/codox/beowulf.reader.parser.html | 2 +- docs/codox/beowulf.reader.simplify.html | 2 +- docs/codox/index.html | 2 +- docs/codox/intro.html | 2 +- docs/codox/mexpr.html | 11 +- docs/codox/values.html | 172 ++++++++++++++- src/beowulf/core.clj | 9 +- src/beowulf/gendoc.clj | 12 +- 23 files changed, 459 insertions(+), 79 deletions(-) diff --git a/doc/lisp1.5.md b/doc/lisp1.5.md index d554bcf..7e53b6b 100644 --- a/doc/lisp1.5.md +++ b/doc/lisp1.5.md @@ -982,7 +982,7 @@ fn: EVAL args: ((CONS (CAR (QUOTE (A. B))) (CDR (QUOTE (C. D)))) NIL) The value of both of these is (A. D). -111. EXTENSION OF THE LISP LANGUAGE +111. ## EXTENSION OF THE LISP LANGUAGE ``` Section I of this manual presented a purely formal mathematical system that we @@ -994,7 +994,7 @@ shall call pure LISP. The elements of this formal system are the following. 3. A formal mapping of M-expressions into S-expressions. 4. A universal function (written ,IS an M-expression) for interpreting the application of any function written as an S-expression to its arguments. -Section I1 introduced the LISP Programming System. The basis of the LISP Pro- +Section II introduced the LISP Programming System. The basis of the LISP Pro- gramming System is the interpreter, or evalquote and its components.. A LISP program in fact consists of pairs of arguments for evalquote which are interpreted in sequence. In this section we shall introduce a number of extensions of elementary LISP. These @@ -1304,7 +1304,7 @@ DEFINE (( (T (TIMES N (FACTORIAL (SUB1 N)))) ))) ``` -4.4 The Array Feature +### 4.4 The Array Feature Provision is made in LISP 1.5 for allocating blocks of storage for data. The data may consist of numbers, atomic symbols or other S-expressions. @@ -1348,18 +1348,14 @@ Arrays use marginal indexing for maximum speed. For most efficient results, specify dimensions in increasing order. ~eta[3;4;5] is better than beta[5;3;4]. Storage for arrays is located in an area of memory called binary program space. -``` -V. THE PROGRAM FEATURE -``` +## V. THE PROGRAM FEATURE -``` The LISP 1 .5 program feature allows the user to write an Algol-like program con- taining LISP statements to be executed. An example of the program feature is the function length, which examines a list and decides how many elements there are in the top level of the list. The value of length is an integer. Length is a function of one argurnentL. The program uses two program variables -``` - u and y, which can be regarded as storage locations whose contents are to be changed by the program. In English the program is written: @@ -2735,37 +2731,31 @@ respectively. The value of not is true if its argument is false, and false otherwise. -``` -Interpreter and Prog Feature +### Interpreter and Prog Feature + These are described elsewhere in the manual: -APPLY, EVAL, EVLIS, QUOTE, LABEL, FUNCTION, PROG, GO, RETURN, SET, -SETQ. -``` +`APPLY, EVAL, EVLIS, QUOTE, LABEL, FUNCTION, PROG, GO, RETURN, SET, SETQ.` -``` -Defining Functions and Functions Useful for Property Lists -``` +### Defining Functions and Functions Useful for Property Lists + +#### define [x] : EXPR pseudo-function + +The argument of `define`, `x`, is a list of pairs + +> ((ul vl) (u2 v2) ... (un vn)) + +where each `u` is a name and each `v` is a λ-expression for a function . For each `pair`, define puts an `EXPR` on the property list for `u` pointing to `v`. The function of `define` puts things on at the front of the property list. The value of `define` is the list of `u`s. + +> define[x] = deflist[x; EXPR] -- define [x] EXPR pseudo-function -``` -The argument of define, x, is a list of pairs -((ul vl) (uz v2) tun vn)) -where each u is a name and each v is a A-expression for a function. For each pair, -define puts an EXPR on the property list for u pointing to v. The function of define -puts things on at the front of the property list. The value of define is the list of u's. -define[x] = deflist[x; EXPR] -``` -``` deflist [x; ind] EXPR pseudo-function The function deflist is a more general defining function. Its first argument is a list of pairs as for define. Its second argument is the indicator that is to be used. After deflist has been executed with (ui vi) among its first argument, the property list of ui will begin: -``` -``` If deflist or define is used twice on the same object with the same indicator, the old value will be replaced by the new one. attrib[x;e] - SUBR pseudo-function diff --git a/doc/mexpr.md b/doc/mexpr.md index 60f9ff3..ee5b8a9 100644 --- a/doc/mexpr.md +++ b/doc/mexpr.md @@ -1,8 +1,9 @@ -# M-Expressions +# Interpreting M-Expressions -M-Expressions ('mexprs') are the grammar which John McCarthy origininally used to write Lisp, and the grammar in which many of the function definitions in the [Lisp 1.5 Programmer's Manual](https://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf) are stated. However, I have not seen anywhere a claim that Lisp 1.5 could *read* M-Expressions, and it is not clear to me whether it was even planned that it should do so. +M-Expressions ('mexprs') are the grammar which John McCarthy origininally used to write Lisp, and the grammar in which many of the function definitions in the [Lisp 1.5 Programmer's Manual](https://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf) are stated. However, I have not seen anywhere a claim that Lisp 1.5 could *read* M-Expressions, and it is not clear to me whether it was even planned that it should do so, although the discussion on [page 10](https://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf#page=18) suggests that it was. -Rather, it seems to me probably that M-Expressions were only ever a grammar intended to be written on paper, like [Backus Naur Form](https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_form), to describe and to reason about algorithms. +Rather, it seems to me possible that M-Expressions were only ever a grammar intended to be written on paper, like [Backus Naur Form](https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_form), to describe and to reason about algorithms. I think at the point +at which the M-Expression grammar was written, the idea of the [universal Lisp function](https://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf#page=18) I set out to make Beowulf read M-Expressions essentially out of curiousity, to see whether it could be done. I had this idea that if it could be done, I could implement most of Lisp 1.5 simply by copying in the M-Expression definitions out of the manual. @@ -45,7 +46,7 @@ is ((QUOTE T) (QUOTE F)))))) ``` -This is certainly more prolix and more awkward, but it also risks being flat wrong. +This is certainly more prolix and more awkward. Is the value of `NIL` the atom `NIL`, or is it the empty list `()`? If the former, then the translation from the M-Expression above is correct. However, that means that recursive functions which recurse down a list seeking the end will fail. So the latter must be the case. @@ -57,8 +58,10 @@ Is the value of `NIL` the atom `NIL`, or is it the empty list `()`? If the forme I think there is an ambiguity in referencing constants which are not bound to themselves in the M-Expression notation as given in the manual. This is particularly problematic with regards to `NIL` and `F`, but there may be others instances. +However, so long as `F` is bound to `NIL`, and `NIL` is also bound to `NIL` (both of which are true by default, although changeable by the user), and `NIL` is the special marker used in the `CDR` of the last cons cell of a flat list, this is a difference which in practice does not make a difference. I still find it worrying, though, that rebinding variables could lead to disaster. + ### Curly braces -The use of curly braces is not defined in the grammar as stated on page 10. They are not used in the initial definition of `APPLY` on page 13, but they are used in the more developed restatement on page 70. I believe they are to be read as indicating a `DO` statement -- a list of function calls to be made sequentially but without strict functional dependence on one another -- but I don't find the exposition here particularly clear and I'm not sure of this. +The use of curly braces is not defined in the grammar as stated on page 10. They are not used in the initial definition of `APPLY` on page 13, but they are used in the more developed restatement on page 70. I believe they are to be read as indicating a section of assembly code to be assembled by the [Lisp Assembly Program](https://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf#page=81) -- but I don't find the exposition here particularly clear and I'm not sure of this. Consequently, the M-Expression interpreter in Beowulf does not interpret curly braces. \ No newline at end of file diff --git a/doc/values.md b/doc/values.md index 0639a01..2c73959 100644 --- a/doc/values.md +++ b/doc/values.md @@ -1,8 +1,109 @@ -# Understanding values and properties +# The properties of the system, and their values: here be dragons -I had had the naive assumption that entries on the object list had their CAR pointing to the symbol and their CDR pointing to the related value. Consequently, I could not work out where the property list went. More careful reading of [the text] implies, but does not explicitly state, that my naive assumption is wrong. +Lisp is the list processing language; that is what its name means. It processes data structures built of lists - which may be lists of lists, or lists of numbers, or lists of any other sort of data item provided for by the designers of the system. -Instead, it appears that the `CAR` points to the symbol, as expected, but the `CAR` points to the property list; and that on the property list there are privileged properties at least as follows: +But how is a list, in a computer, actually implemented? + +They're implemented as pairs, or, as the manual sometimes rather delightfully called them, 'doublets'. Pairs of what? Pairs of pointers. Of the two pointers of a pair, the first points to the current entry of the list, and the second, by default, points to the remainder of the list, or, if the end of the list has been reached, to a special datum known as `NIL` which among other things indicates that the end of the list has been reached. The pair itself is normally referred to as a 'cons cell' for reasons which are nerdy and not important just now (all right, because they are constructed using a function called `cons`, which is in itself believed to be simply an abbreviation of 'construct'). + +Two functions are used to access the two pointers of the cell. In modern Lisps these functions are called `first` and `rest`, because a lot of people who aren't greybeards find these names easier. But they aren't the original names. The original names were `CAR` and `CDR`. + +Why? + +## History + +Lisp was originally written on an [IBM 704 computer at Massachusetts Institute of Technology](https://comphist.dhlab.mit.edu/archives/story/IBM_mechanics), almost seventy years ago. + +The machine had registers which were not eight, or sixteen, or thirty two, or sixty four, bits wide, or any other number which would seem rational to modern computer scientists, but thirty six. Myth - folk memory - tells us that the machine's memory was arranged in pages. As I understand it (but this truly is folk memory) the offset within the page of the word to be fetched was known as the 'decrement', while the serial number of the page in the addressing sequence was known as the 'address'. To fetch a word from memory, you first had to select the page using the 'address', and secondly the word itself using the 'decrement'. So there were specific instructions for selecting the address, and the decrement, from the register separately. + +There were two mnemonics for the machine instructions used to access the content of these registers, respectively: + +CAR + +: *Contents of the Address part of Register*; and + +CDR + +: *Contents of the Decrement part of Register*. + +Is this actually true? + +I think so. If you look at [page 80 of the Lisp 1 Programmer's Manual](https://bitsavers.org/pdf/mit/rle_lisp/LISP_I_Programmers_Manual_Mar60.pdf#page=89), you will see this: + +```IBM704 +TEN (the TEN-Mode is entered) + +O CAR (((A,B),C)) () \ + | +:1 CDR ((D,(E,F))) () | + > Type ins +:2 CONS ((G,H), | + | +230 (I,J)) () / +RLN | | oo 7 a | + +O14 (read lines O and 1) +``` + +Of course, this isn't proof. If `CAR` and `CDR` used here are standard IBM 704 assembler mnemonics -- as I believe they are -- then what is `CONS`? It's used in a syntactically identical way. If it also is an assembler mnemonic, then it's hard to believe that, as legend relates, it is short for 'construct'; on the other hand, if it's a label representing an entry point into a subroutine, then why should `CAR` and `CDR` not also be labels? + +I think that the answer has to be that if `CAR` and `CDR` had been named by the early Lisp team -- John McCarthy and his immediate colleagues -- they would not have been named as they were. If not `FRST` and `REST`, as in more modern Lisps, then something like `P1` and `P2`. `CAR` and `CDR` are distinctive and memorable (and therefore in my opinion worth preserving) because they very specifically name the parts of a cons cell and of nothing else. + +Let's be clear, here: when `CAR` and `CDR` are used in Lisp, they are returning pointers, certainly -- but not in the sense that one points to a page and the other to a word. Each is an offset into a cell array, which is almost certainly an array of single 36 bit words held on a single page. So both are in effect being used as decrements. Their use in Lisp is an overload onto their original semantic meaning; they are no longer being used for the purpose for which they are named. + +As far as I can tell, these names first appear in print in 1960, both in the Lisp 1 Programmer's Manual referenced above, and in McCarthy's paper [Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I](https://web.archive.org/web/20230328222647/http://www-formal.stanford.edu/jmc/recursive.pdf). The paper was published in April so was presumably written in 1959 + +## Grey Anatomy + +### The Object List + +Lisp keeps track of values by associating them with names. It does so by having what is in effect a global registry of all the names it knows to which values are attached. This being a list processing language, that was of course, in early Lisps, a list: a single specialised first class list known as the 'object list', or `oblist` for short. + +Of course, a list need not just be a list of single items, it can be a list of pairs: it can be a list of pairs of the form `(name . value)`. Hold onto that, because I want to talk about another fundamental part of a working Lisp system, the stack. + +### The Stack + +Considering the case of pure interpreter first, let's think about how a function keeps track of the data it's working on. In order to do its work, it probably calls other functions, to which it passes off data, and they in turn probably call further functions. So, when control returns to our first function, how does it know where its data is? The answer is that each function pushes its argument bindings onto the stack when it starts work, and pops them off again when it exits. So when control returns to a function, its own data is still on the top of the stack. Or, to be precise, actually it doesn't; in practice the function [`EVAL`](https://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf#page=79) does it for each function in turn. But it doesn't matter: it gets done. + +What is this stack? Well, it's a list of `(name . value)` pairs. At least, it is in pure Lisps; [Clojure](https://clojure.org/), because it runs on the [Java Virtual Machine](https://en.wikipedia.org/wiki/Java_virtual_machine) and interoperates with other software running on the JVM, uses the JVM stack which is a permanently reserved vector of memory never used for anything else. Consequently it cannot be very large; and the consequence of that is that it's very easy to crash JVM programs because they've run out of stack space. + +The advantage of organising your stack as a vector is that on average it's usually slightly more memory efficient, and that it's somewhat faster to access. The disadvantage is you need a contiguous block of memory for it, and once you've run out, you've at best lost both those advantages but in the normal case your program just crashes. Also, the memory you've reserved for the stack isn't available for any other use, even during the most of the time that the stack isn't using most of it. So of course there's a temptation to keep the amount reserved for the stack as small as possible. + +It's this brutal fragility of vector stacks -- which are used by most modern computer languages -- which makes software people so wary of fully exploiting the beauty and power of recursion, and I really think that's a shame. + +The advantage of organising your stack as a list is that, while there is any memory left on the machine at all, you cannot run out of stack. + +### The Spine + +So, there's an object list where you associate names and values, and there's a stack where you associate names and values. But, why do they have to be different? And why do you have to search in two places to find the value of a name? + +The answer is -- or it was, and arguably it should be -- that you don't. The stack can simply be pushed onto the front of the object list. This has multiple advantages. The first and most obvious is that you only have to search in one place for the value associated with a name. + +The second is more subtle: a function can mask a variable in the object list by binding the same name to a new value, and the functions to which it then calls will only see that new value. This is useful if, for example, printed output is usually sent to the user's terminal, but for a particular operation you want to send it to a line printer or to a file on disk. You simply rebind the name of the standard output stream to your chosen output stream, and call the function whose output you want to redirect. + +So, in summary, there's a lot of merit in making the stack and the object list into a single central structure on which the architecture of our Lisp system is built. But there's more we need to record, and it's important. + +### Fields and Properties + +No, I'm not banging on about [land reform](https://www.journeyman.cc/blog/tags-output/Levelling/) again! I'm not a total monomaniac! + +But there's more than one datum we may want to associate with a single name. A list can be more than a set of bindings between single names and single values. There's more than one thing, for example, that I know about my friend Lucy. I know her age. I know her address. I know her height. I know her children. I know her mother. All of this information -- and much more -- should be associated with her. + +In conventional computing systems we'd use a table. We'd put into the table a field -- a column -- for each datum we wanted to store about a class of things of interest. And we'd reserve space to store that datum for every record, whether every record had something to go there of not. Furthermore, we'd have to reserve space in each field for the maximum size of the datum to be stored in it -- so if we needed to store full names for even some of the people we knew, and one of the people whose full name we needed to store (because he's both very important and very irascible) was *Charles Philip Arthur George Windsor*, then we'd have to reserve space for thirty-six characters for the full name of everyone in our records, even if for most of them half that would be enough. + +But if instead of storing a table for each sort of thing on which we hold data, and a row in that table for each item of that sort on which we store data, we simply tagged each thing on which we hold data with those things which are interesting about them? We could tag my friend Lucy with the fact she's on pilgrimage, and what her pilgrimage route is. Those aren't things we need to know about most people, it would be absurdly wasteful to add a column to a `person` table to record `pilgrimage route`. So in a conventional data system we would lose that data. + +Lisp has had, right back from the days of Lisp 1.5 -- so, for sixty-five years -- a different solution. We can give every symbol arbitrarily many, arbitrarily different, properties. A property is a `(name . value)` pair. We don't have to store the same properties for every object. The values of the properties don't have to have a fixed size, and they don't have to take up space they don't need. It's like having a table with as many fields as we choose, and being able to add more fields at any time. + +So, in summary, I knew, in building Beowulf, that I'd have to implement property lists. I just didn't know how I was going to do it. + +## Archaeology + +What I'm doing with Beowulf is trying to better understand the history of Lisp by reconstructing a very early example; in this case, Lisp 1.5, from about 1962, or sixty one years ago. + +I had had the naive assumption that entries on the object list in early Lisps had their `CAR` pointing to the symbol and their `CDR` pointing to the related value. Consequently, in building [beowulf](https://github.com/simon-brooke/beowulf), I could not work out where the property list went. More careful reading of [the text](https://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf#page=67) implies, but does not explicitly state, that my naive assumption is wrong. + +Instead, it appears that the `CAR` points to the symbol, as expected, but the `CDR` points to the property list; and that on the property list there are privileged properties at least as follows: APVAL : the simple straightforward ordinary value of the symbol, considered as a variable; @@ -11,13 +112,13 @@ EXPR : the definition of the function considered as a normal lambda expression (arguments to be evaluated before applying); FEXPR -: the definition of a function which should be applied to unevaluated arguments; +: the definition of a function which should be applied to unevaluated arguments (what InterLisp and Portable Standard Lisp would call ['*nlambda*'](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=fe0f4f19cee0c607d7b229feab26fc9ed559fc9c#page=9)); SUBR -: the definition of a complied subroutine which should be applied to evaluated arguments; +: the definition of a compiled subroutine which should be applied to evaluated arguments; FSUBR -: the definition of a complied subroutine which should be applied to unevaluated arguments. +: the definition of a compiled subroutine which should be applied to unevaluated arguments. I think there was also another privileged property value which contained the property considered as a constant, but I haven't yet confirmed that. @@ -33,8 +134,134 @@ Essentially the properties are tried in turn, and only the first value found is This means that, while the other potential values can be retrieved from the property list, interpreted definitions (if present) will always be preferred to uninterpreted definitions, and lambda function definitions (which evaluate their arguments), where present, will always be preferred to non-lamda definitions, which don't. -**BUT NOTE THAT** the `APVAL` value is sought only when seeking a variable value for the symbol, and the others only when seeking a function value, so Lisp 1.5 is a 'Lisp 2', not a 'Lisp 1'. +**BUT NOTE THAT** the `APVAL` value is sought only when seeking a variable value for the symbol, while the others are only when seeking a function value, so Lisp 1.5 is a 'Lisp 2', not a 'Lisp 1'. I strongly believe that this is wrong: a function is a value, and should be treated as such. But at the same time I do acknowledge the benefit of being able to store both source and compiled forms of the function as properties of the same symbol. +## The persistent problem +There's a view in modern software theory -- with which I strongly hold -- that data should be immutable. Data that changes under you is the source of all sorts of bugs. And in modern multi threaded systems, the act of reading a datum whilst some other process is writing it, or worse, two processes attempting simultaneously to write the same datum, is a source of data corruption and even crashes. So I'm very wary of mutable data; and, in modern systems where we normally have a great deal of space and a lot of processor power, making fresh copies of data structures containing the change we wanted to make is a reasonable price to pay for avoiding a whole class of bugs. -Versions of Beowulf up to and including 0.2.1 used the naive understanding of the architecture; version 0.3.0 *should* use the corrected version. +But early software was not like that. It was always constrained by the limits of the hardware on which it ran, to a degree that we are not. And the experience that we now have of the problems caused by mutable data, they did not have. So it's core to the design of Lisp 1.5 that its lists are mutable; and, indeed, one of the biggest challenges in writing Beowulf has been [implementing mutable lists in Clojure](https://github.com/simon-brooke/beowulf/blob/master/src/beowulf/cons_cell.clj#L19), a language carefully designed to prevent them. + +But, just because Lisp 1.5 lists can be mutable, should they be? And when should they be? + +The problem here is that [spine of the system](#the_spine) I talked about earlier. If we build the execution stack on top of the oblist -- as at present I do -- then if we make a new version of the oblist with changes in it, the new changes will not be in the copy of the oblist that the stack is built on top of; and consequently, they'll be invisible to the running program. + +What I do at present, and what I think may be good enough, is that each time execution returns to the read-eval-print loop, the REPL, the user's command line, I rebuild a new execution stack on the top of the oblist as it exists now. So, if the last operation modified the oblist, the next operation will see the new, modified version. But if someone tried to run some persistent program which was writing stuff to property values and hoping to read them back in the same computation, that wouldn't work, and it would be a very hard bug to trace down. + +So my options are: + +1. To implement `PUT` and `GET` in Clojure, so that they can operate on the current copy of the object list, not the one at the base of the stack. I'm slightly unwilling to do that, because my objective is to make Beowulf ultimately as self-hosting as possible. +2. To implement `PUT` and `GET` in Lisp, and have them destructively modify the working copy of the object list. + +Neither of these particularly appeal. + +## How property lists should work + +I'm still not fully understanding how property lists in Lisp 1.5 are supposed to work. + +### List format + +Firstly, are they association lists comprising dotted pairs of `(property-name . value)`, i.e.: + +>((property-name1 . value1) (property-name2 . value2) ... (property-namen . valuen)) + +I have assumed so, and that is what I presently intend to implement, but the diagrams on [pages 59 and 60](https://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf#page=67) seem rather to show a flat list of interleaved names and values: + +> (property-name1 value1 property-name2 value2 ... property-namen valuen) + +I cannot see what the benefit of this latter arrangement is, and I'm unwilling to do it, although I think it may be what was done. But if it was done that way, *why* was it done that way? These were bright people, and they certainly knew about association lists. So... I'm puzzled. + +### Function signatures + +To associate the value of a property with a symbol, we need three things: we need the symbol, we need the property name, and we need the value. For this reason, [Portable Standard Lisp](https://www.softwarepreservation.org/projects/LISP/utah/USCP-Portable_Standard_LISP_Users_Manual-TR_10-1984.pdf#page=44) and others has a function `put` with three arguments: + +> `(Put U:id IND:id PROP:any)`: any +> The indicator `IND` with the property `PROP` is placed on the property list of +> the id `U`. If the action of Put occurs, the value of `PROP` is returned. If +> either of `U` and `IND` are not ids the type mismatch error occurs and no +> property is placed. +> `(Put 'Jim 'Height 68)` +> The above returns `68` and places `(Height . 68)` on the property list of the id `Jim` + +Cambridge Lisp is identical to this except in lower case. [InterLisp](https://larrymasinter.net/86-interlisp-manual-opt.pdf#page=37) and several others have `putprop`: + +> `(PUTPROP ATM PROP VAL) [Function]` +> Puts the property `PROP` with value `VAL` on the property list of `ATM`. `VAL` replaces +> any previous value for the property `PROP` on this property list. Returns `VAL`. + +The execrable Common Lisp uses its execrable macro [`setf`](https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node108.html) but really the less said about that the better. + +So I was looking for a function of three arguments to set properties, and I didn't find one. + +There's a function `DEFINE` which takes one argument, an association list of pairs: +```lisp + (function-name . function-definition)` +``` +So how does that work, if what it's doing is setting properties? If all you're passing is pairs of name and definition, where does the property name come from? + +The answer is as follows, taken from [the manual](https://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf#page=66): + +> #### define [x] : EXPR pseudo-function + +> The argument of `define`, `x`, is a list of pairs + +> > ((ul vl) (u2 v2) ... (un vn)) + +> where each `u` is a name and each `v` is a λ-expression for a function . For each `pair`, define puts an `EXPR` on the property list for `u` pointing to `v`. The function of `define` puts things on at the front of the property list. The value of `define` is the list of `u`s. + +So, in fact, the value of the property being set by `define` is fixed: hard wired, not parameterised. That seems an astonishing decision, until you realise that Lisp 1.5's creators weren't creating their functions one by one, in a playful exploration with their system, but entering them in a batch. + +## Learning by doing + +In fact, when I got over my surprise, I realised that that `(name . function-definition)` list is actually very much like this, which is an excerpt from a sysout from a Beowulf prototype: + +```lisp +(... + (MAPLIST LAMBDA (L F) + (COND ((NULL L) NIL) + ((QUOTE T) (CONS (F (CAR L)) (MAPLIST (CDR L) F))))) + (MEMBER LAMBDA (A X) + (COND ((NULL X) (QUOTE F)) + ((EQ A (CAR X)) (QUOTE T)) + ((QUOTE T) (MEMBER A (CDR X))))) + (MINUSP LAMBDA (X) (LESSP X 0)) + (NOT LAMBDA (X) + (COND (X (QUOTE NIL)) + ((QUOTE T) (QUOTE T)))) + (NULL LAMBDA (X) + (COND ((EQUAL X NIL) (QUOTE T)) + (T (QUOTE F)))) +...) +``` + +I was looking at `DEFINE` and thinking, 'why would one ever want to do that?' and then I found that, behind the scenes, I was actually doing it myself. + +Because the point of a sysout is you don't write it. The point about the REPL -- the Read Eval Print Loop which is the heart of the interactive Lisp development cycle, where you sit playing with things and fiddling with them interactively, and where when one thing works you get onto the next without bothering to make some special effort to record it. + +The point of a sysout is that, at the end of the working day, you invoke one function + +| Function | Type | Signature | Implementation | Documentation | +| -------- | ---- | --------- | -------------- | ------------- | +| SYSOUT | Host function | (SYSOUT); (SYSOUT FILEPATH) | SUBR | Dump the current content of the object list to file. If no `filepath` is specified, a file name will be constructed of the symbol `Sysout` and the current date. File paths will be considered relative to the filepath set when starting Lisp. | + +At the start of the next working day, you load that sysout in and continue your session. + +The sysout captures the entire working state of the machine. No-one types it in, as an operation in itself. Instead, data structures -- corpuses of functions among them -- simply build up on the object list almost casually, as a side effect of the fact that you're enjoying exploring your problem and finding elegant ways of solving it. So `SYSOUT` and `SYSIN` seem to me, as someone who all his adult life has worked with Lisp interactively, as just an automatic part of the cycle of the day. + +## The process of discovery + +The thing is, I don't think anyone is ever going to use Beowulf the way Lisp 1.5 was used. I mean, probably, no one is ever going to use Beowulf at all; but if they did they wouldn't use Beowulf the way Lisp 1.5 was used. + +I'm a second generation software person. I have worked, in my career, with two people who personally knew and had worked with [Alan Turing](https://en.wikipedia.org/wiki/Alan_Turing). I have worked with, and to an extent been mentored by, [Chris Burton](https://www.bcs.org/articles-opinion-and-research/manchester-s-place-in-computing-history-marked/), who in his apprenticeship was part of the team that built the Manchester Mark One, and who in his retirement led the team who restored it. But I never knew the working conditions they were accustomed to. In my first year at university we used card punches, and, later, when we had a bit of seniority, teletypewriters (yes, that's what TTY stands for), but by the time I'd completed my undergraduate degree and become a research associate I had a Xerox 1108 workstation with a huge bitmapped graphic screen, and an optical mouse, goddamit, running InterLisp, all to myself. + +People in the heroic age did not have computers all to themselves. They did not have terminals all to themselves. They didn't sit at a terminal experimenting in the REPL. They wrote their algorithms in pencil on paper. When they were certain they'd got it right, they'd use a card punch to punch a deck of cards carrying the text of the program, and then they were certain they'd got *that* right, they'd drop it into the input hopper. Some time later their batch would run, and the operator would put the consequent printout into their pigeon hole for them to collect. + +(They wrote amazingly clean code, those old masters. I could tell you a story about Chris Burton, the train, and the printer driver, that software people of today simply would not believe. But it's true. And I think that what taught them that discipline was the high cost of even small errors.) + +Lisp 1.5 doesn't have `PUT`, `PUTPROP` or `DEFUN` because setting properties individually, defining functions individually one at a time, was not something they ever thought about doing. And in learning that, I've learned more than I ever expected to about the real nature of Lisp 1.5, and the (great) people who wrote it. + +----- + +So what this is about is I've spent most of a whole day procrastinating, because I'm not exactly sure how I'm going to make the change I've got to make. Versions of Beowulf up to and including 0.2.1 used the naive understanding of the architecture; version 0.3.0 *should* use the corrected version. But before it can, I need to be reasonably confident that I understand what the correct solution is. + +I *shall* implement `PUT`, even though it isn't in the spec, because it's a useful building block on which to build `DEFINE` and `DEFLIS`, both of which are. And also, because `PUT` would have been very easy for the Lisp 1.5 implementers to implement, if it had been relevant to their working environment. diff --git a/docs/codox/beowulf.bootstrap.html b/docs/codox/beowulf.bootstrap.html index ed6eb8a..068508a 100644 --- a/docs/codox/beowulf.bootstrap.html +++ b/docs/codox/beowulf.bootstrap.html @@ -1,6 +1,6 @@ -beowulf.bootstrap documentation

Project

Topics

Namespaces

Public Vars

beowulf.bootstrap

Lisp as defined in Chapter 1 (pages 1-14) of the Lisp 1.5 Programmer's Manual; that is to say, a very simple Lisp language, which should, I believe, be sufficient in conjunction with the functions provided by beowulf.host, be sufficient to bootstrap the full Lisp 1.5 interpreter..

+beowulf.bootstrap documentation

Project

Topics

Namespaces

Public Vars

beowulf.bootstrap

Lisp as defined in Chapter 1 (pages 1-14) of the Lisp 1.5 Programmer's Manual; that is to say, a very simple Lisp language, which should, I believe, be sufficient in conjunction with the functions provided by beowulf.host, be sufficient to bootstrap the full Lisp 1.5 interpreter..

The convention is adopted that functions in this file with names in ALLUPPERCASE are Lisp 1.5 functions (although written in Clojure) and that therefore all arguments must be numbers, symbols or beowulf.cons_cell.ConsCell objects.

APPLY

(APPLY function args environment depth)

Apply this function to these arguments in this environment and return the result.

For bootstrapping, at least, a version of APPLY written in Clojure. All args are assumed to be symbols or beowulf.cons-cell/ConsCell objects. See page 13 of the Lisp 1.5 Programmers Manual.

view source

CAAAAR

macro

(CAAAAR x)

TODO: write docs

view source

CAAADR

macro

(CAAADR x)

TODO: write docs

view source

CAAAR

macro

(CAAAR x)

TODO: write docs

view source

CAADAR

macro

(CAADAR x)

TODO: write docs

view source

CAADDR

macro

(CAADDR x)

TODO: write docs

view source

CAADR

macro

(CAADR x)

TODO: write docs

view source

CAAR

macro

(CAAR x)

TODO: write docs

view source

CADAAR

macro

(CADAAR x)

TODO: write docs

view source

CADADR

macro

(CADADR x)

TODO: write docs

view source

CADAR

macro

(CADAR x)

TODO: write docs

view source

CADDAR

macro

(CADDAR x)

TODO: write docs

view source

CADDDR

macro

(CADDDR x)

TODO: write docs

view source

CADDR

macro

(CADDR x)

TODO: write docs

view source

CADR

macro

(CADR x)

TODO: write docs

view source

CDAAAR

macro

(CDAAAR x)

TODO: write docs

view source

CDAADR

macro

(CDAADR x)

TODO: write docs

view source

CDAAR

macro

(CDAAR x)

TODO: write docs

view source

CDADAR

macro

(CDADAR x)

TODO: write docs

view source

CDADDR

macro

(CDADDR x)

TODO: write docs

view source

CDADR

macro

(CDADR x)

TODO: write docs

view source

CDAR

macro

(CDAR x)

TODO: write docs

view source

CDDAAR

macro

(CDDAAR x)

TODO: write docs

view source

CDDADR

macro

(CDDADR x)

TODO: write docs

view source

CDDAR

macro

(CDDAR x)

TODO: write docs

view source

CDDDAR

macro

(CDDDAR x)

TODO: write docs

view source

CDDDDR

macro

(CDDDDR x)

TODO: write docs

view source

CDDDR

macro

(CDDDR x)

TODO: write docs

view source

CDDR

macro

(CDDR x)

TODO: write docs

view source

EVAL

(EVAL expr)(EVAL expr env depth)

Evaluate this expr and return the result. If environment is not passed, it defaults to the current value of the global object list. The depth argument is part of the tracing system and should not be set by user code.

All args are assumed to be numbers, symbols or beowulf.cons-cell/ConsCell objects.

view source

INTEROP

(INTEROP fn-symbol args)

Clojure (or other host environment) interoperation API. fn-symbol is expected to be either

diff --git a/docs/codox/beowulf.cons-cell.html b/docs/codox/beowulf.cons-cell.html index e7a05b4..26fa55a 100644 --- a/docs/codox/beowulf.cons-cell.html +++ b/docs/codox/beowulf.cons-cell.html @@ -1,3 +1,3 @@ -beowulf.cons-cell documentation

Project

Topics

Namespaces

Public Vars

beowulf.cons-cell

The fundamental cons cell on which all Lisp structures are built. Lisp 1.5 lists do not necessarily have a sequence as their CDR, and must have both CAR and CDR mutable, so cannot be implemented on top of Clojure lists.

cons-cell?

(cons-cell? o)

Is this object o a beowulf cons-cell?

view source

F

The canonical false value - different from NIL, which is not canonically false in Lisp 1.5.

view source

make-beowulf-list

(make-beowulf-list x)

Construct a linked list of cons cells with the same content as the sequence x.

view source

make-cons-cell

(make-cons-cell car cdr)

Construct a new instance of cons cell with this car and cdr.

view source

MutableSequence

protocol

Like a sequence, but mutable.

members

getCar

(getCar this)

Return the first element of this sequence.

getCdr

(getCdr this)

like more, q.v., but returns List NIL not Clojure nil when empty.

getUid

(getUid this)

Returns a unique identifier for this object

rplaca

(rplaca this value)

replace the first element of this sequence with this value

rplacd

(rplacd this value)

replace the rest (but-first; cdr) of this sequence with this value

view source

pretty-print

(pretty-print cell)(pretty-print cell width level)

This isn’t the world’s best pretty printer but it sort of works.

view source

T

The canonical true value.

view source
\ No newline at end of file +beowulf.cons-cell documentation

Project

Topics

Namespaces

Public Vars

beowulf.cons-cell

The fundamental cons cell on which all Lisp structures are built. Lisp 1.5 lists do not necessarily have a sequence as their CDR, and must have both CAR and CDR mutable, so cannot be implemented on top of Clojure lists.

cons-cell?

(cons-cell? o)

Is this object o a beowulf cons-cell?

view source

F

The canonical false value - different from NIL, which is not canonically false in Lisp 1.5.

view source

make-beowulf-list

(make-beowulf-list x)

Construct a linked list of cons cells with the same content as the sequence x.

view source

make-cons-cell

(make-cons-cell car cdr)

Construct a new instance of cons cell with this car and cdr.

view source

MutableSequence

protocol

Like a sequence, but mutable.

members

getCar

(getCar this)

Return the first element of this sequence.

getCdr

(getCdr this)

like more, q.v., but returns List NIL not Clojure nil when empty.

getUid

(getUid this)

Returns a unique identifier for this object

rplaca

(rplaca this value)

replace the first element of this sequence with this value

rplacd

(rplacd this value)

replace the rest (but-first; cdr) of this sequence with this value

view source

pretty-print

(pretty-print cell)(pretty-print cell width level)

This isn’t the world’s best pretty printer but it sort of works.

view source

T

The canonical true value.

view source
\ No newline at end of file diff --git a/docs/codox/beowulf.core.html b/docs/codox/beowulf.core.html index b4494b3..660d5f5 100644 --- a/docs/codox/beowulf.core.html +++ b/docs/codox/beowulf.core.html @@ -1,3 +1,3 @@ -beowulf.core documentation

Project

Topics

Namespaces

Public Vars

beowulf.core

Essentially, the -main function and the bootstrap read-eval-print loop.

-main

(-main & opts)

Parse options, print the banner, read the init file if any, and enter the read/eval/print loop.

view source

cli-options

TODO: write docs

view source

repl

(repl prompt)

Read/eval/print loop.

view source

stop-word

TODO: write docs

view source
\ No newline at end of file +beowulf.core documentation

Project

Topics

Namespaces

Public Vars

beowulf.core

Essentially, the -main function and the bootstrap read-eval-print loop.

-main

(-main & opts)

Parse options, print the banner, read the init file if any, and enter the read/eval/print loop.

view source

cli-options

TODO: write docs

view source

repl

(repl prompt)

Read/eval/print loop.

view source

stop-word

TODO: write docs

view source
\ No newline at end of file diff --git a/docs/codox/beowulf.gendoc.html b/docs/codox/beowulf.gendoc.html index 331c022..208efe6 100644 --- a/docs/codox/beowulf.gendoc.html +++ b/docs/codox/beowulf.gendoc.html @@ -1,4 +1,4 @@ -beowulf.gendoc documentation

Project

Topics

Namespaces

Public Vars

beowulf.gendoc

Generate table of documentation of Lisp symbols and functions.

+beowulf.gendoc documentation

Project

Topics

Namespaces

Public Vars

beowulf.gendoc

Generate table of documentation of Lisp symbols and functions.

NOTE: this is very hacky. You almost certainly do not want to use this!

find-documentation

(find-documentation entry)

Find appropriate documentation for this entry from the oblist.

view source

gen-doc-table

(gen-doc-table)

TODO: write docs

view source

gen-index

(gen-index)(gen-index url destination)

TODO: write docs

view source

host-functions

Functions which we can infer are written in Clojure. We need to collect these at run-time, not compile time, hence memoised function, not variable.

view source

infer-implementation

(infer-implementation entry)

TODO: write docs

view source

infer-signature

(infer-signature entry)

Infer the signature of the function value of this oblist entry, if any.

view source

infer-type

(infer-type entry)

Try to work out what this entry from the oblist actually represents.

view source

open-doc

(open-doc symbol)

Open the documentation page for this symbol, if known, in the default web browser.

view source
\ No newline at end of file diff --git a/docs/codox/beowulf.host.html b/docs/codox/beowulf.host.html index 146e37c..c22765d 100644 --- a/docs/codox/beowulf.host.html +++ b/docs/codox/beowulf.host.html @@ -1,6 +1,6 @@ -beowulf.host documentation

Project

Topics

Namespaces

Public Vars

beowulf.host

provides Lisp 1.5 functions which can’t be (or can’t efficiently be) implemented in Lisp 1.5, which therefore need to be implemented in the host language, in this case Clojure.

ADD1

(ADD1 x)

TODO: write docs

view source

AND

(AND & args)

T if and only if none of my args evaluate to either F or NIL, else F.

+beowulf.host documentation

Project

Topics

Namespaces

Public Vars

beowulf.host

provides Lisp 1.5 functions which can’t be (or can’t efficiently be) implemented in Lisp 1.5, which therefore need to be implemented in the host language, in this case Clojure.

ADD1

(ADD1 x)

TODO: write docs

view source

AND

(AND & args)

T if and only if none of my args evaluate to either F or NIL, else F.

In beowulf.host principally because I don’t yet feel confident to define varargs functions in Lisp.

view source

ASSOC

(ASSOC x a)

If a is an association list such as the one formed by PAIRLIS in the above example, then assoc will produce the first pair whose first term is x. Thus it is a table searching function.

All args are assumed to be beowulf.cons-cell/ConsCell objects. See page 12 of the Lisp 1.5 Programmers Manual.

NOTE THAT this function is overridden by an implementation in Lisp, but is currently still present for bootstrapping.

view source

ATOM

(ATOM x)

Returns T if and only if the argument x is bound to an atom; else F. It is not clear to me from the documentation whether (ATOM 7) should return T or F. I’m going to assume T.

view source

ATOM?

macro

(ATOM? x)

The convention of returning F from predicates, rather than NIL, is going to tie me in knots. This is a variant of ATOM which returns NIL on failure.

view source

CAAAAR

macro

(CAAAAR x)

TODO: write docs

view source

CAAADR

macro

(CAAADR x)

TODO: write docs

view source

CAAAR

macro

(CAAAR x)

TODO: write docs

view source

CAADAR

macro

(CAADAR x)

TODO: write docs

view source

CAADDR

macro

(CAADDR x)

TODO: write docs

view source

CAADR

macro

(CAADR x)

TODO: write docs

view source

CAAR

macro

(CAAR x)

TODO: write docs

view source

CADAAR

macro

(CADAAR x)

TODO: write docs

view source

CADADR

macro

(CADADR x)

TODO: write docs

view source

CADAR

macro

(CADAR x)

TODO: write docs

view source

CADDAR

macro

(CADDAR x)

TODO: write docs

view source

CADDDR

macro

(CADDDR x)

TODO: write docs

view source

CADDR

macro

(CADDR x)

TODO: write docs

view source

CADR

macro

(CADR x)

TODO: write docs

view source

CAR

(CAR x)

Return the item indicated by the first pointer of a pair. NIL is treated specially: the CAR of NIL is NIL.

view source

CDAAAR

macro

(CDAAAR x)

TODO: write docs

view source

CDAADR

macro

(CDAADR x)

TODO: write docs

view source

CDAAR

macro

(CDAAR x)

TODO: write docs

view source

CDADAR

macro

(CDADAR x)

TODO: write docs

view source

CDADDR

macro

(CDADDR x)

TODO: write docs

view source

CDADR

macro

(CDADR x)

TODO: write docs

view source

CDAR

macro

(CDAR x)

TODO: write docs

view source

CDDAAR

macro

(CDDAAR x)

TODO: write docs

view source

CDDADR

macro

(CDDADR x)

TODO: write docs

view source

CDDAR

macro

(CDDAR x)

TODO: write docs

view source

CDDDAR

macro

(CDDDAR x)

TODO: write docs

view source

CDDDDR

macro

(CDDDDR x)

TODO: write docs

view source

CDDDR

macro

(CDDDR x)

TODO: write docs

view source

CDDR

macro

(CDDR x)

TODO: write docs

view source

CDR

(CDR x)

Return the item indicated by the second pointer of a pair. NIL is treated specially: the CDR of NIL is NIL.

view source

CONS

(CONS car cdr)

Construct a new instance of cons cell with this car and cdr.

view source

CONSP

(CONSP o)

Return T if object o is a cons cell, else F.

diff --git a/docs/codox/beowulf.io.html b/docs/codox/beowulf.io.html index 1158db6..2d19239 100644 --- a/docs/codox/beowulf.io.html +++ b/docs/codox/beowulf.io.html @@ -1,6 +1,6 @@ -beowulf.io documentation

Project

Topics

Namespaces

Public Vars

beowulf.io

Non-standard extensions to Lisp 1.5 to read and write to the filesystem.

+beowulf.io documentation

Project

Topics

Namespaces

Public Vars

beowulf.io

Non-standard extensions to Lisp 1.5 to read and write to the filesystem.

Lisp 1.5 had only READ, which read one S-Expression at a time, and various forms of PRIN* functions, which printed to the line printer. There was also PUNCH, which wrote to a card punch. It does not seem that there was any concept of an interactive terminal.

See Appendix E, OVERLORD - THE MONITOR, and Appendix F, LISP INPUT AND OUTPUT.

diff --git a/docs/codox/beowulf.manual.html b/docs/codox/beowulf.manual.html index fab729e..bf60737 100644 --- a/docs/codox/beowulf.manual.html +++ b/docs/codox/beowulf.manual.html @@ -1,3 +1,3 @@ -beowulf.manual documentation

Project

Topics

Namespaces

Public Vars

beowulf.manual

Experimental code for accessing the manual online.

*manual-url*

dynamic

TODO: write docs

view source

format-page-references

(format-page-references fn-symbol)

Format page references from the manual index for the function whose name is fn-symbol.

view source

index

This is data extracted from the index pages of Lisp 1.5 Programmer's Manual. It’s here in the hope that we can automatically link to an online PDF link to the manual when the user invokes a function probably called DOC or HELP.

view source

page-url

(page-url page-no)

Format the URL for the page in the manual with this page-no.

view source
\ No newline at end of file +beowulf.manual documentation

Project

Topics

Namespaces

Public Vars

beowulf.manual

Experimental code for accessing the manual online.

*manual-url*

dynamic

TODO: write docs

view source

format-page-references

(format-page-references fn-symbol)

Format page references from the manual index for the function whose name is fn-symbol.

view source

index

This is data extracted from the index pages of Lisp 1.5 Programmer's Manual. It’s here in the hope that we can automatically link to an online PDF link to the manual when the user invokes a function probably called DOC or HELP.

view source

page-url

(page-url page-no)

Format the URL for the page in the manual with this page-no.

view source
\ No newline at end of file diff --git a/docs/codox/beowulf.oblist.html b/docs/codox/beowulf.oblist.html index a71aac0..241acff 100644 --- a/docs/codox/beowulf.oblist.html +++ b/docs/codox/beowulf.oblist.html @@ -1,5 +1,5 @@ -beowulf.oblist documentation

Project

Topics

Namespaces

Public Vars

beowulf.oblist

A namespace mainly devoted to the object list and other top level global variables.

+beowulf.oblist documentation

Project

Topics

Namespaces

Public Vars

beowulf.oblist

A namespace mainly devoted to the object list and other top level global variables.

Yes, this makes little sense, but if you put them anywhere else you end up in cyclic dependency hell.

*options*

dynamic

Command line options from invocation.

view source

NIL

The canonical empty list symbol.

TODO: this doesn’t really work, because (from Clojure) (empty? NIL) throws an exception. It might be better to subclass beowulf.cons_cell.ConsCell to create a new singleton class Nil which overrides the empty method of IPersistentCollection?

view source

oblist

The default environment.

view source
\ No newline at end of file diff --git a/docs/codox/beowulf.read.html b/docs/codox/beowulf.read.html index b985082..115d614 100644 --- a/docs/codox/beowulf.read.html +++ b/docs/codox/beowulf.read.html @@ -1,6 +1,6 @@ -beowulf.read documentation

Project

Topics

Namespaces

Public Vars

beowulf.read

This provides the reader required for boostrapping. It’s not a bad reader - it provides feedback on errors found in the input - but it isn’t the real Lisp reader.

+beowulf.read documentation

Project

Topics

Namespaces

Public Vars

beowulf.read

This provides the reader required for boostrapping. It’s not a bad reader - it provides feedback on errors found in the input - but it isn’t the real Lisp reader.

Intended deviations from the behaviour of the real Lisp reader are as follows:

  1. It reads the meta-expression language MEXPR in addition to the symbolic expression language SEXPR, which I do not believe the Lisp 1.5 reader ever did;
    2. diff --git a/docs/codox/beowulf.reader.char-reader.html b/docs/codox/beowulf.reader.char-reader.html index d607448..8699ea3 100644 --- a/docs/codox/beowulf.reader.char-reader.html +++ b/docs/codox/beowulf.reader.char-reader.html @@ -1,6 +1,6 @@ -beowulf.reader.char-reader documentation

    Project

    Topics

    Namespaces

    Public Vars

    beowulf.reader.char-reader

    Provide sensible line editing, auto completion, and history recall.

    +beowulf.reader.char-reader documentation

    Project

    Topics

    Namespaces

    Public Vars

    beowulf.reader.char-reader

    Provide sensible line editing, auto completion, and history recall.

    None of what’s needed here is really working yet, and a pull request with a working implementation would be greatly welcomed.

    What’s needed (rough specification)

      diff --git a/docs/codox/beowulf.reader.generate.html b/docs/codox/beowulf.reader.generate.html index 65627db..574e61e 100644 --- a/docs/codox/beowulf.reader.generate.html +++ b/docs/codox/beowulf.reader.generate.html @@ -1,6 +1,6 @@ -beowulf.reader.generate documentation

      Project

      Topics

      Namespaces

      Public Vars

      beowulf.reader.generate

      Generating S-Expressions from parse trees.

      +beowulf.reader.generate documentation

      Project

      Topics

      Namespaces

      Public Vars

      beowulf.reader.generate

      Generating S-Expressions from parse trees.

      From Lisp 1.5 Programmers Manual, page 10

      Note that I’ve retyped much of this, since copy/pasting out of PDF is less than reliable. Any typos are mine.

      Quote starts:

      diff --git a/docs/codox/beowulf.reader.macros.html b/docs/codox/beowulf.reader.macros.html index af18089..510db75 100644 --- a/docs/codox/beowulf.reader.macros.html +++ b/docs/codox/beowulf.reader.macros.html @@ -1,5 +1,5 @@ -beowulf.reader.macros documentation

      Project

      Topics

      Namespaces

      Public Vars

      beowulf.reader.macros

      Can I implement reader macros? let’s see!

      +beowulf.reader.macros documentation

      Project

      Topics

      Namespaces

      Public Vars

      beowulf.reader.macros

      Can I implement reader macros? let’s see!

      We don’t need (at least, in the Clojure reader) to rewrite forms like 'FOO, because that’s handled by the parser. But we do need to rewrite things which don’t evaluate their arguments, like SETQ, because (unless LABEL does it, which I’m not yet sure of) we’re not yet able to implement things which don’t evaluate arguments.

      TODO: at this stage, the following should probably also be read macros: DEFINE

      *readmacros*

      dynamic

      TODO: write docs

      view source

      expand-macros

      (expand-macros form)

      TODO: write docs

      view source
      \ No newline at end of file diff --git a/docs/codox/beowulf.reader.parser.html b/docs/codox/beowulf.reader.parser.html index b091fc9..92309e8 100644 --- a/docs/codox/beowulf.reader.parser.html +++ b/docs/codox/beowulf.reader.parser.html @@ -1,3 +1,3 @@ -beowulf.reader.parser documentation

      Project

      Topics

      Namespaces

      Public Vars

      beowulf.reader.parser

      The actual parser, supporting both S-expression and M-expression syntax.

      parse

      Parse a string presented as argument into a parse tree which can then be operated upon further.

      view source
      \ No newline at end of file +beowulf.reader.parser documentation

      Project

      Topics

      Namespaces

      Public Vars

      beowulf.reader.parser

      The actual parser, supporting both S-expression and M-expression syntax.

      parse

      Parse a string presented as argument into a parse tree which can then be operated upon further.

      view source
      \ No newline at end of file diff --git a/docs/codox/beowulf.reader.simplify.html b/docs/codox/beowulf.reader.simplify.html index 55f654a..f2adfb6 100644 --- a/docs/codox/beowulf.reader.simplify.html +++ b/docs/codox/beowulf.reader.simplify.html @@ -1,4 +1,4 @@ -beowulf.reader.simplify documentation

      Project

      Topics

      Namespaces

      Public Vars

      beowulf.reader.simplify

      Simplify parse trees. Be aware that this is very tightly coupled with the parser.

      remove-nesting

      (remove-nesting tree context)

      TODO: write docs

      view source

      remove-optional-space

      (remove-optional-space tree)

      TODO: write docs

      view source

      simplify

      (simplify p)

      Simplify this parse tree p. If p is an instaparse failure object, throw an ex-info, with p as the value of its :failure key. Calls remove-optional-space before processing.

      view source

      simplify-tree

      (simplify-tree p)(simplify-tree p context)

      Simplify this parse tree p. If p is an instaparse failure object, throw an ex-info, with p as the value of its :failure key.

      +beowulf.reader.simplify documentation

      Project

      Topics

      Namespaces

      Public Vars

      beowulf.reader.simplify

      Simplify parse trees. Be aware that this is very tightly coupled with the parser.

      remove-nesting

      (remove-nesting tree context)

      TODO: write docs

      view source

      remove-optional-space

      (remove-optional-space tree)

      TODO: write docs

      view source

      simplify

      (simplify p)

      Simplify this parse tree p. If p is an instaparse failure object, throw an ex-info, with p as the value of its :failure key. Calls remove-optional-space before processing.

      view source

      simplify-tree

      (simplify-tree p)(simplify-tree p context)

      Simplify this parse tree p. If p is an instaparse failure object, throw an ex-info, with p as the value of its :failure key.

      NOTE THAT it is assumed that remove-optional-space has been run on the parse tree BEFORE it is passed to simplify-tree.

      view source
      \ No newline at end of file diff --git a/docs/codox/index.html b/docs/codox/index.html index b6001d0..54a0ed5 100644 --- a/docs/codox/index.html +++ b/docs/codox/index.html @@ -1,3 +1,3 @@ -Beowulf 0.3.0-SNAPSHOT

      Project

      Topics

      Namespaces

      Beowulf 0.3.0-SNAPSHOT

      Released under the GPL-2.0-or-later

      An implementation of LISP 1.5 in Clojure.

      Installation

      To install, add the following dependency to your project or build file:

      [beowulf "0.3.0-SNAPSHOT"]

      Topics

      Namespaces

      beowulf.bootstrap

      Lisp as defined in Chapter 1 (pages 1-14) of the Lisp 1.5 Programmer's Manual; that is to say, a very simple Lisp language, which should, I believe, be sufficient in conjunction with the functions provided by beowulf.host, be sufficient to bootstrap the full Lisp 1.5 interpreter..

      Public variables and functions:

      beowulf.cons-cell

      The fundamental cons cell on which all Lisp structures are built. Lisp 1.5 lists do not necessarily have a sequence as their CDR, and must have both CAR and CDR mutable, so cannot be implemented on top of Clojure lists.

      Public variables and functions:

      beowulf.core

      Essentially, the -main function and the bootstrap read-eval-print loop.

      Public variables and functions:

      beowulf.gendoc

      Generate table of documentation of Lisp symbols and functions.

      Public variables and functions:

      beowulf.host

      provides Lisp 1.5 functions which can’t be (or can’t efficiently be) implemented in Lisp 1.5, which therefore need to be implemented in the host language, in this case Clojure.

      Public variables and functions:

      beowulf.io

      Non-standard extensions to Lisp 1.5 to read and write to the filesystem.

      Public variables and functions:

      beowulf.manual

      Experimental code for accessing the manual online.

      Public variables and functions:

      beowulf.oblist

      A namespace mainly devoted to the object list and other top level global variables.

      Public variables and functions:

      beowulf.read

      This provides the reader required for boostrapping. It’s not a bad reader - it provides feedback on errors found in the input - but it isn’t the real Lisp reader.

      Public variables and functions:

      beowulf.reader.char-reader

      Provide sensible line editing, auto completion, and history recall.

      Public variables and functions:

      beowulf.reader.generate

      Generating S-Expressions from parse trees.

      Public variables and functions:

      beowulf.reader.macros

      Can I implement reader macros? let’s see!

      Public variables and functions:

      beowulf.reader.parser

      The actual parser, supporting both S-expression and M-expression syntax.

      Public variables and functions:

      beowulf.reader.simplify

      Simplify parse trees. Be aware that this is very tightly coupled with the parser.

      Public variables and functions:

      beowulf.scratch

      This namespace is for temporary functions and is intentionally excluded from Git.

      Public variables and functions:

      \ No newline at end of file +Beowulf 0.3.0-SNAPSHOT

      Project

      Topics

      Namespaces

      Beowulf 0.3.0-SNAPSHOT

      Released under the GPL-2.0-or-later

      An implementation of LISP 1.5 in Clojure.

      Installation

      To install, add the following dependency to your project or build file:

      [beowulf "0.3.0-SNAPSHOT"]

      Topics

      Namespaces

      beowulf.bootstrap

      Lisp as defined in Chapter 1 (pages 1-14) of the Lisp 1.5 Programmer's Manual; that is to say, a very simple Lisp language, which should, I believe, be sufficient in conjunction with the functions provided by beowulf.host, be sufficient to bootstrap the full Lisp 1.5 interpreter..

      Public variables and functions:

      beowulf.cons-cell

      The fundamental cons cell on which all Lisp structures are built. Lisp 1.5 lists do not necessarily have a sequence as their CDR, and must have both CAR and CDR mutable, so cannot be implemented on top of Clojure lists.

      Public variables and functions:

      beowulf.core

      Essentially, the -main function and the bootstrap read-eval-print loop.

      Public variables and functions:

      beowulf.gendoc

      Generate table of documentation of Lisp symbols and functions.

      Public variables and functions:

      beowulf.host

      provides Lisp 1.5 functions which can’t be (or can’t efficiently be) implemented in Lisp 1.5, which therefore need to be implemented in the host language, in this case Clojure.

      Public variables and functions:

      beowulf.io

      Non-standard extensions to Lisp 1.5 to read and write to the filesystem.

      Public variables and functions:

      beowulf.manual

      Experimental code for accessing the manual online.

      Public variables and functions:

      beowulf.oblist

      A namespace mainly devoted to the object list and other top level global variables.

      Public variables and functions:

      beowulf.read

      This provides the reader required for boostrapping. It’s not a bad reader - it provides feedback on errors found in the input - but it isn’t the real Lisp reader.

      Public variables and functions:

      beowulf.reader.char-reader

      Provide sensible line editing, auto completion, and history recall.

      Public variables and functions:

      beowulf.reader.generate

      Generating S-Expressions from parse trees.

      Public variables and functions:

      beowulf.reader.macros

      Can I implement reader macros? let’s see!

      Public variables and functions:

      beowulf.reader.parser

      The actual parser, supporting both S-expression and M-expression syntax.

      Public variables and functions:

      beowulf.reader.simplify

      Simplify parse trees. Be aware that this is very tightly coupled with the parser.

      Public variables and functions:

      \ No newline at end of file diff --git a/docs/codox/intro.html b/docs/codox/intro.html index a765e07..9854ad9 100644 --- a/docs/codox/intro.html +++ b/docs/codox/intro.html @@ -1,6 +1,6 @@ -beowulf

      Project

      Topics

      Namespaces

      beowulf

      +beowulf

      Project

      Topics

      Namespaces

      beowulf

      LISP 1.5 is to all Lisp dialects as Beowulf is to English literature.

      What this is

      A work-in-progress towards an implementation of Lisp 1.5 in Clojure. The objective is to build a complete and accurate implementation of Lisp 1.5 as described in the manual, with, in so far as is possible, exactly the same bahaviour - except as documented below.

      diff --git a/docs/codox/mexpr.html b/docs/codox/mexpr.html index 1edfc43..c8d104f 100644 --- a/docs/codox/mexpr.html +++ b/docs/codox/mexpr.html @@ -1,8 +1,8 @@ -M-Expressions

      Project

      Topics

      Namespaces

      M-Expressions

      -

      M-Expressions (‘mexprs’) are the grammar which John McCarthy origininally used to write Lisp, and the grammar in which many of the function definitions in the Lisp 1.5 Programmer’s Manual are stated. However, I have not seen anywhere a claim that Lisp 1.5 could read M-Expressions, and it is not clear to me whether it was even planned that it should do so.

      -

      Rather, it seems to me probably that M-Expressions were only ever a grammar intended to be written on paper, like Backus Naur Form, to describe and to reason about algorithms.

      +Interpreting M-Expressions

      Project

      Topics

      Namespaces

      Interpreting M-Expressions

      +

      M-Expressions (‘mexprs’) are the grammar which John McCarthy origininally used to write Lisp, and the grammar in which many of the function definitions in the Lisp 1.5 Programmer’s Manual are stated. However, I have not seen anywhere a claim that Lisp 1.5 could read M-Expressions, and it is not clear to me whether it was even planned that it should do so, although the discussion on page 10 suggests that it was.

      +

      Rather, it seems to me possible that M-Expressions were only ever a grammar intended to be written on paper, like Backus Naur Form, to describe and to reason about algorithms. I think at the point at which the M-Expression grammar was written, the idea of the universal Lisp function

      I set out to make Beowulf read M-Expressions essentially out of curiousity, to see whether it could be done. I had this idea that if it could be done, I could implement most of Lisp 1.5 simply by copying in the M-Expression definitions out of the manual.

      Consequently, the Beowulf parser can parse the M-Expression grammar as stated in the manual, and generate S-Expressions from it according to the table specified on page 10 of the manual.

      There are two problems with this.

      @@ -28,7 +28,7 @@ ((EQUAL X (QUOTE NIL)) (QUOTE T)) ((QUOTE T) (QUOTE F)))))) -

      This is certainly more prolix and more awkward, but it also risks being flat wrong.

      +

      This is certainly more prolix and more awkward.

      Is the value of NIL the atom NIL, or is it the empty list ()? If the former, then the translation from the M-Expression above is correct. However, that means that recursive functions which recurse down a list seeking the end will fail. So the latter must be the case.

      NULL is described thus (Ibid, p11):

      @@ -36,6 +36,7 @@

      NIL is used explicitly in an M-Expression for example in the definition of intersection (Ibid, p15).

      I think there is an ambiguity in referencing constants which are not bound to themselves in the M-Expression notation as given in the manual. This is particularly problematic with regards to NIL and F, but there may be others instances.

      +

      However, so long as F is bound to NIL, and NIL is also bound to NIL (both of which are true by default, although changeable by the user), and NIL is the special marker used in the CDR of the last cons cell of a flat list, this is a difference which in practice does not make a difference. I still find it worrying, though, that rebinding variables could lead to disaster.

      Curly braces

      -

      The use of curly braces is not defined in the grammar as stated on page 10. They are not used in the initial definition of APPLY on page 13, but they are used in the more developed restatement on page 70. I believe they are to be read as indicating a DO statement – a list of function calls to be made sequentially but without strict functional dependence on one another – but I don’t find the exposition here particularly clear and I’m not sure of this.

      +

      The use of curly braces is not defined in the grammar as stated on page 10. They are not used in the initial definition of APPLY on page 13, but they are used in the more developed restatement on page 70. I believe they are to be read as indicating a section of assembly code to be assembled by the Lisp Assembly Program – but I don’t find the exposition here particularly clear and I’m not sure of this.

      Consequently, the M-Expression interpreter in Beowulf does not interpret curly braces.

      \ No newline at end of file diff --git a/docs/codox/values.html b/docs/codox/values.html index ceebbe3..2dd0ca3 100644 --- a/docs/codox/values.html +++ b/docs/codox/values.html @@ -1,19 +1,79 @@ -Understanding values and properties

      Project

      Topics

      Namespaces

      Understanding values and properties

      -

      I had had the naive assumption that entries on the object list had their CAR pointing to the symbol and their CDR pointing to the related value. Consequently, I could not work out where the property list went. More careful reading of the text implies, but does not explicitly state, that my naive assumption is wrong.

      -

      Instead, it appears that the CAR points to the symbol, as expected, but the CAR points to the property list; and that on the property list there are privileged properties at least as follows:

      +Understanding values and properties

      Project

      Topics

      Namespaces

      Understanding values and properties

      +

      Lisp is the list processing language; that is what it name means. It processes data structures built of lists - which may be lists of lists, or lists of numbers, or lists of any other sort of data item provided for by the designers of the system.

      +

      But how is a list, in a computer, actually implemented?

      +

      They’re implemented as pairs, or, as the manual sometimes rather delightfully called them, ‘doublets’. Pairs of what? Pairs of pointers. Of the two pointers of a pair, the first points to the current entry of the list, and the second, by default, points to the remainder of the list, or, if the end of the list has been reached, to a special datum known as NIL which among other things indicates that the end of the list has been reached. The pair itself is normally referred to as a ‘cons cell’ for reasons which are nerdy and not important just now (all right, because they are constructed using a function called cons, which is in itself simply and abbreviation of ‘construct’).

      +

      Two functions are used to access the two pointers of the cell. In modern Lisps these functions are called first and rest, because a lot of people who aren’t greybeards find these names easier. But they aren’t the original names. The original names were CAR and CDR.

      +

      Why?

      +

      History

      +

      Lisp was originally written on an IBM 704 computer at Massachusetts Institute of Technology, almost seventy years ago.

      +

      The machine had registers which were not eight, or sixteen, or thirty two, or sixty four, bits wide, or any other number which would seem rational to modern computer scientists, but thirty six. Myth - folk memory - tells us that the machine’s memory was arranged in pages. As I understand it (but this truly is folk memory) the offset within the page of the word to be fetched was known as the ‘decrement’, while the serial number of the page in the addressing sequence was known as the ‘address’. To fetch a word from memory, you first had to select the page using the ‘address’, and secondly the word itself using the ‘decrement’. So there were specific instructions for selecting the address, and the decrement, from the register separately.

      +

      There were two mnemonics for the machine instructions used to access the content of these registers, respectively:

      +
      +
      CAR
      +
      +

      Contents of the Address part of Register; and

      +
      CDR
      +
      +

      Contents of the Decrement part of Register.

      +
      +

      Is this actually true?

      +

      I think so. If you look at page 80 of the Lisp 1 Programmer’s Manual, you will see this:

      +
      TEN                       (the TEN-Mode is entered)
+
+O CAR   (((A,B),C)) () \
+                        |
+:1 CDR  ((D,(E,F))) ()  |
+                         > Type ins
+:2 CONS ((G,H),         |
+                        |
+230 (I,J)) ()          /
+RLN | | oo 7 a |
+
+O14 (read lines O and 1)
+
      +

      Of course, this isn’t proof. If CAR and CDR used here are standard IBM 704 assembler mnemonics – as I believe they are – then what is CONS? It’s used in a syntactically identical way. If it also is an assembler mnemonic, then it’s hard to believe that, as legend relates, it is short for ‘construct’; on the other hand, if it’s a label representing an entry point into a subroutine, then why should CAR and CDR not also be labels?

      +

      I think that the answer has to be that if CAR and CDR had been named by the early Lisp team – John McCarthy and his immediate colleagues – they would not have been named as they were. If not FRST and REST, as in more modern Lisps, then something like P1 and P2. CAR and CDR are distinctive and memorable (and therefore in my opinion worth preserving) because they very specifically name the parts of a cons cell and of nothing else.

      +

      Let’s be clear, here: when CAR and CDR are used in Lisp, they are returning pointers, certainly – but not in the sense that one points to a page and the other to a word. Each is an offset into a cell array, which is almost certainly an array of single 36 bit words held on a single page. So both are in effect being used as decrements. Their use in Lisp is an overload onto their original semantic meaning; they are no longer being used for the purpose for which they are named.

      +

      As far as I can tell, these names first appear in print in 1960, both in the Lisp 1 Programmer’s Manual referenced above, and in McCarthy’s paper Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I. The paper was published in April so was presumably written in 1959

      +

      Grey Anatomy

      +

      The Object List

      +

      Lisp keeps track of values by associating them with names. It does so by having what is in effect a global registry of all the names it knows to which values are attached. This being a list processing language, that was of course, in early Lisps, a list: a single specialised first class list known as the ‘object list’, or oblist for short.

      +

      Of course, a list need not just be a list of single items, it can be a list of pairs: it can be a list of pairs of the form (name . value). Hold onto that, because I want to talk about another fundamental part of a working Lisp system, the stack.

      +

      The Stack

      +

      Considering the case of pure interpreter first, let’s think about how a function keeps track of the data it’s working on. In order to do its work, it probably calls other functions, to which it passes off data, and they in turn probably call further functions. So, when control returns to our first function, how does it know where its data is? The answer is that each function pushes its argument bindings onto the stack when it starts work, and pops them off again when it exits. So when control returns to a function, its own data is still on the top of the stack. Or, to be precise, actually it doesn’t; in practice the function EVAL does it for each function in turn. But it doesn’t matter: it gets done.

      +

      What is this stack? Well, it’s a list of (name . value) pairs. At least, it is in pure Lisps; Clojure, because it runs on the Java Virtual Machine and interoperates with other software running on the JVM, uses the JVM stack which is a permanently reserved vector of memory never used for anything else. Consequently it cannot be very large; and the consequence of that is that it’s very easy to crash JVM programs because they’ve run out of stack space.

      +

      The advantage of organising your stack as a vector is that on average it’s usually slightly more memory efficient, and that it’s somewhat faster to access. The disadvantage is you need a contiguous block of memory for it, and once you’ve run out, you’ve at best lost both those advantages but in the normal case your program just crashes. Also, the memory you’ve reserved for the stack isn’t available for any other use, even during the most of the time that the stack isn’t using most of it. So of course there’s a temptation to keep the amount reserved for the stack as small as possible.

      +

      It’s this brutal fragility of vector stacks – which are used by most modern computer languages – which makes software people so wary of fully exploiting the beauty and power of recursion, and I really think that’s a shame.

      +

      The advantage of organising your stack as a list is that, while there is any memory left on the machine at all, you cannot run out of stack.

      +

      ### The Spine

      +

      So, there’s an object list where you associate names and values, and there’s a stack where you associate names and values. But, why do they have to be different? And why do you have to search in two places to find the value of a name?

      +

      The answer is – or it was, and arguably it should be – that you don’t. The stack can simply be pushed onto the front of the object list. This has multiple advantages. The first and most obvious is that you only have to search in one place for the value associated with a name.

      +

      The second is more subtle: a function can mask a variable in the object list by binding the same name to a new value, and the functions to which it then calls will only see that new value. This is useful if, for example, printed output is usually sent to the user’s terminal, but for a particular operation you want to send it to a line printer or to a file on disk. You simply rebind the name of the standard output stream to your chosen output stream, and call the function whose output you want to redirect.

      +

      So, in summary, there’s a lot of merit in making the stack and the object list into a single central structure on which the architecture of our Lisp system is built. But there’s more we need to record, and it’s important.

      +

      ### Fields and Properties

      +

      No, I’m not banging on about land reform again! I’m not a total monomaniac!

      +

      But there’s more than one datum we may want to associate with a single name. A list can be more than a set of bindings between single names and single values. There’s more than one thing, for example, that I know about my friend Lucy. I know her age. I know her address. I know her height. I know her children. I know her mother. All of this information – and much more – should be associated with her.

      +

      In conventional computing systems we’d use a table. We’d put into the table a field – a column – for each datum we wanted to store about a class of things of interest. And we’d reserve space to store that datum for every record, whether every record had something to go there of not. Furthermore, we’d have to reserve space in each field for the maximum size of the datum to be stored in it – so if we needed to store full names for even some of the people we knew, and one of the people whose full name we needed to store (because he’s both very important and very irascible) was Charles Philip Arthur George Windsor, then we’d have to reserve space for thirty-six characters for the full name of everyone in our records, even if for most of them half that would be enough.

      +

      But if instead of storing a table for each sort of thing on which we hold data, and a row in that table for each item of that sort on which we store data, we simply tagged each thing on which we hold data with those things which are interesting about them? We could tag my friend Lucy with the fact she’s on pilgrimage, and what her pilgrimage route is. Those aren’t things we need to know about most people, it would be absurdly wasteful to add a column to a person table to record pilgrimage route. So in a conventional data system we would lose that data.

      +

      Lisp has had, right back from the days of Lisp 1.5 – so, for sixty-five years – a different solution. We can give every symbol arbitrarily many, arbitrarily different, properties. A property is a (name . value) pair. We don’t have to store the same properties for every object. The values of the properties don’t have to have a fixed size, and they don’t have to take up space they don’t need. It’s like having a table with as many fields as we choose, and being able to add more fields at any time.

      +

      So, in summary, I knew, in building Beowulf, that I’d have to implement property lists. I just didn’t know how I was going to do it.

      +

      Archaeology

      +

      What I’m doing with Beowulf is trying to better understand the history of Lisp by reconstructing a very early example; in this case, Lisp 1.5, from about 1962, or sixty one years ago.

      +

      I had had the naive assumption that entries on the object list in early Lisps had their CAR pointing to the symbol and their CDR pointing to the related value. Consequently, in building beowulf, I could not work out where the property list went. More careful reading of the text implies, but does not explicitly state, that my naive assumption is wrong.

      +

      Instead, it appears that the CAR points to the symbol, as expected, but the CDR points to the property list; and that on the property list there are privileged properties at least as follows:

      APVAL
      -
      the simple straightforward ordinary value of the symbol, considered a variable;
      +
      the simple straightforward ordinary value of the symbol, considered as a variable;
      EXPR
      the definition of the function considered as a normal lambda expression (arguments to be evaluated before applying);
      FEXPR
      -
      the definition of a function which should be applied to unevaluated arguments;
      +
      the definition of a function which should be applied to unevaluated arguments (what InterLisp and Portable Standard Lisp would call ‘nlambda’);
      SUBR
      -
      the definition of a complied subroutine which should be applied to evaluated arguments;
      +
      the definition of a compiled subroutine which should be applied to evaluated arguments;
      FSUBR
      -
      the definition of a complied subroutine which should be applied to unevaluated arguments.
      +
      the definition of a compiled subroutine which should be applied to unevaluated arguments.

      I think there was also another privileged property value which contained the property considered as a constant, but I haven’t yet confirmed that.

      From this it would seem that Lisp 1.5 was not merely a ‘Lisp 2’ but in fact a ‘Lisp 6’, with six effectively first class namespaces. In fact it’s not as bad as that, because of the way EVAL is evaluated.

      @@ -26,5 +86,99 @@
    1. FSUBR

    This means that, while the other potential values can be retrieved from the property list, interpreted definitions (if present) will always be preferred to uninterpreted definitions, and lambda function definitions (which evaluate their arguments), where present, will always be preferred to non-lamda definitions, which don’t.

    -

    BUT NOTE THAT the APVAL value is saught only when seeking a variable value for the symbol, and the others only when seeking a function value, so Lisp 1.5 is a ‘Lisp 2’, not a ‘Lisp 1’.

    -

    Versions of Beowulf up to and including 0.2.1 used the naive understanding of the architecture; version 0.3.0 should use the corrected version.

    \ No newline at end of file +

    BUT NOTE THAT the APVAL value is sought only when seeking a variable value for the symbol, while the others are only when seeking a function value, so Lisp 1.5 is a ‘Lisp 2’, not a ‘Lisp 1’. I strongly believe that this is wrong: a function is a value, and should be treated as such. But at the same time I do acknowledge the benefit of being able to store both source and compiled forms of the function as properties of the same symbol.

    +

    The persistent problem

    +

    There’s a view in modern software theory – with which I strongly hold – that data should be immutable. Data that changes under you is the source of all sorts of bugs. And in modern multi threaded systems, the act of reading a datum whilst some other process is writing it, or worse, two processes attempting simultaneously to write the same datum, is a source of data corruption and even crashes. So I’m very wary of mutable data; and, in modern systems where we normally have a great deal of space and a lot of processor power, making fresh copies of data structures containing the change we wanted to make is a reasonable price to pay for avoiding a whole class of bugs.

    +

    But early software was not like that. It was always constrained by the limits of the hardware on which it ran, to a degree that we are not. And the experience that we now have of the problems caused by mutable data, they did not have. So it’s core to the design of Lisp 1.5 that its lists are mutable; and, indeed, one of the biggest challenges in writing Beowulf has been implementing mutable lists in Clojure, a language carefully designed to prevent them.

    +

    But, just because Lisp 1.5 lists can be mutable, should they be? And when should they be?

    +

    The problem here is that spine of the system I talked about earlier.

    +

    How property lists should work

    +

    I’m still not fully understanding how property lists in Lisp 1.5 are supposed to work.

    +

    List format

    +

    Firstly, are they association lists comprising dotted pairs of (property-name . value), i.e.:

    +
    +

    ((property-name1 . value1) (property-name2 . value2) … (property-namen . valuen))

    +
    +

    I have assumed so, and that is what I presently intend to implement, but the diagrams on pages 59 and 60 seem rather to show a flat list of interleaved names and values:

    +
    +

    (property-name1 value1 property-name2 value2 … property-namen valuen)

    +
    +

    I cannot see what the benefit of this latter arrangement is, and I’m unwilling to do it, although I think it may be what was done. But if it was done that way, why was it done that way? These were bright people, and they certainly knew about association lists. So… I’m puzzled.

    +

    Function signatures

    +

    To associate the value of a property with a symbol, we need three things: we need the symbol, we need the property name, and we need the value. For this reason, Portable Standard Lisp and others has a function put with three arguments:

    +
    +

    (Put U:id IND:id PROP:any): any The indicator IND with the property PROP is placed on the property list of the id U. If the action of Put occurs, the value of PROP is returned. If either of U and IND are not ids the type mismatch error occurs and no property is placed. (Put 'Jim 'Height 68) The above returns 68 and places (Height . 68) on the property list of the id Jim

    +
    +

    Cambridge Lisp is identical to this except in lower case. InterLisp and several others have putprop:

    +
    +

    (PUTPROP ATM PROP VAL) [Function] Puts the property PROP with value VAL on the property list of ATM. VAL replaces any previous value for the property PROP on this property list. Returns VAL.

    +
    +

    The execrable Common Lisp uses its execrable macro setf but really the less said about that the better.

    +

    So I was looking for a function of three arguments to set properties, and I didn’t find one.

    +

    There’s a function DEFINE which takes one argument, an association list of pairs:

    +
    	(function-name . function-definition)`
+
    +

    So how does that work, if what it’s doing is setting properties? If all you’re passing is pairs of name and definition, where does the property name come from?

    +

    The answer is as follows, taken from the manual:

    +
    +

    define [x] : EXPR pseudo-function

    +

    The argument of define, x, is a list of pairs

    +
    +

    ((ul vl) (u2 v2) … (un vn))

    +
    +

    where each u is a name and each v is a λ-expression for a function . For each pair, define puts an EXPR on the property list for u pointing to v. The function of define puts things on at the front of the property list. The value of define is the list of us.

    +
    +

    So, in fact, the value of the property being set by define is fixed: hard wired, not parameterised. That seems an astonishing decision, until you realise that Lisp 1.5’s creators weren’t creating their functions one by one, in a playful exploration with their system, but entering them in a batch.

    +

    Learning by doing

    +

    In fact, when I got over my surprise, I realised that that (name . function-definition) list is actually very much like this, which is an excerpt from a sysout from a Beowulf prototype:

    +
    (...
+	(MAPLIST LAMBDA (L F) 
+           (COND ((NULL L) NIL) 
+                 ((QUOTE T) (CONS (F (CAR L)) (MAPLIST (CDR L) F)))))
+  (MEMBER LAMBDA (A X)
+            (COND ((NULL X) (QUOTE F))
+                  ((EQ A (CAR X)) (QUOTE T)) 
+                  ((QUOTE T) (MEMBER A (CDR X)))))
+  (MINUSP LAMBDA (X) (LESSP X 0))
+  (NOT LAMBDA (X) 
+       			(COND (X (QUOTE NIL)) 
+                  ((QUOTE T) (QUOTE T))))
+  (NULL LAMBDA (X) 
+        		(COND ((EQUAL X NIL) (QUOTE T)) 
+                  (T (QUOTE F))))
+...)
+
    +

    I was looking at DEFINE and thinking, ‘why would one ever want to do that?’ and then I found that, behind the scenes, I was actually doing it myself.

    +

    Because the point of a sysout is you don’t write it. The point about the REPL – the Read Eval Print Loop which is the heart of the interactive Lisp development cycle, where you sit playing with things and fiddling with them interactively, and where when one thing works you get onto the next without bothering to make some special effort to record it.

    +

    The point of a sysout is that, at the end of the working day, you invoke one function

    + + + + + + + + + + + + + + + + + + + +
    Function Type Signature Implementation Documentation
    SYSOUT Host function (SYSOUT); (SYSOUT FILEPATH) SUBR Dump the current content of the object list to file. If no filepath is specified, a file name will be constructed of the symbol Sysout and the current date. File paths will be considered relative to the filepath set when starting Lisp.
    +

    At the start of the next working day, you load that sysout in and continue your session.

    +

    The sysout captures the entire working state of the machine. No-one types it in, as an operation in itself. Instead, data structures – corpuses of functions among them – simply build up on the object list almost casually, as a side effect of the fact that you’re enjoying exploring your problem and finding elegant ways of solving it. So SYSOUT and SYSIN seem to me, as someone who all his adult life has worked with Lisp interactively, as just an automatic part of the cycle of the day.

    +

    The process of discovery

    +

    The thing is, I don’t think anyone is ever going to use Beowulf the way Lisp 1.5 was used. I mean, probably, no one is ever going to use Beowulf at all; but if they did they wouldn’t use Beowulf the way Lisp 1.5 was used.

    +

    I’m a second generation software person. I have worked, in my career, with two people who personally knew and had worked with Alan Turing. I have worked with, and to an extent been mentored by, Chris Burton, who in his apprenticeship was part of the team that built the Manchester Mark One, and who in his retirement led the team who restored it. But I never knew the working conditions they were accustomed to. In my first year at university we used card punches, and, later, when we had a bit of seniority, teletypewriters (yes, that’s what TTY stands for), but by the time I’d completed my undergraduate degree and become a research associate I had a Xerox 1108 workstation with a huge bitmapped graphic screen, and an optical mouse, goddamit, running InterLisp, all to myself.

    +

    People in the heroic age did not have computers all to themselves. They did not have terminals all to themselves. They didn’t sit at a terminal experimenting in the REPL. They wrote their algorithms in pencil on paper. When they were certain they’d got it right, they’d use a card punch to punch a deck of cards carrying the text of the program, and then they were certain they’d got that right, they’d drop it into the input hopper. Some time later their batch would run, and the operator would put the consequent printout into their pigeon hole for them to collect.

    +

    (They wrote amazingly clean code, those old masters. I could tell you a story about Chris Burton, the train, and the printer driver, that software people of today simply would not believe. But it’s true. And I think that what taught them that discipline was the high cost of even small errors.)

    +

    Lisp 1.5 doesn’t have PUT, PUTPROP or DEFUN because setting properties individually, defining functions individually one at a time, was not something they ever thought about doing. And in learning that, I’ve learned more than I ever expected to about the real nature of Lisp 1.5, and the (great) people who wrote it.

    +
    +

    So what this is about is I’ve spent most of a whole day procrastinating, because I’m not exactly sure how I’m going to make the change I’ve got to make. Versions of Beowulf up to and including 0.2.1 used the naive understanding of the architecture; version 0.3.0 should use the corrected version. But before it can, I need to be reasonably confident that I understand what the correct solution is.

    +

    I shall implement PUT, even though it isn’t in the spec, because it’s a useful building block on which to build DEFINE and DEFLIS, both of which are. And also, because PUT would have been very easy for the Lisp 1.5 implementers to implement, if it had been relevant to their working environment.

    \ No newline at end of file diff --git a/src/beowulf/core.clj b/src/beowulf/core.clj index 3ff8a62..42e3e16 100644 --- a/src/beowulf/core.clj +++ b/src/beowulf/core.clj @@ -52,6 +52,11 @@ ["-s" "--strict" "Strictly interpret the Lisp 1.5 language, without extensions."] ["-t" "--time" "Time evaluations."]]) +(defn- re + "Like REPL, but it isn't a loop and doesn't print." + [input] + (EVAL (READ input) @oblist 0)) + (defn repl "Read/eval/print loop." [prompt] @@ -65,8 +70,8 @@ (println (str "> " (print-str (if (:time *options*) - (time (EVAL (READ input) @oblist 0)) - (EVAL (READ input) @oblist 0)))))) + (time (re input)) + (re input)))))) (println)) (catch Exception diff --git a/src/beowulf/gendoc.clj b/src/beowulf/gendoc.clj index d81b2f8..994549e 100644 --- a/src/beowulf/gendoc.clj +++ b/src/beowulf/gendoc.clj @@ -8,7 +8,7 @@ *manual-url* page-url]] [beowulf.oblist :refer [NIL oblist]] [clojure.java.browse :refer [browse-url]] - [clojure.string :refer [join replace upper-case]])) + [clojure.string :as s ])) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; @@ -81,13 +81,13 @@ "Format the signature of the Clojure function represented by `symbol` for Lisp documentation." [symbol arglists] - (join + (s/join "; " (doall (map (fn [l] - (join (concat (list "(" symbol " ") - (join " " (map #(upper-case (str %)) l)) (list ")")))) + (s/join (concat (list "(" symbol " ") + (s/join " " (map #(s/upper-case (str %)) l)) (list ")")))) arglists)))) (defn infer-signature @@ -113,7 +113,7 @@ (let [k (keyword (first entry))] (cond (= (count entry) 1) (if-let [doc (get-metadata-for-entry entry :doc)] - (replace doc "\n" " ") + (s/replace doc "\n" " ") "?") (k index) (str "see manual pages " (format-page-references k)) :else "?"))) @@ -126,7 +126,7 @@ ;; (try (SYSIN sysfile) ;; (catch Throwable any ;; (println (.getMessage any) " while reading " sysfile)))) - (join + (s/join "\n" (doall (concat