simon/post-scarcity
1
0
Fork 0
Code Issues 11 Pull requests Projects Releases Packages Wiki Activity Actions

Copied latest roadmap and 0.1.X design document from develop branch

Browse source
This commit is contained in:
Simon Brooke 2026-03-24 16:58:12 +00:00
parent 145a0fe5a7
commit cb84e7ef95
2 changed files with 204 additions and 19 deletions
Download patch file Download diff file Expand all files Collapse all files

111
docs/0-1-0-design-decisions.md Normal file
View file

@ -0,0 +1,111 @@
# Design decisions for 0.1.0
This is a document that is likely to be revisited, probably frequently.
## Retire the 0.0.X codebase
Move the existing codebase out of the compile space altogether; it is to be
treated as a finished rapid prototype, not extended further, and code largely
not copied but learned from.
## Remain open to new substrate languages, but continue in C for now
I'm disappointed with [Zig](https://ziglang.org/). While the language
concepts are beautiful, and if it were stable it would be an excellent tool, it
isn't stable. I'm still open to build some of the 0.1.X prototype in Zig, but
it isn't the main tool.
I haven't yet evaluated [Nim](https://nim-lang.org/). I'm prejudiced against
its syntax, but, again, I'm open to using it for some of this prototype.
But for now, I will continue to work in C.
## Substrate is shallow
In the 0.0.X prototype, I tried to do too much in the substrate. I tried to
write bignums in C, and in this I failed; I would have done much better to
get a very small Lisp working well sooner, and build new features in that.
In 0.1.X the substrate will be much less feature rich, but support the creation
of novel types of data object in Lisp.
## Sysin and sysout are urgent
If a significant proportion of the system is written in Lisp, it must be
possible to save a working Lisp image to file and recover it.
## Compiler is urgent
I still don't know how to write a compiler, and writing a compiler will still
be a major challenge. But I am now much closer to knowing how to write a
compiler than I was. I think it's important to have a compiler, both for
performance and for security. Given that we do not have a separate execute ACL,
if a user can execute an interpreted function, they can also read its source.
Generally this is a good thing. For things low down in the stack, it may not
be.
## Paged Space Objects
Paged space objects will be implemented largely in line with
[this document](Paged-space-objects.md).
## Tags
Tags will continue to be 32 bit objects, which can be considered as unsigned
integer values or as four bytes. However, only the first three bytes will be
mnemonic. The fourth byte will indicate the size class of the object; where
the size class represents the allocation size, *not* the payload size. The
encoding is as in this table:
| Tag | | | Size of payload | |
| ---- | ----------- | --- | --------------- | --------------- |
| Bits | Field value | Hex | Number of words | Number of bytes |
| ---- | ----------- | --- | --------------- | --------------- |
| 0000 | 0 | 0 | 1 | 8 |
| 0001 | 1 | 1 | 2 | 16 |
| 0010 | 2 | 2 | 4 | 32 |
| 0011 | 3 | 3 | 8 | 64 |
| 0100 | 4 | 4 | 16 | 128 |
| 0101 | 5 | 5 | 32 | 256 |
| 0110 | 6 | 6 | 64 | 512 |
| 0111 | 7 | 7 | 128 | 1024 |
| 1000 | 8 | 8 | 256 | 2048 |
| 1001 | 9 | 9 | 512 | 4096 |
| 1010 | 10 | A | 1024 | 8192 |
| 1011 | 11 | B | 2048 | 16384 |
| 1100 | 12 | C | 4096 | 32768 |
| 1101 | 13 | D | 8192 | 65536 |
| 1110 | 14 | E | 16384 | 131072 |
| 1111 | 15 | F | 32768 | 262144 |
Consequently, an object of size class F will have an allocation size of 32,768 words, but a payload size of 32,766 words. This obviously means that size classes 0 and 1 will not exist, since they would not have any payload.
## Page size
Every page will be 1,048,576 bytes.
## Namespaces
Namespaces will be implemented; in addition to the root namespace, there will be at least the following namespaces:
### :bootstrap
Functions written in the substrate language, intended to be replaced for all normal purposes by functions written in Lisp which may call these bootstrap functions. Not ever available to user code.
### :substrate
Functions written in the substrate language which *may* be available to user-written code.
### :system
Functions, written either in Lisp or in the substrate language, which modify system memory in ways that only trusted and privileged users are permitted to do.
## Access control
Obviously, for this to work, access control lists must be implemented and must work.
## Router is deferred to 0.2.X
This generation is about producing a better single thread Lisp (but hopefully
to build it fast); the hypercube topology is deferred.

112
docs/Roadmap.md
View file

@ -1,58 +1,132 @@
# Roadmap # Roadmap
With the release of 0.0.6 close, it's time to look at a plan for the future development of the project. With the release of 0.0.6 close, it's time to look at a plan for the future
development of the project.
I have an almost-working Lisp interpreter, which, as an interpreter, has many of the features of the language I want. It runs in one thread on one processor. I have an almost-working Lisp interpreter, which, as an interpreter, has many
of the features of the language I want. It runs in one thread on one processor.
Given how experimental this all is, I don't think I need it to be a polished interpreter, and polished it isn't. Lots of things are broken. Given how experimental this all is, I don't think I need it to be a polished
interpreter, and polished it isn't. Lots of things are broken.
* garbage collection is pretty broken, and I'n beginning to doubt my whole garbage collection strategy; * garbage collection is pretty broken, and I'n beginning to doubt my whole
garbage collection strategy;
* bignums are horribly broken; * bignums are horribly broken;
* there's something very broken in shallow-bound symbols, and that matters and wil have to be fixed; * there's something very broken in shallow-bound symbols, and that matters
and will have to be fixed;
* there are undoubtedly many other bugs I don't know about. * there are undoubtedly many other bugs I don't know about.
However, while I will fix bugs where I can, it's good enough for other people to play with if they're mad enough, and it's time to move on. However, while I will fix bugs where I can, it's good enough for other people
to play with if they're mad enough, and it's time to move on.
## Next major milestones ## Next major milestones
### New substrate language?
I really don't feel competent to write the substrate in C, and I don't think
that what exists of the substrate is of sufficient quality. It's too big and
too complex. I think what the system needs is a smaller substrate written in
a more modern language.
I propose to evaluate both [Zig](https://ziglang.org/) and
[Rust](https://rust-lang.org/), and see whether I can feel more productive in
either of those.
### Smaller substrate
However, I also think the substrate ought to be smaller. I
do not think the substrate should include things like bignum or ratio
arithmetic, for example. I'm not convinced that it should include things like
hashmaps. If these things are to be written in Lisp, though, it means that
there have to be Lisp functions which manipulate memory a long way below the
'[don't know, don't care](Post-scarcity-software.md#store-name-and-value)'
dictum; this means that these functions have to be system private. But they
can be, because access control lists on arbitrary objects have always been
part of this architecture.
### The 0.1.0 branch
I'm therefore proposing, immediately, to upversion the `develop` branch to
0.1.0, and restart pretty much from scratch. For now, the C code will remain in
the development tree, and I may fix bugs which annoy me (and possibly other
people), but I doubt there now will be a 0.0.7 release, unless I decide that
the new substrate languages are a bust.
So release 0.1.0, which I'll target for 1<sup>st</sup> January 2027, will
essentially be a Lisp interpreter running on the new substrate and memory
architecture, without any significant new features.
See [0.1.0 design decisions](0-1-0-design-decisions.md) for more detail.
### Simulated hypercube ### Simulated hypercube
There is really no point to this whole project while it remains a single thread running on a single processor. Until I can pass off computation to peer neighbours, I can't begin to understand what the right strategies are for when to do so. There is really no point to this whole project while it remains a single thread
running on a single processor. Until I can pass off computation to peer
neighbours, I can't begin to understand what the right strategies are for when
to do so.
`cond` is explicitly sequential, since later clauses should not be executed at all if earlier ones succeed. `progn` is sort of implicitly sequential, since it's the value of the last form in the sequence which will be returned. `cond` is explicitly sequential, since later clauses should not be executed at
all if earlier ones succeed. `progn` is sort of implicitly sequential, since
it's the value of the last form in the sequence which will be returned.
For `mapcar`, the right strategy might be to partition the list argument between each of the idle neighbours, and then reassemble the results that come bask. For `mapcar`, the right strategy might be to partition the list argument
between each of the idle neighbours, and then reassemble the results that come
bask.
For most other things, my hunch is that you pass args which are not self-evaluating to idle neighbours, keeping (at least) one on the originating node to work on while they're busy. For most other things, my hunch is that you pass args which are not
self-evaluating to idle neighbours, keeping (at least) one on the originating
node to work on while they're busy.
But before that can happen, we need a router on each node which can monitor concurrent traffic on six bidirectional links. I think at least initially what gets written across those links is just S-expressions. But before that can happen, we need a router on each node which can monitor
concurrent traffic on six bidirectional links. I think at least initially what
gets written across those links is just S-expressions.
I think a working simulated hypercube is the key milestone for version 0.0.7. I think a working simulated hypercube is the key milestone for version 0.2.0.
### Sysout, sysin, and system persistance ### Sysout, sysin, and system persistance
Doctrine is that the post scarcity computing environment doesn't have a file system, but nevertheless we need some way of making an image of a working system so that, after a catastrophic crash or a power outage, it can be brought back up to a known good state. This also really needs to be in 0.0.7. Doctrine is that the post scarcity computing environment doesn't have a file
system, but nevertheless we need some way of making an image of a working
system so that, after a catastrophic crash or a power outage, it can be brought
back up to a known good state. This really needs to be in 0.1.1.
### Better command line experience ### Better command line experience
The current command line experience is embarrassingly poor. Recallable input history, input line editing, and a proper structure editor are all things that I will need for my comfort. The current command line experience is embarrassingly poor. Recallable input
history, input line editing, and a proper structure editor are all things that
I will need for my comfort.
### Users, groups and ACLs ### Users, groups and ACLs
Allowing multiple users to work together within the same post scarcity computing environment while retaining security and privacy is a major goal. So working out ways for users to sign on and be authenticated, and to configure their own environment, and to set up their own access control lists on objects they create, needs to be another nearish term goal. Probably 0.0.8. Allowing multiple users to work together within the same post scarcity
computing environment while retaining security and privacy is a major goal. So
working out ways for users to sign on and be authenticated, and to configure
their own environment, and to set up their own access control lists on objects
they create, needs to be another nearish term goal. Probably 0.1.2.
### Homogeneities, regularities, slots, migration, permeability ### Homogeneities, regularities, slots, migration, permeability
There are a lot of good ideas about the categorisation and organisation of data which are sketched in my original [Post scarcity software](Post-scarcity-software.md) essay which I've never really developed further because I didn't have the right software environment for them, which now I shall have. It would be good to build them. There are a lot of good ideas about the categorisation and organisation of data
which are sketched in my original
[Post scarcity software](Post-scarcity-software.md) essay which I've never
really developed further because I didn't have the right software environment
for them, which now I shall have. It would be good to build them.
### Compiler ### Compiler
I do want this system to have a compiler. I do want compiled functions to be the default. And I do want to understand how to write my own compiler for a system like this. But until I know what the processor architecture of the system I'm targetting is, worrying too much about a compiler seems premature. I do want this system to have a compiler. I do want compiled functions to be
the default. And I do want to understand how to write my own compiler for a
system like this. But until I know what the processor architecture of the
system I'm targetting is, worrying too much about a compiler seems premature.
### Graphical User Interface ### Graphical User Interface
Ultimately I want a graphical user interface at least as fluid and flexible as what we had on Interlisp machines 40 years ago. It's not a near term goal there. Ultimately I want a graphical user interface at least as fluid and flexible as
what we had on Interlisp machines 40 years ago. It's not a near term goal yet.
### Real hardware ### Real hardware
This machine would be **very** expensive to build, and there's no way I'm ever going to afford more than a sixty-four node machine. But it would be nice to have software which would run effectively on a four billion node machine, if one could ever be built. I think that has to be the target for version 1.0.0. This machine would be **very** expensive to build, and there's no way I'm ever
going to afford more than a sixty-four node machine. But it would be nice to
have software which would run effectively on a four billion node machine, if
one could ever be built. I think that has to be the target for version 1.0.0.