Chez plan: zero-Janet north star, self-host the compiler on Chez

Revise the epic's direction from a minimal Janet shim to ripping Janet
out entirely — Chez becomes the sole substrate. The missing spine: the
compiler pipeline itself only runs on Janet today (the analyzer executes
on the Janet host; the IR->Scheme emitter is host/chez/emit.janet). Phase
3 is re-scoped to self-host the compiler on Chez (emitter -> Clojure
jolt.backend-scheme, reader -> jolt-core, compile-from-source bootstrap
fixpoint). Phase 5 becomes a hard delete of both src/jolt/*.janet and
host/chez/*.janet. Sequencing: core parity first, then self-host, then
delete.
This commit is contained in:
Yogthos 2026-06-18 08:27:22 -04:00
parent 6af3e73595
commit af680ed106

View file

@ -10,22 +10,33 @@ tunable via WPO + stripped boot + AOT-under-petite.
This plan is built around two north stars beyond raw speed: This plan is built around two north stars beyond raw speed:
1. **Minimal host shim.** Every line that stays in Scheme is a line we failed to 1. **Zero Janet — Chez is the sole substrate** (revised 2026-06-18). The goal is
self-host. The port is the forcing function for jolt-tzo/uqi/lcn: shrink the not a minimal Janet shim that coexists with Chez; it is to rip Janet out
Scheme seed to the irreducible primitive set, push everything else into entirely and rely on Chez going forward. Two things must move off Janet: the
portable Clojure (`jolt-core/`). **runtime** (a hand-written Chez Scheme RT replaces the Janet value layer /
vars / evaluator) and the **compiler**. The compiler is the subtle part: the
analyzer/IR are already portable Clojure, but today they *execute on the Janet
host*, and the IR->Scheme emitter + driver are *Janet code* (`host/chez/
emit.janet`), so the current `clojure.core` prelude is a Janet cross-compile.
The end state requires Chez-jolt to run the analyzer itself and the emitter to
become portable Clojure — so Chez-jolt compiles its own `clojure.core` AND the
analyzer from source, with no Janet in the loop (the bootstrap fixpoint). Then
both `src/jolt/*.janet` and `host/chez/*.janet` are deleted. Every line that
stays in Scheme is hand-written Chez RT, not Janet; the forcing function for
jolt-tzo/uqi/lcn still applies (push logic into `jolt-core/`).
2. **Tests are the contract.** The spec/conformance corpus is host-neutral data 2. **Tests are the contract.** The spec/conformance corpus is host-neutral data
(`[name expected-clj actual-clj]` triples compared via jolt's own `=`). It is (`[name expected-clj actual-clj]` triples compared via jolt's own `=`). It is
the acceptance gate for "the port is correct" — Chez-jolt must pass the same the acceptance gate for "the port is correct" — Chez-jolt must pass the same
corpus the Janet host passes, with no regression to the clojure-test-suite corpus the Janet host passes, with no regression to the clojure-test-suite
baseline. baseline.
## The minimal host shim (target end-state) ## The Chez host RT vs portable Clojure (target end-state)
What MUST be Scheme (the irreducible primitive layer the self-hosted core rests What MUST be hand-written Chez Scheme (the irreducible primitive layer the
on) vs what MOVES into portable Clojure: self-hosted core rests on) vs what MOVES into portable Clojure. Nothing here is
Janet — this is the split *after* Janet is gone:
### Stays in Scheme (the shim) ### Stays in Scheme (the Chez host RT)
- **Value primitives that can't bottom out in Clojure without circularity:** - **Value primitives that can't bottom out in Clojure without circularity:**
the `nil` sentinel (distinct from `#f` and `'()` — the classic Lisp-on-Lisp the `nil` sentinel (distinct from `#f` and `'()` — the classic Lisp-on-Lisp
trap), keyword/symbol records (Clojure symbols carry ns + meta), char/string trap), keyword/symbol records (Clojure symbols carry ns + meta), char/string
@ -45,9 +56,11 @@ on) vs what MOVES into portable Clojure:
### Moves into portable Clojure (`jolt-core/`) ### Moves into portable Clojure (`jolt-core/`)
- **The reader** (text -> forms). CLJS self-hosts its reader; ours can too. ~33KB - **The reader** (text -> forms). CLJS self-hosts its reader; ours can too. ~33KB
of Janet leaves the host. Not hot. of Janet leaves the host. Not hot.
- **Analyzer + IR + passes** — already portable Clojure. No change. - **Analyzer + IR + passes** — already portable Clojure source; the change is
- **The backend emitter** — its LOGIC becomes Clojure that emits Scheme forms as that they must EXECUTE on Chez-jolt, not on the Janet host (Phase 3).
data; only `host/compile` crosses the seam. - **The backend emitter** — today it is Janet (`host/chez/emit.janet`); its LOGIC
becomes portable Clojure (`jolt.backend-scheme`) that emits Scheme forms as
data, so it runs on Chez. Only `host/compile` (Chez `eval`) crosses the seam.
- **macros + clojure.core** — finish the jolt-uqi/tzo migration (most already - **macros + clojure.core** — finish the jolt-uqi/tzo migration (most already
Clojure). Clojure).
- **Protocol/multimethod dispatch logic** — over the host tag primitive. - **Protocol/multimethod dispatch logic** — over the host tag primitive.
@ -98,10 +111,17 @@ on) vs what MOVES into portable Clojure:
the Chez RT. Gate: spec + conformance + clojure-test-suite parity with the the Chez RT. Gate: spec + conformance + clojure-test-suite parity with the
Janet baseline. Janet baseline.
**Phase 3 — Self-host expansion (shrink the shim)** **Phase 3 — Self-host the compiler on Chez** (the no-Janet spine)
- Move the reader into jolt-core. Continue core-* leaf migration (jolt-uqi/ded/ - Rewrite the IR->Scheme emitter from Janet (`host/chez/emit.janet` + `driver.
tzo, now targeting Chez). Drop the tree-walking interpreter. Gate: shim equals janet`) into portable Clojure in jolt-core (a `jolt.backend-scheme` target);
the documented minimal set; parity holds. folds jolt-lcn. Move the reader into jolt-core. Stand up Chez compile-from-
source: Chez-jolt reads the `.clj` tiers, runs the analyzer *executing on Chez*
(not Janet), emits Scheme, evals — replacing the Janet cross-compile of the
prelude. Bootstrap fixpoint: Chez-jolt compiles `clojure.core` AND the analyzer
from source with no Janet in the loop; verify stage2==stage3 emitted forms.
Drop the tree-walking interpreter. Continue core-* leaf migration (jolt-uqi/
ded/tzo). Gate: Chez-jolt builds itself from source, full corpus parity holds,
zero Janet invoked.
**Phase 4 — Deployment & optimization modes** (the "optimize specific cases" lever) **Phase 4 — Deployment & optimization modes** (the "optimize specific cases" lever)
- Wire `JOLT_WHOLE_PROGRAM`/direct-link to emit specialized Scheme (fl*/fx*), - Wire `JOLT_WHOLE_PROGRAM`/direct-link to emit specialized Scheme (fl*/fx*),
@ -111,9 +131,12 @@ on) vs what MOVES into portable Clojure:
collections/binary-trees (the GC axes); size + memory measured vs spike collections/binary-trees (the GC axes); size + memory measured vs spike
baseline. baseline.
**Phase 5 — Retire the Janet host** **Phase 5 — Delete the Janet host** (single substrate)
- Chez parity + perf confirmed -> remove `src/jolt/*.janet` seed. jolt-core - Chez self-hosts + parity + perf confirmed -> delete both `src/jolt/*.janet`
unchanged. Janet becomes a historical/alternate host proving portability. (the seed) and `host/chez/*.janet` (the Janet emitter/driver, now superseded by
the Clojure `jolt.backend-scheme`). Chez is the only substrate; no alternate
host retained. jolt-core unchanged. Oracle stays the spec corpus + JVM Clojure.
Net: one host, no Janet, no cross-compile.
## Host interop & the examples acceptance corpus ## Host interop & the examples acceptance corpus