Three pure function combinators into the collection tier, as compositions of
mapv/every?/some/apply. Three more core-* primitives + table entries gone.
Battery 3916 -> 3920 (the canonical defs are more correct than the prior Janet
ones — some-fn returns the matching value, every-pred is properly boolean).
Conformance 218/218 all modes.
New collection tier (core/20-coll.clj): six pure, eager fns expressed as
compositions of frozen primitives (reduce/assoc/get/conj/filter/vec). Six more
core-* primitives + table entries gone from the Janet kernel.
Two semantics to preserve, caught by the suite:
- max-key/min-key use the canonical multi-arity form (strict </> on the first
pair, <=/>= in the fold) to reproduce the JVM IEEE-754 NaN behavior; a single
reduce got the NaN cases wrong.
- frequencies/group-by base on (hash-map), not the {} literal: a struct map
doesn't canonicalize collection keys across representations ({:a 1} literal vs
(hash-map :a 1)), a PHM does — same reason the Janet impl used make-phm.
Conformance 218/218 all modes; battery holds 3916.
Calls compile direct (target value embedded) iff the compiling unit has
direct-linking on, the target isn't ^:redef/^:dynamic, and it's an
already-defined fn; otherwise indirect (live var deref, redefinable). Direct
emission is guarded to function values only — embedding a non-function would make
Janet evaluate it as code.
:aot-core? (init opt, default true; JOLT_AOT_CORE=0 disables) compiles the core
tiers with direct-linking on so core->core calls are direct. User/REPL code
compiles indirect by default, so redefining functions in the REPL works with no
annotation, exactly like Clojure. :aot-core? false makes core indirect too — the
whole language becomes redefinable. ^:redef / ^:dynamic on a target force it
indirect even inside a direct-linked unit.
Also fixes a pre-existing gap this surfaced: compiled def dropped ALL var
metadata (so ^:dynamic was silently lost under compilation, and ^:redef couldn't
work). def metadata now flows analyzer -> IR -> back end (new form-sym-meta host
fn, def-node :meta, var-setter-meta) and is applied to the cell.
Matrix test in test/integration/direct-linking-test.janet. Conformance 218/218
in all three modes under both :aot-core? true and false; battery holds 3916;
binary rebuilt.
First fractal turn of the multi-stage bootstrap (jolt-tzo). The Clojure part of
clojure.core is now loaded in ordered tiers under jolt-core/clojure/core/. The
kernel tier (00-kernel: second/peek/subvec/mapv/update) holds the structural fns
the self-hosted compiler itself uses; in compile mode it is bootstrap-compiled
into clojure.core BEFORE the analyzer is built, so the analyzer binds those names
to the Clojure definitions instead of a not-yet-defined forward ref. That removes
the circularity that previously forced these to stay in Janet — the five core-*
primitives and their init-core! entries are gone.
Mechanism: backend/bootstrap-load-source (generalized from compile-load) builds a
source string into a target ns via the bootstrap; rebuild-compiler! recompiles
the self-hosted compiler against the current core (the rail for future turns,
exercised by the new fixpoint test). api/load-core-overlay! walks the ordered
tiers, bootstrap-loading kernel tiers and self-hosting the rest.
Also fixes a latent evaluator bug surfaced by the move: a fn rebinds current-ns
to its defining ns while it runs and restores on normal return, but a fn that
THREW unwound past its own restore, leaking the ns. try now restores the
try-entry ns on the catch/finally path, so referred-symbol resolution survives a
caught error (this was breaking is/testing in the suite harness after any thrown
assertion). Net battery +3 (3913 -> 3916); conformance 218/218 in all three
modes; binary builds and runs the embedded tiers.
Continue the kernel shrink. last/butlast become canonical Clojure defs
(pure first/next/loop/recur) in the core.clj overlay, dropping two
realize-for-iteration uses from the Janet kernel.
second was the obvious next candidate but can't move: the self-hosted
compiler front-end (analyzer.clj) calls it, and the compiler has to
compile the overlay, so anything it uses must already exist as a Janet
primitive. Documented that as a third clause of the safe-to-move rule.
Suite green, clojure-test-suite battery holds 3913 pass / 58 clean,
binary builds and runs the embedded overlay.
Phase 4 kernel-shrink seam. jolt-core/clojure/core.clj holds the Clojure portion
of clojure.core; api/init loads it into the clojure.core namespace after
init-core! interns the Janet primitives, routed through eval-toplevel so it
compiles via the self-hosted pipeline (or interprets when :compile? is off).
First three fns moved off the Janet side: ffirst, fnext, nnext (leaf, no internal
callers). Same results compiled or interpreted; full suite green; battery holds
3913.
eval-toplevel now routes every compiled form through backend/compile-and-eval
(analyzer -> IR -> Janet back end). The bootstrap compiler (compile-ast) stays
only to bootstrap jolt.ir/jolt.analyzer in backend/compile-load. Dropped the
:selfhost? flag and JOLT_SELFHOST env plumbing — self-host is the default.
Three correctness fixes the full-suite-under-self-host sweep exposed:
- analyzer/back end: recur into a variadic arity now compiles. The arity fn
gets an ordinary positional rest param (the collected seq), with a dispatch
wrapper, so (recur fixed... rest-seq) is a clean self-call like Clojure --
instead of punting to the interpreter, which hangs on this (jolt-4df).
- host_iface: interop heads (.foo/.-field/Foo.) and ns-management forms
(all-ns, create-ns, resolve, ...) now mark uncompilable so they fall back
instead of compiling to a bad call.
- back end: named-fn self-reference ((fn self [n] ... (self ...))) binds the
fn name via a var.
Full unit+integration suite green; clojure-test-suite battery holds 3913.
The self-hosted compiler now genuinely compiles to native Janet bytecode rather
than falling back to the interpreter:
- bootstrap fixes that let it compile the analyzer: (def x) with no init, and an
empty (do) (declare expands to one) no longer error;
- analyzer reordered so only the mutually-recursive is forward declared
(a deeply-nested forward ref to analyze-fn/analyze-try miscompiled to nil);
- back end emits native Janet ops for hot primitives (+,-,*,<,>,<=,>=,inc,dec)
on clojure.core refs, like the bootstrap's core-renames.
Result: self-hosted fib(30) runs in 0.52s (was ~50s interpreted). Full
clojure-test-suite via JOLT_SELFHOST runs compiled at 3869 pass (some compiled-
vs-interpreted divergences remain to chase toward full parity — jolt-4xc).
Default suite green; conformance 218/218; jank-conformance up to 135.
The portable analyzer now refers the host contract + IR ctors unqualified (host
form predicates renamed form-* to dodge core-renames), so the bootstrap compiles
it via its plain :var path — no qualified-ref compilation needed. ensure-analyzer
compile-loads jolt.ir + jolt.analyzer as native bytecode.
Result: the self-hosted pipeline (portable Clojure analyzer -> IR -> Janet back
end) runs the FULL clojure-test-suite at 3913 pass — parity with the interpreter
baseline — and fast (no timeouts), via JOLT_SELFHOST=1. Conformance 218/218.
Also fixes a real interpreter bug: multi-arity dispatch stored the variadic clause
by fixed-count (colliding) and only matched exact counts, so (f a b c..) on an
[a b & more] clause threw. Now the variadic clause dispatches for any count >= its
fixed arity.
Remaining (jolt-4xc): the compiled analyzer still errors analyzing a multi-arity
fn literal (falls back to the interpreter, now correct); compiling that is the
last coverage gap before flipping the self-hosted pipeline on by default.
Analyzer threads an env (locals + recur target) instead of a bare locals set, and
compiles loop*/recur, recur directly in a fixed-arity fn (each arity's name is its
recur target -> self-call), and try/catch/finally (-> Janet try + defer). recur
into a variadic arity isn't a recur target, so it falls back to the interpreter
rather than miscompiling the rest seq. Still 218/218 conformance via the pipeline,
now with these forms compiling natively instead of interpreting.
The Clojure analyzer + Janet back end now pass all 218 conformance cases (vs the
interpreter) through the self-hosted pipeline:
- host contract gains special? (the full special-form/uncompilable set), so the
analyzer falls back for forms it doesn't compile instead of miscompiling them
as calls;
- qualified symbol refs that don't resolve to a var (janet/…, Math/…, host
interop) and tagged literals (#"regex", #inst) and set literals throw
uncompilable -> interpreter;
- the back end's compile-and-eval is now hybrid: only the compile step is
guarded, runtime errors propagate.
Still interpreter-routed (coverage to grow next): loop/recur, recur in fn, try,
sets, destructuring.
The Clojure-in-Clojure front end now compiles and runs a real subset end to end:
arithmetic, if/do/let, vector/map literals, def + name-based global refs, fn,
defn (via macroexpand), recursion through the var cell, multi-arity, variadic,
and higher-order calls. Proven by test/integration/self-host-test.janet.
Pieces:
- jolt-core/jolt/analyzer.clj — PORTABLE Clojure analyzer: reader form -> IR,
depending only on the host contract (jolt.host), never on Janet.
- src/jolt/host_iface.janet — Janet impl of the jolt.host contract (form
introspection + resolve/macro/current-ns), installed into each ctx.
- src/jolt/backend.janet — Janet back end: IR -> Janet form -> eval; resolves
name-based :var nodes to cells and reuses runtime helpers.
Host Janet code lives in src/jolt/ (not a host/janet/ dir): Janet resolves
relative imports per file, so cross-dir ../../src/jolt/* imports load second
instances of compiler/types/core and corrupt state. The portability boundary is
the jolt.host namespace contract + jolt-core/, not the directory.
Notes: gensym in the back end must not be Janet's (shadowed by Jolt's via use);
a jolt bug — (into #{} coll) doesn't add elements — was worked around with reduce
conj in the analyzer (filed separately).
Adds doc/self-hosting-architecture.md: the portability design (portable front
end + host back end + host runtime, seam = a minimal host protocol + the IR),
grounded in the CLJS/Clojure-in-Clojure prior art. Decisions locked: minimal host
protocol as the seam; physical split with a jolt-core/ source root.
Adds jolt-core/ as a portable Clojure source root (dev path + embedded into the
binary alongside the rest of the stdlib) and jolt-core/jolt/ir.clj — host-neutral
IR node constructors (vars referenced by name, no host values embedded). Verified
it loads and runs under both interpreter and compiler.
