recompile-defns! is the defn analog of recompile-macros!: pre/at-kernel
overlay defns (00-syntax's destructure and friends; the kernel tier too in
interpret mode) load as interpreted closures, the evaluator stashes their fn
source on the var (:defn-src, scoped by a flag only api/load-core-overlay!
sets), and the end-of-init pass compiles them and swaps the var root. With
that in place, keys/vals/empty? — the fns the 00-syntax expanders call at
expansion time — move to the top of 00-syntax as raw fn* defs (canonical:
keys/vals project (seq m), so sorted maps come back in comparator order and
(keys {}) is nil; empty? keeps O(1) count dispatch with seq's cell check only
for the lazy/list fallback). The sorted tier drops its now-dead :keys/:vals
ops.
Correctness fixes that surfaced once the gate was run with a REAL exit code
(the previous 'jpm test | grep' gates reported grep's exit and masked spec
failures across #48-#50):
- map conj is strict again: a non-nil/non-map arg must be a 2-element vector
('Vector arg to map conj must be a pair'), and merge inherits it — the
batch-2 canonical merge had silently dropped the validation
- conj onto a lazy seq prepends (it fell into the MAP fallback); upstream
clojure.data/diff relies on (conj seq x) via set/union over keys, so diff
now matches Clojure exactly
- (seq {}) / (seq #{}) / empty phm are nil, not ()
- key/val are strict (a plain vector is not an entry); find mints a REAL
entry as the first entry of a one-entry map, nil values intact
- the sci avoid-method-too-large stub passes its registry map through
instead of returning a raw host table (strict conj rejected it; sci's
clojure-core registry is also no longer discarded)
Test updates: lazy-infinite pins take-nth realization at 5 (was 7 — the
canonical lazy impl realizes fewer); self-host asserts the analyzer IS loaded
in interpret mode (compiled expanders, PR #50) and is NOT in the
:compile-macros? false oracle. 18 new maps-spec rows.
Gate: jpm test exit 0 (verified directly, not through a pipe), conformance
326x3, suite 4698 >= 4660.
init in compile mode is ~2.4 s (tier loading, analyzer self-compile, macro
recompilation), paid by every process that builds a ctx from source — each
jpm-test file, embedders, workers. init-cached marshals the built ctx to a
disk image (same root-env dicts as snapshot/fork) and later processes
unmarshal it in ~5 ms, any process: nothing from the baking process is
needed at load.
The cache key fingerprints the embedded .clj stdlib (which covers jolt-core:
analyzer, IR, core tiers), the .janet seed sources next to the module, the
janet version, the init opts, and the env knobs that shape a ctx (JOLT_PATH/
MUTABLE/AOT_CORE/FEATURES) — any change rebuilds. Corrupt or non-ctx images
fall back to a rebuild (unmarshal of garbage can 'succeed' with a scalar, so
the shape is checked, not just the throw). Writes are atomic (tmp + rename)
so racing cold starts never publish a torn image. JOLT_NO_IMAGE_CACHE=1
opts out; JOLT_IMAGE_CACHE_DIR overrides the location (default TMPDIR).
Test consumers switch to init-cached (harness, suite-worker, conformance,
the behavioral unit/integration tests); tests that validate the bootstrap
itself (bootstrap-fixpoint, staged-bootstrap, aot round-trip, direct-linking)
and the deps tests (tmp-dir :paths would fragment the key) keep real init.
Full jpm test: 2:46 -> 1:58 (~29%). New ctx-image-test covers cold/warm,
cross-process load (subprocess runs defn/redef/macros/protocols/multimethods
off the baked image), per-opts keying, and corrupt-image fallback.
adds self-hosted compiler is functionally:
- The default compile path is the portable pipeline using jolt.analyzer (Clojure) → host-neutral IR → backend.janet.
- The analyzer is itself Clojure, compiled by jolt for true self-hosting.
- bootstrap-fixpoint passes (stage1 == stage2 == stage3): rebuilding the compiler on its own output.
- clojure.core is now self-hosted in the overlay.
- Stateful forms (defmacro/ns/deftype/defmulti/require/in-ns) are interpreted by design.
