core: AOT context image — init-cached recovers the bootstrap cost across processes

init in compile mode is ~2.4 s (tier loading, analyzer self-compile, macro
recompilation), paid by every process that builds a ctx from source — each
jpm-test file, embedders, workers. init-cached marshals the built ctx to a
disk image (same root-env dicts as snapshot/fork) and later processes
unmarshal it in ~5 ms, any process: nothing from the baking process is
needed at load.

The cache key fingerprints the embedded .clj stdlib (which covers jolt-core:
analyzer, IR, core tiers), the .janet seed sources next to the module, the
janet version, the init opts, and the env knobs that shape a ctx (JOLT_PATH/
MUTABLE/AOT_CORE/FEATURES) — any change rebuilds. Corrupt or non-ctx images
fall back to a rebuild (unmarshal of garbage can 'succeed' with a scalar, so
the shape is checked, not just the throw). Writes are atomic (tmp + rename)
so racing cold starts never publish a torn image. JOLT_NO_IMAGE_CACHE=1
opts out; JOLT_IMAGE_CACHE_DIR overrides the location (default TMPDIR).

Test consumers switch to init-cached (harness, suite-worker, conformance,
the behavioral unit/integration tests); tests that validate the bootstrap
itself (bootstrap-fixpoint, staged-bootstrap, aot round-trip, direct-linking)
and the deps tests (tmp-dir :paths would fragment the key) keep real init.
Full jpm test: 2:46 -> 1:58 (~29%). New ctx-image-test covers cold/warm,
cross-process load (subprocess runs defn/redef/macros/protocols/multimethods
off the baked image), per-opts keying, and corrupt-image fallback.
This commit is contained in:
Yogthos 2026-06-10 13:57:37 -04:00
parent bf885078f9
commit 0e3584884f
24 changed files with 222 additions and 64 deletions

View file

@ -3,7 +3,7 @@
(defn ct-eval [ctx s] (normalize-pvecs (eval-string ctx s)))
(print "Phase 6: comprehensive compile-mode tests...")
(let [ctx (init {:compile? true})]
(let [ctx (init-cached {:compile? true})]
(print " collections...")
(assert (= :a (ct-eval ctx "(nth [:a :b :c :d] 0)")) "nth")
@ -126,13 +126,13 @@
# Context isolation: a def in one compiled context is invisible in another. With
# var-indirection each context has its own var cells, so b's `secret` is a
# distinct, unbound var (nil) rather than a's 7.
(let [a (init {:compile? true}) b (init {:compile? true})]
(let [a (init-cached {:compile? true}) b (init-cached {:compile? true})]
(eval-string a "(def secret 7)")
(assert (= 7 (ct-eval a "secret")) "def visible in its own ctx")
(assert (nil? (ct-eval b "secret")) "def isolated to its ctx"))
# Redefinition is visible to already-compiled callers (var-indirection).
(let [c (init {:compile? true})]
(let [c (init-cached {:compile? true})]
(eval-string c "(defn g [] 1)")
(eval-string c "(defn calls-g [] (g))")
(eval-string c "(defn g [] 2)")