test: ~11x faster gate — ctx snapshot/fork + parallel suite battery

The gate spent almost all its time rebuilding identical contexts: init is
~50 ms interpreted / ~900 ms compiled (tier loading, analyzer build, macro
recompilation), and both the conformance harness and the spec harness built a
fresh ctx PER CASE — 269 cases x 3 modes for conformance alone (~285 s), and
~1500 spec cases (~90 s). The suite battery additionally ran its 234 worker
subprocesses sequentially (~100 s incl. 5 x 6 s timeout files).

- api: snapshot/fork — marshal a fully-built ctx once (reverse-lookup dicts
  from root-env, built at module load before any ctx exists), unmarshal cheap
  fully-isolated deep copies (~2 ms). A fork shares nothing mutable with its
  siblings, so per-case isolation is preserved exactly.
- conformance: one init per mode + fork per case (self-host pre-builds the
  analyzer before snapshotting). 285 s -> 5 s, same 269x3 results.
- spec harness: jeval/run-spec/expect-throws fork from one lazy module-level
  snapshot. Spec sweep ~90 s -> 9 s, all pass.
- clojure-test-suite: run the per-file worker subprocesses through a
  token-channel worker pool (default 4, JOLT_SUITE_WORKERS to override,
  capped at 8). 17.5 s with counts identical to the sequential run
  (4046/520/116, 5 timeouts).

Full gate wall-clock: ~8 min -> 43 s, everything green (conformance 269x3,
fallback-zero, fixpoint, self-host, sci, staged, suite >= 4034/67, all
specs+unit).
This commit is contained in:
Yogthos 2026-06-10 09:11:58 -04:00
parent 3e9fe8d0fb
commit 920fafe032
4 changed files with 76 additions and 11 deletions

View file

@ -147,6 +147,34 @@
(load-core-overlay! ctx)
ctx))
# --- Context snapshot/fork (cheap isolated copies) --------------------------
#
# init is expensive (~50 ms interpreted, ~900 ms compiled: tier loading, analyzer
# build, macro recompilation). For workloads that need MANY isolated contexts —
# the test harnesses build a fresh ctx per case — snapshot a fully-built ctx once
# and fork cheap deep copies (~2 ms) from it via Janet marshal/unmarshal. A fork
# shares nothing mutable with the original: defs, protocol extensions, hierarchy
# changes, atom states in one fork are invisible to the others.
#
# The reverse-lookup dicts must be built from root-env (cfunctions and abstract
# values from the Janet runtime marshal by reference through them) BEFORE any ctx
# values exist in scope — module load time here, so user code can't leak into it.
(def- image-load-dict (env-lookup root-env))
(def- image-make-dict (invert image-load-dict))
(defn snapshot
"Marshal a fully-built context into a buffer that fork can cheaply clone.
Build the ctx (init), customize it if needed, then snapshot once."
[ctx]
(marshal ctx image-make-dict))
(defn fork
"A fresh, fully-isolated deep copy of a snapshotted context (~2 ms, vs
re-running init). (fork (snapshot ctx)) behaves exactly like ctx did at
snapshot time; mutations to a fork never affect the original or other forks."
[snap]
(unmarshal snap image-load-dict))
(defn eval-one
"Evaluate a single already-parsed form. Routing (compile when :compile? is set,
stateful forms interpret, interpreter fallback for forms the compiler can't