test: ~11x faster gate — ctx snapshot/fork + parallel suite battery
The gate spent almost all its time rebuilding identical contexts: init is ~50 ms interpreted / ~900 ms compiled (tier loading, analyzer build, macro recompilation), and both the conformance harness and the spec harness built a fresh ctx PER CASE — 269 cases x 3 modes for conformance alone (~285 s), and ~1500 spec cases (~90 s). The suite battery additionally ran its 234 worker subprocesses sequentially (~100 s incl. 5 x 6 s timeout files). - api: snapshot/fork — marshal a fully-built ctx once (reverse-lookup dicts from root-env, built at module load before any ctx exists), unmarshal cheap fully-isolated deep copies (~2 ms). A fork shares nothing mutable with its siblings, so per-case isolation is preserved exactly. - conformance: one init per mode + fork per case (self-host pre-builds the analyzer before snapshotting). 285 s -> 5 s, same 269x3 results. - spec harness: jeval/run-spec/expect-throws fork from one lazy module-level snapshot. Spec sweep ~90 s -> 9 s, all pass. - clojure-test-suite: run the per-file worker subprocesses through a token-channel worker pool (default 4, JOLT_SUITE_WORKERS to override, capped at 8). 17.5 s with counts identical to the sequential run (4046/520/116, 5 timeouts). Full gate wall-clock: ~8 min -> 43 s, everything green (conformance 269x3, fallback-zero, fixpoint, self-host, sci, staged, suite >= 4034/67, all specs+unit).
This commit is contained in:
parent
3e9fe8d0fb
commit
920fafe032
4 changed files with 76 additions and 11 deletions
|
|
@ -147,6 +147,34 @@
|
|||
(load-core-overlay! ctx)
|
||||
ctx))
|
||||
|
||||
# --- Context snapshot/fork (cheap isolated copies) --------------------------
|
||||
#
|
||||
# init is expensive (~50 ms interpreted, ~900 ms compiled: tier loading, analyzer
|
||||
# build, macro recompilation). For workloads that need MANY isolated contexts —
|
||||
# the test harnesses build a fresh ctx per case — snapshot a fully-built ctx once
|
||||
# and fork cheap deep copies (~2 ms) from it via Janet marshal/unmarshal. A fork
|
||||
# shares nothing mutable with the original: defs, protocol extensions, hierarchy
|
||||
# changes, atom states in one fork are invisible to the others.
|
||||
#
|
||||
# The reverse-lookup dicts must be built from root-env (cfunctions and abstract
|
||||
# values from the Janet runtime marshal by reference through them) BEFORE any ctx
|
||||
# values exist in scope — module load time here, so user code can't leak into it.
|
||||
(def- image-load-dict (env-lookup root-env))
|
||||
(def- image-make-dict (invert image-load-dict))
|
||||
|
||||
(defn snapshot
|
||||
"Marshal a fully-built context into a buffer that fork can cheaply clone.
|
||||
Build the ctx (init), customize it if needed, then snapshot once."
|
||||
[ctx]
|
||||
(marshal ctx image-make-dict))
|
||||
|
||||
(defn fork
|
||||
"A fresh, fully-isolated deep copy of a snapshotted context (~2 ms, vs
|
||||
re-running init). (fork (snapshot ctx)) behaves exactly like ctx did at
|
||||
snapshot time; mutations to a fork never affect the original or other forks."
|
||||
[snap]
|
||||
(unmarshal snap image-load-dict))
|
||||
|
||||
(defn eval-one
|
||||
"Evaluate a single already-parsed form. Routing (compile when :compile? is set,
|
||||
stateful forms interpret, interpreter fallback for forms the compiler can't
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue