init in compile mode is ~2.4 s (tier loading, analyzer self-compile, macro
recompilation), paid by every process that builds a ctx from source — each
jpm-test file, embedders, workers. init-cached marshals the built ctx to a
disk image (same root-env dicts as snapshot/fork) and later processes
unmarshal it in ~5 ms, any process: nothing from the baking process is
needed at load.
The cache key fingerprints the embedded .clj stdlib (which covers jolt-core:
analyzer, IR, core tiers), the .janet seed sources next to the module, the
janet version, the init opts, and the env knobs that shape a ctx (JOLT_PATH/
MUTABLE/AOT_CORE/FEATURES) — any change rebuilds. Corrupt or non-ctx images
fall back to a rebuild (unmarshal of garbage can 'succeed' with a scalar, so
the shape is checked, not just the throw). Writes are atomic (tmp + rename)
so racing cold starts never publish a torn image. JOLT_NO_IMAGE_CACHE=1
opts out; JOLT_IMAGE_CACHE_DIR overrides the location (default TMPDIR).
Test consumers switch to init-cached (harness, suite-worker, conformance,
the behavioral unit/integration tests); tests that validate the bootstrap
itself (bootstrap-fixpoint, staged-bootstrap, aot round-trip, direct-linking)
and the deps tests (tmp-dir :paths would fragment the key) keep real init.
Full jpm test: 2:46 -> 1:58 (~29%). New ctx-image-test covers cold/warm,
cross-process load (subprocess runs defn/redef/macros/protocols/multimethods
off the baked image), per-opts keying, and corrupt-image fallback.
resolve-sym's last resort silently resolved any unknown Clojure symbol
against Janet's root environment — leaking Janet builtins with JANET
semantics into Clojure code: (type 1) was Janet's :number, (gensym) returned
Janet symbols (the long-documented (symbol (str (gensym))) macro landmine
existed BECAUSE of this), compare/slurp/int?/any? likewise. The explicit
janet/ prefix is the deliberate interop channel; the implicit fallback is
gone — an unresolved symbol is an error.
What the leak was masking, now proper interned vars:
- gensym: jolt's own (already existed, never interned) — returns real jolt
symbols; the macro landmine is dead
- compare: full Clojure total order (nil-first, numbers, strings, keywords,
symbols by ns/name, booleans, chars, uuid/inst, vectors by length then
elementwise; cross-type throws)
- type: :type metadata override, deftype/record tag as symbol, else a
taxonomy keyword (host-classified)
- int?: core-integer? — which had a latent bug the leak hid: (integer?
##Inf) was true (floor of inf is inf); NaN/infinities now excluded
- any?: constantly true (Clojure 1.9; SCI's namespaces.cljc needs it)
- jolt.interop/janet-type now uses the explicit (janet/type x) channel
- evaluator-test uses init (a bare make-ctx resolved EVERYTHING via the leak)
Suite 4470 -> 4532+ pass / 86-87 clean (proper compare unlocks the sort
files); baselines raised. Conformance 326x3 (+5 rows), +22 predicate spec
rows, stdlib battery green, all specs+unit. Coverage dashboard now counts
previously-leak-resolvable names honestly (missing-portable 19 -> 27).
prefer-method/remove-method/remove-all-methods/get-method/methods become
overlay macros over ctx-capturing *-setup fns (a multimethod's method table
lives on its VAR, so the name passes quoted — the defmulti/defmethod shape).
instance? likewise (class names don't evaluate to values); satisfies? is a
plain ctx-capturing fn (evaluated args). locking and defonce become overlay
macros — locking now also evaluates the monitor expr (the old arm skipped it
and any body form past the first); defonce keeps the existing-root check.
read-string and macroexpand-1 are ctx-capturing fns.
Removed all eleven from the evaluator special arms + special-symbol?,
host_iface special-names, and compiler uncompilable-heads. evaluator-test's
locking/instance? cases use init now (overlay macros need the full env).
Surfaced pre-existing (filed): multimethod dispatch records prefer-method
preferences on the var but never consults them in ambiguous isa dispatch.
Gate: conformance 296x3 (+11 tier-6c cases), fallback-zero 73/3, fixpoint,
self-host, sci, staged, suite 4049>=4034, all specs+unit.
* core: Stage 2 Task 2 tier 5a — compile defmulti + defmethod
defmulti/defmethod become macros (30-macros) over ctx-capturing
clojure.core fns (defmulti-setup/defmethod-setup, interned by
install-stateful-fns!):
- defmulti: (defmulti name dispatch & opts) -> (defmulti-setup 'name
dispatch ~@opts). name quoted; dispatch + opts (:default/:hierarchy)
evaluated. defmulti-setup builds the dispatch closure over the method
table and interns the var (same hierarchy/default/cache behavior).
- defmethod: (defmethod mm dval & fn-tail) -> (defmethod-setup 'mm dval
(fn ~@fn-tail)). The method impl is now a COMPILED (fn …) (was an
interpreted fn* eval). Auto-creates the multimethod if missing.
- removed their special-symbol? entries + eval-list arms, and dropped them
from host_iface special-names + loader stateful-head?.
Both compile + interpret as plain invokes; dispatch incl. :default and
derive/hierarchy works in both modes.
Tests: evaluator-test (defmulti case) + namespace-test now use init (these
forms are overlay macros now, so a bare make-ctx lacks them).
Gate green: conformance 269x3, fallback-zero 38/4, bootstrap-fixpoint
stage1==2==3, self-host, staged-bootstrap, sci-bootstrap, clojure-test-suite
>=4034/67, features 78/78, all unit + spec (multimethods 16/16).
* core: Stage 2 Task 2 tier 5b — compile deftype + defrecord
deftype becomes a macro (30-macros) over make-deftype-ctor (a ctx-capturing
clojure.core fn that bakes the ns-qualified type tag at def time) plus
extend-type for any inline protocol methods — so it compiles as a plain (do …).
Mirrors defrecord's existing field-let/protocol-grouping pattern.
- make-deftype-ctor-impl (evaluator) builds the ctor; interned as a closure.
- removed the deftype special-symbol? entry + eval-list arm; dropped deftype/
defrecord from host_iface special-names + loader stateful-head?.
- defrecord no longer redefines ->name via (Name. …) interop (frozen) — deftype
already provides ->name, so defrecord compiles too (map->name builds via it).
- field-kws spliced into a vector LITERAL ([~@…]) so the analyzer sees a vector
form, not a runtime pvec; type name + fields are unwrapped of ^meta (the reader
yields (with-meta sym m) forms, e.g. sci's (deftype ^{:doc …} Var …)).
With tier 5a, all of deftype/defrecord/defmulti/defmethod compile. The loader's
interpret-only set is now just the frozen host-coupled forms: defmacro/set!/./
new/eval.
Tests: evaluator-test deftype case uses init (deftype is an overlay macro now);
fallback-zero moves deftype off must-punt, adds deftype/defrecord/defmulti/
defmethod to must-compile (43/3).
Gate green (full jpm build + jpm test): conformance 269x3, fallback-zero 43/3,
bootstrap-fixpoint stage1==2==3, self-host, staged-bootstrap, sci-bootstrap
422/0, clojure-test-suite >=4034/67, all unit + spec.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Reorganize the flat 49-file test/ into three layers (jpm test recurses, so all
are still discovered):
- test/unit/ white-box component tests (reader, evaluator, types,
persistent-map, lazy-seq, macro, interop, compiler)
- test/integration/ cross-cutting + regression batteries (conformance, jank,
sci-bootstrap/runtime, features, systematic-coverage, api,
core, namespaces, ported clojure suites) and
.../ports/ ported clojure/cljs test batches pending consolidation
- test/spec/ the behavioral contract (built out in following commits)
- test/support/harness.janet shared defspec table runner (cases compared via
Jolt's own =, with a :throws sentinel) + expect= helpers
Files moved with git mv (history preserved) and import paths fixed for depth.
jpm test green. README Test section updated.
Next: build out test/spec/ to cover the public API area-by-area, mining the
integration batteries and filling gaps.
