core: compiled macro expansion in every mode (+123 suite passes)

Macros are ordinary compiled fns in Clojure's model; compile mode has had
that since the staged bootstrap, but interpret mode — the conformance
battery's default — kept interpreted expanders, so every distinct (and ...)/
(cond ...) call form, and every fresh form produced by a recursive expansion,
ran an interpreted closure. ensure-macros-compiled! now runs in every mode:
interpret-mode init loads the tiers fast-interpreted, then one pass at the
end builds the analyzer (which itself stays interpreted there) and compiles
all stashed expanders; user defmacros after init compile too. The new
:compile-macros? opt (JOLT_INTERPRET_MACROS=1) preserves the fully-
interpreted oracle, and joins the ctx-image cache key.

Battery: 4700 pass / 90 clean files / 7 timeouts, from 4577 / 87 / 9 — two
macro-heavy files stopped timing out and 149 more assertions execute. The
compiled-expander delta proper is +67 passes (oracle mode on the same tree
measures 4633). Baselines raised 4540->4660, clean 86->88. Interpret init
grows 0.12s -> 1.12s for the analyzer build; init-cached amortizes it to ~5ms
per process.

New macro-expansion-test pins: expanders compiled in interpret mode (core +
post-init user defmacros), uncompilable bodies fall back interpreted and
still work, compile mode unchanged, oracle opt-out honored.

Follow-up filed (jolt-4j3): the same staged-recompile treatment for early
overlay DEFNS, which is what still pins keys/vals/empty? to the seed.
This commit is contained in:
Yogthos 2026-06-10 15:37:32 -04:00
parent 1ec2fa4adf
commit dc1f6d9755
5 changed files with 101 additions and 10 deletions

View file

@ -243,13 +243,14 @@
# Opts land in the key via their printed form; an opt that prints unstably
# (e.g. a closure in :namespaces) just degrades to a cache miss, never to a
# wrong hit. Runtime knobs that shape the ctx outside opts ride along too.
(def key (string/format "%q|%q|%q|%q|%q|%q"
(def key (string/format "%q|%q|%q|%q|%q|%q|%q"
(string janet/version "-" janet/build)
opts
(os/getenv "JOLT_PATH")
(os/getenv "JOLT_MUTABLE")
(os/getenv "JOLT_AOT_CORE")
(os/getenv "JOLT_FEATURES")))
(os/getenv "JOLT_FEATURES")
(os/getenv "JOLT_INTERPRET_MACROS")))
(string dir "/jolt-ctx-" (band h 0x7FFFFFFF) "-" len "-" (band (hash key) 0x7FFFFFFF) ".jimg"))
(defn init-cached

View file

@ -510,12 +510,16 @@
n)
(defn ensure-macros-compiled!
"Called once the overlay is fully loaded (api/load-core-overlay!): in compile
mode, ensure the analyzer is built, then run the staged macro-recompile pass so
the early (interpreted-during-bootstrap) macro expanders become compiled. No-op
in interpreter mode (no analyzer, macros stay interpreted by design) and cheap to
call again (recompile-macros! skips already-compiled vars)."
"Called once the overlay is fully loaded (api/load-core-overlay!): ensure the
analyzer is built, then run the staged macro-recompile pass so the early
(interpreted-during-bootstrap) macro expanders become compiled. Runs in EVERY
mode — macro expansion is compiled code even when evaluation is interpreted
(in interpret mode the tiers load fast interpreted, then this one pass builds
the analyzer and compiles all stashed expanders; the analyzer itself stays
interpreted there). :compile-macros? false (JOLT_INTERPRET_MACROS=1) skips it,
keeping the fully-interpreted oracle. Cheap to call again (recompile-macros!
skips already-compiled vars)."
[ctx]
(when (get (ctx :env) :compile?)
(when (get (ctx :env) :compile-macros?)
(ensure-analyzer ctx)
(when (analyzer-built? ctx) (recompile-macros! ctx))))

View file

@ -408,11 +408,20 @@
# (off unless opted in) and load-core-overlay! flips it on around core.
aot-core? (let [o (if opts (get opts :aot-core?) nil)]
(if (nil? o) (not (= "0" (os/getenv "JOLT_AOT_CORE"))) o))
# Macro expanders compile in EVERY mode (macros are ordinary compiled
# fns, as in Clojure) — including interpret mode, where evaluation stays
# interpreted but expansion runs native. :compile-macros? false (or
# JOLT_INTERPRET_MACROS=1) opts back into the fully-interpreted oracle.
compile-macros? (let [o (if opts (get opts :compile-macros?) nil)]
(if (nil? o)
(not (= "1" (os/getenv "JOLT_INTERPRET_MACROS")))
o))
env @{:namespaces @{}
:class->opts @{}
:current-ns "user"
:compile? compile?
:aot-core? aot-core?
:compile-macros? compile-macros?
:direct-linking? (if opts (get opts :direct-linking?) nil)
# Ordered roots searched (after the stdlib) to resolve a namespace
# to a .clj/.cljc file. jolt-core holds the portable Clojure layer