core: compile macro expanders via staged bootstrap (Stage 2 Task 1)

Macros are now compiled, not interpreted, by steady state — matching
Clojure (macros are ordinary compiled fns the Java seed compiles for
clojure.core) and ClojureScript (macros compiled, invoked at compile
time). Neither reference keeps an interpreted-closure fallback.

The early macros (00-syntax) are defined while the self-hosted analyzer
is still being bootstrapped (it builds lazily after the overlay loads),
so macro-compile-hook returns nil and they get an interpreted closure.
The bootstrap compiler.janet can't substitute (it punts on syntax-quote,
which nearly every expander uses).

Fix = staged bootstrap, the same pattern as the compiler fixpoint:
defmacro stashes the expander source on the var (:macro-src) plus a
:macro-uses-env flag; once the overlay + analyzer are fully built,
backend/recompile-macros! (via ensure-macros-compiled! at the end of
load-core-overlay!) compiles each stashed expander through the now-live
analyzer and rebinds the var, marking :macro-compiled. Idempotent;
&env/&form macros keep the interpreted closure (the compiled fn* has no
such params). The interpreter is now a build-time crutch, gone by
steady state.

Rewrote if-not/if-let/if-some/assert from '& [else]' rest-destructuring
(which the analyzer punts on) to a plain rest param + (first rest), so
all 47 overlay macros compile. Analyzer rest-destructuring gap: jolt-f79.

47/47 overlay macros compiled, 0 interpreted; user macros compile
immediately post-init. Gate green: conformance 267x3, fallback-zero
31/5, bootstrap-fixpoint stage1==2==3, self-host, staged-bootstrap,
lazy-infinite 44/44, clojure-test-suite >=4034/67, sci-bootstrap,
features 78/78, all unit+spec, core-bench neutral.
This commit is contained in:
Yogthos 2026-06-09 08:27:00 -04:00
parent 1ded89b47b
commit ad5216a178
4 changed files with 77 additions and 14 deletions

View file

@ -12,25 +12,27 @@
(defmacro comment [& body] nil)
;; Single arglist (Jolt defmacro is single-arity); the optional else defaults nil
;; via rest-destructuring.
(defmacro if-not [test then & [else]]
`(if (not ~test) ~then ~else))
;; Single arglist (Jolt defmacro is single-arity); the optional else is the head of
;; a plain rest param — (first else) is the else form or nil. (A `& [else]`-style
;; rest-destructure reads the same but the analyzer can't yet compile destructuring
;; in a rest param, which would force this expander to stay interpreted — jolt-f79.)
(defmacro if-not [test then & else]
`(if (not ~test) ~then ~(first else)))
;; Conditional binding macros: the name is bound ONLY in the taken branch (the
;; auto-gensym temp# tests the value; the else/empty branch sees the surrounding
;; scope). temp# is a single template-local gensym — referenced twice, same symbol.
(defmacro if-let [bindings then & [else]]
(defmacro if-let [bindings then & else]
(let [form (bindings 0) tst (bindings 1)]
`(let [temp# ~tst]
(if temp# (let [~form temp#] ~then) ~else))))
(if temp# (let [~form temp#] ~then) ~(first else)))))
;; when-let lives in 00-syntax (not here): 20-coll uses it, which loads before this tier.
(defmacro if-some [bindings then & [else]]
(defmacro if-some [bindings then & else]
(let [form (bindings 0) tst (bindings 1)]
`(let [temp# ~tst]
(if (some? temp#) (let [~form temp#] ~then) ~else))))
(if (some? temp#) (let [~form temp#] ~then) ~(first else)))))
(defmacro when-some [bindings & body]
(let [form (bindings 0) tst (bindings 1)]
@ -91,8 +93,9 @@
(partition 2 clauses))]
`(let [~g ~expr ~@(thread-binds g steps)] ~(if (empty? steps) g (last steps)))))
(defmacro assert [x & [message]]
(let [msg (if message message (str "Assert failed: " (pr-str x)))]
(defmacro assert [x & message]
(let [m (first message)
msg (if m m (str "Assert failed: " (pr-str x)))]
`(when-not ~x (throw (ex-info ~msg {})))))
(defmacro delay [& body]

View file

@ -15,9 +15,13 @@
(import ./host_iface :as host)
# A defmacro expander compiles to a native fn (built as (fn* args body...) and run
# through the self-hosted pipeline) so macro expansion is compiled, zero runtime
# cost — instead of an interpreted closure. Returns nil (interpreted fallback) when
# the analyzer isn't built yet or the body isn't compilable.
# through the self-hosted pipeline) so macro expansion is COMPILED code, zero runtime
# cost — instead of an interpreted closure, mirroring Clojure (macros are ordinary
# compiled fns). Returns nil when the analyzer isn't built yet (the early macros,
# expanded WHILE the analyzer is being bootstrapped) or the body isn't compilable; in
# that case defmacro keeps an interpreted closure, and backend/recompile-macros!
# replaces it with a compiled expander once the analyzer comes alive (staged
# bootstrap — the interpreter is a build-time crutch, gone by steady state).
(set macro-compile-hook
(fn [ctx args-form body]
(backend/try-compile-fn ctx
@ -92,7 +96,12 @@
# half-loaded core (which would forward-ref the missing kernel fns to nil).
(when (tier :kernel) (put env :kernel-ready? true))))
(put env :direct-linking? user-dl)
(ctx-set-current-ns ctx saved))
(ctx-set-current-ns ctx saved)
# Staged bootstrap: the early macros (00-syntax) were defined while the analyzer
# was still being built, so their expanders are interpreted closures. Now that the
# full overlay + analyzer are in place, recompile those expanders to native code —
# by steady state no macro expansion runs interpreted (no-op in interpreter mode).
(backend/ensure-macros-compiled! ctx))
(defn init
"Create a new Jolt evaluation context.

View file

@ -372,3 +372,46 @@
(when (compiled 0)
(def r (protect (eval (compiled 1) (comp/ctx-janet-env ctx))))
(when (r 0) (r 1)))))
(def- fn*-sym {:jolt/type :symbol :ns nil :name "fn*"})
(defn recompile-macros!
"Staged-bootstrap second pass: once the self-hosted analyzer is alive, replace
every interpreted macro expander with a COMPILED one. The early macros (00-syntax
etc.) are defined WHILE the analyzer is still being bootstrapped, so their
expanders can't compile yet (the analyzer they'd compile through doesn't exist) —
defmacro gives them an interpreted closure as a build-time crutch and stashes the
source on the var (:macro-src). This pass compiles that source through the now-live
analyzer and rebinds the var, so by steady state no macro expansion is interpreted
— mirroring how a self-hosting compiler recompiles its seed once it can.
Idempotent: a var compiled once is marked :macro-compiled and skipped (so the
refer of a core macro into another ns, or a later rebuild, costs nothing). A macro
whose body uses &env/&form keeps its interpreted closure (the compiled fn* has no
such params). Returns the number of expanders compiled this pass."
[ctx]
(var n 0)
(each ns (all-ns ctx)
(each v (ns :mappings)
(when (and (var? v) (var-macro? v)
(v :macro-src) (not (v :macro-compiled))
(not (v :macro-uses-env)))
(def [args-form body] (v :macro-src))
(def compiled
(try-compile-fn ctx (array/concat @[fn*-sym args-form] body)))
(when compiled
(bind-root v compiled)
(put v :macro-compiled true)
(++ n)))))
n)
(defn ensure-macros-compiled!
"Called once the overlay is fully loaded (api/load-core-overlay!): in compile
mode, ensure the analyzer is built, then run the staged macro-recompile pass so
the early (interpreted-during-bootstrap) macro expanders become compiled. No-op
in interpreter mode (no analyzer, macros stay interpreted by design) and cheap to
call again (recompile-macros! skips already-compiled vars)."
[ctx]
(when (get (ctx :env) :compile?)
(ensure-analyzer ctx)
(when (analyzer-built? ctx) (recompile-macros! ctx))))

View file

@ -805,6 +805,14 @@
ns (ctx-find-ns ctx ns-name)]
(def v (ns-intern ns (name-sym :name) macro-fn))
(put v :macro true)
# Stash the expander source so backend/recompile-macros! can
# compile it once the analyzer is alive (staged bootstrap): a
# macro defined WHILE the analyzer is still being built gets an
# interpreted closure now, a compiled expander later. uses-env
# macros stay interpreted (the compiled fn* has no &env/&form).
(put v :macro-src @[args-form body])
(put v :macro-uses-env uses-env)
(when compiled-fn (put v :macro-compiled true))
# A (re)defined macro invalidates any cached expansions.
(table/clear macro-cache)
(var-get v)))