defprotocol/defrecord/extend-type/extend-protocol/reify + the extend/proxy/
definterface stubs move to 30-macros (user-facing, not used by the compiler). They
emit Jolt's protocol/type special forms (protocol-dispatch/register-method/
make-reified/deftype).
The one subtlety: a protocol value is a :jolt/type-tagged struct, and the
interpreter can't tell an embedded opaque value from a tagged map literal being
constructed. So defprotocol builds it via a make-protocol fn call (exposed from
core) instead of an embedded literal — a fn result evaluates normally and even
compiles. reify emits a {kw (fn* ...)} map literal that make-reified evaluates
(build-eval-map yields a struct it can iterate, unlike a hash-map phm). defrecord
vecs its spliced field-let bindings (a lazy mapcat seq won't splice) and uses an
explicit fresh-sym for the map-> param (auto-gensym doesn't cross nested
syntax-quotes).
protocols-spec 21/21, conformance 228x3, fixpoint, clojure-test-suite 3930, full
suite green.
defn drops an optional leading docstring/attr-map then emits (def name (fn* ...)).
Single- and multi-arity both reduce to (fn* ~@body) so no arity branching is needed.
map? is true for symbol forms in Jolt, so the attr-map strip is guarded with symbol?.
defn- delegates to defn (privacy isn't enforced, as in Clojure's own defn-). Placed
before fresh-sym, which is itself a defn- now.
All five fundamental macros (fn/let/loop/defn/defn-) are now in the overlay.
conformance 228x3, fixpoint, clojure-test-suite 3930, full suite green.
let -> let* with destructuring pre-expanded via destructure (now exposed as a
clojure.core fn, which it is in Clojure too) so the compiler sees plain bindings —
analyze-bindings rejects patterns as uncompilable. loop -> loop* with raw bindings,
matching the prior Janet macro: loop can't pre-destructure without breaking recur
arity, so the interpreter handles pattern loops and the compiler falls back.
conformance 228x3, fixpoint, clojure-test-suite 3930.
fn -> fn* is a one-line head-swap (the analyzer treats fn* as the primitive). It's
in 00-syntax since the analyzer, kernel, and other overlay macro bodies all use fn.
First of the fundamental macros to move.
conformance 228x3, fixpoint, clojure-test-suite 3930.
Port of core-for: desugar a comprehension to nested map/mapcat over the binding
colls, :let -> let*, :while -> take-while on the coll. Lives in 00-syntax because
doseq (already there) expands to it; the macro body uses only kernel/seed fns so it
runs at analyzer-build time. doseq no longer depends on a Janet macro.
Fixed a latent bug the Janet macro had: :when wrapped the inner form in (list ...)
unconditionally, but for an outer binding group the inner form is already a seq, so
mapcat produced a seq-of-seqs instead of flattening. e.g.
(for [x [0 1] :when (odd? x) y [:a :b]] [x y])
gave ((... ...)) instead of ([1 :a] [1 :b]). Now the (list ...) wrap is only applied
to the last group's scalar body; outer groups contribute their seq directly. Added
:let+:when, multi-group :when, and destructuring regression cases.
conformance 228x3, fixpoint, clojure-test-suite 3930, full suite green, bench flat.
Same shortcut as the prior Janet macro — realize a for comprehension with count
for side effects, return nil — so for keeps handling :when/:let/:while and multiple
bindings. Lives in 00-syntax because the analyzer uses doseq (analyze-try). Added
:when/:while/:let/returns-nil regression cases.
conformance 228x3, fixpoint, clojure-test-suite 3930, full suite green.
The real bug behind the case/cond-> fixpoint regression wasn't gensym — it was
build ordering. A top-level defn in a pre-kernel tier (the fresh-sym helper these
macros need) gets compiled in compile mode, and that lazily builds the self-hosted
analyzer via ensure-analyzer. 00-syntax loads before 00-kernel, so the analyzer
was built against a core missing mapv/second/peek/...: its references to them were
interned as forward-ref nil cells in jolt.analyzer. Those nil cells then shadowed
the real clojure.core defs when the analyzer rebuilt itself (stage2), so the
analyzer's own mapv calls went to nil — 'Cannot call nil as a function'. Only
variadic-heavy paths (juxt+mapv) surfaced it.
build-compiler! already documented that the kernel must be loaded first; nothing
enforced it. Gate ensure-analyzer on a :kernel-ready? flag set after the kernel
tier loads. A pre-kernel compile now falls back to the interpreter (compile-and-eval
already handles that) instead of building the analyzer too early. fresh-sym ends up
interpreted during 00-syntax load, which is fine.
With ordering fixed, case and cond-> move to 00-syntax (they need a real gensym, so
syntax-quote auto-gensym alone won't do). conformance 228x3, fixpoint stage1==2==3,
clojure-test-suite 3930, full suite green, bench flat.
Threading macros (recursive; the expand-once cache makes that free) and declare
(a no-op on Jolt — forward refs resolve via pending cells). They live in 00-syntax
because the analyzer itself uses -> and declare; validated by conformance 228x3
(the bootstrap-compiled analyzer expands them).
conformance 228/228 x3, clojure-test-suite 3930.
The hot control macros now live in the Clojure syntax tier. This was blocked
before: as interpreted overlay macros they re-expanded on every eval, timing out
a battery file (3930->3911). With the expand-once macro cache (prior commit) they
expand a single time with zero runtime cost, so moving them is free.
conformance 228/228 x3, clojure-test-suite 3930 (9 timeouts, not 10), full suite
green, bench flat (overlay vs janet macros within noise).
New 00-syntax tier loaded FIRST (before 00-kernel), interpreted defmacros, so the
control macros the compiler and every later tier depend on can live in Clojure.
Validated by moving when: the kernel tier, self-hosted analyzer and seq/coll
tiers all compile against the overlay when. Constraint: syntax-tier macros may use
only special forms + core-renames seed primitives (not second/peek/kernel fns).
conformance 228/228 x3, full suite green.
Phase 3 batch 6 (jolt-1j0). binding installs an array-map var->value thread
frame and restores it on exit via try/finally. Dynamic rebinding is seen by
called fns. This completes the cleanly-portable safe macros (19 total).
conformance 228/228 x3, full suite green.
Phase 3 batch 5 (jolt-1j0). condp with a recursive emit building the nested if
chain, including the test :>> result-fn form and the trailing default. This
exhausts the cleanly-portable safe macros.
conformance 228/228 x3, full suite green.
Phase 3 batch 4 (jolt-1j0), 5 macros. cond->> (thread-last) is safe; cond->
stays (compiler uses it). delay/future expand to make-delay/future-call host
fns; letfn builds its fn* binding via a template (cons/list in a macro body make
a plist the evaluator can't call as a form).
conformance 228/228 x3, clojure-test-suite 3930, full suite green, bench flat.
Phase 3 batch 3 (jolt-1j0), 5 macros. Establishes the macro-body toolkit:
fresh-sym = (symbol (str (gensym))) for a shared explicit jolt symbol (a bare
(gensym) returns a Janet symbol the destructurer rejects), and splicing binding
pairs into a TEMPLATE vector so core-let sees a tuple form, not a runtime pvec.
when-first stays first-based (lazy-safe): Clojure's (seq coll) form realizes an
infinite coll like (repeat nil) under Jolt's eager evaluator and hangs — the
per-batch battery caught it (when_first.cljc). Clojure-correct seq form waits on
Phase 5 laziness.
conformance 228/228 x3, clojure-test-suite 3930, full suite green, bench flat.
Phase 3 batch 2 (jolt-1j0), 5 user-facing macros as defmacro + syntax-quote.
The conditional-binding macros use the auto-gensym temp# idiom so the name binds
only in the taken branch (the else/empty branch sees the outer scope, carrying
forward the earlier scope fix). when-let stays in Janet for now — 20-coll uses
it, so it must remain available before the macro tier loads.
Note: gensym in a macro body resolves to Janet's 0-arity builtin, so explicit
(gensym "prefix") fails — use template auto-gensym (foo#) instead.
conformance 228/228 x3, full suite green (incl. the if-let scope regression spec,
now exercising the overlay macro), clojure-test-suite 3930, bench A/B flat.
New 30-macros tier (registered after 20-coll) for user-facing macros expressed as
defmacro + syntax-quote instead of hand-built Janet form-transformers. Validated
end to end: overlay-defined macros expand in interpret, compile AND self-host
modes (conformance 228/228 x3).
Moved comment and if-not; removed their Janet core-X fns + core-macro-names
entries. Note: Jolt defmacro is single-arity, so multi-arglist macros become one
arglist with & rest / destructuring (if-not uses [test then & [else]]). Macros
used by the compiler or earlier tiers (and/or/when/cond/case/cond->/->/declare/
doseq/when-let) stay in Janet until a load-order story exists for them.
full suite green, clojure-test-suite 3930.
Eighth pure-fn batch (jolt-1j0 phase 2), 2 leaf fns. dedupe is eager consecutive
dedup (no transducer arity, as before); seq-to-map-for-destructuring is the
internal &{:keys} helper. Largely exhausts the easy pure-eager tier.
conformance 228/228 x3, full suite green (incl. destructuring).
Sixth pure-fn batch (jolt-1j0 phase 2), 3 leaf fns, now with regression tests
per the per-batch workflow. reductions is the canonical form, so (reductions f [])
calls (f) -> [0] instead of the prior []; tree-seq is eager pre-order DFS.
conformance 228/228 x3, clojure-test-suite 3929, full suite green, bench A/B flat.
Two correctness bugs found while migrating:
- nthrest returned nil where Clojure returns () : for n>0 the walk yields
(seq xs) wrapped in (or ... ()), so an exhausted/nil walk is () not nil
(only n<=0 returns coll). Both the prior Janet impl and the first overlay
port got this wrong. nthrest.cljc 13/1/0 -> 14/0/0.
- if-let/when-let/if-some/when-some leaked their binding into the else branch:
they wrapped the whole if in (let* [name val] ...), so (let [x 5] (if-let
[x nil] x x)) returned nil instead of 5. Fixed to bind a fresh temp around the
if and rebind the name only inside the taken branch, matching Clojure.
conformance 218/218 x3, clojure-test-suite 3928 -> 3929.
Fifth pure-fn batch (jolt-1j0 phase 2), 6 leaf fns. nthrest uses the canonical
loop form (same neutral 13/1/0 on nthrest.cljc as the prior Janet version; the
1 fail is a pre-existing platform-conditional case). object?/undefined? are
always false on Jolt.
conformance 218/218 x3, clojure-test-suite 3928, core-bench A/B noise-only.
Fourth pure-fn batch (jolt-1j0 phase 2), 6 leaf fns. distinct? now uses
value-semantics (the prior Janet impl keyed a table by identity, so equal
collections compared distinct); nthnext uses the canonical loop form, which
fixes (nthnext nil nil) => nil and the nil-count cases the prior pre-check threw
on. replace preserves nil values via get-with-default.
conformance 218/218 x3, clojure-test-suite 3927 -> 3928 (canonical nthnext),
core-bench A/B flat. Per-batch gate caught a transient -1 from the first
nthnext rewrite; fixed before commit.
Second pure-fn batch (jolt-1j0 phase 2). All leaf fns; canonical Clojure defs.
split-with now returns seqs for the parts (drop-while is a seq), matching
Clojure rather than the prior all-vector result — value-equal either way.
conformance 218/218 x3, clojure-test-suite 3927, core-bench 2333ms (~baseline).
First pure-fn migration batch. Both are leaf fns (no internal Janet callers);
the Clojure defs (not . some / not . every?) match the prior Janet behavior.
Removed the core-* defns and their core-bindings entries.
conformance 218/218 x3, clojure-test-suite 3927, core-bench 2340ms vs 2336ms
baseline (noise).
Classify the 421 core-* fns into seed/macro/host/lazy/movable buckets and record
a compile-mode benchmark baseline to catch regressions as fns move from native
Janet to the self-hosted overlay. Worklist in jolt-core/clojure/core/MIGRATION.md.
Finding: after seed + host, the self-hosted compiler uses no clojure.core fns
beyond the existing kernel tier (+ atom/swap!/reset! host primitives), so the
compiler-dep kernel tier is already complete — confirmed by self-host
conformance passing.
Three pure function combinators into the collection tier, as compositions of
mapv/every?/some/apply. Three more core-* primitives + table entries gone.
Battery 3916 -> 3920 (the canonical defs are more correct than the prior Janet
ones — some-fn returns the matching value, every-pred is properly boolean).
Conformance 218/218 all modes.
New collection tier (core/20-coll.clj): six pure, eager fns expressed as
compositions of frozen primitives (reduce/assoc/get/conj/filter/vec). Six more
core-* primitives + table entries gone from the Janet kernel.
Two semantics to preserve, caught by the suite:
- max-key/min-key use the canonical multi-arity form (strict </> on the first
pair, <=/>= in the fold) to reproduce the JVM IEEE-754 NaN behavior; a single
reduce got the NaN cases wrong.
- frequencies/group-by base on (hash-map), not the {} literal: a struct map
doesn't canonicalize collection keys across representations ({:a 1} literal vs
(hash-map :a 1)), a PHM does — same reason the Janet impl used make-phm.
Conformance 218/218 all modes; battery holds 3916.
First fractal turn of the multi-stage bootstrap (jolt-tzo). The Clojure part of
clojure.core is now loaded in ordered tiers under jolt-core/clojure/core/. The
kernel tier (00-kernel: second/peek/subvec/mapv/update) holds the structural fns
the self-hosted compiler itself uses; in compile mode it is bootstrap-compiled
into clojure.core BEFORE the analyzer is built, so the analyzer binds those names
to the Clojure definitions instead of a not-yet-defined forward ref. That removes
the circularity that previously forced these to stay in Janet — the five core-*
primitives and their init-core! entries are gone.
Mechanism: backend/bootstrap-load-source (generalized from compile-load) builds a
source string into a target ns via the bootstrap; rebuild-compiler! recompiles
the self-hosted compiler against the current core (the rail for future turns,
exercised by the new fixpoint test). api/load-core-overlay! walks the ordered
tiers, bootstrap-loading kernel tiers and self-hosting the rest.
Also fixes a latent evaluator bug surfaced by the move: a fn rebinds current-ns
to its defining ns while it runs and restores on normal return, but a fn that
THREW unwound past its own restore, leaking the ns. try now restores the
try-entry ns on the catch/finally path, so referred-symbol resolution survives a
caught error (this was breaking is/testing in the suite harness after any thrown
assertion). Net battery +3 (3913 -> 3916); conformance 218/218 in all three
modes; binary builds and runs the embedded tiers.
Continue the kernel shrink. last/butlast become canonical Clojure defs
(pure first/next/loop/recur) in the core.clj overlay, dropping two
realize-for-iteration uses from the Janet kernel.
second was the obvious next candidate but can't move: the self-hosted
compiler front-end (analyzer.clj) calls it, and the compiler has to
compile the overlay, so anything it uses must already exist as a Janet
primitive. Documented that as a third clause of the safe-to-move rule.
Suite green, clojure-test-suite battery holds 3913 pass / 58 clean,
binary builds and runs the embedded overlay.
Phase 4 kernel-shrink seam. jolt-core/clojure/core.clj holds the Clojure portion
of clojure.core; api/init loads it into the clojure.core namespace after
init-core! interns the Janet primitives, routed through eval-toplevel so it
compiles via the self-hosted pipeline (or interprets when :compile? is off).
First three fns moved off the Janet side: ffirst, fnext, nnext (leaf, no internal
callers). Same results compiled or interpreted; full suite green; battery holds
3913.
