Commit graph

281 commits

Author SHA1 Message Date
Yogthos
4eb2cf5c46 core: move let and loop to the syntax tier
let -> let* with destructuring pre-expanded via destructure (now exposed as a
clojure.core fn, which it is in Clojure too) so the compiler sees plain bindings —
analyze-bindings rejects patterns as uncompilable. loop -> loop* with raw bindings,
matching the prior Janet macro: loop can't pre-destructure without breaking recur
arity, so the interpreter handles pattern loops and the compiler falls back.

conformance 228x3, fixpoint, clojure-test-suite 3930.
2026-06-07 17:29:39 -04:00
Yogthos
85cdb97320 core: move fn to the syntax tier
fn -> fn* is a one-line head-swap (the analyzer treats fn* as the primitive). It's
in 00-syntax since the analyzer, kernel, and other overlay macro bodies all use fn.
First of the fundamental macros to move.

conformance 228x3, fixpoint, clojure-test-suite 3930.
2026-06-07 17:22:30 -04:00
Yogthos
023fe637ce core: move for to the syntax tier (and fix multi-group :when)
Port of core-for: desugar a comprehension to nested map/mapcat over the binding
colls, :let -> let*, :while -> take-while on the coll. Lives in 00-syntax because
doseq (already there) expands to it; the macro body uses only kernel/seed fns so it
runs at analyzer-build time. doseq no longer depends on a Janet macro.

Fixed a latent bug the Janet macro had: :when wrapped the inner form in (list ...)
unconditionally, but for an outer binding group the inner form is already a seq, so
mapcat produced a seq-of-seqs instead of flattening. e.g.
  (for [x [0 1] :when (odd? x) y [:a :b]] [x y])
gave ((... ...)) instead of ([1 :a] [1 :b]). Now the (list ...) wrap is only applied
to the last group's scalar body; outer groups contribute their seq directly. Added
:let+:when, multi-group :when, and destructuring regression cases.

conformance 228x3, fixpoint, clojure-test-suite 3930, full suite green, bench flat.
2026-06-07 16:55:21 -04:00
Yogthos
40b0f525f3 core: move doseq to the syntax tier
Same shortcut as the prior Janet macro — realize a for comprehension with count
for side effects, return nil — so for keeps handling :when/:let/:while and multiple
bindings. Lives in 00-syntax because the analyzer uses doseq (analyze-try). Added
:when/:while/:let/returns-nil regression cases.

conformance 228x3, fixpoint, clojure-test-suite 3930, full suite green.
2026-06-07 16:40:24 -04:00
Yogthos
646744ca1d self-host: gate the analyzer build on the kernel tier; move case/cond-> to the overlay
The real bug behind the case/cond-> fixpoint regression wasn't gensym — it was
build ordering. A top-level defn in a pre-kernel tier (the fresh-sym helper these
macros need) gets compiled in compile mode, and that lazily builds the self-hosted
analyzer via ensure-analyzer. 00-syntax loads before 00-kernel, so the analyzer
was built against a core missing mapv/second/peek/...: its references to them were
interned as forward-ref nil cells in jolt.analyzer. Those nil cells then shadowed
the real clojure.core defs when the analyzer rebuilt itself (stage2), so the
analyzer's own mapv calls went to nil — 'Cannot call nil as a function'. Only
variadic-heavy paths (juxt+mapv) surfaced it.

build-compiler! already documented that the kernel must be loaded first; nothing
enforced it. Gate ensure-analyzer on a :kernel-ready? flag set after the kernel
tier loads. A pre-kernel compile now falls back to the interpreter (compile-and-eval
already handles that) instead of building the analyzer too early. fresh-sym ends up
interpreted during 00-syntax load, which is fine.

With ordering fixed, case and cond-> move to 00-syntax (they need a real gensym, so
syntax-quote auto-gensym alone won't do). conformance 228x3, fixpoint stage1==2==3,
clojure-test-suite 3930, full suite green, bench flat.
2026-06-07 16:29:50 -04:00
Yogthos
d0d48f0ebd core: move ->/->>/declare to the syntax tier
Threading macros (recursive; the expand-once cache makes that free) and declare
(a no-op on Jolt — forward refs resolve via pending cells). They live in 00-syntax
because the analyzer itself uses -> and declare; validated by conformance 228x3
(the bootstrap-compiled analyzer expands them).

conformance 228/228 x3, clojure-test-suite 3930.
2026-06-07 10:18:56 -04:00
Yogthos
613aaa5451 core: move and/or/cond/when-not to the overlay (enabled by expand-once cache)
The hot control macros now live in the Clojure syntax tier. This was blocked
before: as interpreted overlay macros they re-expanded on every eval, timing out
a battery file (3930->3911). With the expand-once macro cache (prior commit) they
expand a single time with zero runtime cost, so moving them is free.

conformance 228/228 x3, clojure-test-suite 3930 (9 timeouts, not 10), full suite
green, bench flat (overlay vs janet macros within noise).
2026-06-07 10:08:10 -04:00
Yogthos
29a79f34f4 interpreter: expand each macro form once (cache), zero runtime cost
The interpreter re-expanded a macro call on every evaluation — so a macro in a
hot loop re-expanded each iteration (the runtime cost, and why moving the hot
macros to the overlay timed out battery files). Now each macro CALL form expands
once, keyed by form identity, and the macro-free expansion is reused; this is the
proper Lisp model (macroexpansion is a compile-time step). Also gives compile-once
gensym semantics — a foo# auto-gensym is now fixed across calls instead of fresh
per re-expansion. Cache cleared when a macro is (re)defined.

conformance 228/228 x3, full suite green.
2026-06-07 10:02:05 -04:00
Yogthos
56a746bb73 compiler: compile defmacro expanders (native-speed macro expansion)
A defmacro now compiles its expander to a native Janet fn — built as
(fn* args body...) and run through the self-hosted pipeline (its backtick body is
now compilable, prior commit) — instead of an interpreted closure. Macro
expansion happens at compile time with zero runtime cost, the proper Lisp model.

Wiring: evaluator exposes a macro-compile-hook the api sets to backend/
try-compile-fn; the defmacro handler uses the compiled expander when available,
falling back to the interpreted closure when the analyzer isn't built yet (early
tiers) or the body uses &env/&form (no such params on the compiled fn).

conformance 228/228 x3 (incl. REPL defmacro + gensym hygiene), full suite green.
2026-06-07 09:41:51 -04:00
Yogthos
b438835cf6 compiler: lower syntax-quote to construction code so backtick compiles
Following the canonical read->macroexpand->compile model (Clojure LispReader /
tools.reader): a syntax-quote is lowered to plain construction code instead of
being interpreted. Adds form builders (__sqcat -> array, __sqvec -> tuple,
__sqmap, __sq1) and syntax-quote-lower (mirrors the interpreter's syntax-quote*/
sq-symbol exactly: foo# auto-gensym, core/special unqualified, else qualify;
~ -> expr, ~@ -> splice). The analyzer now handles syntax-quote as a special:
lower then analyze the result. The interpreter keeps the self-contained
syntax-quote* (no core dependency, works in a bare ctx); the two are cross-checked
by conformance interpret-vs-compile.

Effect: a backtick body compiles (1.96s -> 1.19s/200k) instead of falling back to
the interpreter. Next: compile defmacro expanders for the full macro speedup.

conformance 228/228 x3, full suite green.
2026-06-07 09:32:52 -04:00
Yogthos
ed118b9ce6 migration doc: phase 3 status + the hot-macro expansion-speed finding 2026-06-07 08:47:04 -04:00
Yogthos
d2d33a2ea9 core: syntax tier — move when to the overlay ahead of the kernel (jolt-1j0 phase 3)
New 00-syntax tier loaded FIRST (before 00-kernel), interpreted defmacros, so the
control macros the compiler and every later tier depend on can live in Clojure.
Validated by moving when: the kernel tier, self-hosted analyzer and seq/coll
tiers all compile against the overlay when. Constraint: syntax-tier macros may use
only special forms + core-renames seed primitives (not second/peek/kernel fns).

conformance 228/228 x3, full suite green.
2026-06-07 08:37:19 -04:00
Yogthos
3dafa60e65 core: move binding macro to overlay
Phase 3 batch 6 (jolt-1j0). binding installs an array-map var->value thread
frame and restores it on exit via try/finally. Dynamic rebinding is seen by
called fns. This completes the cleanly-portable safe macros (19 total).

conformance 228/228 x3, full suite green.
2026-06-07 01:13:22 -04:00
Yogthos
394bbe07c3 core: move condp macro to overlay
Phase 3 batch 5 (jolt-1j0). condp with a recursive emit building the nested if
chain, including the test :>> result-fn form and the trailing default. This
exhausts the cleanly-portable safe macros.

conformance 228/228 x3, full suite green.
2026-06-07 01:08:44 -04:00
Yogthos
24b217d314 core: move cond->>/assert/delay/future/letfn macros to overlay
Phase 3 batch 4 (jolt-1j0), 5 macros. cond->> (thread-last) is safe; cond->
stays (compiler uses it). delay/future expand to make-delay/future-call host
fns; letfn builds its fn* binding via a template (cons/list in a macro body make
a plist the evaluator can't call as a form).

conformance 228/228 x3, clojure-test-suite 3930, full suite green, bench flat.
2026-06-07 01:04:21 -04:00
Yogthos
31a257ce70 core: move as->/some->/some->>/doto/when-first macros to overlay
Phase 3 batch 3 (jolt-1j0), 5 macros. Establishes the macro-body toolkit:
fresh-sym = (symbol (str (gensym))) for a shared explicit jolt symbol (a bare
(gensym) returns a Janet symbol the destructurer rejects), and splicing binding
pairs into a TEMPLATE vector so core-let sees a tuple form, not a runtime pvec.

when-first stays first-based (lazy-safe): Clojure's (seq coll) form realizes an
infinite coll like (repeat nil) under Jolt's eager evaluator and hangs — the
per-batch battery caught it (when_first.cljc). Clojure-correct seq form waits on
Phase 5 laziness.

conformance 228/228 x3, clojure-test-suite 3930, full suite green, bench flat.
2026-06-07 00:55:19 -04:00
Yogthos
e611034550 core: move if-let/if-some/when-some/while/dotimes macros to overlay
Phase 3 batch 2 (jolt-1j0), 5 user-facing macros as defmacro + syntax-quote.
The conditional-binding macros use the auto-gensym temp# idiom so the name binds
only in the taken branch (the else/empty branch sees the outer scope, carrying
forward the earlier scope fix). when-let stays in Janet for now — 20-coll uses
it, so it must remain available before the macro tier loads.

Note: gensym in a macro body resolves to Janet's 0-arity builtin, so explicit
(gensym "prefix") fails — use template auto-gensym (foo#) instead.

conformance 228/228 x3, full suite green (incl. the if-let scope regression spec,
now exercising the overlay macro), clojure-test-suite 3930, bench A/B flat.
2026-06-07 00:38:02 -04:00
Yogthos
330a3a23d9 core: start macro tier — move comment/if-not to the Clojure overlay (jolt-1j0 phase 3)
New 30-macros tier (registered after 20-coll) for user-facing macros expressed as
defmacro + syntax-quote instead of hand-built Janet form-transformers. Validated
end to end: overlay-defined macros expand in interpret, compile AND self-host
modes (conformance 228/228 x3).

Moved comment and if-not; removed their Janet core-X fns + core-macro-names
entries. Note: Jolt defmacro is single-arity, so multi-arglist macros become one
arglist with & rest / destructuring (if-not uses [test then & [else]]). Macros
used by the compiler or earlier tiers (and/or/when/cond/case/cond->/->/declare/
doseq/when-let) stay in Janet until a load-order story exists for them.

full suite green, clojure-test-suite 3930.
2026-06-07 00:29:34 -04:00
Yogthos
f0e111563b core: move dedupe + seq-to-map-for-destructuring to overlay (+ tests)
Eighth pure-fn batch (jolt-1j0 phase 2), 2 leaf fns. dedupe is eager consecutive
dedup (no transducer arity, as before); seq-to-map-for-destructuring is the
internal &{:keys} helper. Largely exhausts the easy pure-eager tier.

conformance 228/228 x3, full suite green (incl. destructuring).
2026-06-07 00:14:40 -04:00
Yogthos
fec5a10ccf core: move some/flatten/interleave/rationalize to overlay (+ spec tests)
Seventh pure-fn batch (jolt-1j0 phase 2), 4 leaf fns. flatten is the canonical
tree-seq form, which also flattens lists (sequential?) — the prior Janet impl's
seqish? predicate missed them. some/interleave eager; rationalize identity (no
ratio type).

conformance 228/228 x3, clojure-test-suite 3929 -> 3930 (flatten lists fix),
full suite green, bench A/B flat.
2026-06-07 00:09:27 -04:00
Yogthos
921b395dfd core: move comparator/reductions/tree-seq to overlay (+ spec tests)
Sixth pure-fn batch (jolt-1j0 phase 2), 3 leaf fns, now with regression tests
per the per-batch workflow. reductions is the canonical form, so (reductions f [])
calls (f) -> [0] instead of the prior []; tree-seq is eager pre-order DFS.

conformance 228/228 x3, clojure-test-suite 3929, full suite green, bench A/B flat.
2026-06-06 23:58:11 -04:00
Yogthos
c35cf7bdbc migration doc: make regression tests part of the per-batch workflow 2026-06-06 23:51:40 -04:00
Yogthos
e646af123a test: regression coverage for the core migration + the two bug fixes
Spec suites (interpret) for the overlay-migrated fns and both fixes:
  - control-flow: if-let/when-let/if-some/when-some else-scope (9 cases)
  - predicates: not-any?/idents/numeric preds/NaN?/abs/object?/... (24 cases)
  - sequences: nthrest()/nthnext/distinct? value-eq/replace nil/take-last/... (23)
Plus 10 cases added to the all-3-modes conformance set for the highest-risk
fixes (if-let scope, nthrest (), distinct? value-eq), so compile + self-host are
guarded too: conformance 218 -> 228 in all three modes.

Full suite green.
2026-06-06 23:51:13 -04:00
Yogthos
f22eb660d3 fix nthrest empty-list semantics + if-let/when-let/if-some/when-some scoping
Two correctness bugs found while migrating:

- nthrest returned nil where Clojure returns () : for n>0 the walk yields
  (seq xs) wrapped in (or ... ()), so an exhausted/nil walk is () not nil
  (only n<=0 returns coll). Both the prior Janet impl and the first overlay
  port got this wrong. nthrest.cljc 13/1/0 -> 14/0/0.

- if-let/when-let/if-some/when-some leaked their binding into the else branch:
  they wrapped the whole if in (let* [name val] ...), so (let [x 5] (if-let
  [x nil] x x)) returned nil instead of 5. Fixed to bind a fresh temp around the
  if and rebind the name only inside the taken branch, matching Clojure.

conformance 218/218 x3, clojure-test-suite 3928 -> 3929.
2026-06-06 23:42:08 -04:00
Yogthos
185cf30c1d core: move nthrest/abs/NaN?/object?/undefined?/keyword-identical? to overlay
Fifth pure-fn batch (jolt-1j0 phase 2), 6 leaf fns. nthrest uses the canonical
loop form (same neutral 13/1/0 on nthrest.cljc as the prior Janet version; the
1 fail is a pre-existing platform-conditional case). object?/undefined? are
always false on Jolt.

conformance 218/218 x3, clojure-test-suite 3928, core-bench A/B noise-only.
2026-06-06 23:34:31 -04:00
Yogthos
4aec21e603 core: move distinct?/replace/nthnext/bounded-count/run!/completing to overlay
Fourth pure-fn batch (jolt-1j0 phase 2), 6 leaf fns. distinct? now uses
value-semantics (the prior Janet impl keyed a table by identity, so equal
collections compared distinct); nthnext uses the canonical loop form, which
fixes (nthnext nil nil) => nil and the nil-count cases the prior pre-check threw
on. replace preserves nil values via get-with-default.

conformance 218/218 x3, clojure-test-suite 3927 -> 3928 (canonical nthnext),
core-bench A/B flat. Per-batch gate caught a transient -1 from the first
nthnext rewrite; fixed before commit.
2026-06-06 23:26:36 -04:00
Yogthos
b1daa4bcbd core: move numeric predicates + replicate/take-last/drop-last to overlay
Third pure-fn batch (jolt-1j0 phase 2), 9 leaf fns: ratio?/decimal? (always
false on Jolt), rational?/nat-int?/neg-int?/pos-int? (reduce to int?), replicate,
take-last, drop-last. Canonical Clojure defs over already-frozen primitives.

conformance 218/218 x3, clojure-test-suite 3927. core-bench is load-sensitive in
absolute terms; A/B under identical load shows this batch == prior (no regression).
2026-06-06 23:14:49 -04:00
Yogthos
681b007b7a core: move split-at/split-with/ident?/qualified-ident?/simple-ident? to overlay
Second pure-fn batch (jolt-1j0 phase 2). All leaf fns; canonical Clojure defs.
split-with now returns seqs for the parts (drop-while is a seq), matching
Clojure rather than the prior all-vector result — value-equal either way.

conformance 218/218 x3, clojure-test-suite 3927, core-bench 2333ms (~baseline).
2026-06-06 23:05:24 -04:00
Yogthos
16cd617bd6 core: move not-any?/not-every? to the Clojure overlay (jolt-1j0 phase 2)
First pure-fn migration batch. Both are leaf fns (no internal Janet callers);
the Clojure defs (not . some / not . every?) match the prior Janet behavior.
Removed the core-* defns and their core-bindings entries.

conformance 218/218 x3, clojure-test-suite 3927, core-bench 2340ms vs 2336ms
baseline (noise).
2026-06-06 23:01:25 -04:00
Yogthos
6282ef0a39 core migration: phase 0 audit + perf baseline (jolt-1j0)
Classify the 421 core-* fns into seed/macro/host/lazy/movable buckets and record
a compile-mode benchmark baseline to catch regressions as fns move from native
Janet to the self-hosted overlay. Worklist in jolt-core/clojure/core/MIGRATION.md.

Finding: after seed + host, the self-hosted compiler uses no clojure.core fns
beyond the existing kernel tier (+ atom/swap!/reset! host primitives), so the
compiler-dep kernel tier is already complete — confirmed by self-host
conformance passing.
2026-06-06 22:54:07 -04:00
Yogthos
14d3cb1de4 test: lock in nil-valued map preservation (jolt-c7h)
Add a maps-spec defspec covering nil values through the reader (literals), the
construction path, and the op surface (assoc/merge/merge-with/into/conj/zipmap/
select-keys/get-in/dissoc/reduce-kv, nil keys). Raise the clojure-test-suite
baseline 3919 -> 3926 to guard the assertions the fix unlocked.
2026-06-06 22:35:26 -04:00
Yogthos
c43f261993 core: nil-correct the rest of the map ops
With nil now preserved in maps, the ops that iterated via (keys (phm-to-struct
m)) or built struct/table results were still dropping nil-valued entries. Route
them through map-entries-of (all [k v] pairs incl nil-valued keys) and build
results via core-assoc / kvs->map / map-assoc1 (phm when a key/value is nil):

  - merge, conj (map onto struct or phm), transient conj!
  - merge-with: also use presence (contains?), not nil-of-value, to decide
    whether to combine — a present nil triggers (f existing v), like Clojure
  - reduce-kv: iterate phm-entries so nil-valued keys are visited
  - select-keys / zipmap: collect entries, build via kvs->map
  - get-in: walk with a sentinel so a present nil value isn't read as missing

conformance 218/218 x3, clojure.data/diff handles nil, suite green,
clojure-test-suite 3926 -> 3927.
2026-06-06 22:29:55 -04:00
Yogthos
c280b10b29 maps: preserve nil keys/values (jolt-c7h)
Janet structs drop nil, so {:b nil}, (assoc {} :b nil) and friends silently
lost the entry — diverging from Clojure, where a nil value is still a present
key. Map construction now uses a phm (which preserves nil) whenever a key or
value is nil; the nil-free case stays a fast struct.

Touched the whole construction path: the reader (a nil-bearing map literal form
is a phm, not a struct), eval-form (evaluates phm map-literal forms, rebuilds
preserving nil), build-map-literal (compiled path), core-assoc (promotes on a
nil key/value), and the host map-form predicates h-map?/h-map-pairs (recognize
a phm form) so the self-hosted analyzer handles {:b nil} literals.

Safe now that the back end densifies phm IR nodes and the IR is nil-field-clean
(prior commits): const-nil nodes go phm but read back correctly, and the common
nodes stay structs, so fib stays compiled (0.08s, no interpret fallback).

Also fixes clojure.set/rename-keys, which deleted the old key via (assoc m old
nil) — a trick that only worked while nil was dropped; now uses dissoc.

conformance 218/218 x3, suite green, clojure-test-suite 3919 -> 3926.
2026-06-06 22:19:35 -04:00
Yogthos
b2ff65972b ir/analyzer: omit absent fields instead of writing nil
def-node carried :meta nil when a def had no metadata, fn-node carried :name
nil for anonymous fns, analyze-arity carried :rest nil for fixed arities, and
empty-env carried :recur nil. These all mean genuine absence, and every
consumer reads an absent key back as nil — so writing the nil is gratuitous.

Dropping them keeps the common nodes (defs, fns, fixed arities) nil-free, so
once user maps preserve nil they stay plain structs instead of becoming phm.
Faithful nil values (a nil const :val) are left as-is; the back end handles
those. No-op today since structs already drop nil: suite, conformance 218/218
x3, fixpoint, fib all unchanged.
2026-06-06 22:05:52 -04:00
Yogthos
22ff737f8b backend: densify phm IR nodes to structs at the emitter boundary
The back end reads IR node fields with raw (node :key) access, which assumes a
Janet struct. Once user maps preserve nil (coming), a node that carries a
nil-valued field — anonymous fn :name, nil const :val, def with no :meta, arity
with no :rest — is built as a phm instead, whose fields live under :buckets and
read back nil. norm-node densifies such a node via phm-to-struct, which drops
exactly those nil fields — precisely what the back end wants, since it already
treats an absent field as nil. Applied where a node first reaches the emitter
(emit entry, fn arities, invoke callee); structs pass through untouched.

No representation change yet, so this is a no-op today: full suite, conformance
218/218 x3, fixpoint, fib all unchanged. Keeps IR access a host-layer concern
so jolt.ir/jolt.analyzer stay portable.
2026-06-06 22:00:21 -04:00
Yogthos
bd76eddb74 phm: preserve nil-valued keys in get/keys/vals
phm-get returned the default for a key present with a nil value; fix it to
presence-check the bucket so a nil value reads back as nil, not the default.
core-keys/core-vals went through phm-to-struct, which drops nil-valued keys —
read phm-entries directly instead so those keys survive.

Isolated phm correctness only; no map-representation change. Battery holds at
3919, conformance 218/218 in all three modes. The earlier -8 came from
assoc's struct->phm promotion, not these accessors.
2026-06-06 21:54:07 -04:00
Yogthos
0878c803d7 compiler: IFn collections held in a var/local are callable (jolt-vh5)
The back end emitted a direct Janet call (f args) for every :var/:local callee,
but Janet calling a pvec-table does get (no integer key -> nil), a keyword does
the wrong thing, etc. — so (let [v [1 2 3]] (v 0)), a keyword/meta-vector in a
binding, all returned nil in compile mode (interpret was fine). Now a direct call
is emitted only when the callee is provably a function: :fn / :host always, and a
:var whose current root is a function; everything else routes through jolt-call,
which dispatches IFn correctly (function fast-path first). User/core fn calls stay
direct, so no hot-path regression — fixpoint stage1==stage2==stage3 holds.
Conformance 218/218 all modes; full suite green. Spec cases added.
2026-06-06 20:03:49 -04:00
Yogthos
d189ee2ded spec: regression cases for with-meta nil and multi-coll map nil elements
Document the two fixed behaviors in the contract spec: (with-meta x nil) clears
metadata, and a multi-collection map keeps nil elements (they're values, not
end-of-seq). The reader grammar (doc/grammar.ebnf) is unchanged — this session
touched semantics/stdlib, not surface syntax.
2026-06-06 19:52:54 -04:00
Yogthos
711a938943 edn: construct set/nested values from reader forms (49/1, only #uuid left)
The reader yields set literals as a form ({:jolt/type :jolt/set ...}); edn now
walks the parsed result and builds actual sets (recursing into maps/vectors;
lists stay lists, never evaluated). Untagged-struct guard so symbols/chars/tagged
literals pass through. 47 -> 49 in the battery; only #uuid (no uuid type) remains.
2026-06-06 19:49:07 -04:00
Yogthos
1bd676c242 edn: port read-string onto the reader; fix special-form ns shadowing
clojure.edn/read-string now delegates to clojure.core/read-string (the
reader-based special form — parses, never evals) via a private helper, with the
opts-map :eof arity and nil/blank-input handling. 43 -> 47 in the stdlib battery.

This required an evaluator fix: special forms were matched by name only, so
clojure.edn/read-string dispatched the core read-string SPECIAL FORM instead of
the var. Now a head qualified to a non-core namespace resolves to its var; only
unqualified or clojure.core-qualified heads are special forms. Conformance
218/218 all modes; full suite green.

Remaining edn gaps (jolt-b7y): set literals read as a set FORM not a constructed
set, and #uuid has no real type.
2026-06-06 19:45:30 -04:00
Yogthos
fc88ecdc74 data: keep mapv (vector result) — list-with-nil destructures wrong positionally 2026-06-06 19:30:20 -04:00
Yogthos
bfb49f23ab core: multi-collection map handles nil elements in a lazy-seq input (jolt-dzh)
The multi-coll map step detected end-of-seq with (nil? (ls-first cur)), which
also fired on a legitimate nil element — so mapping over a lazy-seq containing
nils truncated at the first nil. That's why (doall (map merge a b)) collapsed to
() as a reduce accumulator. Use seq-done? to detect exhaustion and push the value
(which may be nil). doall is unchanged; clojure.data restored to the canonical
(doall (map ...)) form. Conformance 218/218; battery 3919.
2026-06-06 19:28:47 -04:00
Yogthos
e623968b45 stdlib: port clojure.data, rewrite clojure.zip; fix with-meta nil; add battery
clojure.data: ported from the ClojureScript impl (jolt-native equality-partition
fn instead of host-type protocol dispatch). Verified against Clojure's own
data-test (12/13 canonical; the 13th needs nil-valued-map support, jolt-c7h).
Used mapv over the reference's (doall (map …)) — jolt's lazy multi-coll map +
doall don't force as a reduce accumulator (jolt-dzh).

clojure.zip: jolt's was a broken custom reimplementation; replaced with a port of
real clojure.zip (metadata-based locs). Uses (nth loc …) instead of (loc …)
because meta-bearing vectors aren't invocable as fns (jolt-vh5).

core: with-meta now accepts nil metadata (Clojure allows (with-meta x nil)); it
crashed calling keys on nil. This is what unblocked zip's make-node.

New vendored battery (test/clojure-stdlib/, from clojurust's suite with fixtures
corrected to match real Clojure): walk 34, zip 33, data 61 all clean; edn guarded
at 43 (still a stub — jolt-b7y). Conformance 218/218 all modes; full suite green.
2026-06-06 19:02:26 -04:00
Yogthos
c975f5d1c3 perf: generation-guarded cache for multimethod hierarchy dispatch
The expensive multimethod path is the hierarchy fallback: when a dispatch value
isn't a direct method key, mm-fn walks every key with isa? (derive-based
dispatch). That resolution is now cached per dispatch value on the multimethod
var (:jolt/dispatch-cache), cleared in place by defmethod/prefer-method/
remove-method/remove-all-methods so a redefinition can't be served stale. Direct
(get methods dv) hits stay uncached — already a single lookup.

Test cases added to dispatch-cache-test (compile, interpret, aot-core off):
hierarchy hit is cached, a newly-added specific method is seen, removal re-exposes
the fallback. Conformance 218/218; full suite green.
2026-06-06 18:36:11 -04:00
Yogthos
e02ccab4c0 test: bootstrap fixpoint (stage1 == stage2 == stage3 behavioral parity)
Soundness gate for self-hosting (jolt-d0r). stage1 = analyzer built by the Janet
bootstrap; stage2 = analyzer rebuilt by compiling jolt.ir + jolt.analyzer through
stage1 (self-host); stage3 = the same self-rebuild again. A corpus of programs
must produce identical results across all three.

Compared BEHAVIORALLY rather than by emitted code: emitted Janet forms embed live
setter/getter closures (identity varies per compile) and the IR carries
representation-level gensyms and pvec internals, so structural equality fights the
representation. Behavioral parity is the property that actually matters and is
representation-independent.

Also corrects the battery baseline 3920 -> 3919: 3920 was a lucky run; 3919 is the
stable value (the 9 timeout-prone tests can shift the count by one).
2026-06-06 18:27:14 -04:00
Yogthos
3638521e2d core: move juxt/every-pred/some-fn to Clojure
Three pure function combinators into the collection tier, as compositions of
mapv/every?/some/apply. Three more core-* primitives + table entries gone.
Battery 3916 -> 3920 (the canonical defs are more correct than the prior Janet
ones — some-fn returns the matching value, every-pred is properly boolean).
Conformance 218/218 all modes.
2026-06-06 18:09:42 -04:00
Yogthos
57b8ffcb9b core: move frequencies/group-by/not-empty/filterv/max-key/min-key to Clojure
New collection tier (core/20-coll.clj): six pure, eager fns expressed as
compositions of frozen primitives (reduce/assoc/get/conj/filter/vec). Six more
core-* primitives + table entries gone from the Janet kernel.

Two semantics to preserve, caught by the suite:
- max-key/min-key use the canonical multi-arity form (strict </> on the first
  pair, <=/>= in the fold) to reproduce the JVM IEEE-754 NaN behavior; a single
  reduce got the NaN cases wrong.
- frequencies/group-by base on (hash-map), not the {} literal: a struct map
  doesn't canonicalize collection keys across representations ({:a 1} literal vs
  (hash-map :a 1)), a PHM does — same reason the Janet impl used make-phm.

Conformance 218/218 all modes; battery holds 3916.
2026-06-06 18:02:34 -04:00
Yogthos
279c8e176b perf: generation-guarded inline cache for host-value protocol dispatch
Host-value protocol dispatch (a protocol extended to Number/String/… called on a
host value) recomputed the candidate type-tag list and walked the registry — up
to ~15 map lookups plus an array alloc — on every call. It now caches
(most-specific-host-tag, protocol, method) -> fn, guarded by a registry
generation that bumps in register-protocol-method; re-extending a protocol
invalidates the cache. The single-lookup fast paths (deftype protocol dispatch,
direct multimethod dispatch) are untouched — they're already cheap, so no cache
overhead there. The multimethod hierarchy-fallback cache (narrower: only
derive-based dispatch) is deferred.

Test in test/integration/dispatch-cache-test.janet covers correctness under
re-extension in all three modes. Conformance 218/218; battery 3916.
2026-06-06 17:38:44 -04:00
Yogthos
db08cecde8 perf: var-deref fast path; drop the memoized-getter indirection
var-get short-circuits to the root when no dynamic bindings are active (the
common case — the binding stack is only non-empty inside a `binding` block),
skipping the stack walk on every global deref.

The compiled indirect deref now emits (var-get (quote cell)) instead of routing
through a per-var memoized getter closure — one fewer call and no per-var closure
alloc. The cell is quoted so it embeds by reference: a bare table in argument
position is re-evaluated as a constructor, which deep-copies the cell (and any
atom in its root) on every call — that silently broke swap!/atoms/self-ref
lazy-seqs until the quote. Conformance 218/218 all three modes; battery 3916.
2026-06-06 17:31:16 -04:00
Yogthos
a3cc035782 compiler: direct-linking (call-site/unit, Clojure model) + :aot-core? flag
Calls compile direct (target value embedded) iff the compiling unit has
direct-linking on, the target isn't ^:redef/^:dynamic, and it's an
already-defined fn; otherwise indirect (live var deref, redefinable). Direct
emission is guarded to function values only — embedding a non-function would make
Janet evaluate it as code.

:aot-core? (init opt, default true; JOLT_AOT_CORE=0 disables) compiles the core
tiers with direct-linking on so core->core calls are direct. User/REPL code
compiles indirect by default, so redefining functions in the REPL works with no
annotation, exactly like Clojure. :aot-core? false makes core indirect too — the
whole language becomes redefinable. ^:redef / ^:dynamic on a target force it
indirect even inside a direct-linked unit.

Also fixes a pre-existing gap this surfaced: compiled def dropped ALL var
metadata (so ^:dynamic was silently lost under compilation, and ^:redef couldn't
work). def metadata now flows analyzer -> IR -> back end (new form-sym-meta host
fn, def-node :meta, var-setter-meta) and is applied to the cell.

Matrix test in test/integration/direct-linking-test.janet. Conformance 218/218
in all three modes under both :aot-core? true and false; battery holds 3916;
binary rebuilt.
2026-06-06 17:13:54 -04:00