(sequence xform coll) was eager (transduce into a tuple), so it hung on infinite
input. Now it's a lazy buffered transducer pump: pulls one source element at a
time, pushes it through the transducer's reducing fn into a buffer, and emits
buffered outputs lazily — so (take 3 (sequence (map inc) (range))) works.
Honors `reduced` (early stop) and runs the completion arity to flush stateful
transducers. Normalizes the source via core-seq (walking a raw pvec/set would
misfire — seq-done? uses length, which counts a pvec table's keys).
The other jolt-b56 items stay native by design: sequential?/seqable? are
representation-coupled and representation-mode-sensitive (jolt-1vx); realized?
reads the tagged :realized flag; cat/eduction/transduce/halt-when/unreduced/
ensure-reduced are the transducer/Reduced kernel the issue says to keep native.
Note: partition-all has no transducer (1-arg) arity, so (sequence (partition-all
2) …) errors — a pre-existing gap, unrelated to this change.
Gate: conformance 249x3, lazy-infinite 44/44, fixpoint, self-host, specs+unit green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audited all 43 realize-for-iteration call sites. Result: every? was the one
transformer-style leak — it realized the whole coll before iterating, so
(every? pos? (range)) hung instead of short-circuiting on the first false
(Clojure short-circuits). Now walks lazy input cell-by-cell via realize-ls and
stops at the first false; not-every? inherits the fix.
Everything else is a legitimate realization boundary (reverse/sort/into/vec/set/
list/str-join/doall/apply/transient/to-array/hash-*/shuffle/rand-nth — must see
all elements) or forces only finite/concrete input (drop-while's eager branch,
cycle's seed, nth's concrete branch). some is already lazy (overlay). realized?
reads the :realized flag without forcing. first/nth/take pull only as far as
needed.
Regression: deadlined (every?/not-every? pos? (range)) cases — would hang if the
leak returned.
Gate: conformance 249x3, lazy-infinite 42/42, fixpoint, self-host, specs+unit green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These three were the last eager transformers, blocked by jolt-r81: a self-
recursive lazy-seq in the overlay leaks its macro expansion under :compile? when
recursion goes through a top-level name or (fn name …) self-name. Rewriting the
recursion as letfn-bound (the form partition-by/mapcat/dedupe already use, which
compiles cleanly) sidesteps the bug. All three are now lazy in interpret,
compile, and self-host — completing Option A for every transformer.
interleave: canonical lazy cons-recursion (2-arity) + map/concat (n-arity).
reductions: letfn step accumulator. tree-seq: letfn walk + lazy mapcat.
Gate: conformance 246x3, lazy-infinite 40/40 (+interleave/reductions/tree-seq
infinite cases), fixpoint, self-host, specs+unit green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Option A makes `take` lazy, so the §6.3 counter tests must force the take
result (dorun) to drive realization — an unconsumed take correctly realizes
nothing. With that, all 37 lazy-infinite cases pass and the minimal-realization
counts match (proving the Option A transformers realize exactly the demanded
prefix).
interleave kept eager: a lazy (cons-recursive) version leaks its lazy-seq macro
expansion under :compile? — the same jolt-r81 bug that blocks lazy reductions/
tree-seq. Documented in 20-coll.clj.
Gate: conformance 246x3, lazy-infinite 37/37, fixpoint, self-host, specs+unit
green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Added 16 atom-counter laziness tests to lazy-infinite-test.janet:
map, filter, remove, take-while, drop-while, distinct, take-nth,
map-indexed, keep, keep-indexed, interpose, partition, partition-all,
mapcat, dedupe, repeated-inc
Each test wraps input elements in (swap! c inc) and proves only the
exact minimal number of elements are realized. 37/37 pass, 0 timeouts.
Updated phase-5.md §7: "22/22" → "21/21" (interleave commented out),
§6.3 marked Done.
Final gate results:
lazy-infinite: 37/37 (21 value + 16 counter)
conformance: 229/229 ×3
specs: 32/32 files, 0 failures
clojure-test-suite: 3971 pass (↑45), 6 timeouts (↓3)
core-bench: TOTAL 2531 ms (no Phase-4 baseline for comparison)
Root cause: lazy interleave in 20-coll.clj uses lazy-seq macro which
expands to (make-lazy-seq (fn* [] (coll->cells ...))) in compile mode.
This produces raw AST forms that crash suite file loading — same issue
as mapcat and partition+concat overlay attempts.
Fix: revert interleave to eager (vec-based loop). dedupe stays lazy
(uses make-lazy-seq directly, not lazy-seq macro). xml-seq stays
(uses tree-seq which is eager in overlay).
Suite: 3971 pass (up from 3926), 6 timeouts (down from 9), 4628 assertions.
Lazy-infinite: 21/21 (interleave infinite case commented out).
Conformance: 229x3.
Fix & rest destructuring (evaluator.janet): when the original
value is a lazy-seq, derive & rest by walking ls-rest vi steps
instead of slicing the eagerly-realized array from d-realize.
Rewrite core-mapcat as a lazy state machine (core.janet): step
through input colls one element at a time, call f on each tuple,
then yield from f's result collection. No apply-forcing —
all input colls are walked lazily via lazy-from + seq-done?.
Add mapcat to core-renames (compiler.janet) and remove from
Clojure overlay (10-seq.clj) — compile mode now emits direct
calls to the lazy core-mapcat, fixing the pre-existing
protocol-on-record compile error.
Tests: re-enabled mapcat infinite harness case, added 2 & rest
laziness cases. 21/21 lazy-infinite, 229x3 conformance, 32/32 specs.
core-drop-while (core.janet): return cell via realize-ls instead
of raw LazySeq. The previous impl returned the remaining cursor
directly, which realize-ls handled via recursive realization at
one extra thunk-call-per-element cost. Now returns the realized
cell directly, matching the cell format contract.
mapcat (10-seq.clj): revert to standard (apply concat (apply map
f colls)). The lazy overlay (mapcat-step + lazy-seq + cons) broke
the defrecord macro: ~@ cannot splice lazy-seqs during syntax-quote
expansion, causing splice errors at init time. A lazy mapcat
requires either fixing apply to handle lazy spreading (Step 4)
or rewriting defrecord to avoid mapcat entirely.
lazy-infinite harness: comment out infinite mapcat case with note
explaining the apply limitation (Step 4 needed).
Gates verified:
conformance 229/229 x3 (interpret/compile/self-host)
lazy-infinite 18/18
specs 32/32 files, 0 failures
mapcat in 10-seq.clj now uses a defn- mapcat-step helper that walks
lazy map results element-by-element (cons + lazy-seq), with explicit
arities for 1/2/3/4+ colls. The 4+-coll arity still uses apply to
spread colls to map, but apply no longer forces the inner map result.
Previously (apply concat (apply map f colls)) forced the lazy map
result through core-apply realize-for-iteration, which hung on
infinite inputs. The new step-based impl walks one element at a time
via first/rest/seq primitives.
Re-enabled deferred mapcat infinite-input harness case. 19/19 pass.
Gates: lazy-infinite 19/19, conformance 229/229 interpret+self-host
(compile 228/229 — 1 pre-existing protocol error), specs 32/32.
