Collection fns: JVM-faithful return types + laziness (#219)

A type-aware audit (~190 collection expressions vs reference Clojure) found four
divergences the corpus missed — value-equality (= [0 1] '(0 1)) hides type and
laziness differences. Fixed, with type-predicate + over-infinite corpus rows that
pin them.

- partition-all [n coll] built vector chunks; JVM chunks are seqs. (The [n step
  coll] arity was already correct, as is the partition-all transducer, whose
  chunks are vectors in JVM too.) Now builds seq chunks.
- replace always returned a vector (mapv) and was eager; JVM is type-preserving —
  a vector maps to a vector, any other seqable to a lazy seq.
- sequence eagerly realized its source (into-xform), so (first (sequence (map inc)
  (range))) hung. Rewrote as a transformer iterator: pull one input at a time,
  buffer the step outputs, emit lazily, run the completion to flush a stateful
  xform. eduction builds on it (lazy, no longer an eager vector).
- mapcat and (apply concat coll-of-colls) hung over an infinite source because
  jolt-apply seq->lists the trailing arg and mapcat seq->lists the map result.
  Added lazy-concat-seq (lazily flatten a seq of colls); mapcat uses it directly,
  and apply special-cases concat (its result is lazy) to route through it.

Docs: a cross-cutting return-type + laziness contract in docs/spec/09-core-library;
SPEC.md notes that = masks type/laziness so they need predicate / over-infinite
rows. EBNF is reader syntax only — unaffected.

Seed change (partition-all/replace/eduction are clojure.core overlay) -> re-mint;
selfhost holds. make test + shakesmoke + buildsmoke green, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
This commit is contained in:
Dmitri Sotnikov 2026-06-26 03:01:36 +00:00 committed by GitHub
parent 8180c85393
commit f3084f8043
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 810 additions and 679 deletions

View file

@ -101,7 +101,9 @@
(define (jolt-mapcat f . colls)
(if (null? colls)
(td-mapcat f)
(apply jolt-concat (seq->list (apply jolt-map f colls)))))
;; lazily concat the per-element results — no seq->list, so mapcat over an
;; infinite source stays lazy.
(lazy-concat-seq (apply jolt-map f colls))))
;; take-while / drop-while: 1-arg -> transducer; 2-arg -> a seq over the coll.
(define (take-while-seq pred s)

View file

@ -38,13 +38,54 @@
(res (reduce-seq xf init (jolt-seq coll))))
(jolt-invoke xf res)))))
;; (sequence coll) -> a seq; (sequence xform coll) -> coll transformed by xform.
;; Materialized eagerly through into-xform then seq'd (corpus inputs are finite; a
;; fully-lazy pull is future work). Honors reduced via into-xform/reduce-seq.
;; (sequence coll) -> a seq; (sequence xform coll) -> a LAZY seq of coll transformed
;; by xform. A transformer iterator (mirrors clojure.core's TransformerIterator):
;; pull one input at a time through (xform rf), where rf buffers each emitted value;
;; emit the buffer lazily, pulling more input only when it drains. So an infinite or
;; expensive source is consumed incrementally — (first (sequence (map inc) (range)))
;; returns at once. Honors `reduced` (stop pulling) and runs the 1-arg completion to
;; flush a stateful xform (partition-all / dedupe / a trailing partition).
(define (sequence-xf xform coll)
(let* ((buf (box '())) ; emitted values for the current step, reversed
(rf (case-lambda
(() jolt-nil)
((acc) acc)
((acc x) (set-box! buf (cons x (unbox buf))) acc)))
(xrf (jolt-invoke xform rf)))
;; advance the source until buf holds output or the input is drained+completed.
(define (fill src acc completed)
(let loop ((src src) (acc acc) (completed completed))
(cond
((pair? (unbox buf)) (values src acc completed))
(completed (values src acc #t))
((jolt-reduced? acc)
(jolt-invoke xrf (jolt-reduced-val acc)) ; completion may flush
(loop src (jolt-reduced-val acc) #t))
(else
(let ((s (jolt-seq src)))
(if (jolt-nil? s)
(begin (jolt-invoke xrf acc) (loop src acc #t)) ; complete -> flush
(loop (seq-more s) (jolt-invoke xrf acc (seq-first s)) completed)))))))
;; Resolve the next chunk now (one fill pulls just enough input to emit or to
;; exhaust), so the result is a real cseq | empty — `empty` is jolt-empty-list
;; at the top (so an empty result still prints "()") and jolt-nil inside a tail
;; (the cseq terminator). The TAILS stay lazy, so an infinite source is fine.
(define (step src acc completed empty)
(let-values (((src2 acc2 comp2) (fill src acc completed)))
(let ((out (reverse (unbox buf))))
(set-box! buf '())
(if (null? out)
empty
(let build ((o out))
(if (null? (cdr o))
(cseq-lazy (car o) (lambda () (step src2 acc2 comp2 jolt-nil)))
(cseq-lazy (car o) (lambda () (build (cdr o))))))))))
(step coll jolt-nil #f jolt-empty-list)))
(define jolt-sequence
(case-lambda
((coll) (jolt-seq coll))
((xform coll) (jolt-seq (into-xform (jolt-vector) xform coll)))))
((xform coll) (sequence-xf xform coll))))
(def-var! "clojure.core" "transduce" jolt-transduce)
(def-var! "clojure.core" "sequence" jolt-sequence)

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View file

@ -269,10 +269,25 @@
((null? (cdr colls)) (jolt-seq (car colls)))
(else (concat2 (jolt-seq (car colls)) (lambda () (apply jolt-concat (cdr colls)))))))
;; (apply f a b ... coll): spread the trailing seqable into the call.
;; Lazily concatenate a (possibly infinite) SEQ of colls — what (apply concat ss)
;; means, but without realizing ss. Pulls one coll at a time, concatenating it with
;; a lazy tail, so mapcat / (apply concat …) over an infinite source stays lazy.
(define (lazy-concat-seq ss)
(let ((s (jolt-seq ss)))
(if (jolt-nil? s)
jolt-empty-list
(jolt-concat (seq-first s)
(jolt-make-lazy-seq (lambda () (lazy-concat-seq (seq-more s))))))))
;; (apply f a b ... coll): spread the trailing seqable into the call. concat is
;; special-cased: it produces a LAZY result, so spreading an infinite tail through
;; a Scheme variadic (which must realize it) would hang — route to lazy-concat-seq,
;; prepending any fixed leading colls.
(define (jolt-apply f . args)
(let* ((r (reverse args)) (spread (seq->list (jolt-seq (car r)))) (fixed (reverse (cdr r))))
(apply jolt-invoke f (append fixed spread))))
(let* ((r (reverse args)) (tail (car r)) (fixed (reverse (cdr r))))
(if (eq? f jolt-concat)
(lazy-concat-seq (fold-right jolt-cons (jolt-seq tail) fixed))
(apply jolt-invoke f (append fixed (seq->list (jolt-seq tail)))))))
;; ============================================================================
;; numeric predicates / identity — usable in fn AND value position (map/filter).