simon/jolt
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chunk-first: pull the trie leaf instead of flattening the whole vector

Browse source 
A pvec is a 32-way trie, but na-chunk-first built each block by calling
pvec-v on the full backing vector — materializing all n elements to a
flat Scheme vector — then copying the 32-wide window out of it. That made
chunk-first O(n), so walking a vector chunk-by-chunk (Clojure's real
chunked map/filter fast path) was O(n^2): a ported chunked map over 500K
elements took 39s, superlinear to ~700s at 2M.

na-chunk-size equals pv-width and blocks are 32-aligned, so a block is
exactly one trie leaf — pv-chunk-for hands it back in O(log n). Copy that
leaf directly; fall back to per-index reads for the rare window that
crosses a leaf boundary. Chunked map is now linear, ~133x faster at 500K
(293ms) and within ~2.3x of the native seq loop, which makes a
clojure-in-clojure seq tier viable.

Corpus rows pin chunk-first window contents + chunk-rest boundaries
against JVM; fixed a stale 'always false' chunked-seq? label.
This commit is contained in:
Yogthos 2026-06-28 01:56:26 -04:00
parent 6940b2c7f5
commit 6d441e2d00
2 changed files with 22 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

7
test/chez/corpus.edn
View file

@ -1640,7 +1640,7 @@
{:suite "predicates / tagged-value" :label "tagged-literal? yes" :expected "true" :actual "(tagged-literal? (tagged-literal (quote inst) \"2020\"))"}
{:suite "predicates / tagged-value" :label "tagged-literal? no" :expected "false" :actual "(tagged-literal? 1)"}
{:suite "predicates / tagged-value" :label "reader-conditional? no" :expected "false" :actual "(reader-conditional? 1)"}
{:suite "predicates / tagged-value" :label "chunked-seq? always false" :expected "true" :actual "(chunked-seq? (seq [1 2 3]))"}
{:suite "predicates / tagged-value" :label "chunked-seq? true for a vector seq" :expected "true" :actual "(chunked-seq? (seq [1 2 3]))"}
{:suite "predicates / equality & identity" :label "= same" :expected "true" :actual "(= 1 1)"}
{:suite "predicates / equality & identity" :label "= vectors" :expected "true" :actual "(= [1 2] [1 2])"}
{:suite "predicates / equality & identity" :label "= vector & list" :expected "true" :actual "(= [1 2] (list 1 2))"}
@ -2709,6 +2709,11 @@
{:suite "untested / chunk family (eager equivalents) + cat" :label "halt-when" :expected "4" :actual "(transduce (halt-when even?) conj [] [1 3 4 5])"}
{:suite "untested / chunk family (eager equivalents) + cat" :label "chunk-next exhausted" :expected "nil" :actual "(let [cb (chunk-buffer 2)] (chunk-append cb 1) (chunk-next (chunk-cons (chunk cb) nil)))"}
{:suite "untested / chunk family (eager equivalents) + cat" :label "chunk-rest seqable" :expected "[]" :actual "(let [cb (chunk-buffer 2)] (chunk-append cb 1) (vec (chunk-rest (chunk-cons (chunk cb) nil))))"}
{:suite "chunked-seq / over a vector" :label "chunk-first is a 32-element block" :expected "32" :actual "(count (chunk-first (seq (vec (range 100)))))"}
{:suite "chunked-seq / over a vector" :label "chunked-seq? true for vector seq" :expected "true" :actual "(chunked-seq? (seq (vec (range 100))))"}
{:suite "chunked-seq / over a vector" :label "chunked-seq? false for a list seq" :expected "false" :actual "(chunked-seq? (seq (list 1 2 3)))"}
{:suite "chunked-seq / over a vector" :label "chunk-first contents + chunk-rest boundary" :expected "[32 0 31 32]" :actual "(let [s (seq (vec (range 50))) c (chunk-first s)] [(count c) (nth c 0) (nth c 31) (first (chunk-rest s))])"}
{:suite "chunked-seq / over a vector" :label "chunk-first window past the first block" :expected "[32 33 34 35 36 37 38 39]" :actual "(vec (chunk-first (chunk-rest (seq (vec (range 40))))))"}
{:suite "untested / JVM-shape stubs (documented jolt behavior)" :label "class number" :expected "java.lang.Long" :actual "(class 1)"}
{:suite "untested / JVM-shape stubs (documented jolt behavior)" :label "class string" :expected "java.lang.String" :actual "(class \"s\")"}
{:suite "untested / JVM-shape stubs (documented jolt behavior)" :label "class keyword" :expected "clojure.lang.Keyword" :actual "(class :k)"}