feat: persistent singly-linked lists with O(1) conj/cons prepend

Round 3 of the persistent-collections work. Lists were immutable Janet arrays,
so conj/cons-prepend was an O(n) copy (O(n^2) to build a list) — a large perf
gap vs Clojure's PersistentList.

Add src/jolt/plist.janet: an immutable cons-cell list (first/rest/count), same
algorithm as Clojure/CLJS/jank PersistentList. conj/cons onto a list now creates
an O(1) node that shares the existing list as its tail (no copy), with a cached
O(1) count. Repeated conj is O(n) total instead of O(n^2).

Hooked plist through first/rest/next/seq/count/peek/pop/nth/empty/empty?, the
predicates (list?/seq?/coll?/sequential?), realize-for-iteration, =, coll->cells
(concat/lazy), both printers, destructuring, and instance? tags. (list ...) and
quoted lists stay arrays; only conj/cons introduce plist nodes, so the surface
and risk stay small.

Verified: reduce-conj of 200k elements runs in ~0.4s (was effectively O(n^2)).
conformance 206/206, features 78/78 (+7 list regressions), jank 120 (+1).
This commit is contained in:
Yogthos 2026-06-04 19:20:27 -04:00
parent e43f39bc36
commit a0c9696900
9 changed files with 142 additions and 22 deletions

View file

@ -75,7 +75,7 @@ Jolt targets Clojure semantics but runs on Janet, not the JVM. The notable diver
- **Host platform.** No JVM and no Java interop — `import`, `gen-class`, `proxy` of Java classes, and `java.*` are unavailable. `instance?` recognizes a small set of built-in types (`clojure.lang.Atom`, `Number`, `String`, …).
- **Numbers.** Janet integers and doubles only — no bignums, ratios, or `BigDecimal`. `(/ 1 3)` is `0.3333…`, large products lose precision, and there are no auto-promoting `+'`/`*'`. `quot`/`rem`/`mod` follow Clojure's sign rules. `bigint`, `rational?`, and `class` are not provided.
- **Collections.** By default Jolt uses immutable persistent data structures: vectors are 32-way branching tries (structural-sharing persistent vectors with O(log₃₂ n) `conj`/`assoc`/`nth`), lists are immutable, and maps/sets are persistent hash structures. Value equality and sequence operations are Clojure-compatible, but hash-map/hash-set iteration order is unspecified and differs from Clojure — use `sorted-map`/`sorted-set` when order matters.
- **Collections.** By default Jolt uses immutable persistent data structures: vectors are 32-way branching tries (structural-sharing persistent vectors with O(log₃₂ n) `conj`/`assoc`/`nth`), lists are persistent singly-linked cons cells (O(1) `conj`/`cons` prepend with structural sharing), and maps/sets are persistent hash structures. Value equality and sequence operations are Clojure-compatible, but hash-map/hash-set iteration order is unspecified and differs from Clojure — use `sorted-map`/`sorted-set` when order matters.
- **Mutable build mode.** Jolt can be compiled to use fast Janet-native *mutable* collections instead, via a build-time flag: `JOLT_MUTABLE=1 jpm build` (default `jpm build` is immutable). In mutable mode vectors and lists share one mutable array representation (so `conj` mutates in place and appends, and `vector?`/`list?` no longer distinguish them) — a performance/looseness trade-off. The default immutable build has full Clojure value semantics.
- **Concurrency / STM.** Single-threaded. No refs, `dosync`, agents, or `send`; `locking` evaluates its body without real locking. Atoms, volatiles, and delays are supported.
- **Regex.** Compiled to Janet's PEG engine (Janet has no regex). Supported: capturing groups (`[whole g1 …]`), greedy and lazy quantifiers with backtracking, `(?:…)`, lookahead `(?=…)`/`(?!…)`, alternation, anchors `^ $ \b \B`, character classes, and the `(?i)` flag. Not supported: lookbehind, backreferences (`\1`), and named groups (`(?<name>…)`).

View file

@ -3,6 +3,7 @@
(use ./types)
(use ./pv)
(use ./plist)
(use ./reader)
(use ./evaluator)
(use ./core)
@ -17,6 +18,7 @@
[x]
(cond
(pvec? x) (tuple ;(map normalize-pvecs (pv->array x)))
(plist? x) (tuple ;(map normalize-pvecs (pl->array x)))
(tuple? x) (tuple ;(map normalize-pvecs x))
(array? x) (tuple ;(map normalize-pvecs x))
x))

View file

@ -6,6 +6,7 @@
(use ./regex)
(use ./config)
(use ./pv)
(use ./plist)
# ------------------------------------------------------------
# Vector representation helpers
@ -45,6 +46,7 @@
loop on infinite lazy-seqs. Terminates on the empty cell, not on nil."
(cond
(pvec? c) (pv->array c)
(plist? c) (pl->array c)
(set? c) (phs-seq c)
(lazy-seq? c)
(do
@ -80,8 +82,8 @@
(defn core-symbol? [x] (and (struct? x) (= :symbol (x :jolt/type))))
(defn core-vector? [x] (jvec? x))
(defn core-map? [x] (or (phm? x) (struct? x) (if (and (table? x) (get x :jolt/deftype)) true false)))
(defn core-seq? [x] (or (array? x) (tuple? x) (pvec? x)))
(defn core-coll? [x] (or (array? x) (tuple? x) (pvec? x) (struct? x) (phm? x) (set? x) (lazy-seq? x)))
(defn core-seq? [x] (or (array? x) (tuple? x) (pvec? x) (plist? x) (lazy-seq? x)))
(defn core-coll? [x] (or (array? x) (tuple? x) (pvec? x) (plist? x) (struct? x) (phm? x) (set? x) (lazy-seq? x)))
(defn core-true? [x] (= true x))
(defn core-false? [x] (= false x))
@ -95,16 +97,17 @@
(defn core-integer? [x] (and (number? x) (= x (math/floor x))))
(defn core-boolean? [x] (or (= x true) (= x false)))
(defn core-list? [x] (and (array? x) (not (get x :jolt/type))))
(defn core-list? [x] (or (plist? x) (and (array? x) (not (get x :jolt/type)))))
(defn core-empty? [coll]
(if (nil? coll) true
(if (set? coll) (= 0 (coll :cnt))
(if (phm? coll) (= 0 (coll :cnt))
(if (pvec? coll) (= 0 (pv-count coll))
(if (plist? coll) (pl-empty? coll)
(if (lazy-seq? coll) (nil? (ls-first coll))
(if (struct? coll) (= 0 (length (keys coll)))
(= 0 (length coll)))))))))
(= 0 (length coll))))))))))
(defn core-every? [pred coll]
(var result true)
@ -161,6 +164,7 @@
(cond
(lazy-seq? x) (realize-for-iteration x)
(pvec? x) (pv->array x)
(plist? x) (pl->array x)
(tuple? x) x
(array? x) x
nil))
@ -233,20 +237,18 @@
(defn core-conj [coll & xs]
(if (pvec? coll)
(do (var result coll) (each x xs (set result (pv-conj result x))) result)
(if (plist? coll)
# list: prepend, O(1) per element via structural sharing
(do (var result coll) (each x xs (set result (pl-cons x result))) result)
(if (tuple? coll)
(tuple/slice (tuple ;(array/concat (array/slice coll) xs)))
(if (array? coll)
(if mutable?
# mutable mode: arrays are vectors — append in place
(do (each x xs (array/push coll x)) coll)
# immutable mode: arrays are lists — prepend onto a copy
(do
(var result (array/slice coll))
(var i 0)
(while (< i (length xs))
(set result (array/insert result 0 (xs i)))
(++ i))
result))
# immutable mode: arrays are lists — prepend onto a persistent cons node,
# sharing the original array as the tail (O(1) per element, no copy)
(do (var result coll) (each x xs (set result (pl-cons x result))) result))
(if (set? coll)
(apply phs-conj coll xs)
(if (phm? coll)
@ -265,7 +267,7 @@
(let [pair (xs i)]
(set result (merge result {(vnth pair 0) (vnth pair 1)})))
(++ i))
result)))))))
result))))))))
(defn core-assoc [m & kvs]
(cond
@ -385,6 +387,7 @@
(core-sorted-set? coll) (length (coll :items))
(lazy-seq? coll) (ls-count coll)
(pvec? coll) (pv-count coll)
(plist? coll) (pl-count coll)
(set? coll) (coll :cnt)
(phm? coll) (coll :cnt)
(and (table? coll) (get coll :jolt/deftype)) (- (length (keys coll)) 1)
@ -396,6 +399,7 @@
(core-sorted-set? coll) (let [i (coll :items)] (if (empty? i) nil (in i 0)))
(lazy-seq? coll) (ls-first coll)
(pvec? coll) (if (= 0 (pv-count coll)) nil (pv-nth coll 0))
(plist? coll) (if (pl-empty? coll) nil (pl-first coll))
(or (nil? coll) (= 0 (length coll))) nil
(string? coll) (make-char (in coll 0))
(in coll 0)))
@ -403,6 +407,7 @@
(defn core-rest [coll]
(cond
(lazy-seq? coll) (ls-rest coll)
(plist? coll) (pl-rest coll)
(pvec? coll) (let [a (pv->array coll)] (if (<= (length a) 1) @[] (array/slice a 1)))
(or (nil? coll) (= 0 (length coll))) @[]
(string? coll) (tuple ;(map make-char (string/bytes (string/slice coll 1))))
@ -414,8 +419,12 @@
(if (= 0 (length r)) nil r)))
(defn core-cons [x coll]
"Returns a lazy-seq compatible cons cell [first, rest-thunk]."
@[x (fn [] coll)])
"Prepend x onto coll. For concrete collections this is an O(1) persistent cons
node; for lazy-seqs it stays a lazy cell so laziness is preserved."
(cond
(lazy-seq? coll) @[x (fn [] coll)]
(or (nil? coll) (plist? coll) (array? coll) (tuple? coll)) (pl-cons x coll)
(pl-cons x (realize-for-iteration coll))))
(defn core-seq [coll]
(cond
@ -424,6 +433,7 @@
(or (nil? coll) (and (or (tuple? coll) (array? coll)) (= 0 (length coll)))) nil
(lazy-seq? coll) (ls-seq coll)
(pvec? coll) (if (= 0 (pv-count coll)) nil (tuple ;(pv->array coll)))
(plist? coll) (if (pl-empty? coll) nil (tuple ;(pl->array coll)))
(set? coll) (phs-seq coll)
(phm? coll) (tuple ;(phm-entries coll))
(tuple? coll) (tuple/slice coll)
@ -782,6 +792,7 @@
If the result is already a cell (array of [val, function]), return it directly."
(if (nil? c) nil
(if (pvec? c) (coll->cells (pv->array c))
(if (plist? c) (coll->cells (pl->array c))
(if (function? c)
(let [r (c)]
(if (and (indexed? r) (= 2 (length r)) (function? (in r 1)))
@ -799,7 +810,7 @@
(if (tuple? c) (tuple/slice c 1) (array/slice c 1))
nil)]
@[f (fn [] (coll->cells rest))])))
nil))))))
nil)))))))
(defn core-concat [& colls]
"Truly lazy concatenation. `step` returns a 0-arg thunk that is only forced
@ -875,6 +886,10 @@
(defn core-nth
"Return the nth element of a sequential collection."
[coll idx &opt default]
(if (plist? coll)
(let [a (pl->array coll)]
(if (and (>= idx 0) (< idx (length a))) (in a idx)
(if (nil? default) (error (string "Index " idx " out of bounds, length: " (length a))) default)))
(if (pvec? coll)
(if (and (>= idx 0) (< idx (pv-count coll)))
(pv-nth coll idx)
@ -896,7 +911,7 @@
(if (string? c) (make-char (in c idx)) (in c idx))
(if (nil? default)
(error (string "Index " idx " out of bounds, length: " (length c)))
default))))))
default)))))))
(defn core-sort
"(sort coll) or (sort comparator coll). Comparator may return a boolean or a
@ -1052,6 +1067,7 @@
(cond
(nil? coll) nil
(lazy-seq? coll) (ls-first coll)
(plist? coll) (if (pl-empty? coll) nil (pl-first coll)) # list: first
(pvec? coll) (if (= 0 (pv-count coll)) nil (pv-nth coll (- (pv-count coll) 1))) # vector: last
(= 0 (length coll)) nil
(tuple? coll) (in coll (- (length coll) 1)) # vector: last
@ -1061,6 +1077,7 @@
(defn core-pop [coll]
(cond
(nil? coll) nil
(plist? coll) (pl-rest coll) # list: drop first
(pvec? coll) (pv-pop coll) # vector: drop last
(tuple? coll) (tuple/slice coll 0 (- (length coll) 1)) # vector: drop last
(array? coll) (array/slice coll 1) # list: rest
@ -1294,6 +1311,7 @@
(set? v) (pr-render-seq buf (phs-seq v) "#{" "}")
(phm? v) (pr-render-pairs buf (phm-entries v))
(pvec? v) (pr-render-seq buf (pv->array v) "[" "]")
(plist? v) (pr-render-seq buf (pl->array v) "(" ")")
(and (table? v) (get v :jolt/deftype)) (buffer/push-string buf (string v))
(tuple? v) (pr-render-seq buf v "[" "]")
# mutable mode: arrays are vectors -> print with [] (else lists -> ())
@ -2541,6 +2559,7 @@
(cond
(phm? coll) (make-phm)
(set? coll) (make-phs)
(plist? coll) EMPTY-PLIST
(pvec? coll) (make-vec @[])
(struct? coll) (struct)
(tuple? coll) (make-vec @[])
@ -2575,7 +2594,7 @@
(each p preds (each x xs (when (and (nil? hit) (truthy? (p x))) (set hit (p x)))))
hit))
(defn core-sequential? [x] (or (tuple? x) (array? x) (pvec? x) (lazy-seq? x)))
(defn core-sequential? [x] (or (tuple? x) (array? x) (pvec? x) (plist? x) (lazy-seq? x)))
(defn core-associative? [x] (or (phm? x) (struct? x) (tuple? x) (array? x) (pvec? x) (and (table? x) (not (set? x)))))
(defn core-ifn? [x]
(or (function? x) (cfunction? x) (keyword? x) (phm? x) (set? x) (tuple? x) (array? x) (pvec? x)

View file

@ -4,6 +4,7 @@
(use ./types)
(use ./phm)
(use ./pv)
(use ./plist)
(use ./config)
(use ./reader)
(use ./regex)
@ -374,6 +375,7 @@
"Realize a lazy-seq to an array for positional destructuring; pass others through."
[val]
(if (pvec? val) (pv->array val)
(if (plist? val) (pl->array val)
(if (lazy-seq? val)
(do
(var items @[]) (var cur val) (var go true)
@ -385,7 +387,7 @@
(let [rt (in cell 1)]
(if (nil? rt) (set go false) (set cur (make-lazy-seq rt))))))))
items)
val)))
val))))
(defn- d-get
"Look up key k in a map-like value (phm/struct/table/nil)."
@ -510,6 +512,7 @@
(keyword? obj) ["Keyword" "Object"]
(and (struct? obj) (= :jolt/char (get obj :jolt/type))) ["Character" "Object"]
(and (struct? obj) (= :symbol (get obj :jolt/type))) ["Symbol" "Object"]
(plist? obj) ["PersistentList" "IPersistentList" "IPersistentCollection" "ISeq" "Object"]
(or (tuple? obj) (array? obj) (pvec? obj)) ["PersistentVector" "IPersistentVector" "IPersistentCollection" "ISeq" "Object"]
(or (function? obj) (cfunction? obj)) ["IFn" "Fn" "Object"]
(nil? obj) ["nil" "Object"]

View file

@ -5,6 +5,7 @@
(use ./types)
(use ./phm)
(use ./pv)
(use ./plist)
(use ./config)
(use ./reader)
@ -36,6 +37,17 @@
(++ i)))
(push-str buf "]"))
(plist? v)
(do
(push-str buf "(")
(let [a (pl->array v) n (length a)]
(var i 0)
(while (< i n)
(write-value (in a i) buf)
(when (< (+ i 1) n) (push-str buf " "))
(++ i)))
(push-str buf ")"))
(tuple? v)
(do
(push-str buf "[")

73
src/jolt/plist.janet Normal file
View file

@ -0,0 +1,73 @@
# Persistent list — an immutable singly-linked cons cell, modeled on Clojure's
# PersistentList. The whole point is O(1) prepend (conj/cons): a new node simply
# points at the existing list as its tail, sharing all of it, so building a list
# with repeated conj is O(n) total instead of O(n²) array copies.
#
# A node is:
# @{:jolt/type :jolt/plist
# :first x head element
# :rest r tail: another plist, or a Janet array/tuple (a list that
# was conj'd onto), or nil for the empty tail
# :count n} element count, or nil when unknown (cons onto a lazy tail)
#
# `:rest` may be a plain array/tuple so `(conj some-list x)` needn't copy the
# original list — the node just references it. pl->array materializes the chain.
(defn plist? [x]
(and (table? x) (= :jolt/plist (get x :jolt/type))))
(defn- counted
"Count of a tail value if known in O(1), else nil."
[r]
(cond
(nil? r) 0
(plist? r) (get r :count)
(or (array? r) (tuple? r) (string? r) (buffer? r)) (length r)
nil))
(defn pl-cons
"Prepend x onto tail r (a plist / array / tuple / nil). O(1)."
[x r]
(def c (counted r))
@{:jolt/type :jolt/plist :first x :rest r :count (if c (+ c 1) nil)})
(def EMPTY-PLIST @{:jolt/type :jolt/plist :first nil :rest nil :count 0})
(defn pl-empty? [p] (= 0 (get p :count)))
(defn pl-first [p] (get p :first))
(defn pl->array
"Materialize the cons chain to a fresh Janet array."
[p]
(def out @[])
(var cur p)
(while (plist? cur)
(if (= 0 (get cur :count))
(set cur nil)
(do (array/push out (get cur :first)) (set cur (get cur :rest)))))
# cur is now a non-plist tail (array/tuple) or nil
(when (and (not (nil? cur)) (or (array? cur) (tuple? cur)))
(each x cur (array/push out x)))
out)
(defn pl-count [p]
(def c (get p :count))
(if (nil? c) (length (pl->array p)) c))
(defn pl-rest
"The tail as a seqable (array). Returns an empty array for a one-element list."
[p]
(if (or (= 0 (get p :count)) (nil? (get p :rest)))
@[]
(let [r (get p :rest)]
(if (plist? r) r r))))
(defn pl-from-indexed
"Build a plist from a Janet array/tuple, preserving order. O(n)."
[xs]
(var p EMPTY-PLIST)
(var i (- (length xs) 1))
(while (>= i 0)
(set p (pl-cons (in xs i) p))
(-- i))
p)

View file

@ -1,14 +1,16 @@
(use ../src/jolt/types)
(use ../src/jolt/pv)
(use ../src/jolt/plist)
(use ../src/jolt/reader)
(use ../src/jolt/evaluator)
(use ../src/jolt/core)
# Normalize jolt collection results to Janet tuples so Janet-level deep=/= can
# compare against tuple literals regardless of the (pvec) representation.
# compare against tuple literals regardless of the (pvec/plist) representation.
(defn norm [x]
(cond
(pvec? x) (tuple ;(map norm (pv->array x)))
(plist? x) (tuple ;(map norm (pl->array x)))
(tuple? x) (tuple ;(map norm x))
(array? x) (tuple ;(map norm x))
x))

View file

@ -118,6 +118,15 @@
["for :let" "(quote (1 4 9))" "(for [x [1 2 3] :let [sq (* x x)]] sq)"]
["for :while" "(quote (0 1 2))" "(for [x (range 10) :while (< x 3)] x)"]
### 13b. Persistent lists — O(1) conj-prepend, immutable, value semantics
["list conj prepends" "(quote (0 1 2 3))" "(conj (list 1 2 3) 0)"]
["list conj multi" "(quote (:c :b :a))" "(conj (quote ()) :a :b :c)"]
["list immutable" "true" "(let [l (list 1 2 3) l2 (conj l 9)] (and (= l (quote (1 2 3))) (= l2 (quote (9 1 2 3)))))"]
["list? after conj" "true" "(list? (conj (list 1 2) 0))"]
["list = vector elts" "true" "(= (quote (1 2 3)) [1 2 3])"]
["reduce conj list" "(quote (2 1 0))" "(reduce conj (list) (range 3))"]
["cons onto list" "(quote (0 1 2 3))" "(cons 0 (list 1 2 3))"]
### 14. Janet interop
["interop method" "\"v=41\"" "(. {:value 41 :describe (fn [self] (str \"v=\" (:value self)))} describe)"]
["interop field" "41" "(.-value {:value 41})"]

View file

@ -12,7 +12,7 @@
# Baseline: the number of pass-tests Jolt currently handles. Raise this as Jolt
# gains features so regressions (a previously-passing test breaking) are caught.
(def baseline 119)
(def baseline 120)
# Tests that loop forever under Jolt's eager evaluation (skipped to avoid hangs;
# tracked as known gaps — variadic-recur arity selection and var-quote calls).