Chez Phase 1 (increment 3a): persistent collections on the Chez RT

Broaden the Scheme back end past the numeric/functional subset to vectors,
maps, and sets. host/chez/collections.ss adds a copy-on-write persistent
vector and a bitmap HAMT (the structure 0c measured self-hostable) backing
both maps and sets, keyed by jolt-hash and compared by jolt=. emit.janet
emits :vector/:map/:set literals to the rt constructors and lowers the leaf
ops (conj/get/nth/count/assoc/dissoc/contains?/empty?/peek/pop) via the
native-ops path, with a per-op arity gate.

Also: keyword/map literals in fn position lower to jolt-get ((:k m), ({:k v} k));
arity-1 comparisons emit the vacuous jolt truth (Scheme < rejects a non-number
even at arity 1); count returns a flonum and vector indices coerce from flonum,
both consequences of the all-double number model; values.ss = / hash and the
rt printer learn collections (maps/sets render in HAMT order, so the probe
compares unordered values via =, not printed form).

Subset parity 182 -> 433/436 compiled cases (2219/2655 out of subset), 0 new
divergences. The 3 known divergences are dynamic IFn dispatch (a keyword/vector
held in a local, called as a fn) — deferred to the IFn/protocol increment and
allowlisted in the probe. emit-test 31/31, full run-tests green (125 files).
This commit is contained in:
Yogthos 2026-06-17 14:33:57 -04:00
parent 9bbcc07c8f
commit 5c5d2cd1fc
7 changed files with 473 additions and 18 deletions

View file

@ -48,9 +48,17 @@ compile-time signal) and are counted "out of subset", not as divergences.
JOLT_CHEZ_CORPUS=1 janet test/chez/run-corpus-chez.janet
Baseline (2026-06-17): **182/182 compiled cases pass, 0 divergences**; 2473/2655
out of subset (await core on Chez). It's a slow report (a Chez subprocess per
case), so it's gated behind `JOLT_CHEZ_CORPUS` out of the default suite, like the
benches. `test/chez/emit-test.janet` is the fast Phase-1 unit gate (real
analyzer → Chez parity for fib/mandelbrot + regressions); both skip cleanly when
Baseline after inc 3a (persistent collections, jolt-wgbz): **433/436 compiled
cases pass**, 3 known divergences, 0 NEW; 2219/2655 out of subset (await the seq
tier + core on Chez). The 3 known divergences are dynamic IFn dispatch — a
keyword/vector held in a LOCAL and called as a fn (`(let [k :a] (k m))`); the
STATIC literal forms (`(:a m)`, `({:a 1} :a)`) are supported. They're
allowlisted in the probe; it exits non-zero on a NEW divergence.
(Prior, inc 2 baseline: 182/182 compiled, 0 divergences, 2473 out of subset.)
It's a slow report (a Chez subprocess per case), so it's gated behind
`JOLT_CHEZ_CORPUS` out of the default suite, like the benches.
`test/chez/emit-test.janet` is the fast Phase-1 unit gate (real analyzer → Chez
parity for fib/mandelbrot + collections + regressions); both skip cleanly when
`chez` isn't on PATH.

View file

@ -25,6 +25,17 @@
(def oracle-ctx (api/init {:compile? true}))
(defn oracle [src] (string (api/load-string oracle-ctx src)))
# Canonical CLI oracle (the run-corpus gate's boundary): collection values don't
# round-trip through (string value) — they need jolt's real `-e` printer. Take
# the last non-empty stdout line, exactly like run-corpus.janet.
(defn cli-oracle [src]
(def proc (os/spawn ["build/jolt" "-e" src] :p {:out :pipe :err :pipe}))
(def out (ev/read (proc :out) 0x100000))
(ev/read (proc :err) 0x100000)
(os/proc-wait proc)
(def lines (filter (fn [l] (not (empty? l))) (string/split "\n" (string/trim (if out (string out) "")))))
(if (empty? lines) "" (last lines)))
(def ctx (d/make-ctx))
# 1) constant-folded arithmetic: (+ 1 2) -> the analyzer folds to const 3.
@ -73,6 +84,43 @@
(let [[code out err] (d/run-on-chez ctx src)]
(ok label (and (= code 0) (= out (oracle src))) (string "chez=" out " janet=" (oracle src) " | " err))))
# 3c) persistent collections (jolt-wgbz): vector/map/set literals + leaf ops.
# Maps/sets print in jolt's INTERNAL hash order, which a Scheme HAMT won't
# reproduce — so unordered cases are checked via `(= ...)` (prints true/false,
# exactly how the run-corpus gate compares them), and only ORDERED vectors are
# compared by printed form. Parity is still vs the Janet oracle in both shapes.
(each src [# ordered: direct printed-form parity
"[1 2 3]"
"(conj [1 2] 3)"
"(count [1 2 3])"
"(nth [10 20 30] 1)"
"(get [10 20 30] 0)"
"(peek [1 2 3])"
"(pop [1 2 3])"
# unordered / boolean: equality-wrapped, order-independent
"(= {:a 1 :b 2} {:b 2 :a 1})"
"(= {:a 1 :b 2} (assoc {:a 1} :b 2))"
"(= 1 (get {:a 1} :a))"
"(= 2 (count {:a 1 :b 2}))"
"(= 99 (get {:a 1} :z 99))"
"(= {:a 1} (dissoc {:a 1 :b 2} :b))"
"(= #{1 2 3} (conj #{1 2} 3))"
"(= #{1 2} (conj #{1 2} 2))"
"(contains? #{1 2} 1)"
"(contains? #{1 2} 9)"
"(contains? {:a 1} :a)"
"(empty? [])"
"(empty? [1])"
"(empty? {})"
"(= [1 2] [1 2])"
"(= [1 2] [1 3])"
"(= #{1 2} #{2 1})"
"(= {1 2} {1 3})"]
(let [[code out err] (d/run-on-chez ctx src)
want (cli-oracle src)]
(ok (string "coll: " src) (and (= code 0) (= out want))
(string "chez=" out " janet=" want " | " err))))
# 4) perf signal: emitted fib(30) in-Scheme timing (excludes Chez startup), to
# track against the spike ceiling (hand-Scheme fib ~5ms). Informational — the
# jolt-truthy? wrapper (~3x) and flonum modeling are known Phase-4 levers.

View file

@ -27,9 +27,21 @@
(seq [i :range [0 (length corpus) stride]] (in corpus i)))
corpus))
# Known subset divergences: cases that compile but need a feature beyond the
# current increment. Dynamic IFn dispatch — a keyword/vector held in a LOCAL or
# var then called as a fn ((let [k :a] (k m))) — is runtime dispatch on the
# invoke mechanism, deferred to the IFn/protocol increment. The STATIC literal
# forms ((:a m), ({:a 1} :a)) ARE supported. Allowlisted by label; the gate fails
# only on a NEW divergence.
(def known-divergences
{"param holding a keyword (IFn leftover)" true
"vector-in-local as fn" true
"keyword-in-local as fn" true})
(def ctx (d/make-ctx))
(var compiled 0) (var pass 0) (var out-of-subset 0)
(def diverged @[])
(def known-hit @[])
(each row cases
(def {:expected e :actual a :label l} row)
@ -44,15 +56,17 @@
(++ out-of-subset)
(let [[code out] res]
(++ compiled)
(defn record-div [m] (if (known-divergences l) (array/push known-hit l) (array/push diverged [l m])))
(cond
(not= code 0) (array/push diverged [l (string "exit " code)])
(not= code 0) (record-div (string "exit " code))
(= out "true") (++ pass)
(array/push diverged [l (string "got " out)])))))))
(record-div (string "got " out))))))))
(printf "\nChez subset parity: %d/%d compiled cases pass (%d/%d corpus out of subset)"
pass compiled out-of-subset (length cases))
(printf "\nChez subset parity: %d/%d compiled cases pass (%d/%d out of subset, %d known divergences)"
pass compiled out-of-subset (length cases) (length known-hit))
(when (> (length diverged) 0)
(printf "%d divergence(s) within the compiled subset:" (length diverged))
(printf "%d NEW divergence(s) within the compiled subset:" (length diverged))
(each [l m] (slice diverged 0 (min 25 (length diverged)))
(printf " [%s] %s" l m)))
(flush)
(os/exit (if (> (length diverged) 0) 1 0))