corpus.edn :expected is now the value reference JVM Clojure produces, set by the
new test/conformance/regen-corpus.clj (one JVM process, per-row thread watchdog).
167 rows moved to the JVM value: ratios (/ 1 2)=>1/2, doubles (double 3)=>3.0,
shared-heap concurrency (the future/pmap/agent cases), clojure.math doubles. The
JVM is the spec; jolt is measured against it.
known-divergences.edn shrinks to the rows whose JVM value is an opaque host object
that can't round-trip to source (Java arrays, transients, atoms, beans, proxies,
chunks all print as #object[..@addr]) plus (fn* foo) and a few racy concurrency
cases (:flaky). The zero-janet gate's allowlist becomes the set of host gaps vs the
JVM spec (no Class/array/BigDecimal, :jolt reader, jolt's own printing).
Math/clojure.math sqrt/pow/floor/trig now return doubles (Chez returns exact for
exact args, e.g. (sqrt 9)=>3); JVM always returns a double.
extract-corpus.janet no longer writes corpus.edn unless asked (the test runner
imported it and was silently overwriting the JVM corpus with the spec sources'
placeholder answers). The prelude parity gate is deleted — the zero-janet spine +
certify.clj are the oracles.
zero-janet 2678 (0 new divergences), certify 0 new / 0 stale, emit-test 330/330.
jolt was all-flonum (one :number type, inherited from Janet whose only number
type is a double). The Chez runtime has a full numeric tower, so the zero-Janet
path now carries it = JVM Clojure semantics:
(/ 1 2) => 1/2 (exact Ratio, was 0.5)
(integer? 3) => true (integer? 3.0) => false (float? 3.0) => true
(ratio? (/ 1 2)) => true (= 3 3.0) => false (== 3 3.0) => true
(+ 1 2) => 3 (exact) (/ 1.0 2) => 0.5 (double)
jolt= was already exactness-aware (values.ss) and == is value-equality, so
=/== match the JVM split. The reader preserves exactness (integer literals exact,
a/b ratios exact rationals, decimals/exponents flonums); backend_scheme emit-const
renders exact ints/ratios and flonums faithfully; the value-position arithmetic,
count, int, compare, bit ops, parseLong, string .length/.indexOf, range,
timestamps, and array bytes return exact integers (= JVM int/long) instead of
coercing to flonum. double/parseDouble/clojure.math floor|ceil|signum stay double.
Only the zero-Janet path carries the tower (the Janet reader loses exactness into
a double before emit). The prelude/all-flonum path is unaffected for compiled code;
the runtime reader is shared, so a couple of all-flonum reader assertions become
value (==) assertions. ~16 numeric corpus cases now give the JVM tower value vs the
Janet-era :expected and are allowlisted as tower divergences (Chez == reference
JVM) pending the corpus flip to JVM (jolt-ecz0). No BigDecimal type (1M).
Re-minted. zero-janet 2682 (floor 2698->2682, the reclassified tower cases), 0 new
divergences; fixpoint 10/10, bootstrap 6/6, spine 35/35, cli 49/49; Janet gate 155
files 0 failed.
Two Chez reader bugs, both JVM-parity gaps:
inc'/+'/foo' (trailing apostrophe) were mis-read as a symbol followed by a
quote macro, because the reader treated ' as a terminator. In Clojure ' is a
NON-terminating macro char (constituent after the first char). Since the seed
is minted on Chez, (def inc' inc) became (def inc 'inc), clobbering inc's var
cell with its own symbol -- so (var-get (var inc)) returned the symbol, not the
fn. Drop ' from the token terminator set; a leading ' still quotes.
^bytes [b] / ^String [x y] return-type hints: the Chez reader lowered ^meta on
a collection to a (with-meta vec meta) form, but emitted a QUALIFIED
clojure.core/with-meta while the Janet reader emits a bare with-meta -- so the
fn/defn macros' unwrap logic (matching the bare head) slipped past it and choked
on a non-vector arglist. Emit bare with-meta to match Janet, and unwrap a
(with-meta <vec> _) arglist in analyze-fn as a backstop.
Re-minted the seed. zero-janet 2699, prelude 2652, Janet gate 155/0, fixpoint
10/10, bootstrap 6/6, all 0 new divergences.
realized? threw 'not supported on' for a jolt-lazyseq record (the overlay
reads :jolt/type); add a jolt-lazyseq? arm to the post-prelude wrapper
reading the record's own realized? flag.
conj! 1-arity (conj! coll) is the transducer-completion arity and returns
coll as-is on the JVM, no transient check — we threw 'not a transient'.
Both gates: zero-janet 2696->2698, prelude 2649->2652, 0 new divergences.
Audit of the janet.*/jolt.interop/STM corpus cases vs Chez equivalents: the Janet
FFI-bridge cases (janet/string, janet/type, janet.math/sqrt, janet.string/ascii-
upper) test functionality already covered by PORTABLE corpus cases (str 29, type 9,
sqrt 3, upper-case 9), so they're safe to delete in Phase 5. Two needed a Chez
equivalent so they pass instead of being lost:
- clojure.lang.LockingTransaction/isRunning -> false (no STM on jolt).
- line-seq: the corpus case used janet/spit setup but was the SOLE line-seq
coverage (0 other cases). Ported janet/spit->spit, and added a native Chez
line-seq (drain a jhost io/reader + split on newline; no trailing empty line)
that delegates a Janet map-reader to the overlay version.
zero-Janet 2694->2696, prelude floor 2648; self-host + Janet gate + JVM cert green.
jolt-cf1q.7 jolt-0obq
hc-list? required cseq-list? (a reader-built list), so a form built at runtime via
concat/map/cons — a lazy cseq with list?=#f — was rejected as "unsupported form".
In Clojure any seq is a valid call form, so accept any cseq. Lights up macros that
build their expansion with concat/list and (eval seq-form).
zero-Janet 2692->2694, 0 new divergences; self-host + Janet gate + JVM cert green.
jolt-cf1q.7
Date/time (inst-time.ss): java.util.Date / java.sql.Timestamp ctors accept ms or
another date value (ms-of) -> a jinst; java.text.SimpleDateFormat (pattern + .format
via the existing format-ms UTC engine; .setTimeZone accepted); java.util.TimeZone/
getTimeZone. instance? answers Date true / Timestamp false for a jinst (a Date is
not a Timestamp on the JVM).
clojure.edn/read over a reader (io.ss + post-prelude): the overlay edn.clj's
drain-reader is janet/type-coupled, so Chez drains the jhost StringReader/
PushbackReader to a string and reads the first EDN form. Unblocks jolt-uicd.
Native Chez throughout (no vendoring): Chez date arithmetic + string ports. zero-Janet
2688->2692, 0 new divergences; self-host + Janet gate + JVM cert green.
jolt-cf1q.7 jolt-dcmm jolt-7t3l jolt-uicd
deftype binds the type name as a VAR (the make-deftype-ctor closure), but (P. 5)
lowers to (host-new "P"), which only checked class-ctors-tbl -> "No constructor".
Fall back to resolving the class name as a var in the current ns / clojure.core
and invoking it — so (P. args) constructs the same jrec as the ->P factory, and
protocol method dispatch (.m / .-field) over it works.
zero-Janet 2685->2688 (no-constructor 5->2); prelude floor bumped to 2641 (the
delay batch's run). Self-host + Janet gates + JVM cert green.
jolt-cf1q.7
Add a thread-safe delay type (concurrency.ss): make-delay wraps a thunk; deref/
force run it once under a lock and cache (JVM delays are thread-safe + memoized).
delay? and realized?-on-a-delay are native; the overlay's `delay` macro
(-> make-delay) and `force` (-> deref) now work. realized? wrapper (post-prelude)
and the deref chain gain a delay arm. Removed the np-delay? stub from
natives-parity.ss (the real type lives in concurrency.ss).
Seed unchanged (no re-mint). zero-Janet 2673->2685, 0 new divergences; Janet gate
+ JVM cert green.
jolt-cf1q.7
The zero-Janet runner wrapped each case as (= EXPECTED ACTUAL) and checked the
result was true. That nests ACTUAL's top-level (do ...), so a case like
(do (defmacro m ...) (m 1)) can't use the macro it just defined — the analyzer
punted defmacro -> "uncompilable" (35 cases).
Match certify.clj's eval-isolated instead: carry EXPECTED and ACTUAL as separate
sources and evaluate ACTUAL as its own top-level program (jolt-compile-eval unrolls
the top-level do, so a macro defined earlier is usable later), then compare to
EXPECTED with =. Evaluating ACTUAL from SOURCE (not (eval (quote A))) preserves the
reader's map-literal source order, so the eval-order cases still pass.
eval-corpus-zero-janet / program-corpus-zero-janet now use a 3-field TSV
(label/expected/actual); run-corpus-zero-janet's per-case debug path evals both
sides too. Only run-corpus-zero-janet uses these (the prelude gate is untouched).
zero-Janet 2642->2673 (analyzer "uncompilable" 35->4); 1 new allowlisted divergence
(`{:a ~x :b ~y} syntax-quote map construction doesn't preserve source eval order —
pmap is unordered). Janet gate + JVM cert green.
jolt-cf1q.7
host/chez/natives-array.ss: a jolt-array over a mutable Chez vector + object/typed
constructors (object-array/int-array/.../byte-array/make-array/into-array/to-array/
aclone), typed aset-*, byte/short coercions, bytes?/bytes/ints/..., and eager
chunk-buffer/chunk-* (Jolt doesn't chunk). count/nth/seq/get and jolt.host/ref-put!
are extended to see a jolt-array, so the overlay's aget/aset/alength work over it.
Numbers it produces are flonums (jolt's rep) so exactness-aware = holds. char-array
stays in io.ss (a char-SEQ that io/reader/str/slurp consume).
natives-parity.ss adds: __reader-features / -set!, reader-conditional, re-matcher,
delay? (stub — no delay type yet), macroexpand / macroexpand-1 (via the host-contract
macro seams).
A Chez array is a DISTINCT object (= the JVM), not a seq, so comparing a BARE array
to a list diverges from the Janet array-as-seq stub — those cases are allowlisted on
both gates (element ops aget/aset/alength/seq/vec pass). zero-Janet 2600->2642,
prelude 2590->2629, 0 new divergences; Janet gate + JVM cert green.
jolt-cf1q.7
Native Chez shims for clojure.core fns that live in the Janet seed but had none
on the zero-Janet spine (they resolved to nil -> "not a fn"):
- host/chez/natives-parity.ss: hash / hash-combine / hash-ordered-coll /
hash-unordered-coll (24-bit masked like core_extra), transient?, rseq (vectors +
sorted colls), cat (transducer).
- jolt-invoke now dispatches a TRANSIENT vec/map/set as a fn (callable on the JVM):
((transient [10 20 30]) 1) -> 20.
- ns.ss: ns-resolve, ns-imports, remove-ns, intern, alias, ns-unalias, refer,
ns-refers, refer-clojure, alter-meta!, reset-meta!, and a real ns-aliases (was a
stub returning {}).
- runtime (require ...)/(use ...) now register :as/:refer into the Chez ns tables
(was a no-op). The Chez analyzer already pre-registers at analyze time, but when
the JANET analyzer compiled the form (prelude path) the Chez tables stayed empty,
so ns-aliases/ns-resolve over an alias diverged — this fixes both paths.
Seed unchanged (overlay doesn't reference these at mint time). zero-Janet corpus
2567->2600, prelude 2557->2590, 0 new divergences on either; Janet gate + JVM cert
green. Filed jolt-vgrp for the pre-existing var-get-of-scalar-native-op quirk.
jolt-cf1q.7
Replace the Janet synchronous agent shim (agent = atom, send applies inline) with
JVM-style async agents: send/send-off enqueue an action and a single worker thread
per agent applies them in order; deref reads the current (maybe not-yet-updated)
state without blocking; await blocks until the queue drains. A validator rejection
or a thrown action puts the agent in an error state (agent-error) and halts the
queue; restart-agent clears it. send and send-off share one serialized worker (a
superset of the JVM's fixed/cached pool split). Native versions re-asserted in
post-prelude over the overlay; await/restart-agent are new.
Corpus: the two "send/send-off applies" cases do (send a f) (deref a) with no
await, so they now read state before the action runs — diverging like the JVM
(the suite was literally "synchronous shim"). Allowlisted on both gates; floors
-2 (zero-Janet 2569->2567, prelude 2559->2557). cli-test covers async agents via
await (ordered 100-send dispatch, error capture) — 49/49. Janet gate + JVM cert
green; 0 new divergences on either corpus.
jolt-byjr
future/future-call run the body on a native thread (fork-thread) over the SAME
heap — JVM semantics, not Janet's isolated-heap snapshot. deref blocks on a
mutex+condition latch; timed (deref f ms val) uses an absolute deadline.
promise is a real blocking promise (deref parks until deliver), replacing the
Janet non-blocking atom shim. future?/future-done?/future-cancelled?/future-cancel
/realized? are native (the overlay versions read Janet map keys); re-asserted in
post-prelude over the overlay. pmap/pcalls/pvalues (overlay, over future) light
up for free.
Thread-safety this forces:
- atoms get a per-atom mutex; swap!/swap-vals! are a JVM-style CAS loop (f runs
outside the lock, so a watch/validator can deref the same atom); reset!/
compare-and-set! are atomic.
- the dynamic binding stack becomes a Chez thread-parameter, so each future/thread
has its own; Chez inherits it at fork, giving binding conveyance (the shim also
installs an explicit snapshot).
- Thread/sleep really sleeps now (a worker sleeping doesn't block the parent).
Re-minted the seed: future-call now resolves at compile time, so pmap compiles to
a var-deref instead of the host-static-call fallback that crashed. image.ss
unchanged.
Corpus: the 2 snapshot cases now match the JVM (shared) not Janet (isolated) —
allowlisted on both Chez gates; the two racy future-cancel cases allowlisted;
"promise undelivered" (blocks on JVM/Chez, profile :bucket :timeout) skipped like
:throws. Zero-Janet corpus 2544 -> 2569, 0 new divergences, floor raised. Full
Janet gate + JVM cert green.
jolt-byjr
The compiler image is already resident at runtime on the Chez spine, so eval
and load-string are just wiring: make them clojure.core functions instead of
analyzer special forms.
- eval / load-string are now functions, not special forms. Dropped "eval" from
the host-contract special-symbol lists so it resolves as an ordinary var, and
def-var! both in compile-eval.ss. eval takes an already-read form (e.g. from
quote/list) and compiles+evals it in the current ns; load-string reads every
form from a source string and evals each, returning the last.
- Runtime defmacro: jolt-compile-eval-form intercepts a (defmacro ...) form
before analysis, defs the expander fn + mark-macro!s the var, exactly as
emit-image.ss does at build time. The two helpers (macro-form? / defmacro->fn)
move to compile-eval.ss and emit-image.ss reuses them.
- Top-level (do ...) is now unrolled form-by-form, like Clojure, so a defmacro
or def in an earlier subform is visible (macro flag set / var interned) before
a later subform is analyzed. This is what makes multi-form -e with a macro work.
Seed is byte-identical (no source references eval), so no re-mint; bootstrap-test
still passes. Zero-Janet corpus 2534 -> 2544 (eval/load-string cases now run),
0 new divergences; floor raised. Prelude corpus, JVM cert, full Janet gate green.
jolt-r8ku
The corpus certifier (test/conformance) flagged four cases where jolt's
hand-written :expected matched a real defect rather than Clojure. Fixed in the
jolt-core overlay, corrected the spec :expected, re-certified against JVM Clojure:
- ex-message: returns nil for a non-throwable (dropped the lenient string branch);
still returns the message for ex-info. (jolt-l8e8)
- munge: preserves the argument's type — a symbol munges to a symbol, not a string.
(jolt-hc35)
- print: (print nil) emits "nil", not "" (top-level nil guard; str yields "").
(jolt-pqio)
- bounded-count: uses the counted? fast path (full count), else counts up to n via
seq — was (min n (count coll)), wrong for counted colls. Added an uncounted-coll
spec case. (jolt-2507)
Removed the 4 :bug entries from known-divergences.edn (now certified), regenerated
corpus + profile, re-minted the Chez bootstrap seed (clojure.core changed). Gates:
Janet 155/0, JVM certify clean, both Chez corpus gates 2534 (floors raised),
bootstrap 6/6, fixpoint intact.
The corpus only ever saw test/spec/*-spec.janet; the 355 hand-written cases in
test/integration/conformance-test.janet (inline Janet, the lazy-seq / IFn /
destructuring / transducer essentials) were invisible to it and to any non-Janet
runtime. extract-corpus.janet now also pulls that (def cases ...) vector, deduped
by :actual, organized into 41 'conformance / <section>' suites recovered from the
file's ### headers. Corpus 2658 -> 2919 rows (+261 unique).
JVM certification: only 1 new divergence ((/ 2) => 0.5 vs 1/2, the all-double
numeric model) — classified. Chez gates: +1 known host gap (instance? Atom, atom
class identity, Phase 4) allowlisted in both runners; parity rose 2295 -> 2533 on
both, floors raised. Janet gate 155 files 0 failed; certifier green (0 new/stale).
Deferred: 41 non-literal core-async spec rows ((a "src") async-harness wrapper)
need harness context the corpus format doesn't carry — left for inc3.
(require '[clojure.string :refer [blank?]]) then an unqualified blank? now
resolves. chez-register-spec! registers :refer names (in addition to :as) into a
refer table; hc-resolve-cell's unqualified branch consults it before clojure.core.
Zero-Janet corpus parity 2293 -> 2295 = the Janet-hosted oracle's exact pass
count. The self-hosted Chez compiler (read -> analyze -> emit -> eval, no Janet)
now compiles every corpus case the Janet-hosted compiler does, with 0
divergences. Remaining failures are shared runtime breadth (host interop,
futures, runtime eval) deferred to Phase 4 / jolt-r8ku. Floor 2295.
The batched zero-Janet runner wrote one case per TSV line, so a multi-line
source (e.g. a ;comment\n inside a map literal) split the line and the case was
truncated -> a false "apply non-procedure" crash. Escape \n/\t/\\ when writing
the cases file and unescape in the runner before eval.
Zero-Janet corpus parity 2288 -> 2293 (the 5 comment-in-map cases), 0
divergences — now within 2 of the Janet-hosted oracle (2295). Floor 2293.
Reader gaps the Chez-hosted analyzer hit where the Janet reader didn't:
- ##Inf / ##-Inf / ##NaN symbolic literals (## dispatch -> flonum).
- #(...) anonymous fn shorthand -> (fn* [p__N#] body), with % / %N / %& and the
max-positional arity rule; scans + rewrites list/vector/set/map bodies.
- #?(...) reader conditional: feature set {:jolt :default}, first matching clause
wins. #?@ splicing not yet supported (one niche case allowlisted).
- (ns name (:require [a :as x])) — the require pre-scan now also reads aliases
from an ns form's :require/:use clauses, not just bare (require ...).
Zero-Janet corpus parity 2240 -> 2288, 0 divergences (2 now-reachable cases
allowlisted: str of Infinity inside a collection — same as the prelude gate —
and #?@ splice). spine-test 35/35; prelude parity 2295 unchanged, 0 new
divergences.
(require '[clojure.string :as s]) then s/foo crashed "Unknown class s": the
Chez analyzer resolved a qualified symbol's ns literally, and there was no
alias table (ns.ss jolt-ns-aliases was a stub, require a no-op). Add an alias
table + chez-register-spec! (parses [ns :as a]) in ns.ss; hc-resolve-cell now
resolves a qualified ns through it; compile-eval pre-registers a form's
require/use :as aliases before analysis (analysis precedes the runtime require,
so the runtime require staying a no-op is fine). The batched gate runner clears
the alias table between cases.
Zero-Janet corpus parity 2159 -> 2240, 0 divergences. spine-test 35/35.
Point the Chez-HOSTED analyzer at the full parity corpus (read -> analyze ->
emit -> eval, all on Chez, no Janet) and close the divergences so the
self-hosted compiler is faithful: 0 divergences, 2159/2494 pass.
Keystone: the on-Chez emitter ran with prelude-mode off, so every call to a
non-native clojure.core fn tripped the "unsupported stdlib fn" out-of-subset
guard. The zero-Janet spine always has the full prelude loaded, so turn
prelude mode on in compile-eval.ss (22% -> 84% pass on a sample).
Faithfulness fixes (each was the Chez host/reader diverging from the Janet
analyzer; fixed in the keeper, not the seed):
- emit-const read a char's codepoint via (get v :ch) — the Janet rep; on Chez a
char is native. Route through a new form-char-code host-contract fn (41 cases).
- next over a lazy seq returned the empty-list terminator (truthy), not nil, so
butlast and other (if (next s) ...) loops ran one step too far — broke
some->/some->>/cond->>.
- reader: radix literals (2r1010/16rFF/36rZ), #^ deprecated metadata, ^meta on
collections (lowers to a runtime with-meta form like the Janet reader),
map-literal source order (values eval left-to-right), and nested syntax-quote
over a literal collapses at read time.
- keyword "a/b" splits into ns/name like the seed (destructure {:keys [x/y]}).
- form-syntax-quote-lower implemented on Chez (was a throwing stub).
7 divergences allowlisted: the same print-method-multimethod / host-class set
the prelude gate defers. 328 crashes remain = shared runtime breadth (host
interop, missing core fns, eval/load-string) deferred to Phase 4 / jolt-r8ku,
not compiler faithfulness.
Gate + speedup: test/chez/run-corpus-zero-janet.janet (floor 2159). Its batched
runner (driver/eval-corpus-zero-janet) runs every case in ONE chez process —
load the runtime once, guard + reset the user namespace per case — instead of a
fresh process per case: 1379s -> 1.6s.
spine-test 35/35; Janet gate 151/0; prelude parity 2295/2494 unchanged, 0 new
divergences.
