jolt/test/conformance/SPEC.md

The jolt conformance spec

This directory defines jolt's behavior as a host-neutral, executable language specification: a data file of cases, certified against reference Clojure, with a feature profile that lets any runtime declare a conformance level. The goal is to make hosting jolt on a new runtime (and proving it correct) a mechanical exercise: read one data file, run each case, compare, report.

The artifacts

File	Role	Generated by
test/chez/corpus.edn	The spec. ~2900 cases of {:suite :label :expected :actual}, :expected sourced from reference JVM Clojure.	test/conformance/regen-corpus.clj
test/conformance/profile.edn	Per-case feature classification — which non-portable cases need which host capability.	certify.clj --profile
test/conformance/known-divergences.edn	The few rows whose JVM value is an opaque host object that can't round-trip to readable source (Java arrays/transients/atoms/beans/proxies print as #object[..@addr]), so the corpus keeps jolt's value.	regen-corpus.clj leftovers, hand-checked
test/conformance/regen-corpus.clj	Sources every :expected from reference JVM Clojure in one process.	—
test/conformance/certify.clj	Certifies :expected against reference JVM Clojure; gates on new/stale divergences; emits the profile.	—

corpus.edn is JVM-sourced: regen-corpus.clj evaluates each case's :actual on reference JVM Clojure and writes the JVM value as :expected. corpus.edn is the canonical, frozen contract: it is what every runtime consumes, what certify.clj certifies, and where new cases are authored directly.

Row schema

{:suite       "numbers / arithmetic"   ; grouping; "<suite> :: <label>" is the case id
 :label       "integer add"            ; unique within a suite
 :actual      "(+ 1 2)"                 ; Clojure source to evaluate
 :expected    "3"                       ; Clojure source whose value it must equal,
                                        ; or the keyword :throws
 :portability :common}                  ; :common = portable Clojure any dialect
                                        ; must satisfy; :jvm = exercises host
                                        ; interop (java.*/clojure.lang.* classes,
                                        ; dot forms, ctors, statics, arrays,
                                        ; proxy/bean) — skip on a non-JVM-shaped
                                        ; dialect

• [:suite :label] is the canonical, unique case id (the generator disambiguates duplicate labels with (N)).
• Comparison is value-equality (=), never string/printed-form — so map/set iteration order never matters.
• Because comparison is =, a type or laziness difference is invisible to a plain value row: (= [0 1] '(0 1)) is true, so a fn returning a vector where Clojure returns a seq still passes. Pin those explicitly — container/element type with a predicate row ((seq? …), (vector? …), (every? seq? …)), and laziness with a (take n (… (range))) row over an infinite source (it hangs, not just diverges, if the fn isn't lazy). The seq / lazy over infinite suite does both.
• :expected :throws asserts evaluating :actual raises.

The oracle: reference JVM Clojure

Historically every :expected was hand-written. certify.clj removes that weakness: it evaluates every :actual (and :expected) on JVM Clojure in a fresh user namespace and checks jolt's :expected against what real Clojure produces. Of ~2740 vanilla-certifiable rows, >2730 match reference Clojure exactly. The rest are classified (see below) — none are silently wrong.

clojure -M test/conformance/certify.clj                                  # gate
clojure -M test/conformance/certify.clj test/chez/corpus.edn --edn r.edn # + report
clojure -M test/conformance/certify.clj test/chez/corpus.edn --profile test/conformance/profile.edn

The gate fails only on a new (unclassified) divergence or a stale allowlist entry; flaky timing-dependent cases (future-cancel) are tolerated.

Conformance levels & the feature profile

Not every case is portable: some assume a host capability jolt has on one runtime but not another (Java interop, real threads, BigDecimal). profile.edn classifies each non-portable case by the feature it requires. Cases not in the profile are portable — they must pass on any faithful Clojure.

A runtime's conformance level = portable cases + the feature families it implements. Current profile (≈2735 portable, ≈167 non-portable):

Feature	Meaning
:numerics/double-only	all-double numeric model — no Ratio/BigDecimal/float; (/ 1 2) ⇒ 0.5
:concurrency/snapshot	isolated-heap futures/agents/pmap — captured atoms are snapshotted, not shared
:host/jvm-interop	Java classes / instance? on host classes / proxy / bean / definterface
:host/arrays	Java arrays (into-array, int-array, …)
:async/core-async	clojure.core.async channels/go
:runtime/eval	runtime eval / load-string
:reader/jolt	jolt reader features (#?(:jolt …)) + syntax-quote literal collapse
:printer/jolt	jolt's rendering of transients/atoms/print-method overrides
:strictness/jolt	intentionally stricter (throws on odd assoc! args, etc.)
:impl/representation	representation detail (e.g. syntax-quote yields a list?, not a Cons)
:bug	a known defect (tracked bead) — not a host difference

Seq semantics

Values alone don't pin laziness — an eager map and a lazy map return the same elements. The spec certifies seq semantics by reducing them to values with a side-effect counter, so the corpus catches a laziness regression the value comparison would miss.

Laziness (certified — jolt matches JVM). The whole producer family (map/filter/remove/take/drop/concat/take-while/drop-while/mapcat/ partition/partition-all/partition-by/keep/keep-indexed/map-indexed/ distinct/interpose/interleave/take-nth/reductions/tree-seq/replace) is lazy at construction: building over a side-effecting source realizes zero elements (lazy / family is lazy at construction). Realization order is left-to-right, take/nth/drop realize exactly as far as demanded, a lazy seq memoizes (realize-once across walks), and next realizes head + one lookahead while rest realizes only the head (lazy / realization order & count, lazy / realization is memoized, lazy / realization timing). A lazy result is clojure.lang.LazySeq.

Accepted divergences. jolt is a simpler, finer-grained superset of JVM seq behavior; two classes diverge by representation, never by value, and are allowlisted in known-divergences.edn:

• :seq-type-model (seq-type-model / … suite, jolt-aei7) — jolt reifies every seq as PersistentList (eager) or LazySeq (deferred). JVM has a specialized class per producer (Cons, Iterate, LongRange, Repeat, Cycle, PersistentVector$ChunkedSeq, StringSeq, KeySeq/ValSeq, RSeq, ArraySeq, SubVector), so (class …) differs. instance? clojure.lang.ISeq/Sequential and all values/laziness are correct.
• :chunking-model (chunking-model / … suite, jolt-mm6v) — jolt seqs are unchunked: forcing one element realizes one, where JVM realizes a ~32-element chunk; mapcat/dedupe realize 0 at construction where JVM forces the first chunk. Strictly finer-grained laziness, decided after the chunk fast path (jolt-j9dz) was made O(n).

Narrow integer types

jolt unifies every integer as one exact-integer type (:integer-box-model, jolt-k9sw). (byte n)/(short n)/(int n) produce value-correct integers — arithmetic, =, and hash behave exactly as the JVM — but report Long, not Byte/Short/Integer, so (class (byte 5)) and (instance? Byte (byte 5)) diverge. This is substrate-inherent: a Chez fixnum is an immediate identical? to the plain integer (nothing to tag, and numbers carry no metadata), so the only faithful representation is a boxed type — which would crash the compiled arithmetic fast path (both operands Chez numbers → the raw Chez op) or force every +/-/* through an unwrapping dispatcher, de-optimizing all arithmetic. Same shape as the accepted BigInt-vs-Long unification.

The cast RANGE contract is full parity (corpus casts / *): byte/short/ int/long/char range-check like RT.byteCast — an out-of-range value is IllegalArgumentException "Value out of range for byte: 128". A double operand range-checks ITSELF before truncating ((byte 1.1) is 1, (byte 127.000001) throws), NaN casts to 0, ratios and bigdecs truncate, a non-number is ClassCastException. float range-checks against Float/MAX_VALUE. The unchecked-* casts wrap and sign-fold like the JVM primitive conversions ((unchecked-byte 200) is -56; a double saturates instead of wrapping). What jolt does NOT model is a distinct single-float type: (float x) keeps the double VALUE, so a double below Float/MIN_VALUE stays nonzero and float rounding does not occur (the accepted no-single-float residue, baselined with :integer-box-model's class residue).

Number operations

Binary arithmetic and comparisons follow the JVM's Numbers.ops(x, y) category dispatch. Every position (call, value, higher-order) funnels a binary op through one seam (host/chez/seq.ss): operands inside Chez's tower take the native op with the JVM contagion rules patched in; an operand outside it (BigDecimal) falls to a slow hook the numeric shim extends (host/chez/java/bigdec.ss); a non-numeric operand throws ClassCastException. The rules the corpus pins (numbers / ops dispatch, numbers / with-precision, numbers / rationalize):

• A double operand wins: (* 1.0 0) is 0.0 (Chez's exact-zero shortcut must not leak), (* ##Inf 0) is ##NaN, (+ 1.5M 2.0) is 3.5.
• Division: an exact zero divisor throws ArithmeticException; a double zero divisor yields ##Inf/##-Inf/##NaN. (/ 1M 3M) with no bound *math-context* throws (non-terminating); under with-precision it rounds.
• quot/rem/mod cover the full tower (ratios truncate; doubles keep double; mod takes the divisor's sign; zero divisor throws in both worlds).
• min/max return the original operand ((min 1 2.0) is 1, exact); a ##NaN operand wins.
• with-precision binds *math-context*; BigDecimal results round to the precision with the java.math.RoundingMode semantics (default HALF_UP, UNNECESSARY throws).
• rationalize routes a double through its shortest decimal print (BigDecimal.valueOf), so (rationalize 1.1) is 11/10, not the exact binary expansion.

Hosting jolt on a new runtime

1. Implement the reader + analyzer + a backend for your runtime (see the Chez port under host/chez/ for a worked example).

2. Write a ~30-line harness that, for each corpus row, evaluates :actual and :expected and compares by value-equality (skip :throws rows to an expect-raises check). Pseudocode:

   (doseq [{:keys [suite label actual expected]} (read-edn "test/chez/corpus.edn")]
     (let [feats (profile-features [suite label])]   ; from profile.edn
       (when (subset? feats my-implemented-features)  ; only cases I claim to support
         (record! [suite label]
                  (if (= :throws expected)
                    (raises? actual)
                    (value= (eval actual) (eval expected)))))))
   

3. Run it. Your conformance level is the set of feature families with no failures. Portable-only is the floor; each feature you implement raises it.

The reference harness does exactly this on Chez: host/chez/run-corpus.ss (the analyzer runs on Chez → Chez runtime), with a regression floor. Run it via make corpus.

Maintaining the spec

• Add/change cases: edit test/chez/corpus.edn directly, then re-source the answers with regen-corpus.clj.
• Re-certify: clojure -M test/conformance/certify.clj. A new divergence is either a real bug (file it, mark the allowlist entry :bug + :bead) or a deliberate delta (classify it in known-divergences.edn).
• Refresh the profile: re-run with --profile test/conformance/profile.edn.
• Re-floor the runtime gate when parity rises (host/chez/run-corpus.ss).

clojure-test-suite baseline traceability

Every residual entry in test/chez/cts-known-failures.txt traces to one documented model divergence — nothing in the baseline is an unexplained bug:

• :integer-box-model (this file, above): every big-int?/instance? Byte…BigInt class check, the overflow-throw rows ((+ max-long 1) is a bignum, not ArithmeticException — abs/inc/dec/minus/plus/star/quot/rem/mod/ bit-set and the +'/*' promotion-identity namespaces), boxed-identity rows ((identical? (Boolean. "true") true), (= x x) on a boxed NaN — jolt numbers are immediates, there is no box to distinguish), and num's primitive-overload reflection rows.
• no single float (Narrow integer types, above): (float Double/MIN_VALUE) keeps the double value instead of rounding to 0.0f; (double? (float x)) is true.
• RFC 0003 transients: (transient sorted/list/lazy-seq) succeeds through the copy-on-write fallback (a deliberate superset; non-collections now throw like the JVM), and double-transient is idempotent rather than throwing.
• :seq-type-model/:chunking-model (Seq semantics, above): realized? on the rest of a realized chain (a plain seq cell on jolt, a cached LazySeq on the JVM), p/lazy-seq? on forced rest chains, and chunk-granularity realization counts (lazy-seq namespace).
• stm-refs (coverage.md): the (ref …)/dosync sections of the watch namespaces (add-watch/remove-watch) — refs are out of scope pending the concurrency design note.
• parse-uuid strictness (spec §9, parse-uuid S3): jolt is deliberately strict where the reference's java.util.UUID accepts non-canonical forms like "0-0-0-0-0".
• vec of an array copies: the reference ADOPTS an Object array (mutating the array mutates the vector); jolt copies — immutable semantics win over the implementation leak (vec namespace, one row).
• eval of JVM shapes (rand-nth/eval residue): rows needing JVM-only evaluation shapes (e.g. evaluating a Java array literal).

A future change that makes any of these rows pass will fail the cts gate as STALE, forcing this section and the baseline to be updated together.