jolt/test/conformance/SPEC.md

# The jolt conformance spec

This directory defines jolt's behavior as a **host-neutral, executable language
specification**: a data file of cases, certified against reference Clojure, with a
feature profile that lets any runtime declare a conformance *level*. The goal is to
make hosting jolt on a new runtime (and proving it correct) a mechanical exercise:
read one data file, run each case, compare, report.

## The artifacts

| File | Role | Generated by |
|------|------|--------------|
| `test/chez/corpus.edn` | **The spec.** ~2900 cases of `{:suite :label :expected :actual}`, `:expected` **sourced from reference JVM Clojure**. | `test/conformance/regen-corpus.clj` |
| `test/conformance/profile.edn` | Per-case **feature classification** — which non-portable cases need which host capability. | `certify.clj --profile` |
| `test/conformance/known-divergences.edn` | The few rows whose JVM value is an opaque host object that can't round-trip to readable source (Java arrays/transients/atoms/beans/proxies print as `#object[..@addr]`), so the corpus keeps jolt's value. | `regen-corpus.clj` leftovers, hand-checked |
| `test/conformance/regen-corpus.clj` | Sources every `:expected` from reference **JVM Clojure** in one process. | — |
| `test/conformance/certify.clj` | Certifies `:expected` against reference **JVM Clojure**; gates on new/stale divergences; emits the profile. | — |

`corpus.edn` is **JVM-sourced**: `regen-corpus.clj` evaluates each case's `:actual`
on reference JVM Clojure and writes the JVM value as `:expected`. **`corpus.edn` is
the canonical, frozen contract**: it is what every runtime consumes, what
`certify.clj` certifies, and where new cases are authored directly.

## Row schema

```edn
{:suite       "numbers / arithmetic"   ; grouping; "<suite> :: <label>" is the case id
 :label       "integer add"            ; unique within a suite
 :actual      "(+ 1 2)"                 ; Clojure source to evaluate
 :expected    "3"                       ; Clojure source whose value it must equal,
                                        ; or the keyword :throws
 :portability :common}                  ; :common = portable Clojure any dialect
                                        ; must satisfy; :jvm = exercises host
                                        ; interop (java.*/clojure.lang.* classes,
                                        ; dot forms, ctors, statics, arrays,
                                        ; proxy/bean) — skip on a non-JVM-shaped
                                        ; dialect
```

- `[:suite :label]` is the **canonical, unique case id** (the generator
  disambiguates duplicate labels with ` (N)`).
- Comparison is **value-equality** (`=`), never string/printed-form — so map/set
  iteration order never matters.
- Because comparison is `=`, a **type** or **laziness** difference is invisible to a
  plain value row: `(= [0 1] '(0 1))` is true, so a fn returning a vector where
  Clojure returns a seq still passes. Pin those explicitly — container/element type
  with a predicate row (`(seq? …)`, `(vector? …)`, `(every? seq? …)`), and laziness
  with a `(take n (… (range)))` row over an infinite source (it hangs, not just
  diverges, if the fn isn't lazy). The `seq / lazy over infinite` suite does both.
- `:expected :throws` asserts evaluating `:actual` raises.

## The oracle: reference JVM Clojure

Historically every `:expected` was hand-written. `certify.clj` removes that
weakness: it evaluates every `:actual` (and `:expected`) on **JVM Clojure** in a
fresh `user` namespace and checks jolt's `:expected` against what real Clojure
produces. Of ~2740 vanilla-certifiable rows, **>2730 match reference Clojure
exactly**. The rest are classified (see below) — none are silently wrong.

```sh
clojure -M test/conformance/certify.clj                                  # gate
clojure -M test/conformance/certify.clj test/chez/corpus.edn --edn r.edn # + report
clojure -M test/conformance/certify.clj test/chez/corpus.edn --profile test/conformance/profile.edn
```

The gate fails only on a **new** (unclassified) divergence or a **stale**
allowlist entry; flaky timing-dependent cases (`future-cancel`) are tolerated.

## Conformance levels & the feature profile

Not every case is portable: some assume a host capability jolt has on one runtime
but not another (Java interop, real threads, BigDecimal). `profile.edn` classifies
each **non-portable** case by the feature it requires. Cases *not* in the profile
are **portable** — they must pass on any faithful Clojure.

A runtime's **conformance level** = portable cases + the feature families it
implements. Current profile (≈2735 portable, ≈167 non-portable):

| Feature | Meaning |
|---------|---------|
| `:numerics/double-only` | all-double numeric model — no Ratio/BigDecimal/float; `(/ 1 2)` ⇒ `0.5` |
| `:concurrency/snapshot` | isolated-heap futures/agents/pmap — captured atoms are snapshotted, not shared |
| `:host/jvm-interop` | Java classes / `instance?` on host classes / proxy / bean / definterface |
| `:host/arrays` | Java arrays (`into-array`, `int-array`, …) |
| `:async/core-async` | `clojure.core.async` channels/`go` |
| `:runtime/eval` | runtime `eval` / `load-string` |
| `:reader/jolt` | jolt reader features (`#?(:jolt …)`) + syntax-quote literal collapse |
| `:printer/jolt` | jolt's rendering of transients/atoms/`print-method` overrides |
| `:strictness/jolt` | intentionally stricter (throws on odd `assoc!` args, etc.) |
| `:impl/representation` | representation detail (e.g. syntax-quote yields a `list?`, not a `Cons`) |
| `:bug` | a *known defect* (tracked bead) — not a host difference |

## Seq semantics

Values alone don't pin laziness — an eager `map` and a lazy `map` return the same
elements. The spec certifies seq *semantics* by reducing them to values with a
side-effect counter, so the corpus catches a laziness regression the value
comparison would miss.

**Laziness (certified — jolt matches JVM).** The whole producer family
(`map`/`filter`/`remove`/`take`/`drop`/`concat`/`take-while`/`drop-while`/`mapcat`/
`partition`/`partition-all`/`partition-by`/`keep`/`keep-indexed`/`map-indexed`/
`distinct`/`interpose`/`interleave`/`take-nth`/`reductions`/`tree-seq`/`replace`)
is lazy at construction: building over a side-effecting source realizes **zero**
elements (`lazy / family is lazy at construction`). Realization order is
left-to-right, `take`/`nth`/`drop` realize exactly as far as demanded, a lazy seq
memoizes (realize-once across walks), and `next` realizes head + one lookahead
while `rest` realizes only the head (`lazy / realization order & count`,
`lazy / realization is memoized`, `lazy / realization timing`). A lazy result is
`clojure.lang.LazySeq`.

**Accepted divergences.** jolt is a simpler, finer-grained superset of JVM seq
behavior; two classes diverge by representation, never by value, and are
allowlisted in `known-divergences.edn`:

- **`:seq-type-model`** (`seq-type-model / …` suite, jolt-aei7) — jolt reifies
  every seq as `PersistentList` (eager) or `LazySeq` (deferred). JVM has a
  specialized class per producer (`Cons`, `Iterate`, `LongRange`, `Repeat`,
  `Cycle`, `PersistentVector$ChunkedSeq`, `StringSeq`, `KeySeq`/`ValSeq`, `RSeq`,
  `ArraySeq`, `SubVector`), so `(class …)` differs. `instance?
  clojure.lang.ISeq/Sequential` and all values/laziness are correct.
- **`:chunking-model`** (`chunking-model / …` suite, jolt-mm6v) — jolt seqs are
  unchunked: forcing one element realizes one, where JVM realizes a ~32-element
  chunk; `mapcat`/`dedupe` realize 0 at construction where JVM forces the first
  chunk. Strictly finer-grained laziness, decided after the chunk fast path
  (jolt-j9dz) was made O(n).

## Narrow integer types

jolt unifies every integer as one exact-integer type (`:integer-box-model`,
jolt-k9sw). `(byte n)`/`(short n)`/`(int n)` produce value-correct integers —
arithmetic, `=`, and `hash` behave exactly as the JVM — but report `Long`, not
`Byte`/`Short`/`Integer`, so `(class (byte 5))` and `(instance? Byte (byte 5))`
diverge. This is substrate-inherent: a Chez fixnum is an immediate `identical?`
to the plain integer (nothing to tag, and numbers carry no metadata), so the only
faithful representation is a boxed type — which would crash the compiled
arithmetic fast path (both operands Chez numbers → the raw Chez op) or force
every `+`/`-`/`*` through an unwrapping dispatcher, de-optimizing all
arithmetic. Same shape as the accepted BigInt-vs-Long unification.

The cast RANGE contract is full parity (corpus `casts / *`): `byte`/`short`/
`int`/`long`/`char` range-check like `RT.byteCast` — an out-of-range value is
IllegalArgumentException "Value out of range for byte: 128". A double operand
range-checks ITSELF before truncating (`(byte 1.1)` is `1`, `(byte 127.000001)`
throws), NaN casts to 0, ratios and bigdecs truncate, a non-number is
ClassCastException. `float` range-checks against Float/MAX_VALUE. The
`unchecked-*` casts wrap and sign-fold like the JVM primitive conversions
(`(unchecked-byte 200)` is `-56`; a double saturates instead of wrapping).
What jolt does NOT model is a distinct single-float type: `(float x)` keeps
the double VALUE, so a double below Float/MIN_VALUE stays nonzero and float
rounding does not occur (the accepted no-single-float residue, baselined with
`:integer-box-model`'s class residue).

## Number operations

Binary arithmetic and comparisons follow the JVM's `Numbers.ops(x, y)` category
dispatch. Every position (call, value, higher-order) funnels a binary op through
one seam (`host/chez/seq.ss`): operands inside Chez's tower take the native op
with the JVM contagion rules patched in; an operand outside it (BigDecimal)
falls to a slow hook the numeric shim extends (`host/chez/java/bigdec.ss`); a
non-numeric operand throws `ClassCastException`. The rules the corpus pins
(`numbers / ops dispatch`, `numbers / with-precision`, `numbers / rationalize`):

- A double operand wins: `(* 1.0 0)` is `0.0` (Chez's exact-zero shortcut must
  not leak), `(* ##Inf 0)` is `##NaN`, `(+ 1.5M 2.0)` is `3.5`.
- Division: an exact zero divisor throws `ArithmeticException`; a double zero
  divisor yields `##Inf`/`##-Inf`/`##NaN`. `(/ 1M 3M)` with no bound
  `*math-context*` throws (non-terminating); under `with-precision` it rounds.
- `quot`/`rem`/`mod` cover the full tower (ratios truncate; doubles keep double;
  `mod` takes the divisor's sign; zero divisor throws in both worlds).
- `min`/`max` return the *original* operand (`(min 1 2.0)` is `1`, exact); a
  `##NaN` operand wins.
- `with-precision` binds `*math-context*`; BigDecimal results round to the
  precision with the `java.math.RoundingMode` semantics (default `HALF_UP`,
  `UNNECESSARY` throws).
- `rationalize` routes a double through its shortest decimal print
  (`BigDecimal.valueOf`), so `(rationalize 1.1)` is `11/10`, not the exact
  binary expansion.

## Hosting jolt on a new runtime

1. Implement the reader + analyzer + a backend for your runtime (see the Chez port
   under `host/chez/` for a worked example).
2. Write a ~30-line harness that, for each corpus row, evaluates `:actual` and
   `:expected` and compares by value-equality (skip `:throws` rows to an
   expect-raises check). Pseudocode:

   ```
   (doseq [{:keys [suite label actual expected]} (read-edn "test/chez/corpus.edn")]
     (let [feats (profile-features [suite label])]   ; from profile.edn
       (when (subset? feats my-implemented-features)  ; only cases I claim to support
         (record! [suite label]
                  (if (= :throws expected)
                    (raises? actual)
                    (value= (eval actual) (eval expected)))))))
   ```

3. Run it. Your **conformance level** is the set of feature families with no
   failures. Portable-only is the floor; each feature you implement raises it.

The reference harness does exactly this on Chez: `host/chez/run-corpus.ss` (the
analyzer runs on Chez → Chez runtime), with a regression floor. Run it via `make
corpus`.

## Maintaining the spec

- **Add/change cases**: edit `test/chez/corpus.edn` directly, then re-source the
  answers with `regen-corpus.clj`.
- **Re-certify**: `clojure -M test/conformance/certify.clj`. A new divergence is
  either a real bug (file it, mark the allowlist entry `:bug` + `:bead`) or a
  deliberate delta (classify it in `known-divergences.edn`).
- **Refresh the profile**: re-run with `--profile test/conformance/profile.edn`.
- **Re-floor the runtime gate** when parity rises (`host/chez/run-corpus.ss`).

## clojure-test-suite baseline traceability

Every residual entry in `test/chez/cts-known-failures.txt` traces to one
documented model divergence — nothing in the baseline is an unexplained bug:

- `:integer-box-model` (this file, above): every `big-int?`/`instance?
  Byte…BigInt` class check, the overflow-throw rows (`(+ max-long 1)` is a
  bignum, not ArithmeticException — abs/inc/dec/minus/plus/star/quot/rem/mod/
  bit-set and the `+'`/`*'` promotion-identity namespaces), boxed-identity
  rows (`(identical? (Boolean. "true") true)`, `(= x x)` on a boxed NaN — jolt
  numbers are immediates, there is no box to distinguish), and `num`'s
  primitive-overload reflection rows.
- **no single float** (Narrow integer types, above): `(float Double/MIN_VALUE)`
  keeps the double value instead of rounding to 0.0f; `(double? (float x))`
  is true.
- **RFC 0003 transients**: `(transient sorted/list/lazy-seq)` succeeds through
  the copy-on-write fallback (a deliberate superset; non-collections now throw
  like the JVM), and double-transient is idempotent rather than throwing.
- `:seq-type-model`/`:chunking-model` (Seq semantics, above): `realized?` on
  the rest of a realized chain (a plain seq cell on jolt, a cached LazySeq on
  the JVM), `p/lazy-seq?` on forced rest chains, and chunk-granularity
  realization counts (lazy-seq namespace).
- **stm-refs** (`coverage.md`): the `(ref …)`/`dosync` sections of the watch
  namespaces (add-watch/remove-watch) — refs are out of scope pending the
  concurrency design note.
- **parse-uuid strictness** (spec §9, parse-uuid S3): jolt is deliberately
  strict where the reference's java.util.UUID accepts non-canonical forms
  like `"0-0-0-0-0"`.
- **vec of an array copies**: the reference ADOPTS an Object array (mutating
  the array mutates the vector); jolt copies — immutable semantics win over
  the implementation leak (`vec` namespace, one row).
- **eval of JVM shapes** (`rand-nth`/`eval` residue): rows needing JVM-only
  evaluation shapes (e.g. evaluating a Java array literal).

A future change that makes any of these rows pass will fail the cts gate as
STALE, forcing this section and the baseline to be updated together.