ce8e89ca86
Auditing the remaining cts baseline for R7 exposed real contract gaps hiding among the model residue — all fixed to reference behavior: - stale no-ratio-era stubs: numerator/denominator now work over jolt's exact rationals (non-ratio is the Ratio cast failure); rational? includes decimals - casts and pending: peek/pop demand an IPersistentStack (pop nil is nil), realized? demands an IPending (a plain list/range throws), transient demands an editable COLLECTION (non-colls throw; the RFC 0003 sorted/list/seq superset keeps the copy-on-write fallback), empty on a plain record throws - nil and empties: (nth nil i) is nil, (nth nil i d) is d, a nil index is NPE, keys/vals of anything empty are nil, (conj nil) is nil - lookups: contains? on a string is index-only (other keys IAE), get on an array is lenient (nth still throws), a VECTOR invocation has nth semantics (([1 2] 5) throws — call position and jolt-invoke both) - into only transients editable collections; a PersistentQueue/sorted target folds through conj (RT's IEditableCollection split) - numbers: number?/num accept BigDecimal, quot/rem throw on an Infinite/NaN quotient, even?/odd? demand integers - ordering: keywords compare namespace-first with nil first (Symbol.compareTo) - misc: run! honors reduced, eval self-evaluates non-form values, intern demands an existing namespace, counted? excludes strings, seqable? includes arrays, shuffle rejects maps, sort-by rejects a collection comparator, when-let demands one binding pair, case*/deftype*/letfn*/reify*/& are special symbols Two mis-certified corpus rows fixed (they threw on the JVM too and hid in the tolerated bucket): a raw \d string escape and duplicate literal map keys. SPEC.md gains the baseline-traceability section: every one of the 146 remaining suite failures maps to a documented divergence (integer-box, no-single-float, RFC 0003 transients, seq/chunking model, stm-refs, parse-uuid strictness, vec-array adoption). cts baseline 5955 -> 6042 pass, 5 errors, 30 namespaces. 9 JVM-certified corpus rows.
244 lines
14 KiB
Markdown
244 lines
14 KiB
Markdown
# The jolt conformance spec
|
|
|
|
This directory defines jolt's behavior as a **host-neutral, executable language
|
|
specification**: a data file of cases, certified against reference Clojure, with a
|
|
feature profile that lets any runtime declare a conformance *level*. The goal is to
|
|
make hosting jolt on a new runtime (and proving it correct) a mechanical exercise:
|
|
read one data file, run each case, compare, report.
|
|
|
|
## The artifacts
|
|
|
|
| File | Role | Generated by |
|
|
|------|------|--------------|
|
|
| `test/chez/corpus.edn` | **The spec.** ~2900 cases of `{:suite :label :expected :actual}`, `:expected` **sourced from reference JVM Clojure**. | `test/conformance/regen-corpus.clj` |
|
|
| `test/conformance/profile.edn` | Per-case **feature classification** — which non-portable cases need which host capability. | `certify.clj --profile` |
|
|
| `test/conformance/known-divergences.edn` | The few rows whose JVM value is an opaque host object that can't round-trip to readable source (Java arrays/transients/atoms/beans/proxies print as `#object[..@addr]`), so the corpus keeps jolt's value. | `regen-corpus.clj` leftovers, hand-checked |
|
|
| `test/conformance/regen-corpus.clj` | Sources every `:expected` from reference **JVM Clojure** in one process. | — |
|
|
| `test/conformance/certify.clj` | Certifies `:expected` against reference **JVM Clojure**; gates on new/stale divergences; emits the profile. | — |
|
|
|
|
`corpus.edn` is **JVM-sourced**: `regen-corpus.clj` evaluates each case's `:actual`
|
|
on reference JVM Clojure and writes the JVM value as `:expected`. **`corpus.edn` is
|
|
the canonical, frozen contract**: it is what every runtime consumes, what
|
|
`certify.clj` certifies, and where new cases are authored directly.
|
|
|
|
## Row schema
|
|
|
|
```edn
|
|
{:suite "numbers / arithmetic" ; grouping; "<suite> :: <label>" is the case id
|
|
:label "integer add" ; unique within a suite
|
|
:actual "(+ 1 2)" ; Clojure source to evaluate
|
|
:expected "3"} ; Clojure source whose value it must equal,
|
|
; or the keyword :throws
|
|
```
|
|
|
|
- `[:suite :label]` is the **canonical, unique case id** (the generator
|
|
disambiguates duplicate labels with ` (N)`).
|
|
- Comparison is **value-equality** (`=`), never string/printed-form — so map/set
|
|
iteration order never matters.
|
|
- Because comparison is `=`, a **type** or **laziness** difference is invisible to a
|
|
plain value row: `(= [0 1] '(0 1))` is true, so a fn returning a vector where
|
|
Clojure returns a seq still passes. Pin those explicitly — container/element type
|
|
with a predicate row (`(seq? …)`, `(vector? …)`, `(every? seq? …)`), and laziness
|
|
with a `(take n (… (range)))` row over an infinite source (it hangs, not just
|
|
diverges, if the fn isn't lazy). The `seq / lazy over infinite` suite does both.
|
|
- `:expected :throws` asserts evaluating `:actual` raises.
|
|
|
|
## The oracle: reference JVM Clojure
|
|
|
|
Historically every `:expected` was hand-written. `certify.clj` removes that
|
|
weakness: it evaluates every `:actual` (and `:expected`) on **JVM Clojure** in a
|
|
fresh `user` namespace and checks jolt's `:expected` against what real Clojure
|
|
produces. Of ~2740 vanilla-certifiable rows, **>2730 match reference Clojure
|
|
exactly**. The rest are classified (see below) — none are silently wrong.
|
|
|
|
```sh
|
|
clojure -M test/conformance/certify.clj # gate
|
|
clojure -M test/conformance/certify.clj test/chez/corpus.edn --edn r.edn # + report
|
|
clojure -M test/conformance/certify.clj test/chez/corpus.edn --profile test/conformance/profile.edn
|
|
```
|
|
|
|
The gate fails only on a **new** (unclassified) divergence or a **stale**
|
|
allowlist entry; flaky timing-dependent cases (`future-cancel`) are tolerated.
|
|
|
|
## Conformance levels & the feature profile
|
|
|
|
Not every case is portable: some assume a host capability jolt has on one runtime
|
|
but not another (Java interop, real threads, BigDecimal). `profile.edn` classifies
|
|
each **non-portable** case by the feature it requires. Cases *not* in the profile
|
|
are **portable** — they must pass on any faithful Clojure.
|
|
|
|
A runtime's **conformance level** = portable cases + the feature families it
|
|
implements. Current profile (≈2735 portable, ≈167 non-portable):
|
|
|
|
| Feature | Meaning |
|
|
|---------|---------|
|
|
| `:numerics/double-only` | all-double numeric model — no Ratio/BigDecimal/float; `(/ 1 2)` ⇒ `0.5` |
|
|
| `:concurrency/snapshot` | isolated-heap futures/agents/pmap — captured atoms are snapshotted, not shared |
|
|
| `:host/jvm-interop` | Java classes / `instance?` on host classes / proxy / bean / definterface |
|
|
| `:host/arrays` | Java arrays (`into-array`, `int-array`, …) |
|
|
| `:async/core-async` | `clojure.core.async` channels/`go` |
|
|
| `:runtime/eval` | runtime `eval` / `load-string` |
|
|
| `:reader/jolt` | jolt reader features (`#?(:jolt …)`) + syntax-quote literal collapse |
|
|
| `:printer/jolt` | jolt's rendering of transients/atoms/`print-method` overrides |
|
|
| `:strictness/jolt` | intentionally stricter (throws on odd `assoc!` args, etc.) |
|
|
| `:impl/representation` | representation detail (e.g. syntax-quote yields a `list?`, not a `Cons`) |
|
|
| `:bug` | a *known defect* (tracked bead) — not a host difference |
|
|
|
|
## Seq semantics
|
|
|
|
Values alone don't pin laziness — an eager `map` and a lazy `map` return the same
|
|
elements. The spec certifies seq *semantics* by reducing them to values with a
|
|
side-effect counter, so the corpus catches a laziness regression the value
|
|
comparison would miss.
|
|
|
|
**Laziness (certified — jolt matches JVM).** The whole producer family
|
|
(`map`/`filter`/`remove`/`take`/`drop`/`concat`/`take-while`/`drop-while`/`mapcat`/
|
|
`partition`/`partition-all`/`partition-by`/`keep`/`keep-indexed`/`map-indexed`/
|
|
`distinct`/`interpose`/`interleave`/`take-nth`/`reductions`/`tree-seq`/`replace`)
|
|
is lazy at construction: building over a side-effecting source realizes **zero**
|
|
elements (`lazy / family is lazy at construction`). Realization order is
|
|
left-to-right, `take`/`nth`/`drop` realize exactly as far as demanded, a lazy seq
|
|
memoizes (realize-once across walks), and `next` realizes head + one lookahead
|
|
while `rest` realizes only the head (`lazy / realization order & count`,
|
|
`lazy / realization is memoized`, `lazy / realization timing`). A lazy result is
|
|
`clojure.lang.LazySeq`.
|
|
|
|
**Accepted divergences.** jolt is a simpler, finer-grained superset of JVM seq
|
|
behavior; two classes diverge by representation, never by value, and are
|
|
allowlisted in `known-divergences.edn`:
|
|
|
|
- **`:seq-type-model`** (`seq-type-model / …` suite, jolt-aei7) — jolt reifies
|
|
every seq as `PersistentList` (eager) or `LazySeq` (deferred). JVM has a
|
|
specialized class per producer (`Cons`, `Iterate`, `LongRange`, `Repeat`,
|
|
`Cycle`, `PersistentVector$ChunkedSeq`, `StringSeq`, `KeySeq`/`ValSeq`, `RSeq`,
|
|
`ArraySeq`, `SubVector`), so `(class …)` differs. `instance?
|
|
clojure.lang.ISeq/Sequential` and all values/laziness are correct.
|
|
- **`:chunking-model`** (`chunking-model / …` suite, jolt-mm6v) — jolt seqs are
|
|
unchunked: forcing one element realizes one, where JVM realizes a ~32-element
|
|
chunk; `mapcat`/`dedupe` realize 0 at construction where JVM forces the first
|
|
chunk. Strictly finer-grained laziness, decided after the chunk fast path
|
|
(jolt-j9dz) was made O(n).
|
|
|
|
## Narrow integer types
|
|
|
|
jolt unifies every integer as one exact-integer type (`:integer-box-model`,
|
|
jolt-k9sw). `(byte n)`/`(short n)`/`(int n)` produce value-correct integers —
|
|
arithmetic, `=`, and `hash` behave exactly as the JVM — but report `Long`, not
|
|
`Byte`/`Short`/`Integer`, so `(class (byte 5))` and `(instance? Byte (byte 5))`
|
|
diverge. This is substrate-inherent: a Chez fixnum is an immediate `identical?`
|
|
to the plain integer (nothing to tag, and numbers carry no metadata), so the only
|
|
faithful representation is a boxed type — which would crash the compiled
|
|
arithmetic fast path (both operands Chez numbers → the raw Chez op) or force
|
|
every `+`/`-`/`*` through an unwrapping dispatcher, de-optimizing all
|
|
arithmetic. Same shape as the accepted BigInt-vs-Long unification.
|
|
|
|
The cast RANGE contract is full parity (corpus `casts / *`): `byte`/`short`/
|
|
`int`/`long`/`char` range-check like `RT.byteCast` — an out-of-range value is
|
|
IllegalArgumentException "Value out of range for byte: 128". A double operand
|
|
range-checks ITSELF before truncating (`(byte 1.1)` is `1`, `(byte 127.000001)`
|
|
throws), NaN casts to 0, ratios and bigdecs truncate, a non-number is
|
|
ClassCastException. `float` range-checks against Float/MAX_VALUE. The
|
|
`unchecked-*` casts wrap and sign-fold like the JVM primitive conversions
|
|
(`(unchecked-byte 200)` is `-56`; a double saturates instead of wrapping).
|
|
What jolt does NOT model is a distinct single-float type: `(float x)` keeps
|
|
the double VALUE, so a double below Float/MIN_VALUE stays nonzero and float
|
|
rounding does not occur (the accepted no-single-float residue, baselined with
|
|
`:integer-box-model`'s class residue).
|
|
|
|
## Number operations
|
|
|
|
Binary arithmetic and comparisons follow the JVM's `Numbers.ops(x, y)` category
|
|
dispatch. Every position (call, value, higher-order) funnels a binary op through
|
|
one seam (`host/chez/seq.ss`): operands inside Chez's tower take the native op
|
|
with the JVM contagion rules patched in; an operand outside it (BigDecimal)
|
|
falls to a slow hook the numeric shim extends (`host/chez/java/bigdec.ss`); a
|
|
non-numeric operand throws `ClassCastException`. The rules the corpus pins
|
|
(`numbers / ops dispatch`, `numbers / with-precision`, `numbers / rationalize`):
|
|
|
|
- A double operand wins: `(* 1.0 0)` is `0.0` (Chez's exact-zero shortcut must
|
|
not leak), `(* ##Inf 0)` is `##NaN`, `(+ 1.5M 2.0)` is `3.5`.
|
|
- Division: an exact zero divisor throws `ArithmeticException`; a double zero
|
|
divisor yields `##Inf`/`##-Inf`/`##NaN`. `(/ 1M 3M)` with no bound
|
|
`*math-context*` throws (non-terminating); under `with-precision` it rounds.
|
|
- `quot`/`rem`/`mod` cover the full tower (ratios truncate; doubles keep double;
|
|
`mod` takes the divisor's sign; zero divisor throws in both worlds).
|
|
- `min`/`max` return the *original* operand (`(min 1 2.0)` is `1`, exact); a
|
|
`##NaN` operand wins.
|
|
- `with-precision` binds `*math-context*`; BigDecimal results round to the
|
|
precision with the `java.math.RoundingMode` semantics (default `HALF_UP`,
|
|
`UNNECESSARY` throws).
|
|
- `rationalize` routes a double through its shortest decimal print
|
|
(`BigDecimal.valueOf`), so `(rationalize 1.1)` is `11/10`, not the exact
|
|
binary expansion.
|
|
|
|
## Hosting jolt on a new runtime
|
|
|
|
1. Implement the reader + analyzer + a backend for your runtime (see the Chez port
|
|
under `host/chez/` for a worked example).
|
|
2. Write a ~30-line harness that, for each corpus row, evaluates `:actual` and
|
|
`:expected` and compares by value-equality (skip `:throws` rows to an
|
|
expect-raises check). Pseudocode:
|
|
|
|
```
|
|
(doseq [{:keys [suite label actual expected]} (read-edn "test/chez/corpus.edn")]
|
|
(let [feats (profile-features [suite label])] ; from profile.edn
|
|
(when (subset? feats my-implemented-features) ; only cases I claim to support
|
|
(record! [suite label]
|
|
(if (= :throws expected)
|
|
(raises? actual)
|
|
(value= (eval actual) (eval expected)))))))
|
|
```
|
|
|
|
3. Run it. Your **conformance level** is the set of feature families with no
|
|
failures. Portable-only is the floor; each feature you implement raises it.
|
|
|
|
The reference harness does exactly this on Chez: `host/chez/run-corpus.ss` (the
|
|
analyzer runs on Chez → Chez runtime), with a regression floor. Run it via `make
|
|
corpus`.
|
|
|
|
## Maintaining the spec
|
|
|
|
- **Add/change cases**: edit `test/chez/corpus.edn` directly, then re-source the
|
|
answers with `regen-corpus.clj`.
|
|
- **Re-certify**: `clojure -M test/conformance/certify.clj`. A new divergence is
|
|
either a real bug (file it, mark the allowlist entry `:bug` + `:bead`) or a
|
|
deliberate delta (classify it in `known-divergences.edn`).
|
|
- **Refresh the profile**: re-run with `--profile test/conformance/profile.edn`.
|
|
- **Re-floor the runtime gate** when parity rises (`host/chez/run-corpus.ss`).
|
|
|
|
## clojure-test-suite baseline traceability
|
|
|
|
Every residual entry in `test/chez/cts-known-failures.txt` traces to one
|
|
documented model divergence — nothing in the baseline is an unexplained bug:
|
|
|
|
- `:integer-box-model` (this file, above): every `big-int?`/`instance?
|
|
Byte…BigInt` class check, the overflow-throw rows (`(+ max-long 1)` is a
|
|
bignum, not ArithmeticException — abs/inc/dec/minus/plus/star/quot/rem/mod/
|
|
bit-set and the `+'`/`*'` promotion-identity namespaces), boxed-identity
|
|
rows (`(identical? (Boolean. "true") true)`, `(= x x)` on a boxed NaN — jolt
|
|
numbers are immediates, there is no box to distinguish), and `num`'s
|
|
primitive-overload reflection rows.
|
|
- **no single float** (Narrow integer types, above): `(float Double/MIN_VALUE)`
|
|
keeps the double value instead of rounding to 0.0f; `(double? (float x))`
|
|
is true.
|
|
- **RFC 0003 transients**: `(transient sorted/list/lazy-seq)` succeeds through
|
|
the copy-on-write fallback (a deliberate superset; non-collections now throw
|
|
like the JVM), and double-transient is idempotent rather than throwing.
|
|
- `:seq-type-model`/`:chunking-model` (Seq semantics, above): `realized?` on
|
|
the rest of a realized chain (a plain seq cell on jolt, a cached LazySeq on
|
|
the JVM), `p/lazy-seq?` on forced rest chains, and chunk-granularity
|
|
realization counts (lazy-seq namespace).
|
|
- **stm-refs** (`coverage.md`): the `(ref …)`/`dosync` sections of the watch
|
|
namespaces (add-watch/remove-watch) — refs are out of scope pending the
|
|
concurrency design note.
|
|
- **parse-uuid strictness** (spec §9, parse-uuid S3): jolt is deliberately
|
|
strict where the reference's java.util.UUID accepts non-canonical forms
|
|
like `"0-0-0-0-0"`.
|
|
- **vec of an array copies**: the reference ADOPTS an Object array (mutating
|
|
the array mutates the vector); jolt copies — immutable semantics win over
|
|
the implementation leak (`vec` namespace, one row).
|
|
- **eval of JVM shapes** (`rand-nth`/`eval` residue): rows needing JVM-only
|
|
evaluation shapes (e.g. evaluating a Java array literal).
|
|
|
|
A future change that makes any of these rows pass will fail the cts gate as
|
|
STALE, forcing this section and the baseline to be updated together.
|