59cfa5f53f
A 62-case jolt-vs-JVM probe across seq type identity, chunking granularity, eagerness, and realization timing. Findings: the whole producer family is lazy at construction (no eager bugs remain), and the 26 divergences fall into two classes that diverge by representation, not value. Lock in the laziness contract as certified corpus rows: construction=0 for keep/keep-indexed/map-indexed/distinct/partition-by/partition-all/ interpose/interleave/take-nth/reductions/tree-seq/replace, sequence realizes 1, next realizes 2, rest realizes 1. Pin the two accepted divergence classes (allowlisted, gate-guarded): - seq-type-model: jolt reifies seqs as PersistentList/LazySeq vs JVM's Cons/Iterate/LongRange/Repeat/Cycle/ChunkedSeq/StringSeq/KeySeq/RSeq/ ArraySeq/SubVector (jolt-aei7) - chunking-model: unchunked, realizes one where JVM realizes a 32-chunk; mapcat/dedupe fully lazy at construction (jolt-mm6v) known-divergences.edn gains both categories; SPEC.md documents the seq semantics contract. Data/doc only, no re-mint. certify 0 new / 0 stale.
155 lines
8.8 KiB
Markdown
155 lines
8.8 KiB
Markdown
# The jolt conformance spec
|
|
|
|
This directory defines jolt's behavior as a **host-neutral, executable language
|
|
specification**: a data file of cases, certified against reference Clojure, with a
|
|
feature profile that lets any runtime declare a conformance *level*. The goal is to
|
|
make hosting jolt on a new runtime (and proving it correct) a mechanical exercise:
|
|
read one data file, run each case, compare, report.
|
|
|
|
## The artifacts
|
|
|
|
| File | Role | Generated by |
|
|
|------|------|--------------|
|
|
| `test/chez/corpus.edn` | **The spec.** ~2900 cases of `{:suite :label :expected :actual}`, `:expected` **sourced from reference JVM Clojure**. | `test/conformance/regen-corpus.clj` |
|
|
| `test/conformance/profile.edn` | Per-case **feature classification** — which non-portable cases need which host capability. | `certify.clj --profile` |
|
|
| `test/conformance/known-divergences.edn` | The few rows whose JVM value is an opaque host object that can't round-trip to readable source (Java arrays/transients/atoms/beans/proxies print as `#object[..@addr]`), so the corpus keeps jolt's value. | `regen-corpus.clj` leftovers, hand-checked |
|
|
| `test/conformance/regen-corpus.clj` | Sources every `:expected` from reference **JVM Clojure** in one process. | — |
|
|
| `test/conformance/certify.clj` | Certifies `:expected` against reference **JVM Clojure**; gates on new/stale divergences; emits the profile. | — |
|
|
|
|
`corpus.edn` is **JVM-sourced**: `regen-corpus.clj` evaluates each case's `:actual`
|
|
on reference JVM Clojure and writes the JVM value as `:expected`. **`corpus.edn` is
|
|
the canonical, frozen contract**: it is what every runtime consumes, what
|
|
`certify.clj` certifies, and where new cases are authored directly.
|
|
|
|
## Row schema
|
|
|
|
```edn
|
|
{:suite "numbers / arithmetic" ; grouping; "<suite> :: <label>" is the case id
|
|
:label "integer add" ; unique within a suite
|
|
:actual "(+ 1 2)" ; Clojure source to evaluate
|
|
:expected "3"} ; Clojure source whose value it must equal,
|
|
; or the keyword :throws
|
|
```
|
|
|
|
- `[:suite :label]` is the **canonical, unique case id** (the generator
|
|
disambiguates duplicate labels with ` (N)`).
|
|
- Comparison is **value-equality** (`=`), never string/printed-form — so map/set
|
|
iteration order never matters.
|
|
- Because comparison is `=`, a **type** or **laziness** difference is invisible to a
|
|
plain value row: `(= [0 1] '(0 1))` is true, so a fn returning a vector where
|
|
Clojure returns a seq still passes. Pin those explicitly — container/element type
|
|
with a predicate row (`(seq? …)`, `(vector? …)`, `(every? seq? …)`), and laziness
|
|
with a `(take n (… (range)))` row over an infinite source (it hangs, not just
|
|
diverges, if the fn isn't lazy). The `seq / lazy over infinite` suite does both.
|
|
- `:expected :throws` asserts evaluating `:actual` raises.
|
|
|
|
## The oracle: reference JVM Clojure
|
|
|
|
Historically every `:expected` was hand-written. `certify.clj` removes that
|
|
weakness: it evaluates every `:actual` (and `:expected`) on **JVM Clojure** in a
|
|
fresh `user` namespace and checks jolt's `:expected` against what real Clojure
|
|
produces. Of ~2740 vanilla-certifiable rows, **>2730 match reference Clojure
|
|
exactly**. The rest are classified (see below) — none are silently wrong.
|
|
|
|
```sh
|
|
clojure -M test/conformance/certify.clj # gate
|
|
clojure -M test/conformance/certify.clj test/chez/corpus.edn --edn r.edn # + report
|
|
clojure -M test/conformance/certify.clj test/chez/corpus.edn --profile test/conformance/profile.edn
|
|
```
|
|
|
|
The gate fails only on a **new** (unclassified) divergence or a **stale**
|
|
allowlist entry; flaky timing-dependent cases (`future-cancel`) are tolerated.
|
|
|
|
## Conformance levels & the feature profile
|
|
|
|
Not every case is portable: some assume a host capability jolt has on one runtime
|
|
but not another (Java interop, real threads, BigDecimal). `profile.edn` classifies
|
|
each **non-portable** case by the feature it requires. Cases *not* in the profile
|
|
are **portable** — they must pass on any faithful Clojure.
|
|
|
|
A runtime's **conformance level** = portable cases + the feature families it
|
|
implements. Current profile (≈2735 portable, ≈167 non-portable):
|
|
|
|
| Feature | Meaning |
|
|
|---------|---------|
|
|
| `:numerics/double-only` | all-double numeric model — no Ratio/BigDecimal/float; `(/ 1 2)` ⇒ `0.5` |
|
|
| `:concurrency/snapshot` | isolated-heap futures/agents/pmap — captured atoms are snapshotted, not shared |
|
|
| `:host/jvm-interop` | Java classes / `instance?` on host classes / proxy / bean / definterface |
|
|
| `:host/arrays` | Java arrays (`into-array`, `int-array`, …) |
|
|
| `:async/core-async` | `clojure.core.async` channels/`go` |
|
|
| `:runtime/eval` | runtime `eval` / `load-string` |
|
|
| `:reader/jolt` | jolt reader features (`#?(:jolt …)`) + syntax-quote literal collapse |
|
|
| `:printer/jolt` | jolt's rendering of transients/atoms/`print-method` overrides |
|
|
| `:strictness/jolt` | intentionally stricter (throws on odd `assoc!` args, etc.) |
|
|
| `:impl/representation` | representation detail (e.g. syntax-quote yields a `list?`, not a `Cons`) |
|
|
| `:bug` | a *known defect* (tracked bead) — not a host difference |
|
|
|
|
## Seq semantics
|
|
|
|
Values alone don't pin laziness — an eager `map` and a lazy `map` return the same
|
|
elements. The spec certifies seq *semantics* by reducing them to values with a
|
|
side-effect counter, so the corpus catches a laziness regression the value
|
|
comparison would miss.
|
|
|
|
**Laziness (certified — jolt matches JVM).** The whole producer family
|
|
(`map`/`filter`/`remove`/`take`/`drop`/`concat`/`take-while`/`drop-while`/`mapcat`/
|
|
`partition`/`partition-all`/`partition-by`/`keep`/`keep-indexed`/`map-indexed`/
|
|
`distinct`/`interpose`/`interleave`/`take-nth`/`reductions`/`tree-seq`/`replace`)
|
|
is lazy at construction: building over a side-effecting source realizes **zero**
|
|
elements (`lazy / family is lazy at construction`). Realization order is
|
|
left-to-right, `take`/`nth`/`drop` realize exactly as far as demanded, a lazy seq
|
|
memoizes (realize-once across walks), and `next` realizes head + one lookahead
|
|
while `rest` realizes only the head (`lazy / realization order & count`,
|
|
`lazy / realization is memoized`, `lazy / realization timing`). A lazy result is
|
|
`clojure.lang.LazySeq`.
|
|
|
|
**Accepted divergences.** jolt is a simpler, finer-grained superset of JVM seq
|
|
behavior; two classes diverge by representation, never by value, and are
|
|
allowlisted in `known-divergences.edn`:
|
|
|
|
- **`:seq-type-model`** (`seq-type-model / …` suite, jolt-aei7) — jolt reifies
|
|
every seq as `PersistentList` (eager) or `LazySeq` (deferred). JVM has a
|
|
specialized class per producer (`Cons`, `Iterate`, `LongRange`, `Repeat`,
|
|
`Cycle`, `PersistentVector$ChunkedSeq`, `StringSeq`, `KeySeq`/`ValSeq`, `RSeq`,
|
|
`ArraySeq`, `SubVector`), so `(class …)` differs. `instance?
|
|
clojure.lang.ISeq/Sequential` and all values/laziness are correct.
|
|
- **`:chunking-model`** (`chunking-model / …` suite, jolt-mm6v) — jolt seqs are
|
|
unchunked: forcing one element realizes one, where JVM realizes a ~32-element
|
|
chunk; `mapcat`/`dedupe` realize 0 at construction where JVM forces the first
|
|
chunk. Strictly finer-grained laziness, decided after the chunk fast path
|
|
(jolt-j9dz) was made O(n).
|
|
|
|
## Hosting jolt on a new runtime
|
|
|
|
1. Implement the reader + analyzer + a backend for your runtime (see the Chez port
|
|
under `host/chez/` for a worked example).
|
|
2. Write a ~30-line harness that, for each corpus row, evaluates `:actual` and
|
|
`:expected` and compares by value-equality (skip `:throws` rows to an
|
|
expect-raises check). Pseudocode:
|
|
|
|
```
|
|
(doseq [{:keys [suite label actual expected]} (read-edn "test/chez/corpus.edn")]
|
|
(let [feats (profile-features [suite label])] ; from profile.edn
|
|
(when (subset? feats my-implemented-features) ; only cases I claim to support
|
|
(record! [suite label]
|
|
(if (= :throws expected)
|
|
(raises? actual)
|
|
(value= (eval actual) (eval expected)))))))
|
|
```
|
|
|
|
3. Run it. Your **conformance level** is the set of feature families with no
|
|
failures. Portable-only is the floor; each feature you implement raises it.
|
|
|
|
The reference harness does exactly this on Chez: `host/chez/run-corpus.ss` (the
|
|
analyzer runs on Chez → Chez runtime), with a regression floor. Run it via `make
|
|
corpus`.
|
|
|
|
## Maintaining the spec
|
|
|
|
- **Add/change cases**: edit `test/chez/corpus.edn` directly, then re-source the
|
|
answers with `regen-corpus.clj`.
|
|
- **Re-certify**: `clojure -M test/conformance/certify.clj`. A new divergence is
|
|
either a real bug (file it, mark the allowlist entry `:bug` + `:bead`) or a
|
|
deliberate delta (classify it in `known-divergences.edn`).
|
|
- **Refresh the profile**: re-run with `--profile test/conformance/profile.edn`.
|
|
- **Re-floor the runtime gate** when parity rises (`host/chez/run-corpus.ss`).
|