# The jolt conformance spec

This directory defines jolt's behavior as a **host-neutral, executable language
specification**: a data file of cases, certified against reference Clojure, with a
feature profile that lets any runtime declare a conformance *level*. The goal is to
make hosting jolt on a new runtime (and proving it correct) a mechanical exercise:
read one data file, run each case, compare, report.

## The artifacts

| File | Role | Generated by |
|------|------|--------------|
| `test/chez/corpus.edn` | **The spec.** ~2900 cases of `{:suite :label :expected :actual}`. | `test/chez/extract-corpus.janet` |
| `test/conformance/profile.edn` | Per-case **feature classification** — which non-portable cases need which host capability. | `certify.clj --profile` |
| `test/conformance/known-divergences.edn` | Curated allowlist of cases whose `:expected` deliberately differs from JVM Clojure (+ tracked bugs). | hand-maintained |
| `test/conformance/certify.clj` | Certifies `:expected` against reference **JVM Clojure**; gates on new/stale divergences; emits the profile. | — |

The corpus is *generated* from `test/spec/*-spec.janet` and the inline cases in
`test/integration/conformance-test.janet` — those are the authoring convenience.
**`corpus.edn` is the canonical contract**: it is what every runtime consumes, and
what `certify.clj` certifies. A new runtime never needs to read Janet.

## Row schema

```edn
{:suite    "numbers / arithmetic"      ; grouping; "<suite> :: <label>" is the case id
 :label    "integer add"               ; unique within a suite
 :actual   "(+ 1 2)"                    ; Clojure source to evaluate
 :expected "3"}                         ; Clojure source whose value it must equal,
                                        ; or the keyword :throws
```

- `[:suite :label]` is the **canonical, unique case id** (the generator
  disambiguates duplicate labels with ` (N)`).
- Comparison is **value-equality** (`=`), never string/printed-form — so map/set
  iteration order never matters.
- `:expected :throws` asserts evaluating `:actual` raises.

## The oracle: reference JVM Clojure

Historically every `:expected` was hand-written. `certify.clj` removes that
weakness: it evaluates every `:actual` (and `:expected`) on **JVM Clojure** in a
fresh `user` namespace and checks jolt's `:expected` against what real Clojure
produces. Of ~2740 vanilla-certifiable rows, **>2660 match reference Clojure
exactly**. The rest are classified (see below) — none are silently wrong.

```sh
clojure -M test/conformance/certify.clj                                  # gate
clojure -M test/conformance/certify.clj test/chez/corpus.edn --edn r.edn # + report
clojure -M test/conformance/certify.clj test/chez/corpus.edn --profile test/conformance/profile.edn
```

The gate fails only on a **new** (unclassified) divergence or a **stale**
allowlist entry; flaky timing-dependent cases (`future-cancel`) are tolerated.

## Conformance levels & the feature profile

Not every case is portable: some assume a host capability jolt has on one runtime
but not another (Java interop, real threads, BigDecimal). `profile.edn` classifies
each **non-portable** case by the feature it requires. Cases *not* in the profile
are **portable** — they must pass on any faithful Clojure.

A runtime's **conformance level** = portable cases + the feature families it
implements. Current profile (≈2670 portable, ≈249 non-portable):

| Feature | Meaning |
|---------|---------|
| `:numerics/double-only` | all-double numeric model — no Ratio/BigDecimal/float; `(/ 1 2)` ⇒ `0.5` |
| `:concurrency/snapshot` | isolated-heap futures/agents/pmap — captured atoms are snapshotted, not shared |
| `:host/jvm-interop` | Java classes / `instance?` on host classes / proxy / bean / definterface |
| `:host/arrays` | Java arrays (`into-array`, `int-array`, …) |
| `:host/janet` | Janet host interop (`janet.*`) |
| `:async/core-async` | `clojure.core.async` channels/`go` |
| `:runtime/eval` | runtime `eval` / `load-string` |
| `:reader/jolt` | jolt reader features (`#?(:jolt …)`) + syntax-quote literal collapse |
| `:printer/jolt` | jolt's rendering of transients/atoms/`print-method` overrides |
| `:strictness/jolt` | intentionally stricter (throws on odd `assoc!` args, etc.) |
| `:impl/representation` | representation detail (e.g. syntax-quote yields a `list?`, not a `Cons`) |
| `:bug` | a *known defect* (tracked bead) — not a host difference |

## Hosting jolt on a new runtime

1. Implement the reader + analyzer + a backend for your runtime (see the Chez port
   under `host/chez/` for a worked example).
2. Write a ~30-line harness that, for each corpus row, evaluates `:actual` and
   `:expected` and compares by value-equality (skip `:throws` rows to an
   expect-raises check). Pseudocode:

   ```
   (doseq [{:keys [suite label actual expected]} (read-edn "test/chez/corpus.edn")]
     (let [feats (profile-features [suite label])]   ; from profile.edn
       (when (subset? feats my-implemented-features)  ; only cases I claim to support
         (record! [suite label]
                  (if (= :throws expected)
                    (raises? actual)
                    (value= (eval actual) (eval expected)))))))
   ```

3. Run it. Your **conformance level** is the set of feature families with no
   failures. Portable-only is the floor; each feature you implement raises it.

The two reference harnesses already do exactly this on Chez:
`test/chez/run-corpus-prelude.janet` (Janet analyzer → Chez runtime) and
`test/chez/run-corpus-zero-janet.janet` (Chez analyzer → Chez runtime), both with a
regression floor.

## Maintaining the spec

- **Add/change cases**: edit `test/spec/*-spec.janet` or `conformance-test.janet`,
  then `janet test/chez/extract-corpus.janet` to regenerate `corpus.edn`.
- **Re-certify**: `clojure -M test/conformance/certify.clj`. A new divergence is
  either a real bug (file it, mark the allowlist entry `:bug` + `:bead`) or a
  deliberate delta (classify it in `known-divergences.edn`).
- **Refresh the profile**: re-run with `--profile test/conformance/profile.edn`.
- **Re-floor the runtime gates** when parity rises (`run-corpus-*.janet`).