750ce05716
Rewrite the README, CLAUDE.md build/architecture sections, test/chez/README, and conformance SPEC for the Janet-free world: bin/joltc + make test, the self-hosting bootstrap, the frozen JVM-sourced corpus. CI installs Chez + JDK/ Clojure and runs 'make test' (was Janet/jpm). jolt-cf1q.6
115 lines
6.3 KiB
Markdown
115 lines
6.3 KiB
Markdown
# The jolt conformance spec
|
|
|
|
This directory defines jolt's behavior as a **host-neutral, executable language
|
|
specification**: a data file of cases, certified against reference Clojure, with a
|
|
feature profile that lets any runtime declare a conformance *level*. The goal is to
|
|
make hosting jolt on a new runtime (and proving it correct) a mechanical exercise:
|
|
read one data file, run each case, compare, report.
|
|
|
|
## The artifacts
|
|
|
|
| File | Role | Generated by |
|
|
|------|------|--------------|
|
|
| `test/chez/corpus.edn` | **The spec.** ~2900 cases of `{:suite :label :expected :actual}`, `:expected` **sourced from reference JVM Clojure**. | `test/conformance/regen-corpus.clj` |
|
|
| `test/conformance/profile.edn` | Per-case **feature classification** — which non-portable cases need which host capability. | `certify.clj --profile` |
|
|
| `test/conformance/known-divergences.edn` | The few rows whose JVM value is an opaque host object that can't round-trip to readable source (Java arrays/transients/atoms/beans/proxies print as `#object[..@addr]`), so the corpus keeps jolt's value. | `regen-corpus.clj` leftovers, hand-checked |
|
|
| `test/conformance/regen-corpus.clj` | Sources every `:expected` from reference **JVM Clojure** in one process. | — |
|
|
| `test/conformance/certify.clj` | Certifies `:expected` against reference **JVM Clojure**; gates on new/stale divergences; emits the profile. | — |
|
|
|
|
`corpus.edn` is **JVM-sourced**: `regen-corpus.clj` evaluates each case's `:actual`
|
|
on reference JVM Clojure and writes the JVM value as `:expected`. **`corpus.edn` is
|
|
the canonical, frozen contract**: it is what every runtime consumes, what
|
|
`certify.clj` certifies, and where new cases are authored directly.
|
|
|
|
## Row schema
|
|
|
|
```edn
|
|
{:suite "numbers / arithmetic" ; grouping; "<suite> :: <label>" is the case id
|
|
:label "integer add" ; unique within a suite
|
|
:actual "(+ 1 2)" ; Clojure source to evaluate
|
|
:expected "3"} ; Clojure source whose value it must equal,
|
|
; or the keyword :throws
|
|
```
|
|
|
|
- `[:suite :label]` is the **canonical, unique case id** (the generator
|
|
disambiguates duplicate labels with ` (N)`).
|
|
- Comparison is **value-equality** (`=`), never string/printed-form — so map/set
|
|
iteration order never matters.
|
|
- `:expected :throws` asserts evaluating `:actual` raises.
|
|
|
|
## The oracle: reference JVM Clojure
|
|
|
|
Historically every `:expected` was hand-written. `certify.clj` removes that
|
|
weakness: it evaluates every `:actual` (and `:expected`) on **JVM Clojure** in a
|
|
fresh `user` namespace and checks jolt's `:expected` against what real Clojure
|
|
produces. Of ~2740 vanilla-certifiable rows, **>2660 match reference Clojure
|
|
exactly**. The rest are classified (see below) — none are silently wrong.
|
|
|
|
```sh
|
|
clojure -M test/conformance/certify.clj # gate
|
|
clojure -M test/conformance/certify.clj test/chez/corpus.edn --edn r.edn # + report
|
|
clojure -M test/conformance/certify.clj test/chez/corpus.edn --profile test/conformance/profile.edn
|
|
```
|
|
|
|
The gate fails only on a **new** (unclassified) divergence or a **stale**
|
|
allowlist entry; flaky timing-dependent cases (`future-cancel`) are tolerated.
|
|
|
|
## Conformance levels & the feature profile
|
|
|
|
Not every case is portable: some assume a host capability jolt has on one runtime
|
|
but not another (Java interop, real threads, BigDecimal). `profile.edn` classifies
|
|
each **non-portable** case by the feature it requires. Cases *not* in the profile
|
|
are **portable** — they must pass on any faithful Clojure.
|
|
|
|
A runtime's **conformance level** = portable cases + the feature families it
|
|
implements. Current profile (≈2670 portable, ≈249 non-portable):
|
|
|
|
| Feature | Meaning |
|
|
|---------|---------|
|
|
| `:numerics/double-only` | all-double numeric model — no Ratio/BigDecimal/float; `(/ 1 2)` ⇒ `0.5` |
|
|
| `:concurrency/snapshot` | isolated-heap futures/agents/pmap — captured atoms are snapshotted, not shared |
|
|
| `:host/jvm-interop` | Java classes / `instance?` on host classes / proxy / bean / definterface |
|
|
| `:host/arrays` | Java arrays (`into-array`, `int-array`, …) |
|
|
| `:host/janet` | Janet host interop (`janet.*`) |
|
|
| `:async/core-async` | `clojure.core.async` channels/`go` |
|
|
| `:runtime/eval` | runtime `eval` / `load-string` |
|
|
| `:reader/jolt` | jolt reader features (`#?(:jolt …)`) + syntax-quote literal collapse |
|
|
| `:printer/jolt` | jolt's rendering of transients/atoms/`print-method` overrides |
|
|
| `:strictness/jolt` | intentionally stricter (throws on odd `assoc!` args, etc.) |
|
|
| `:impl/representation` | representation detail (e.g. syntax-quote yields a `list?`, not a `Cons`) |
|
|
| `:bug` | a *known defect* (tracked bead) — not a host difference |
|
|
|
|
## Hosting jolt on a new runtime
|
|
|
|
1. Implement the reader + analyzer + a backend for your runtime (see the Chez port
|
|
under `host/chez/` for a worked example).
|
|
2. Write a ~30-line harness that, for each corpus row, evaluates `:actual` and
|
|
`:expected` and compares by value-equality (skip `:throws` rows to an
|
|
expect-raises check). Pseudocode:
|
|
|
|
```
|
|
(doseq [{:keys [suite label actual expected]} (read-edn "test/chez/corpus.edn")]
|
|
(let [feats (profile-features [suite label])] ; from profile.edn
|
|
(when (subset? feats my-implemented-features) ; only cases I claim to support
|
|
(record! [suite label]
|
|
(if (= :throws expected)
|
|
(raises? actual)
|
|
(value= (eval actual) (eval expected)))))))
|
|
```
|
|
|
|
3. Run it. Your **conformance level** is the set of feature families with no
|
|
failures. Portable-only is the floor; each feature you implement raises it.
|
|
|
|
The reference harness does exactly this on Chez: `host/chez/run-corpus.ss` (the
|
|
analyzer runs on Chez → Chez runtime), with a regression floor. Run it via `make
|
|
corpus`.
|
|
|
|
## Maintaining the spec
|
|
|
|
- **Add/change cases**: edit `test/chez/corpus.edn` directly, then re-source the
|
|
answers with `regen-corpus.clj`.
|
|
- **Re-certify**: `clojure -M test/conformance/certify.clj`. A new divergence is
|
|
either a real bug (file it, mark the allowlist entry `:bug` + `:bead`) or a
|
|
deliberate delta (classify it in `known-divergences.edn`).
|
|
- **Refresh the profile**: re-run with `--profile test/conformance/profile.edn`.
|
|
- **Re-floor the runtime gate** when parity rises (`host/chez/run-corpus.ss`).
|