33eff7c7d8
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the seed-baked layer) and update the loader / emit-image / doc paths. Drop dead code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves to tools/), the Janet-era jolt.http binding, and the orphaned persistent_vector.clj whose ns/path didn't even match. Strip porting residue from comments and docstrings across host/chez, jolt-core, stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs Janet" historical exposition, leaving present-tense descriptions and the real JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and corpus are untouched. Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the conformance profile. Add docs/host-interop.md (the JVM shims and how to register your own host class from a library) and a writing-style note in CLAUDE.md. Stabilize the four racy concurrency corpus cases (future-cancel and agent send/send-off): give the future a sleeping body and the agent a slow action, so cancel reliably catches an in-flight future and deref reliably reads the pre-update snapshot. They certify deterministically now, so drop their :flaky allowlist entries and the orphaned legend.
63 lines
3.2 KiB
Markdown
63 lines
3.2 KiB
Markdown
# Conformance: certifying the corpus against reference Clojure
|
|
|
|
> **See [SPEC.md](SPEC.md)** for the full host-neutral language-spec contract: the
|
|
> corpus schema, conformance levels, the feature profile, and how to host jolt on a
|
|
> new runtime. This README covers the certification tooling specifically.
|
|
|
|
|
|
The corpus (`test/chez/corpus.edn`) is jolt's host-neutral behavioral suite — one
|
|
row per case: `{:suite :label :expected :actual}`, where `:actual` is a Clojure
|
|
source expression and `:expected` its result (or `:throws`). The runtime harness
|
|
(`host/chez/run-corpus.ss`, invoked by `make corpus`) replays it on Chez and
|
|
compares by value-equality.
|
|
|
|
Every `:expected` is sourced from reference JVM Clojure, so the corpus is both a
|
|
regression suite and a *specification* certified against Clojure rather than
|
|
against its authors' beliefs. This directory holds the certification tooling that
|
|
closes that gap.
|
|
|
|
## What's here
|
|
|
|
- **`certify.clj`** — runs every corpus row's `:actual` and `:expected` through
|
|
**reference JVM Clojure** (each in a fresh `user` namespace, output/stdin sunk, a
|
|
5s per-case watchdog) and compares with Clojure's `=`. It buckets each row:
|
|
- `certified` / `certified-throws` — jolt's `:expected` matches real Clojure
|
|
- `divergent` — both evaluate but jolt's `:expected` disagrees with Clojure
|
|
- `throws-mismatch` — jolt and Clojure disagree on whether it throws
|
|
- `jvm-error` — `:actual` isn't runnable on vanilla Clojure (host-coupled /
|
|
jolt-specific) — informational, not certifiable
|
|
- `read-error` / `timeout` — won't read on the JVM reader, or ran too long
|
|
|
|
- **`known-divergences.edn`** — every current divergence, classified. Most are
|
|
**deliberate** jolt-specific or host-model deltas (see `:legend`): the all-double
|
|
numeric model, snapshot-heap concurrency, the no-JVM host model, jolt reader
|
|
features, the jolt printer, intentional strictness. A few are genuine **`:bug`**
|
|
entries with a tracked bead. These categories become the `:features` flags in
|
|
conformance inc3.
|
|
|
|
`make certify` is the gate wrapper. It skips cleanly when `clojure` (JVM) is not
|
|
installed; otherwise it runs `certify.clj` and fails the build on a **NEW**
|
|
(unclassified) divergence or a **stale** allowlist entry. Flaky entries (JVM
|
|
result is timing-dependent, e.g. `future-cancel`) are tolerated either way.
|
|
|
|
## Running
|
|
|
|
```sh
|
|
make certify # the gate wrapper (skips if clojure absent)
|
|
clojure -M test/conformance/certify.clj # gate directly (exit≠0 on new/stale)
|
|
clojure -M test/conformance/certify.clj test/chez/corpus.edn --edn /tmp/report.edn # full machine-readable report
|
|
```
|
|
|
|
## Current state
|
|
|
|
Of ~2740 vanilla-certifiable rows, **>2730 match reference Clojure exactly**; the
|
|
handful of divergences are all classified (deliberate deltas plus a few tracked
|
|
bugs). The corpus is trustworthy as a spec, with the host-specific deltas made
|
|
explicit rather than hidden.
|
|
|
|
## Adding / changing cases
|
|
|
|
When you add corpus rows or change behavior, re-run the certifier. A NEW divergence
|
|
means either a real bug (file it, tag the allowlist entry `:bug` + `:bead`) or a
|
|
deliberate delta (classify it). A stale entry means a divergence was fixed — remove
|
|
it from `known-divergences.edn`.
|