Docs + CI for the Chez-only substrate
Rewrite the README, CLAUDE.md build/architecture sections, test/chez/README, and conformance SPEC for the Janet-free world: bin/joltc + make test, the self-hosting bootstrap, the frozen JVM-sourced corpus. CI installs Chez + JDK/ Clojure and runs 'make test' (was Janet/jpm). jolt-cf1q.6
This commit is contained in:
parent
58d03d67be
commit
750ce05716
5 changed files with 169 additions and 386 deletions
|
|
@ -1,60 +1,41 @@
|
|||
# Chez test harness
|
||||
|
||||
The correctness gate for jolt on Chez (epic jolt-cf1q). Correctness is judged
|
||||
against the JVM-sourced conformance spec — no Janet decides pass/fail. The spec
|
||||
itself lives in `test/conformance/` (see its `SPEC.md`); this directory holds the
|
||||
Chez runners and unit tests.
|
||||
The correctness gate for jolt. Pure Chez (+ Clojure for the JVM oracle), no Janet.
|
||||
Correctness is judged against the JVM-sourced conformance spec; the spec itself
|
||||
lives in `test/conformance/` (see its `SPEC.md`). Run the whole gate with `make
|
||||
test` from the repo root.
|
||||
|
||||
## The spec corpus
|
||||
|
||||
- `corpus.edn` — the contract: ~2920 rows `{:suite :label :expected :actual}`,
|
||||
with `:expected` sourced from reference JVM Clojure. Valid as both EDN and Janet
|
||||
data. Generated by `test/conformance/regen-corpus.clj` (the answers) over the
|
||||
case list from `extract-corpus.janet` (the `:actual` strings, pulled from
|
||||
`test/spec/*.janet` and `conformance-test.janet`).
|
||||
- `extract-corpus.janet` — re-derives the case list when spec rows change. Writes
|
||||
only when `JOLT_EXTRACT_CORPUS_OUT` is set, so a gate run never clobbers the
|
||||
JVM-sourced `corpus.edn`. After re-deriving, re-source the answers with
|
||||
`regen-corpus.clj`.
|
||||
`corpus.edn` is the contract: ~2920 rows `{:suite :label :expected :actual}`, with
|
||||
`:expected` sourced from reference JVM Clojure by `test/conformance/regen-corpus.clj`.
|
||||
It is frozen (the canonical source) — add or change cases here, then re-source the
|
||||
answers with `regen-corpus.clj` and re-certify with `test/conformance/certify.clj`.
|
||||
|
||||
## The standing gate
|
||||
## The gate runners (`host/chez/`)
|
||||
|
||||
`run-corpus-zero-janet.janet` runs every corpus case through the zero-Janet
|
||||
spine: read → analyze → IR → emit → eval, all on Chez (the analyzer is
|
||||
`jolt.analyzer` cross-compiled to Scheme over `host-contract.ss`). Each result is
|
||||
compared by value-equality against the JVM `:expected`. A `known-fail` allowlist
|
||||
covers cases jolt can't match because Chez has no JVM host (Java classes, arrays,
|
||||
`BigDecimal`, opaque host-object printers, …); the gate fails only on a NEW
|
||||
divergence or if the pass count drops below the floor.
|
||||
- `run-corpus.ss` — runs every corpus case through the spine (read → analyze → IR →
|
||||
emit → eval, all on Chez), comparing each result by value-equality against the JVM
|
||||
`:expected`. A `known-fail` allowlist covers cases jolt can't match because Chez has
|
||||
no JVM host (Java classes, arrays, `BigDecimal`, opaque host-object printers, …);
|
||||
the gate fails only on a NEW divergence or if the pass count drops below the floor.
|
||||
|
||||
JOLT_CHEZ_ZEROJANET_CORPUS=1 janet test/chez/run-corpus-zero-janet.janet
|
||||
JOLT_CORPUS_LIMIT=200 … # every-Nth stride, fast iteration
|
||||
chez --script host/chez/run-corpus.ss
|
||||
JOLT_CORPUS_LIMIT=200 … # every-Nth stride, fast iteration
|
||||
JOLT_CHEZ_ZJ_FLOOR=N … # override the floor (default 2678)
|
||||
|
||||
Floor: 2678 (`JOLT_CHEZ_ZJ_FLOOR` overrides). Raise it as host gaps close.
|
||||
- `run-unit.ss` — host-specific unit cases (`test/chez/unit.edn`) that aren't in the
|
||||
JVM-portable corpus: dot-forms, java statics, io, reader, walk, vars/namespaces,
|
||||
refs. Each `:expr` is evaluated in-process and its printed value compared to a baked
|
||||
`:expected` (`:throws` asserts a raise).
|
||||
|
||||
## Unit tests
|
||||
- `selfcheck.sh` — self-host fixpoint: `bootstrap.ss` rebuild byte-equals the
|
||||
checked-in seed (`host/chez/seed/`).
|
||||
- `smoke.sh` — real `bin/joltc -e` CLI smoke.
|
||||
|
||||
`_*.janet` drive `bin/joltc` (the pure-Chez CLI; `JOLT_BIN` overrides) one
|
||||
subprocess per case and compare the last stdout line against a baked expected
|
||||
value. `:throws` asserts a non-zero exit. Each file targets one area —
|
||||
`_type`/`_class`/`_seqpred` (taxonomy + predicates), `_str`/`_strns`/`_reader`
|
||||
(strings + reader), `_io`/`_ioreader`/`_insttime`/`_javastatic` (host interop),
|
||||
`_dynbind`/`_var_meta`/`_ns` (vars + namespaces), `_atomwatch`/`_walk`/`_dotform`
|
||||
/`_stdlib` (refs, walk, dot-forms, stdlib). Run one with
|
||||
`janet test/chez/_type.janet`.
|
||||
## Other Chez tests
|
||||
|
||||
## Self-host gates
|
||||
- `values-test.ss` — the value model (nil/truthiness/collections). `make values`.
|
||||
- `bench-chez.ss` — compute bench through the pipeline (opt-in; not in the gate).
|
||||
|
||||
- `spine-test.janet` — the zero-Janet compile/eval spine end to end.
|
||||
- `cli-test.janet` — `bin/joltc -e` behavior.
|
||||
- `bootstrap-test.janet` — `host/chez/bootstrap.ss` rebuilds the prelude + image
|
||||
from source on Chez and matches the checked-in seed (`host/chez/seed/`).
|
||||
- `fixpoint-test.janet` — the on-Chez compiler reproduces itself (stage2 == stage3,
|
||||
prelude pstage3 == pstage4).
|
||||
|
||||
## Bench
|
||||
|
||||
`bench-pipeline.janet` (opt-in via `JOLT_CHEZ_BENCH=1`) times fib + mandelbrot
|
||||
through the real pipeline against the spike ceiling. Skipped in the gate.
|
||||
|
||||
All Chez runners skip cleanly when `chez` is not on PATH.
|
||||
All runners assume `chez` on PATH.
|
||||
|
|
|
|||
|
|
@ -17,10 +17,9 @@ read one data file, run each case, compare, report.
|
|||
| `test/conformance/certify.clj` | Certifies `:expected` against reference **JVM Clojure**; gates on new/stale divergences; emits the profile. | — |
|
||||
|
||||
`corpus.edn` is **JVM-sourced**: `regen-corpus.clj` evaluates each case's `:actual`
|
||||
on reference JVM Clojure and writes the JVM value as `:expected`. The case *list*
|
||||
(the `:actual` strings) comes from `test/spec/*-spec.janet` via `extract-corpus.janet`,
|
||||
but the *answers* are the JVM's. **`corpus.edn` is the canonical contract**: it is
|
||||
what every runtime consumes and what `certify.clj` certifies.
|
||||
on reference JVM Clojure and writes the JVM value as `:expected`. **`corpus.edn` is
|
||||
the canonical, frozen contract**: it is what every runtime consumes, what
|
||||
`certify.clj` certifies, and where new cases are authored directly.
|
||||
|
||||
## Row schema
|
||||
|
||||
|
|
@ -101,17 +100,16 @@ implements. Current profile (≈2670 portable, ≈249 non-portable):
|
|||
3. Run it. Your **conformance level** is the set of feature families with no
|
||||
failures. Portable-only is the floor; each feature you implement raises it.
|
||||
|
||||
The two reference harnesses already do exactly this on Chez:
|
||||
`test/chez/run-corpus-prelude.janet` (Janet analyzer → Chez runtime) and
|
||||
`test/chez/run-corpus-zero-janet.janet` (Chez analyzer → Chez runtime), both with a
|
||||
regression floor.
|
||||
The reference harness does exactly this on Chez: `host/chez/run-corpus.ss` (the
|
||||
analyzer runs on Chez → Chez runtime), with a regression floor. Run it via `make
|
||||
corpus`.
|
||||
|
||||
## Maintaining the spec
|
||||
|
||||
- **Add/change cases**: edit `test/spec/*-spec.janet` or `conformance-test.janet`,
|
||||
then `janet test/chez/extract-corpus.janet` to regenerate `corpus.edn`.
|
||||
- **Add/change cases**: edit `test/chez/corpus.edn` directly, then re-source the
|
||||
answers with `regen-corpus.clj`.
|
||||
- **Re-certify**: `clojure -M test/conformance/certify.clj`. A new divergence is
|
||||
either a real bug (file it, mark the allowlist entry `:bug` + `:bead`) or a
|
||||
deliberate delta (classify it in `known-divergences.edn`).
|
||||
- **Refresh the profile**: re-run with `--profile test/conformance/profile.edn`.
|
||||
- **Re-floor the runtime gates** when parity rises (`run-corpus-*.janet`).
|
||||
- **Re-floor the runtime gate** when parity rises (`host/chez/run-corpus.ss`).
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue