simon/jolt
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
jolt/test/conformance
History
Yogthos 58d03d67be Delete the Janet host — Chez is the sole substrate 
Remove the Janet seed (src/jolt/*.janet: reader, value layer, vars/ns, the
tree-walking interpreter, the Janet backend, the optimizing compiler), the
Janet->Scheme cross-compiler (host/chez/{driver,emit,jolt-chez}.janet),
bin/jolt-chez, the jpm build (project.janet) and the Janet test runner
(run-tests.janet), plus the entire Janet test suite. jolt now builds and runs
on Chez alone: bin/joltc off the checked-in seed, bootstrap.ss to rebuild it.

The portable Clojure stays: jolt-core/**, host/chez/**.ss, and the stdlib +
tooling under src/jolt/clojure + src/jolt/jolt (read by the seed build, no
Janet). The gate is 'make test' (self-host, corpus, unit, cli smoke, certify).
Drop the sci and clojure-test-suite submodules (used only by deleted Janet
integration tests); irregex stays.

Filesystem corpus/unit cases that probed project.janet now probe README.md.

jolt-cf1q.6
2026-06-21 11:29:03 -04:00
..
certify.clj Conformance inc3: promote the corpus to a documented portable spec 2026-06-20 10:21:09 -04:00
known-divergences.edn Source the conformance corpus from JVM Clojure; retire the prelude gate 2026-06-21 01:45:04 -04:00
profile.edn Fix 4 clojure.core bugs surfaced by JVM certification 2026-06-20 11:06:33 -04:00
README.md Conformance inc3: promote the corpus to a documented portable spec 2026-06-20 10:21:09 -04:00
regen-corpus.clj Source the conformance corpus from JVM Clojure; retire the prelude gate 2026-06-21 01:45:04 -04:00
SPEC.md Source the conformance corpus from JVM Clojure; retire the prelude gate 2026-06-21 01:45:04 -04:00

README.md

Conformance: certifying the corpus against reference Clojure

See SPEC.md for the full host-neutral language-spec contract: the corpus schema, conformance levels, the feature profile, and how to host jolt on a new runtime. This README covers the certification tooling specifically.

The corpus (test/chez/corpus.edn) is jolt's host-neutral behavioral suite — one row per case: {:suite :label :expected :actual}, where :actual is a Clojure source expression and :expected its result (or :throws). Runtime harnesses (test/chez/run-corpus-prelude.janet, run-corpus-zero-janet.janet) replay it on each host and compare by value-equality.

Historically every :expected was hand-written by jolt developers. That makes the corpus a fine regression suite but a weak specification: it certifies jolt against its authors' beliefs, not against Clojure. This directory closes that gap.

What's here

  • certify.clj — runs every corpus row's :actual and :expected through reference JVM Clojure (each in a fresh user namespace, output/stdin sunk, a 5s per-case watchdog) and compares with Clojure's =. It buckets each row:

    • certified / certified-throws — jolt's :expected matches real Clojure
    • divergent — both evaluate but jolt's :expected disagrees with Clojure
    • throws-mismatch — jolt and Clojure disagree on whether it throws
    • jvm-error — :actual isn't runnable on vanilla Clojure (host-coupled / jolt-specific) — informational, not certifiable
    • read-error / timeout — won't read on the JVM reader, or ran too long

  • known-divergences.edn — every current divergence, classified. Most are deliberate jolt-specific or host-model deltas (see :legend): the all-double numeric model, snapshot-heap concurrency, the no-JVM host model, jolt reader features, the jolt printer, intentional strictness. A few are genuine :bug entries with a tracked bead. These categories become the :features flags in conformance inc3.

  • certify-test.janet — gate wrapper. Skips cleanly when clojure (JVM) is not installed; otherwise runs certify.clj and fails the build on a NEW (unclassified) divergence or a stale allowlist entry. Flaky entries (JVM result is timing-dependent, e.g. future-cancel) are tolerated either way.

Running

clojure -M test/conformance/certify.clj                      # gate (exit≠0 on new/stale)
clojure -M test/conformance/certify.clj test/chez/corpus.edn --edn /tmp/report.edn  # full machine-readable report
janet test/conformance/certify-test.janet                    # the gate wrapper

Current state

Of ~2487 vanilla-certifiable rows, >2410 match reference Clojure exactly; the ~70 divergences are all classified (deliberate deltas + 4 tracked bugs). The corpus is trustworthy as a spec, with the host-specific deltas made explicit rather than hidden.

Adding / changing cases

When you add corpus rows or change behavior, re-run the certifier. A NEW divergence means either a real bug (file it, tag the allowlist entry :bug + :bead) or a deliberate delta (classify it). A stale entry means a divergence was fixed — remove it from known-divergences.edn.