Commit graph

2 commits

Author SHA1 Message Date
Yogthos
635bbbcc27 Conformance inc2: fold the 355 inline conformance cases into the corpus
The corpus only ever saw test/spec/*-spec.janet; the 355 hand-written cases in
test/integration/conformance-test.janet (inline Janet, the lazy-seq / IFn /
destructuring / transducer essentials) were invisible to it and to any non-Janet
runtime. extract-corpus.janet now also pulls that (def cases ...) vector, deduped
by :actual, organized into 41 'conformance / <section>' suites recovered from the
file's ### headers. Corpus 2658 -> 2919 rows (+261 unique).

JVM certification: only 1 new divergence ((/ 2) => 0.5 vs 1/2, the all-double
numeric model) — classified. Chez gates: +1 known host gap (instance? Atom, atom
class identity, Phase 4) allowlisted in both runners; parity rose 2295 -> 2533 on
both, floors raised. Janet gate 155 files 0 failed; certifier green (0 new/stale).

Deferred: 41 non-literal core-async spec rows ((a "src") async-harness wrapper)
need harness context the corpus format doesn't carry — left for inc3.
2026-06-20 10:10:32 -04:00
Yogthos
b520eefa7a Conformance inc1: JVM-certify the corpus against reference Clojure
The corpus carried hand-written :expected values — a regression suite but a weak
spec (it checked jolt against its authors, not against Clojure). certify.clj runs
every corpus row's :actual and :expected through reference JVM Clojure 1.12.5 (fresh
user ns per case, output/stdin sunk, 5s per-case watchdog) and compares with =.

Result: of ~2487 vanilla-certifiable rows, 2416 match real Clojure exactly. The 71
divergences are all classified in known-divergences.edn — mostly deliberate
jolt-specific/host-model deltas (all-double numerics, snapshot concurrency, no-JVM
host model, jolt reader features, printer, strictness), plus 4 genuine bugs filed
as beads (jolt-l8e8 ex-message, jolt-hc35 munge, jolt-pqio print-nil,
jolt-2507 bounded-count).

certify-test.janet gates it: skips without clojure on PATH, else fails only on a
NEW (unclassified) divergence or stale allowlist entry; flaky timing-dependent
cases (future-cancel) tolerated either way. Full gate 155 files 0 failed.
2026-06-20 09:38:31 -04:00