Vendor clojure-test-suite as a standing gate (make cts)

jank-lang/clojure-test-suite (per-core-fn clojure.test suites shared across
Clojure dialects) joins the default gate as vendor/clojure-test-suite, run by
host/chez/cts.sh: one joltc process per test namespace (a hang or crash is
contained by a per-process timeout), through the test/chez/cts-app project and
its cts-run runner, parallel workers.

Gating is exact per namespace against test/chez/cts-known-failures.txt, like
certify's allowlist: a namespace doing worse than the baseline fails, and one
doing better also fails as stale until the baseline is updated in the same
change. JOLT_CTS_WRITE_BASELINE=1 regenerates it; JOLT_CTS_NS runs a subset
verbosely.

Current standing: 243 namespaces, 5302 assertions pass, 340 fail + 236 error
across 88 namespaces pinned in the baseline (dominant clusters: BigDecimal
arithmetic operands, derive/ancestors hierarchy, transients, special-symbol?,
clojure.string case fns, the accepted narrow-int and seq-type-model
divergences). Two consecutive full runs produce identical counts. Wired into
make ci; skips cleanly when the submodule isn't checked out.
This commit is contained in:
Yogthos 2026-07-02 03:46:57 -04:00
parent 8ffc2f68c5
commit edfd67a322
6 changed files with 249 additions and 2 deletions

View file

@ -0,0 +1,2 @@
{:paths ["src" "../../../vendor/clojure-test-suite/test"]
:aliases {:cts {:main-opts ["-m" "cts-run"]}}}

View file

@ -0,0 +1,18 @@
(ns cts-run
"Runner for the vendored jank-lang/clojure-test-suite (vendor/clojure-test-suite):
requires each test namespace given on the command line and runs its clojure.test
tests, printing one machine-readable result line per namespace. Driven per-process
by host/chez/cts.sh so a hang or crash in one namespace can't take out the run."
(:require [clojure.test :as t]))
(defn -main [& nses]
(doseq [n nses]
(let [ns-sym (symbol n)]
(try
(require ns-sym)
(let [r (t/run-tests ns-sym)]
(println "CTS-RESULT" n (:pass r 0) (:fail r 0) (:error r 0)))
(catch Throwable e
(println "CTS-RESULT" n 0 0 1
(str "LOAD: " (.getName (class e)) ": " (.getMessage e)))))))
(System/exit 0))