Conformance inc1: JVM-certify the corpus against reference Clojure
The corpus carried hand-written :expected values — a regression suite but a weak spec (it checked jolt against its authors, not against Clojure). certify.clj runs every corpus row's :actual and :expected through reference JVM Clojure 1.12.5 (fresh user ns per case, output/stdin sunk, 5s per-case watchdog) and compares with =. Result: of ~2487 vanilla-certifiable rows, 2416 match real Clojure exactly. The 71 divergences are all classified in known-divergences.edn — mostly deliberate jolt-specific/host-model deltas (all-double numerics, snapshot concurrency, no-JVM host model, jolt reader features, printer, strictness), plus 4 genuine bugs filed as beads (jolt-l8e8 ex-message, jolt-hc35 munge, jolt-pqio print-nil, jolt-2507 bounded-count). certify-test.janet gates it: skips without clojure on PATH, else fails only on a NEW (unclassified) divergence or stale allowlist entry; flaky timing-dependent cases (future-cancel) tolerated either way. Full gate 155 files 0 failed.
This commit is contained in:
Yogthos
parent
d0fce540ed
commit
b520eefa7a
4 changed files with 528 additions and 0 deletions
57
test/conformance/README.md
Normal file
57
test/conformance/README.md
Normal file
|
|
@ -0,0 +1,57 @@
|
||||||
|
# Conformance: certifying the corpus against reference Clojure
|
||||||
|
|
||||||
|
The corpus (`test/chez/corpus.edn`) is jolt's host-neutral behavioral suite — one
|
||||||
|
row per case: `{:suite :label :expected :actual}`, where `:actual` is a Clojure
|
||||||
|
source expression and `:expected` its result (or `:throws`). Runtime harnesses
|
||||||
|
(`test/chez/run-corpus-prelude.janet`, `run-corpus-zero-janet.janet`) replay it on
|
||||||
|
each host and compare by value-equality.
|
||||||
|
|
||||||
|
Historically every `:expected` was **hand-written by jolt developers**. That makes
|
||||||
|
the corpus a fine regression suite but a weak *specification*: it certifies jolt
|
||||||
|
against its authors' beliefs, not against Clojure. This directory closes that gap.
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
- **`certify.clj`** — runs every corpus row's `:actual` and `:expected` through
|
||||||
|
**reference JVM Clojure** (each in a fresh `user` namespace, output/stdin sunk, a
|
||||||
|
5s per-case watchdog) and compares with Clojure's `=`. It buckets each row:
|
||||||
|
- `certified` / `certified-throws` — jolt's `:expected` matches real Clojure
|
||||||
|
- `divergent` — both evaluate but jolt's `:expected` disagrees with Clojure
|
||||||
|
- `throws-mismatch` — jolt and Clojure disagree on whether it throws
|
||||||
|
- `jvm-error` — `:actual` isn't runnable on vanilla Clojure (host-coupled /
|
||||||
|
jolt-specific) — informational, not certifiable
|
||||||
|
- `read-error` / `timeout` — won't read on the JVM reader, or ran too long
|
||||||
|
|
||||||
|
- **`known-divergences.edn`** — every current divergence, classified. Most are
|
||||||
|
**deliberate** jolt-specific or host-model deltas (see `:legend`): the all-double
|
||||||
|
numeric model, snapshot-heap concurrency, the no-JVM host model, jolt reader
|
||||||
|
features, the jolt printer, intentional strictness. A few are genuine **`:bug`**
|
||||||
|
entries with a tracked bead. These categories become the `:features` flags in
|
||||||
|
conformance inc3.
|
||||||
|
|
||||||
|
- **`certify-test.janet`** — gate wrapper. Skips cleanly when `clojure` (JVM) is
|
||||||
|
not installed; otherwise runs `certify.clj` and fails the build on a **NEW**
|
||||||
|
(unclassified) divergence or a **stale** allowlist entry. Flaky entries (JVM
|
||||||
|
result is timing-dependent, e.g. `future-cancel`) are tolerated either way.
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
```sh
|
||||||
|
clojure -M test/conformance/certify.clj # gate (exit≠0 on new/stale)
|
||||||
|
clojure -M test/conformance/certify.clj test/chez/corpus.edn --edn /tmp/report.edn # full machine-readable report
|
||||||
|
janet test/conformance/certify-test.janet # the gate wrapper
|
||||||
|
```
|
||||||
|
|
||||||
|
## Current state
|
||||||
|
|
||||||
|
Of ~2487 vanilla-certifiable rows, **>2410 match reference Clojure exactly**; the
|
||||||
|
~70 divergences are all classified (deliberate deltas + 4 tracked bugs). The corpus
|
||||||
|
is trustworthy as a spec, with the host-specific deltas made explicit rather than
|
||||||
|
hidden.
|
||||||
|
|
||||||
|
## Adding / changing cases
|
||||||
|
|
||||||
|
When you add corpus rows or change behavior, re-run the certifier. A NEW divergence
|
||||||
|
means either a real bug (file it, tag the allowlist entry `:bug` + `:bead`) or a
|
||||||
|
deliberate delta (classify it). A stale entry means a divergence was fixed — remove
|
||||||
|
it from `known-divergences.edn`.
|
||||||
42
test/conformance/certify-test.janet
Normal file
42
test/conformance/certify-test.janet
Normal file
|
|
@ -0,0 +1,42 @@
|
||||||
|
# Conformance inc1 (jolt-xsfe) — gate wrapper for the JVM corpus certifier.
|
||||||
|
#
|
||||||
|
# test/conformance/certify.clj evaluates every test/chez/corpus.edn row through
|
||||||
|
# reference JVM Clojure and checks jolt's hand-written :expected against what real
|
||||||
|
# Clojure produces. Divergences are classified in test/conformance/known-divergences.edn
|
||||||
|
# (deliberate jolt-specific / host-model deltas + tracked bugs); the certifier exits
|
||||||
|
# nonzero only on a NEW (unclassified) divergence or a stale allowlist entry.
|
||||||
|
#
|
||||||
|
# This wrapper runs it in the Janet gate and skips cleanly when `clojure` (JVM) is
|
||||||
|
# not installed — same pattern as the chez tests skipping without `chez`.
|
||||||
|
#
|
||||||
|
# janet test/conformance/certify-test.janet
|
||||||
|
|
||||||
|
(defn- have-clojure? []
|
||||||
|
(def p (try (os/spawn ["clojure" "--version"] :p {:out :pipe :err :pipe}) ([_] nil)))
|
||||||
|
(if (nil? p) false
|
||||||
|
(do (def out (ev/read (p :out) :all)) (def err (ev/read (p :err) :all))
|
||||||
|
(zero? (os/proc-wait p)))))
|
||||||
|
|
||||||
|
(unless (have-clojure?)
|
||||||
|
(print "clojure (JVM) not on PATH — skipping corpus certification")
|
||||||
|
(os/exit 0))
|
||||||
|
|
||||||
|
(def proc (os/spawn ["clojure" "-M" "test/conformance/certify.clj"] :p {:out :pipe :err :pipe}))
|
||||||
|
(def out (string (ev/read (proc :out) :all)))
|
||||||
|
(def err (string (or (ev/read (proc :err) :all) "")))
|
||||||
|
(def code (os/proc-wait proc))
|
||||||
|
|
||||||
|
# Echo the summary lines so the gate log shows the certification status.
|
||||||
|
(each line (string/split "\n" out)
|
||||||
|
(when (or (string/find "certif" line) (string/find "allowlist" line)
|
||||||
|
(string/find "NEW" line) (string/find "STALE" line) (string/find "DIVERGENT" line))
|
||||||
|
(print line)))
|
||||||
|
|
||||||
|
(when (not= code 0)
|
||||||
|
(eprint "corpus certification FAILED (new/stale divergence vs reference Clojure):")
|
||||||
|
(eprint out)
|
||||||
|
(unless (= "" err) (eprint err))
|
||||||
|
(os/exit 1))
|
||||||
|
|
||||||
|
(print "corpus certification: OK (all divergences classified)")
|
||||||
|
(os/exit 0)
|
||||||
191
test/conformance/certify.clj
Normal file
191
test/conformance/certify.clj
Normal file
|
|
@ -0,0 +1,191 @@
|
||||||
|
;; certify.clj (jolt-xsfe) — certify the jolt corpus against reference JVM Clojure.
|
||||||
|
;;
|
||||||
|
;; The corpus (test/chez/corpus.edn) carries hand-written :expected values. This
|
||||||
|
;; script runs each row's :actual and :expected through REAL JVM Clojure and checks
|
||||||
|
;; whether jolt's :expected matches what canonical Clojure produces. It turns the
|
||||||
|
;; corpus from "our test cases" into "a suite pinned to the reference implementation"
|
||||||
|
;; — and surfaces any row where the hand-written answer is actually wrong.
|
||||||
|
;;
|
||||||
|
;; Each row is evaluated in a throwaway namespace so top-level defs don't leak.
|
||||||
|
;; Buckets per row:
|
||||||
|
;; :certified jolt :expected == JVM result (the good case)
|
||||||
|
;; :certified-throws :expected is :throws and JVM also throws
|
||||||
|
;; :divergent both evaluate but jolt :expected != JVM result (CORPUS BUG)
|
||||||
|
;; :throws-mismatch :expected :throws but JVM did NOT throw (or vice versa)
|
||||||
|
;; :jvm-error :actual errors on vanilla Clojure (jolt-specific / host-coupled
|
||||||
|
;; / not certifiable against the JVM) — informational, not a bug
|
||||||
|
;; :read-error :actual or :expected won't even read on the JVM reader
|
||||||
|
;;
|
||||||
|
;; Run from the repo root:
|
||||||
|
;; clojure -M test/conformance/certify.clj [corpus.edn] [--edn out.edn]
|
||||||
|
(ns certify
|
||||||
|
(:require [clojure.edn :as edn]
|
||||||
|
[clojure.string :as str]
|
||||||
|
[clojure.set]
|
||||||
|
[clojure.pprint :as pp]))
|
||||||
|
|
||||||
|
(def corpus-path
|
||||||
|
(or (first (remove #(str/starts-with? % "--") *command-line-args*))
|
||||||
|
"test/chez/corpus.edn"))
|
||||||
|
|
||||||
|
(def edn-out
|
||||||
|
(let [args (vec *command-line-args*)
|
||||||
|
i (.indexOf args "--edn")]
|
||||||
|
(when (and (>= i 0) (< (inc i) (count args))) (nth args (inc i)))))
|
||||||
|
|
||||||
|
;; Classified allowlist of known divergences (deliberate jolt-specific / host-model
|
||||||
|
;; differences + tracked bugs). The gate fails only on a NEW (unlisted) divergence
|
||||||
|
;; or throws-mismatch. Keyed by [suite label].
|
||||||
|
(def allowlist-path "test/conformance/known-divergences.edn")
|
||||||
|
(def allowlist-entries
|
||||||
|
(if (.exists (java.io.File. allowlist-path))
|
||||||
|
(:entries (edn/read-string (slurp allowlist-path)))
|
||||||
|
[]))
|
||||||
|
;; Non-flaky known divergences: gated for both NEW and STALE.
|
||||||
|
(def known
|
||||||
|
(->> allowlist-entries (remove :flaky) (map (juxt :suite :label)) set))
|
||||||
|
;; Flaky entries: the JVM result is inherently nondeterministic (e.g. future-cancel
|
||||||
|
;; racing future completion), so they are always tolerated whether or not they
|
||||||
|
;; diverge on a given run — never NEW, never stale.
|
||||||
|
(def flaky
|
||||||
|
(->> allowlist-entries (filter :flaky) (map (juxt :suite :label)) set))
|
||||||
|
|
||||||
|
;; Read a Clojure source string into a single form, wrapping multi-form bodies in
|
||||||
|
;; (do ...) so a case like "(def x 1) (inc x)" evaluates as one program. Reader
|
||||||
|
;; conditionals are allowed (a few corpus rows carry #?(:clj ...)).
|
||||||
|
(defn read-program [src]
|
||||||
|
(read-string {:read-cond :allow} (str "(do " src ")")))
|
||||||
|
|
||||||
|
;; Evaluate a Clojure source string in a FRESH `user` namespace, with output and
|
||||||
|
;; stdin sunk (a case may println, (time ...), or (read) — none should touch the
|
||||||
|
;; report or block on the terminal). The ns must be named `user` (not a gensym),
|
||||||
|
;; because that is jolt's default ns and the corpus :expected values bake it in:
|
||||||
|
;; *ns*, record print tags (#user.Pt), syntax-quote qualification (user/foo), and
|
||||||
|
;; var print (#'user/v) all render the current ns name. Recreating `user` per case
|
||||||
|
;; both names it correctly AND drops the previous case's defs. Never throws;
|
||||||
|
;; returns [:ok value] / [:throw throwable] / [:read-error throwable].
|
||||||
|
(defn eval-isolated [src]
|
||||||
|
(let [sink (java.io.StringWriter.)
|
||||||
|
empty-in (java.io.PushbackReader. (java.io.StringReader. ""))]
|
||||||
|
(try
|
||||||
|
(remove-ns 'user)
|
||||||
|
(let [the-ns (create-ns 'user)]
|
||||||
|
(binding [*ns* the-ns *out* sink *err* sink *in* empty-in]
|
||||||
|
(clojure.core/refer-clojure)
|
||||||
|
(let [form (try (read-program src)
|
||||||
|
(catch Throwable t (throw (ex-info "read" {::read t}))))]
|
||||||
|
[:ok (eval form)])))
|
||||||
|
(catch clojure.lang.ExceptionInfo e
|
||||||
|
(if (::read (ex-data e)) [:read-error (::read (ex-data e))] [:throw e]))
|
||||||
|
(catch Throwable t [:throw t]))))
|
||||||
|
|
||||||
|
(def ^:const case-timeout-ms 5000)
|
||||||
|
|
||||||
|
;; Per-case wall-clock guard: an infinite lazy seq forced, a blocking read, or a
|
||||||
|
;; deadlocked future would otherwise hang the whole run. Returns [:timeout nil] if
|
||||||
|
;; the case exceeds the budget (the worker thread is cancelled best-effort).
|
||||||
|
(defn eval-safe [src]
|
||||||
|
(let [f (future (eval-isolated src))
|
||||||
|
r (deref f case-timeout-ms ::timeout)]
|
||||||
|
(if (= r ::timeout)
|
||||||
|
(do (future-cancel f) [:timeout nil])
|
||||||
|
r)))
|
||||||
|
|
||||||
|
(defn classify [row]
|
||||||
|
(let [{:keys [expected actual]} row
|
||||||
|
throws? (= expected :throws)
|
||||||
|
a (eval-safe actual)]
|
||||||
|
(cond
|
||||||
|
;; actual exceeded the per-case time budget (infinite seq / blocking / deadlock)
|
||||||
|
(= (first a) :timeout)
|
||||||
|
{:bucket :timeout :detail (str "exceeded " case-timeout-ms "ms")}
|
||||||
|
|
||||||
|
;; actual won't read on the JVM
|
||||||
|
(= (first a) :read-error)
|
||||||
|
{:bucket :read-error :detail (str "actual read: " (.getMessage ^Throwable (second a)))}
|
||||||
|
|
||||||
|
throws?
|
||||||
|
(if (= (first a) :throw)
|
||||||
|
{:bucket :certified-throws}
|
||||||
|
{:bucket :throws-mismatch
|
||||||
|
:detail (str "jolt says :throws, JVM returned " (pr-str (second a)))})
|
||||||
|
|
||||||
|
;; actual threw on the JVM — either jolt-specific/host-coupled (informational)
|
||||||
|
(= (first a) :throw)
|
||||||
|
{:bucket :jvm-error
|
||||||
|
:detail (let [m (.getMessage ^Throwable (second a))]
|
||||||
|
(if m (str/replace m #"\s+" " ") (str (class (second a)))))}
|
||||||
|
|
||||||
|
:else
|
||||||
|
(let [e (eval-safe expected)]
|
||||||
|
(cond
|
||||||
|
(= (first e) :read-error)
|
||||||
|
{:bucket :read-error :detail (str "expected read: " (.getMessage ^Throwable (second e)))}
|
||||||
|
(= (first e) :throw)
|
||||||
|
{:bucket :jvm-error :detail (str "expected eval threw: " (.getMessage ^Throwable (second e)))}
|
||||||
|
(= (second e) (second a))
|
||||||
|
{:bucket :certified}
|
||||||
|
:else
|
||||||
|
{:bucket :divergent
|
||||||
|
:detail (str "jolt-expected=" (pr-str (second e))
|
||||||
|
" JVM-result=" (pr-str (second a)))})))))
|
||||||
|
|
||||||
|
(defn -main [& _]
|
||||||
|
(let [corpus (edn/read-string (slurp corpus-path))
|
||||||
|
results (mapv (fn [row] (assoc (classify row) :row row)) corpus)
|
||||||
|
by (group-by :bucket results)
|
||||||
|
n (count results)
|
||||||
|
cnt #(count (get by % []))]
|
||||||
|
(println (format "Certifying %d corpus rows against JVM Clojure %s\n" n (clojure-version)))
|
||||||
|
(println (format " certified %5d (jolt expected == JVM)" (cnt :certified)))
|
||||||
|
(println (format " certified-throws %5d (:throws, JVM also throws)" (cnt :certified-throws)))
|
||||||
|
(println (format " jvm-error %5d (actual not certifiable on vanilla Clojure)" (cnt :jvm-error)))
|
||||||
|
(println (format " read-error %5d (won't read on JVM reader)" (cnt :read-error)))
|
||||||
|
(println (format " timeout %5d (exceeded %dms — infinite/blocking)" (cnt :timeout) case-timeout-ms))
|
||||||
|
(println (format " throws-mismatch %5d <-- jolt/JVM disagree on throwing" (cnt :throws-mismatch)))
|
||||||
|
(println (format " DIVERGENT %5d <-- corpus :expected disagrees with JVM" (cnt :divergent)))
|
||||||
|
(let [certifiable (+ (cnt :certified) (cnt :certified-throws) (cnt :divergent) (cnt :throws-mismatch))]
|
||||||
|
(println (format "\n certifiable rows: %d (certified %d / divergent %d / throws-mismatch %d)"
|
||||||
|
certifiable (+ (cnt :certified) (cnt :certified-throws))
|
||||||
|
(cnt :divergent) (cnt :throws-mismatch))))
|
||||||
|
|
||||||
|
;; Partition divergences/throws-mismatches into known (allowlisted) vs NEW.
|
||||||
|
(let [flagged (concat (get by :divergent []) (get by :throws-mismatch []))
|
||||||
|
key-of (fn [{:keys [row]}] [(:suite row) (:label row)])
|
||||||
|
new? (fn [r] (let [k (key-of r)] (and (not (known k)) (not (flaky k)))))
|
||||||
|
news (filter new? flagged)
|
||||||
|
flagged-keys (set (map key-of flagged))
|
||||||
|
stale (clojure.set/difference known flagged-keys)]
|
||||||
|
(println (format "\n allowlist: %d entries (%d flaky); %d of %d divergences known, %d NEW, %d stale"
|
||||||
|
(+ (count known) (count flaky)) (count flaky)
|
||||||
|
(- (count flagged) (count news)) (count flagged) (count news) (count stale)))
|
||||||
|
(when (seq news)
|
||||||
|
(println "\n=== NEW divergences (not in allowlist) — gate FAILS ===")
|
||||||
|
(doseq [{:keys [row detail]} news]
|
||||||
|
(println (format " [%s] %s\n actual: %s\n %s"
|
||||||
|
(:suite row) (:label row) (:actual row) detail))))
|
||||||
|
(when (seq stale)
|
||||||
|
(println "\n=== STALE allowlist entries (no longer diverging — remove them) ===")
|
||||||
|
(doseq [[s l] (sort stale)] (println (format " [%s] %s" s l))))
|
||||||
|
(def new-divergences news)
|
||||||
|
(def stale-entries stale))
|
||||||
|
|
||||||
|
;; Full per-row divergence detail goes to the --edn report (for triage); the
|
||||||
|
;; console stays quiet about KNOWN divergences (the NEW/STALE sections above are
|
||||||
|
;; what matters for the gate).
|
||||||
|
(when edn-out
|
||||||
|
(spit edn-out (with-out-str
|
||||||
|
(pp/pprint {:corpus corpus-path
|
||||||
|
:clojure-version (clojure-version)
|
||||||
|
:counts (into {} (map (fn [[k v]] [k (count v)]) by))
|
||||||
|
:divergent (mapv (fn [r] (assoc (:row r) :detail (:detail r))) (get by :divergent))
|
||||||
|
:throws-mismatch (mapv (fn [r] (assoc (:row r) :detail (:detail r))) (get by :throws-mismatch))})))
|
||||||
|
(println (format "\nwrote machine-readable report to %s" edn-out)))
|
||||||
|
|
||||||
|
;; Gate: fail only on a NEW (unlisted) divergence or a stale allowlist entry.
|
||||||
|
;; Every current divergence is either intentional (classified in the allowlist)
|
||||||
|
;; or a tracked bug — so a clean run means the corpus matches reference Clojure
|
||||||
|
;; everywhere it claims to, modulo the documented jolt-specific deltas.
|
||||||
|
(System/exit (if (or (seq new-divergences) (seq stale-entries)) 1 0))))
|
||||||
|
|
||||||
|
(apply -main *command-line-args*)
|
||||||
238
test/conformance/known-divergences.edn
Normal file
238
test/conformance/known-divergences.edn
Normal file
|
|
@ -0,0 +1,238 @@
|
||||||
|
{:doc
|
||||||
|
"Known divergences of test/chez/corpus.edn :expected from JVM Clojure, classified. Most are deliberate jolt-specific or host-model differences (-> :features in conformance inc3). :bug entries are genuine and tracked. The certifier (certify.clj) gates on NEW (unlisted) divergences only. Keyed by [suite label].",
|
||||||
|
:legend
|
||||||
|
{:bug
|
||||||
|
"genuine jolt/corpus bug — tracked bead, fix in jolt-core + correct :expected",
|
||||||
|
:numeric-model
|
||||||
|
"jolt is all-double: no Ratio/BigDecimal/float; (/ 1 2)=>0.5, 3.0 prints 3",
|
||||||
|
:concurrency-model
|
||||||
|
"Janet isolated-heap snapshot futures/agents/pmap; atoms snapshot, not shared",
|
||||||
|
:host-model
|
||||||
|
"no JVM host: classes->name strings, no Java arrays, type->symbol, *in* is a map, inline-impl extenders, duck-typed with-open close",
|
||||||
|
:reader-model
|
||||||
|
"jolt reader features (:jolt) + syntax-quote literal collapse (spec 2.4 S25); map source-order preserved",
|
||||||
|
:printer-model
|
||||||
|
"jolt printer renders transients/atoms/print-method overrides differently from JVM #object",
|
||||||
|
:strictness
|
||||||
|
"jolt is intentionally stricter: throws on odd assoc! args / defn with no param vector",
|
||||||
|
:impl-detail
|
||||||
|
"representation detail: syntax-quote yields a list? (JVM yields a Cons)"},
|
||||||
|
:entries
|
||||||
|
[{:suite "clojure.core / leaf batch (complement fnil munge etc.)",
|
||||||
|
:label "munge symbol",
|
||||||
|
:category :bug,
|
||||||
|
:bead "jolt-hc35"}
|
||||||
|
{:suite "exceptions / ex-info",
|
||||||
|
:label "ex-message of string",
|
||||||
|
:category :bug,
|
||||||
|
:bead "jolt-l8e8"}
|
||||||
|
{:suite "io / print family (overlay)",
|
||||||
|
:label "print nil arg",
|
||||||
|
:category :bug,
|
||||||
|
:bead "jolt-pqio"}
|
||||||
|
{:suite "seq / overlay-migrated fns",
|
||||||
|
:label "bounded-count",
|
||||||
|
:category :bug,
|
||||||
|
:bead "jolt-2507"}
|
||||||
|
{:suite "clojure.core / futures — predicates",
|
||||||
|
:label "cancel an in-flight future returns true",
|
||||||
|
:category :concurrency-model,
|
||||||
|
:flaky true}
|
||||||
|
{:suite "clojure.core / futures — snapshot (copy) semantics",
|
||||||
|
:label "captured atom is snapshotted, not shared",
|
||||||
|
:category :concurrency-model}
|
||||||
|
{:suite "clojure.core / pmap family",
|
||||||
|
:label "snapshot semantics",
|
||||||
|
:category :concurrency-model}
|
||||||
|
{:suite "state / agents (synchronous shim)",
|
||||||
|
:label "send applies",
|
||||||
|
:category :concurrency-model}
|
||||||
|
{:suite "state / agents (synchronous shim)",
|
||||||
|
:label "send-off applies",
|
||||||
|
:category :concurrency-model}
|
||||||
|
{:suite "core / extenders",
|
||||||
|
:label "lists extended type",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "core / extenders",
|
||||||
|
:label "seq of tags",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "core / with-open",
|
||||||
|
:label "close on throw",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "host-interop / class tokens & readers",
|
||||||
|
:label "class name evaluates to canonical string",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "host-interop / exception + HashMap shims",
|
||||||
|
:label "getMessage on a thrown string",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "io / *in* + with-in-str + read-line",
|
||||||
|
:label "*in* is bound",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "metadata / def metadata",
|
||||||
|
:label "^Type tag on var",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "metadata / type hints",
|
||||||
|
:label "symbol hint -> :tag",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "predicates / compare, type, any? (stage 3)",
|
||||||
|
:label "type of record",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "predicates / tagged-value (Phase 4)",
|
||||||
|
:label "chunked-seq? always false",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / JVM-shape stubs (documented jolt behavior)",
|
||||||
|
:label "bean is the map",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / JVM-shape stubs (documented jolt behavior)",
|
||||||
|
:label "class keyword",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / JVM-shape stubs (documented jolt behavior)",
|
||||||
|
:label "class number",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / JVM-shape stubs (documented jolt behavior)",
|
||||||
|
:label "class string",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / JVM-shape stubs (documented jolt behavior)",
|
||||||
|
:label "definterface defines",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / JVM-shape stubs (documented jolt behavior)",
|
||||||
|
:label "proxy resolves nil",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / array stubs (vectors + host buffers)",
|
||||||
|
:label "boolean-array",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / array stubs (vectors + host buffers)",
|
||||||
|
:label "double-array",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / array stubs (vectors + host buffers)",
|
||||||
|
:label "float-array",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / array stubs (vectors + host buffers)",
|
||||||
|
:label "int-array",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / array stubs (vectors + host buffers)",
|
||||||
|
:label "into-array",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / array stubs (vectors + host buffers)",
|
||||||
|
:label "long-array",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / array stubs (vectors + host buffers)",
|
||||||
|
:label "short-array",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / array stubs (vectors + host buffers)",
|
||||||
|
:label "to-array",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / chunk family (eager equivalents) + cat",
|
||||||
|
:label "chunk round-trip",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / ns + REPL machinery",
|
||||||
|
:label "*in* bound",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / ns + REPL machinery",
|
||||||
|
:label "ns-imports empty user",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "untested / unchecked-* are plain ops",
|
||||||
|
:label "unchecked-char",
|
||||||
|
:category :host-model}
|
||||||
|
{:suite "macros / defmacro",
|
||||||
|
:label "macroexpand-1",
|
||||||
|
:category :impl-detail}
|
||||||
|
{:suite "clojure.core / leaf batch (complement fnil munge etc.)",
|
||||||
|
:label "bigdec",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "numbers / arithmetic",
|
||||||
|
:label "divide to fraction",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "numbers / literal syntax",
|
||||||
|
:label "bigdec int M",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "numbers / literal syntax",
|
||||||
|
:label "bigdec suffix M",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "numbers / literal syntax",
|
||||||
|
:label "neg ratio",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "numbers / literal syntax",
|
||||||
|
:label "ratio -> double",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "numbers / literal syntax",
|
||||||
|
:label "ratio 3/4",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "numbers / printing of inf & nan",
|
||||||
|
:label "inf inside coll",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "numbers / printing of inf & nan",
|
||||||
|
:label "pr-str Infinity",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "reader / scalar literals",
|
||||||
|
:label "bigdec suffix M",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "reader / scalar literals",
|
||||||
|
:label "ratio -> double",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "untested / primed + division + bit ops",
|
||||||
|
:label "/ ratio-as-double",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "untested / typed coercion views",
|
||||||
|
:label "double",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "untested / typed coercion views",
|
||||||
|
:label "float",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "untested / unchecked-* are plain ops",
|
||||||
|
:label "unchecked-double",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "untested / unchecked-* are plain ops",
|
||||||
|
:label "unchecked-float",
|
||||||
|
:category :numeric-model}
|
||||||
|
{:suite "io / cold tagged types via print-method",
|
||||||
|
:label "transient map",
|
||||||
|
:category :printer-model}
|
||||||
|
{:suite "io / cold tagged types via print-method",
|
||||||
|
:label "transient vector",
|
||||||
|
:category :printer-model}
|
||||||
|
{:suite "io / print-method multimethod",
|
||||||
|
:label "defmethod fires through prn",
|
||||||
|
:category :printer-model}
|
||||||
|
{:suite "io / print-method multimethod",
|
||||||
|
:label "defmethod overrides a record, top level",
|
||||||
|
:category :printer-model}
|
||||||
|
{:suite "io / cold tagged types via print-method",
|
||||||
|
:label "atom override fires nested",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "io / print-method multimethod",
|
||||||
|
:label "defmethod fires nested in a map",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "maps / literal construction",
|
||||||
|
:label "source order through syntax-quote",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "reader / dispatch & sugar",
|
||||||
|
:label "reader cond :jolt",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "reader / dispatch & sugar",
|
||||||
|
:label "reader cond no match",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "reader / dispatch & sugar",
|
||||||
|
:label "reader cond splice",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "reader / dispatch & sugar",
|
||||||
|
:label "reader cond splice no match",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "reader / dispatch & sugar",
|
||||||
|
:label "reader conditional",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "reader / syntax-quote literal collapse (spec 2.4 S25)",
|
||||||
|
:label "bool nested",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "reader / syntax-quote literal collapse (spec 2.4 S25)",
|
||||||
|
:label "nil nested",
|
||||||
|
:category :reader-model}
|
||||||
|
{:suite "forms / fn",
|
||||||
|
:label "no param vector",
|
||||||
|
:category :strictness}
|
||||||
|
{:suite "transient / assoc! odd args throw",
|
||||||
|
:label "map dangling key",
|
||||||
|
:category :strictness}
|
||||||
|
{:suite "transient / assoc! odd args throw",
|
||||||
|
:label "vector dangling",
|
||||||
|
:category :strictness}]}
|
||||||
Loading…
Add table
Add a link
Reference in a new issue