Clean up codebase: rename stdlib layer, strip porting residue, fix tooling

Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.

Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.

Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.

Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
This commit is contained in:
Yogthos 2026-06-22 22:18:00 -04:00
parent c18f8087f0
commit 33eff7c7d8
112 changed files with 970 additions and 1621 deletions

View file

@ -17,12 +17,12 @@ absolute reference.
| Benchmark | Axis | Pass it exercises | Source |
|---|---|---|---|
| `binary-trees` | allocation / GC pressure (escaping short-lived records) | jolt-15jq scalar-replace, jolt-8flj escape analysis | CLBG |
| `dispatch` | polymorphic (**megamorphic**) protocol dispatch | jolt-41m devirt, inline-cache | AWFY-style |
| `mono-dispatch` | **monomorphic** protocol dispatch (devirt/inline-cache *can* fire) | jolt-41m devirt, jolt-ez5h inline-cache | AWFY-style |
| `collections` | persistent map/vector churn (HAMT / 32-way tries) | persistent structures (jolt-684u/0hbr), transients | CLBG k-nucleotide-style |
| `mandelbrot` | pure float compute (tight arith loops, no alloc/dispatch) | jolt-3pl native arith, loop codegen | CLBG |
| `fib` | recursion: function-call + integer-arith overhead | jolt-3pl native arith, jolt-826 small-fn inlining | CLBG |
| `binary-trees` | allocation / GC pressure (escaping short-lived records) | scalar-replace, escape analysis | CLBG |
| `dispatch` | polymorphic (**megamorphic**) protocol dispatch | devirt, inline-cache | AWFY-style |
| `mono-dispatch` | **monomorphic** protocol dispatch (devirt/inline-cache *can* fire) | devirt, inline-cache | AWFY-style |
| `collections` | persistent map/vector churn (HAMT / 32-way tries) | persistent structures, transients | CLBG k-nucleotide-style |
| `mandelbrot` | pure float compute (tight arith loops, no alloc/dispatch) | native arith, loop codegen | CLBG |
| `fib` | recursion: function-call + integer-arith overhead | native arith, small-fn inlining | CLBG |
What the ray tracer does **not** capture and these do: allocation as the
bottleneck (~7% there), megamorphic *and* monomorphic dispatch (its dispatch is
@ -35,17 +35,17 @@ control with record state), k-nucleotide proper.
## Holistic scorecard
`JVM=1 bench/run.sh` runs each benchmark on jolt **and** JVM Clojure and prints
the jolt/JVM ratio — the epic's (jolt-ffn) absolute-reference scorecard. As of
the jolt/JVM ratio — the absolute-reference scorecard. As of
the broadening (2026-06-16), ratios cluster by axis:
- **pure compute** (`mandelbrot`) is the floor, ~15× — native arith (jolt-3pl)
- **pure compute** (`mandelbrot`) is the floor, ~15× — native arith
already gets jolt closest to the JVM.
- **collections** ~28×, **fib** ~37×.
- **dispatch** ~75× (megamorphic), and `mono-dispatch` is *worse* (~110×): the
JVM inline-caches a runtime-monomorphic call site to near-free, while jolt does
a full registry dispatch regardless (devirt only fires on *statically* proven
receivers, which `reduce` over a vector doesn't give). This is the signal for
the call-site inline cache (jolt-ez5h).
the call-site inline cache.
- **allocation** (`binary-trees`) is the widest gap — but also the most inflated
by host memory pressure, so read it as "alloc is the worst axis," not a precise
multiple. Numbers are machine-specific; regenerate with `JVM=1 bench/run.sh`.

View file

@ -1,7 +1,7 @@
;; binary-trees (Computer Language Benchmarks Game) — an ALLOCATION/GC stress
;; test. Builds and discards millions of short-lived `Node` records; the nodes
;; ESCAPE (stored in the tree, walked later), so this is the regime jolt-8flj
;; (escape analysis) targets and the ray tracer never exercises (~7% alloc).
;; ESCAPE (stored in the tree, walked later), so this is the regime escape analysis
;; targets and the ray tracer never exercises (~7% alloc).
;;
;; Portable Clojure: runs on jolt and JVM Clojure for cross-impl comparison.
;; jolt -m binary-trees 14 (JOLT_DIRECT_LINK=1 JOLT_WHOLE_PROGRAM=1)

View file

@ -1,7 +1,7 @@
;; dispatch — a POLYMORPHIC-DISPATCH stress test. A protocol method is called in
;; a hot loop over a heterogeneous (megamorphic) collection of record types, with
;; minimal per-call work, so protocol dispatch dominates. This is the regime
;; jolt-41m (devirtualization) and the inline-cache target, and the one the ray
;; devirtualization and the inline-cache target, and the one the ray
;; tracer can't reveal — its dispatch is monomorphic and a small fraction of the
;; float-math cost (devirt measured FLAT there).
;;

View file

@ -1,7 +1,7 @@
;; fib — naive recursive Fibonacci: pure function-call + integer-arithmetic
;; throughput, with no allocation, dispatch, or collections. Isolates call
;; overhead and native integer arith (jolt-3pl), and is the natural target for
;; single-call-site / small-fn inlining (jolt-826) and self-call direct-linking.
;; overhead and native integer arith, and is the natural target for
;; single-call-site / small-fn inlining and self-call direct-linking.
;;
;; Portable Clojure (jolt + JVM Clojure).
;; jolt -m fib 32 (JOLT_DIRECT_LINK=1 JOLT_WHOLE_PROGRAM=1)

View file

@ -3,7 +3,7 @@
;; dispatch, no collections in the hot loop — just double arithmetic and tight
;; recur loops. This isolates the irreducible-math axis the ray tracer is bound
;; on (where devirt/alloc passes measured flat), so it tracks native-arith codegen
;; (jolt-3pl) and loop quality directly.
;; and loop quality directly.
;;
;; Portable Clojure (jolt + JVM Clojure).
;; jolt -m mandelbrot 1000 (JOLT_DIRECT_LINK=1 JOLT_WHOLE_PROGRAM=1)

View file

@ -1,5 +1,5 @@
;; mono-dispatch — protocol dispatch where every call site sees ONE record type
;; (monomorphic). This is the regime where devirtualization (jolt-41m) and a
;; (monomorphic). This is the regime where devirtualization and a
;; call-site inline cache CAN fire — the megamorphic `dispatch` bench deliberately
;; defeats them, so this is its complement: it measures how close a proven/cached
;; monomorphic dispatch gets to a direct call. Same per-call work as `dispatch`.

View file

@ -3,7 +3,7 @@
#
# Each benchmark isolates an axis the ray tracer (float-compute-bound) doesn't
# capture — see README.md. Run back-to-back against `main` to measure a pass's
# impact (the same protocol as test/bench/core-bench.janet).
# impact.
#
# bench/run.sh # default sizes, whole-program optimization on
# JOLT_WHOLE_PROGRAM=0 bench/run.sh # compare with WP off
@ -20,12 +20,12 @@ export JOLT_PATH="$PWD"
# name:default-arg (arg sized to run in a few seconds each). Axes: allocation
# (binary-trees), megamorphic vs monomorphic dispatch, persistent-collection
# churn (collections — now O(log n) via the HAMT, jolt-684u, so sized up), pure
# churn (collections — now O(log n) via the HAMT, so sized up), pure
# float compute (mandelbrot), call+arith recursion (fib).
BENCHES="binary-trees:14 dispatch:2000 mono-dispatch:2000 collections:30000 mandelbrot:200 fib:30"
# JVM=1 also runs each bench on JVM Clojure and prints a jolt/JVM ratio — the
# holistic absolute-reference scorecard for the optimization epic (jolt-ffn).
# holistic absolute-reference scorecard for the optimization work.
run_one() {
ns="${1%%:*}"; arg="${2:-${1##*:}}"
jmean=$(jolt -m "$ns" "$arg" 2>&1 | awk '/^mean:/{print $2}')