bench: refresh the jolt/JVM scorecard (#236)

The type-proving / native-record / bare-field-read / inference work collapsed
the dispatch + allocation gaps by an order of magnitude (binary-trees ~140x->~10x,
mono-dispatch ~330x->~48x, dispatch ~130x->~12x) and brought compute to parity
(fib now beats JVM, collections ~4x, mandelbrot ~7.5x). mono-dispatch is now the
standout gap — a runtime-monomorphic call site the JVM inline-caches to near-free.

Co-authored-by: Yogthos <yogthos@gmail.com>
This commit is contained in:
Dmitri Sotnikov 2026-06-26 18:20:11 +00:00 committed by GitHub
parent f920ff6ea2
commit f923d52cad
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -40,29 +40,32 @@ source — the jolt/JVM scorecard. jolt's optimizing passes fire only in a build
`joltc run -m` is unoptimized, so the harness always builds. `joltc run -m` is unoptimized, so the harness always builds.
Indicative ratios (M-series, single isolated run — numbers are machine-specific, Indicative ratios (M-series, single isolated run — numbers are machine-specific,
regenerate locally). They cluster into two regimes: regenerate locally), ascending:
| benchmark | ratio | axis | | benchmark | ratio | axis |
|---|---|---| |---|---|---|
| `mandelbrot` | ~8× | pure float compute | | `fib` | ~0.6× | call + integer arith |
| `fib` | ~9× | call + integer arith | | `collections` | ~4× | persistent map/vector churn |
| `collections` | ~9× | persistent map/vector churn | | `mandelbrot` | ~7.5× | pure float compute |
| `dispatch` | ~130× | megamorphic protocol dispatch | | `binary-trees` | ~10× | escaping short-lived records (allocation/GC) |
| `binary-trees` | ~140× | escaping short-lived records (allocation/GC) | | `dispatch` | ~12× | megamorphic protocol dispatch |
| `mono-dispatch` | ~330× | monomorphic protocol dispatch | | `mono-dispatch` | ~48× | monomorphic protocol dispatch |
- **Compute (~89×)** is the substrate floor: Chez is a native-compiling AOT - **Compute (~0.67.5×)** is the substrate floor: Chez is a native-compiling AOT
Scheme, not a profiling JIT, so it can't match HotSpot on hot loops. Native arith Scheme, not a profiling JIT. With native arith + direct-linking + inlining jolt
already gets jolt closest here. is at parity here — `fib` runs *faster* than JVM Clojure (no JIT warmup over a
- **Dispatch & allocation (~130330×)** are the architectural gaps. jolt does a short run), `collections` is within ~4×, and `mandelbrot` (~7.5×) is the
full protocol-registry lookup on every call; the JVM inline-caches a pure-tight-loop float ceiling that only native codegen moves further.
runtime-monomorphic site to near-free — which is why `mono-dispatch` is *worse* - **Dispatch & allocation (~1048×)** are the remaining architectural gaps, though
than megamorphic. devirt only fires on *statically proven* receivers (which the type-proving / native-record / bare-field-read work has collapsed them by an
`reduce`/`mapv` over a heterogeneous vector never gives), so the passes don't order of magnitude (`binary-trees` ~140×→~10×). jolt still does a full protocol-
engage; a call-site inline cache is the missing lever. `binary-trees` nodes registry lookup on every call; the JVM inline-caches a runtime-monomorphic site
escape into the tree, so scalar-replace can't remove them — this is GC pressure. to near-free — which is why `mono-dispatch` is *worse* than megamorphic and is now
- The optimization passes move these benchmarks <10% vs the unoptimized run, so the the standout gap. devirt fires only on a *statically proven* receiver; whole-
gaps are not a missing-flag problem; they're the dispatch/GC/JIT-floor work. program inference now proves more of them, but a value iterated out of a vector
still needs one — a call-site inline cache is the missing lever. `binary-trees`
nodes escape into the tree, so scalar-replace can't remove them — residual GC
pressure.
## Running ## Running