Broaden the benchmark suite; add jolt-vs-JVM scorecard (#140)

The ray tracer is compute-bound and the three existing benches only cover
alloc / megamorphic-dispatch / collections. Add three axes the epic needs to
judge itself holistically:

- mono-dispatch: monomorphic protocol dispatch. Its jolt/JVM ratio (~110x) is
  *worse* than megamorphic (~76x) — the JVM inline-caches a runtime-monomorphic
  call site to near-free while jolt does a full registry dispatch (devirt only
  fires on statically-proven receivers). Points at the call-site inline cache.
- mandelbrot: pure float compute, no alloc/dispatch. The floor at ~15x — native
  arith already gets close to the JVM.
- fib: recursion, call + integer-arith overhead.

run.sh gains JVM=1, which runs each bench on JVM Clojure too and prints the
jolt/JVM ratio. collections sized up now that the map is a HAMT (jolt-684u).
README documents the axes and the current scorecard.

Co-authored-by: Yogthos <yogthos@gmail.com>
This commit is contained in:
Dmitri Sotnikov 2026-06-16 14:50:38 +00:00 committed by GitHub
parent f0293fb4ee
commit 7d0b1d5695
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 166 additions and 15 deletions

View file

@ -18,15 +18,24 @@ export JOLT_WHOLE_PROGRAM="${JOLT_WHOLE_PROGRAM:-1}"
export JOLT_APP_PATHS="$PWD"
export JOLT_PATH="$PWD"
# name:default-arg (arg sized to run in a few seconds each)
# NOTE collections is small because the persistent map is O(n)/assoc (jolt-684u);
# raise it once that's fixed to a HAMT.
BENCHES="binary-trees:14 dispatch:2000 collections:1500"
# name:default-arg (arg sized to run in a few seconds each). Axes: allocation
# (binary-trees), megamorphic vs monomorphic dispatch, persistent-collection
# churn (collections — now O(log n) via the HAMT, jolt-684u, so sized up), pure
# float compute (mandelbrot), call+arith recursion (fib).
BENCHES="binary-trees:14 dispatch:2000 mono-dispatch:2000 collections:30000 mandelbrot:200 fib:30"
# JVM=1 also runs each bench on JVM Clojure and prints a jolt/JVM ratio — the
# holistic absolute-reference scorecard for the optimization epic (jolt-ffn).
run_one() {
ns="${1%%:*}"; arg="${2:-${1##*:}}"
printf '%-16s ' "$ns"
jolt -m "$ns" "$arg" 2>&1 | awk '/^mean:/{print}'
jmean=$(jolt -m "$ns" "$arg" 2>&1 | awk '/^mean:/{print $2}')
if [ -n "$JVM" ]; then
vmean=$(clojure -Sdeps '{:paths ["."]}' -M -m "$ns" "$arg" 2>&1 | awk '/^mean:/{print $2}')
ratio=$(awk "BEGIN{ if ($vmean+0>0) printf \"%.1f\", ($jmean+0)/($vmean+0); else printf \"-\" }")
printf '%-16s jolt %9s ms jvm %8s ms %sx\n' "$ns" "${jmean:--}" "${vmean:--}" "$ratio"
else
printf '%-16s %9s ms\n' "$ns" "${jmean:--}"
fi
}
if [ -n "$1" ]; then
@ -34,6 +43,6 @@ if [ -n "$1" ]; then
[ "${spec%%:*}" = "$1" ] && run_one "$spec" "$2"
done
else
echo "jolt benchmark suite (WP=$JOLT_WHOLE_PROGRAM)"
echo "jolt benchmark suite (WP=$JOLT_WHOLE_PROGRAM${JVM:+, vs JVM Clojure})"
for spec in $BENCHES; do run_one "$spec"; done
fi