7d0b1d5695
The ray tracer is compute-bound and the three existing benches only cover alloc / megamorphic-dispatch / collections. Add three axes the epic needs to judge itself holistically: - mono-dispatch: monomorphic protocol dispatch. Its jolt/JVM ratio (~110x) is *worse* than megamorphic (~76x) — the JVM inline-caches a runtime-monomorphic call site to near-free while jolt does a full registry dispatch (devirt only fires on statically-proven receivers). Points at the call-site inline cache. - mandelbrot: pure float compute, no alloc/dispatch. The floor at ~15x — native arith already gets close to the JVM. - fib: recursion, call + integer-arith overhead. run.sh gains JVM=1, which runs each bench on JVM Clojure too and prints the jolt/JVM ratio. collections sized up now that the map is a HAMT (jolt-684u). README documents the axes and the current scorecard. Co-authored-by: Yogthos <yogthos@gmail.com>
48 lines
2 KiB
Bash
Executable file
48 lines
2 KiB
Bash
Executable file
#!/bin/sh
|
|
# Run the jolt benchmark suite and print mean ms per benchmark.
|
|
#
|
|
# Each benchmark isolates an axis the ray tracer (float-compute-bound) doesn't
|
|
# capture — see README.md. Run back-to-back against `main` to measure a pass's
|
|
# impact (the same protocol as test/bench/core-bench.janet).
|
|
#
|
|
# bench/run.sh # default sizes, whole-program optimization on
|
|
# JOLT_WHOLE_PROGRAM=0 bench/run.sh # compare with WP off
|
|
# bench/run.sh binary-trees # one benchmark
|
|
#
|
|
# Needs `jolt` on PATH (build with `jpm build`; export PATH="$PWD/build:$PATH").
|
|
set -e
|
|
cd "$(dirname "$0")"
|
|
|
|
export JOLT_DIRECT_LINK="${JOLT_DIRECT_LINK:-1}"
|
|
export JOLT_WHOLE_PROGRAM="${JOLT_WHOLE_PROGRAM:-1}"
|
|
export JOLT_APP_PATHS="$PWD"
|
|
export JOLT_PATH="$PWD"
|
|
|
|
# name:default-arg (arg sized to run in a few seconds each). Axes: allocation
|
|
# (binary-trees), megamorphic vs monomorphic dispatch, persistent-collection
|
|
# churn (collections — now O(log n) via the HAMT, jolt-684u, so sized up), pure
|
|
# float compute (mandelbrot), call+arith recursion (fib).
|
|
BENCHES="binary-trees:14 dispatch:2000 mono-dispatch:2000 collections:30000 mandelbrot:200 fib:30"
|
|
|
|
# JVM=1 also runs each bench on JVM Clojure and prints a jolt/JVM ratio — the
|
|
# holistic absolute-reference scorecard for the optimization epic (jolt-ffn).
|
|
run_one() {
|
|
ns="${1%%:*}"; arg="${2:-${1##*:}}"
|
|
jmean=$(jolt -m "$ns" "$arg" 2>&1 | awk '/^mean:/{print $2}')
|
|
if [ -n "$JVM" ]; then
|
|
vmean=$(clojure -Sdeps '{:paths ["."]}' -M -m "$ns" "$arg" 2>&1 | awk '/^mean:/{print $2}')
|
|
ratio=$(awk "BEGIN{ if ($vmean+0>0) printf \"%.1f\", ($jmean+0)/($vmean+0); else printf \"-\" }")
|
|
printf '%-16s jolt %9s ms jvm %8s ms %sx\n' "$ns" "${jmean:--}" "${vmean:--}" "$ratio"
|
|
else
|
|
printf '%-16s %9s ms\n' "$ns" "${jmean:--}"
|
|
fi
|
|
}
|
|
|
|
if [ -n "$1" ]; then
|
|
for spec in $BENCHES; do
|
|
[ "${spec%%:*}" = "$1" ] && run_one "$spec" "$2"
|
|
done
|
|
else
|
|
echo "jolt benchmark suite (WP=$JOLT_WHOLE_PROGRAM${JVM:+, vs JVM Clojure})"
|
|
for spec in $BENCHES; do run_one "$spec"; done
|
|
fi
|