Spike: localize the mandelbrot 15x floor (jolt-5vsp) (#143)

The jolt-vs-hand-Janet-vs-JVM mandelbrot comparison splits the 15.4x floor
into two layers: a Janet-VM floor (~10.8x JVM, optimal while-loop Janet over
unboxed doubles — only native codegen moves it) plus a ~1.43x jolt loop-
lowering overhead on top. The overhead is entirely the loop/recur -> recursive-
closure-called-per-iteration lowering; hand-Janet written the same way matches
jolt, while a while+var/set version is 1.43x faster. So a cheap backend win
(jolt-v28u) sits above the structural native-codegen lever.

Adds the spike artifacts under bench/ and the results writeup; marks the spike
done in the handoff. No source changes.

Co-authored-by: Yogthos <yogthos@gmail.com>

docs/foundational-runtime-handoff.md

@@ -85,7 +85,16 @@ Every other axis adds structural overhead **on top** of that floor.
    (jolt-4x9 element types + jolt-t6r). Helps dispatch. Bounded ceiling (still
    bytecode underneath).
 
-## START HERE — the spike (do this before committing to any lever)
+## START HERE — the spike (DONE — see results)
+
+**The spike ran 2026-06-16. Results: `docs/foundational-runtime-spike-results.md`.**
+Outcome in one line: the 15.4× floor decomposes into a **Janet-VM floor ≈10.8×
+JVM** (the dominant ~70%; only native codegen / lever 1 moves it) plus a **jolt
+loop-lowering ≈1.43×** on top (cheap backend win — `loop`/`recur` is lowered to a
+recursive closure called per iteration; emit Janet `while`+`var`/`set` instead;
+bead **jolt-v28u**). Janet numbers are already unboxed (not a lever). Next: the
+lever-1 jolt-IR→C spike for one hot fn (confirm Janet's incremental native-module
+path first). The original spike instructions are preserved below for context.
 
 **Localize the 15× floor.** Build three `mandelbrot` implementations and compare:
 

docs/foundational-runtime-spike-results.md

@@ -0,0 +1,92 @@
+# Foundational Runtime Spike — Results (the 15× floor, localized)
+
+**Epic:** jolt-5vsp · **Date:** 2026-06-16
+**Spike:** the START HERE section of `docs/foundational-runtime-handoff.md` —
+jolt vs hand-written-Janet vs JVM `mandelbrot`, to localize the ~15× compute
+floor before committing to native codegen (lever 1) vs a backend fix.
+
+## Setup
+
+Three implementations of the same nested mandelbrot loop, all returning the
+identical result (3288753 at n=200, confirming correctness across all legs):
+
+- **jolt-compiled** — `bench/mandelbrot.clj` (`jolt -m mandelbrot 200`, WP + direct-link on)
+- **hand-Janet (`while`)** — `bench/mandelbrot-hand.janet` (idiomatic Janet: `while` + `var`/`set`)
+- **JVM Clojure** — `bench/mandelbrot.clj` on the JVM
+
+Plus a diagnostic fourth leg:
+
+- **hand-Janet (recursive)** — `bench/mandelbrot-hand-rec.janet`: hand Janet that
+  *mirrors jolt's loop lowering* (self-recursive local closure called per
+  iteration), to test whether the loop lowering alone explains jolt's overhead.
+
+Numbers are stable and sandwiched (A/B/A/B); machine noise < 1%.
+
+## The numbers (n=200, mean of 3, after warmup)
+
+| Leg | mean | × JVM |
+|---|---|---|
+| JVM Clojure | 14.2 ms | 1.0× |
+| **hand-Janet (`while`)** | **153.4 ms** | **10.8×** |
+| hand-Janet (recursive, mirrors jolt) | 215.3 ms | 15.2× |
+| **jolt-compiled** | **219.0 ms** | **15.4×** |
+
+## What this localizes
+
+The 15.4× floor **decomposes into two distinct layers**:
+
+1. **Janet VM floor ≈ 10.8× JVM** (70% of the gap). Optimal hand-written Janet —
+   pure `while` loop over unboxed doubles, zero allocation — is still ~11× slower
+   than JVM Clojure. This is the cost of the Janet bytecode VM itself (no JIT, no
+   native code). **Only native codegen (lever 1) can touch this.** It is the
+   dominant share and validates lever 1 as the big structural lever.
+
+2. **jolt backend loop-lowering ≈ 1.43× on top** (the remaining 30%). jolt is
+   `219 / 153 = 1.43×` slower than optimal Janet. The diagnostic leg pins this
+   *entirely* to one cause: jolt lowers every `loop`/`recur` to a **self-recursive
+   local closure called once per iteration**, not a `while` loop. Hand-Janet
+   written that same way (recursive leg) lands at **215 ms ≈ jolt's 219 ms** —
+   so the recursive-closure lowering accounts for essentially all of jolt's
+   backend overhead on pure-compute code.
+
+   See the emitted Janet (`bench/dump-mandelbrot-emit.janet`): `emit-loop`
+   (`src/jolt/backend.janet:210`) produces
+   `(do (var L nil) (set L (fn (i zr zi) … (L (+ i 1) …))) (let (…) (L …)))`
+   and `emit-recur` (`:228`) produces the per-iteration call `(L …)`. It relies
+   on Janet TCO for stack safety, but each iteration still pays a function
+   invocation (frame setup + arg bind) that a `while` loop skips.
+
+## Decision
+
+The handoff posed it as binary (Janet-VM floor *or* backend headroom). It is
+**both**, now sized:
+
+- **Native codegen (lever 1) is the only thing that moves the dominant ~70%.**
+  Confirmed as the big lever. Pursue the incremental jolt-IR→C spike for one hot
+  fn next, per the handoff.
+- **A cheap, localized ~30% win sits in the backend**, independent of any new
+  runtime: lower tail-position `loop`/`recur` with scalar bindings to a Janet
+  `while` + `var`/`set` instead of a recursive closure. Closes the 1.43×, taking
+  `mandelbrot` from 15.4× → ~10.8× JVM. Filed separately (see epic children).
+
+## Open questions answered
+
+- **Are Janet numbers boxed?** No — already unboxed. The `while` leg does pure
+  double arithmetic at a steady 153 ms with no allocation and no GC stutter, and
+  matches the other legs bit-for-bit. Janet's `number` is an immediate IEEE
+  double (stored inline in the Janet value, not heap-allocated). **Unboxing is
+  not a lever; it's done.**
+- **GC share of `binary-trees`** — not measured here (the dev machine swaps
+  heavily, which distorts alloc-heavy benches; the handoff flags this). Size
+  lever 2 on a clean machine. The `mandelbrot` legs are alloc-free so are
+  unaffected and trustworthy.
+- **Janet native-module / incremental C path** — not yet confirmed; this is the
+  gating question for the lever-1 spike (hot fns → C, rest → bytecode).
+
+## Artifacts (kept in `bench/`)
+
+- `mandelbrot-hand.janet` — optimal `while` Janet (the Janet VM floor reference)
+- `mandelbrot-hand-rec.janet` — recursive-closure Janet (the loop-lowering diagnostic)
+- `dump-mandelbrot-emit.janet` — dumps the Janet jolt emits for the hot fns
+
+The bench harness (`bench/run.sh`) ignores these (it iterates a fixed bench list).