jolt/docs/foundational-runtime-handoff.md

# Foundational Runtime Epic — Handoff

**Epic:** jolt-5vsp · **Predecessor:** jolt-ffn (targeted specialization — concluded)
**Date:** 2026-06-16

This is a cold-start handoff. Read it top to bottom before touching code. Its
whole point is to keep the fresh session from re-running the experiments that
already came back flat, and to start from the one measurement that actually
tells us where to invest.

## Why this epic exists

The targeted-specialization epic (jolt-ffn) tried to close jolt's constant-factor
gap vs JVM Clojure with per-form compiler passes. Three independent attempts all
came back flat:

| Attempt | Bead | Result |
|---|---|---|
| Record field-read guard removal (bare field reads) | jolt-3ko | ~3% on dispatch (shipped #141 — kept for correctness, not speed) |
| Protocol inline cache (runtime, per-method) | jolt-ez5h | ~0% — the per-dispatch gen-check exactly cancels the find-protocol-method saving; `find` was never the bottleneck |
| Record-ctor descriptor-baking (fewer allocs/record) | jolt-p7fo | flat on binary-trees + broke the gate; reverted |

The conclusion: **the gap is structural to jolt-on-Janet, not a missing
optimization.** Targeted passes remove only the cheap parts; the structural floor
remains.

## The scorecard (jolt / JVM Clojure)

Regenerate any time with `JVM=1 bench/run.sh` (the absolute-reference mode).

| Axis | Bench | jolt/JVM |
|---|---|---|
| Pure float compute | `mandelbrot` | **~15×  ← THE FLOOR** |
| Persistent collections (HAMT) | `collections` | ~28× |
| Recursion (call + arith) | `fib` | ~37× |
| Megamorphic dispatch | `dispatch` | ~76× |
| Monomorphic dispatch | `mono-dispatch` | ~109× |
| Allocation / GC | `binary-trees` | ~314× (≈150× at depth 10) |

`mandelbrot` is the floor: pure tight arithmetic loops — no dispatch, no
allocation, no collections — and native arith already fires (jolt-3pl). So ~15×
is what jolt's *execution substrate* costs on the simplest possible workload.
Every other axis adds structural overhead **on top** of that floor.

**Machine caveat:** the dev machine swaps heavily (~13 GB). Alloc-heavy benches
(`binary-trees`, `collections`) inflate badly; light benches (`mandelbrot`,
`fib`, `dispatch`) are trustworthy. Get absolute alloc numbers on a clean machine.

## The four structural walls

1. **Bytecode-VM execution.** jolt's backend emits **Janet** (a register-bytecode
   VM) and runs it on the Janet interpreter loop — no JIT, no native code. Every
   op is bytecode dispatch. This is the `mandelbrot` 15× floor.
2. **Mark-sweep GC.** Janet's GC scans all live objects each cycle (no
   generations). Live-data + alloc-heavy workloads (`binary-trees` retains the
   tree) pay O(live) per GC. The JVM's generational GC makes young-object churn
   nearly free.
3. **Indirect calls.** Protocol dispatch and fn calls go through indirection
   (closures, the protocol registry). The JVM inlines/devirtualizes. jolt's
   devirt (jolt-41m) only fires on *statically*-proven monomorphic sites;
   `reduce`/`mapv` over a collection doesn't give that proof, so the common
   runtime-monomorphic case pays full dispatch (that's why `mono-dispatch` is
   *worse* than megamorphic — the JVM inline-caches it to near-free, jolt doesn't).
4. **Boxed / generic representations.** Records are tuples `[descriptor field…]`;
   field access goes through a tag guard unless the type is proven. Generic ops
   carry runtime type checks. (Open question: are Janet *numbers* boxed? Verify
   in the spike — it decides whether unboxing is a lever or already done.)

## Foundational levers (ranked)

1. **Native codegen — emit C, not Janet bytecode.** The Stalin approach. Compile
   jolt IR → C → machine code via the system compiler. The *only* lever that
   moves the 15× compute floor; could approach C/JVM speed on compute-bound code.
   Massive (a new backend). Plausible incremental shape: a jolt-IR→C compiler for
   *hot* fns with a fallback to the existing bytecode path for unsupported forms —
   mirroring today's interpret/compile hybrid. Needs to confirm Janet's C-API /
   native-module story can be targeted incrementally.
2. **Structural GC-pressure reduction.** Value-type small records (avoid heap),
   transient/editable-node hot paths (RFC 0003 future work — pvec/phm/sorted are
   now tries/HAMT/RB, so O(1) `transient`/`persistent!` via editable nodes is
   open). Helps the alloc-bound axes (`binary-trees`, `collections`). Does **not**
   touch the compute floor.
3. **Deeper devirt + body inline.** Propagate element/return types so devirt
   fires on runtime-monomorphic collections, then inline the method body
   (jolt-4x9 element types + jolt-t6r). Helps dispatch. Bounded ceiling (still
   bytecode underneath).

## STATUS (2026-06-16) — lever 1 (native codegen) built and working

The spike ran and lever 1 is now implemented. Full writeups:
`docs/foundational-runtime-spike-results.md` (floor localization) and
`docs/foundational-runtime-lever1-native-codegen.md` (native codegen).

Done (all merged to main, PRs #143–#148):
- **Floor localized:** the 15.4× decomposes into a **Janet-VM floor ≈10.8× JVM**
  (only native codegen moves it) + a **jolt loop-lowering ≈1.43×** (cheap backend
  win, bead **jolt-v28u**). Janet numbers are already unboxed (not a lever).
- **Native codegen (jolt-ihdp, CLOSED):** `src/jolt/cgen.janet` translates
  numeric-leaf fns (numeric in/out, native-op arithmetic + loop/recur/if/let/do)
  to C. Wired into the backend `:def` emit under **`JOLT_CGEN=1`** (opt-in). The
  `.so` is content-addressed + cached. **mandelbrot 224ms → 12.4ms (~18×)**,
  beats JVM. Leaf-first falls out free (callers stay bytecode, call native fn).
- **Build-time AOT (jolt-a7ds, partial):** `:cgen-collect?` records leaf fns at
  build, `aot-build` compiles them into one `.so` + manifest; `:cgen-prebuilt` +
  `load-aot` install them at deploy with **no cc** (proven with cc off PATH).

Open work under epic jolt-5vsp:
- **jolt-a7ds** — fuse the prebuilt `.so` + manifest into the `jpm` exe for a
  literal single binary (+ a `jolt cgen-build -m app` CLI). The heaviest piece;
  into jpm executable-build, not the compiler.
- **jolt-v28u** — `while`-loop lowering for tail `recur` (cheap ~30%, independent
  of cgen; helps ALL loops, not just cgen candidates).
- **jolt-l1l4** — widen cgen numeric grammar (mod/rem/bit-ops/min/max, mixed fns).
- **jolt-qx70** — hot-fn auto-detection (drop the global `JOLT_CGEN` knob).
- Lever 2 (GC-pressure) and lever 3 (deeper devirt) — untouched; see below.

The original spike instructions are preserved below for context.

**Localize the 15× floor.** Build three `mandelbrot` implementations and compare:

- **jolt-compiled** `mandelbrot` (already in `bench/mandelbrot.clj`),
- **hand-written Janet** `mandelbrot` (the same nested loop, idiomatic Janet —
  write it directly, no jolt),
- **JVM Clojure** `mandelbrot`.

Two ratios fall out:

- **jolt-emitted-Janet vs hand-Janet** → how much overhead jolt's *backend* adds
  over optimal Janet. To see jolt's emitted Janet, use the backend emit path
  (`backend/emit-ir` on the analyzed `run`/`count-point` fns) — note `:arities`
  etc. are jolt pvecs, so introspection is awkward; easier to read the emitted
  Janet via the compile path or just A/B the timings.
- **hand-Janet vs JVM** → the Janet VM's own floor.

Decision:

- If **hand-Janet ≈ jolt** and hand-Janet is ~15× JVM → the floor is **Janet's
  bytecode VM**. Native codegen (lever 1) is the only fix. Commit to the spike of
  a jolt-IR→C path for one hot fn and measure.
- If **jolt ≫ hand-Janet** → jolt's backend emits suboptimal Janet; there's
  headroom in the **backend** (cheaper, no new runtime). Find what it emits that
  hand-Janet doesn't.

Also measure the **GC share** on `binary-trees` (Janet GC stats around the run —
`(gccollect)` / `gcinterval`, or count allocations) to size lever 2 honestly.

## Key files / mechanisms

- **Backend (IR → Janet emit):** `src/jolt/backend.janet`. `native-ops` (~L322)
  emits native Janet arith; `emit-ir` (~L674) runs passes then emits. A native-C
  backend would branch here.
- **Passes / inference:** `jolt-core/jolt/passes.clj` (`run-passes`),
  `jolt-core/jolt/passes/types.clj` (inference; the `:fn` branch ~L527 now seeds
  ^Record param hints — #141), `jolt-core/jolt/passes/inline.clj`
  (scalar-replace, `ctor-shape`).
- **Record representation:** `src/jolt/types_protocols.janet` — `make-record`
  (~L145, the ~5-alloc/record path), `record-shape-for` (~L139, rebuilds its
  cache key every call), `record-tag`. Records are tuples `[descriptor field…]`.
- **Dispatch + ctors:** `src/jolt/eval_runtime.janet` —
  `protocol-dispatch-impl` (~L62), `make-deftype-ctor-impl` (~L382).
- **Config knobs:** `src/jolt/config.janet` — `JOLT_DIRECT_LINK`,
  `JOLT_WHOLE_PROGRAM`, `JOLT_OPTIMIZE`, the `ctx-shaping-env-vars` list (any new
  ctx-shaping env var MUST be added there and to `image-cache-path`).
- **Self-hosting design:** `docs/self-hosting-compiler.md` (the kernel/value-layer
  boundary), `docs/rfc/0003-transients.md` (editable-node future work).

## How to build, run, measure

```sh
jpm build                         # build/jolt (ctx baked, ~20ms startup); from-source is ~8s cold
export PATH="$PWD/build:$PATH"
bench/run.sh                      # jolt only, WP on
JVM=1 bench/run.sh                # jolt vs JVM scorecard (needs `clojure` on PATH)
bench/run.sh mandelbrot 400       # one bench, custom size
JOLT_WHOLE_PROGRAM=0 bench/run.sh # measure what WP buys
```

Gate: `jpm build; janet run-tests.janet` (parallel, ~100s; `JOLT_TEST_JOBS`
overrides). Bench memory hygiene (`bd memories bench-isolation-gotcha`): never run
a perf matrix while other CPU work runs — it starves later configs and produces
bogus numbers. Sandwich A/B/A.

## What NOT to repeat (already flat — see beads for detail)

- Runtime protocol inline cache (jolt-ez5h): gen-check cancels the saving.
- Field-read guard removal as a *speed* play (jolt-3ko): ~3%; machinery dominates.
  (The #141 change is kept for correctness + the `with-meta`-on-symbols fix.)
- `make-record` descriptor-baking (jolt-p7fo): flat — `binary-trees` is dominated
  by the live retained tree + GC, not the short-lived intermediate allocs.

## Open questions for the spike

- Are Janet numbers boxed? (Lever or already done.)
- Does Janet expose a native-module / C-codegen path jolt can target incrementally
  (hot fns → C, rest → bytecode)?
- What fraction of `binary-trees` is GC vs execution?
- Is there a cheaper record representation (Janet struct vs tuple-with-descriptor)
  that lowers field-read + alloc cost without a new backend?