cgen: build-time AOT — native fns without a toolchain on the target (jolt-a7ds) (#148)

Splits native codegen into a build phase (needs cc) and a deploy phase (none):

- gen-c-module/compile-module compile MANY numeric-leaf fns into ONE native
  module (the AOT shape), generalizing the one-fn-per-.so JIT path.
- Backend :cgen-collect? records each numeric-leaf defn's IR while the app loads
  as bytecode; cgen/aot-build compiles them into one module and write-manifest
  persists {sopath, [{ns name sym}]}.
- Backend :cgen-prebuilt + cgen/load-aot: the deploy run loads the prebuilt .so
  (via the native builtin, no cc) and installs each cfunction as the var root
  with the same timing as the JIT path, so callers direct-link to native code.
- toolchain-available? no longer crashes when cc is off PATH (os/execute raises
  on a missing exe) — a toolchain-less target now gets false.

Proven end-to-end in two processes (spike/native/aot-demo.janet): build with cc,
then deploy with cc removed from PATH -> count-point still native, mandelbrot
3288753 at 12.4ms (full 18x). Test: test/integration/cgen-aot-test.janet. Default
path unchanged; the modes are opt-in. Gate green (118 files).

Remaining for a literal single binary: fuse the .so + manifest into the jpm exe.

Co-authored-by: Yogthos <yogthos@gmail.com>
This commit is contained in:
Dmitri Sotnikov 2026-06-16 18:09:22 +00:00 committed by GitHub
parent ce3c7df24b
commit bffb492c1c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 282 additions and 43 deletions

View file

@ -96,6 +96,28 @@ Known limitation: building *core* with `JOLT_CGEN=1` would try to cgen core
numeric-leaf fns into the cached ctx image, where embedded cfunctions may not
serialize — keep cgen for app/user code until image-cache interaction is handled.
## Build-time AOT: native speed without a toolchain on the target (jolt-a7ds)
The JIT path above runs `cc` at runtime. The AOT path moves compilation to build
time so the deploy target needs no `cc`/`janet.h`:
- **Build phase** (`:cgen-collect?`, needs cc): loading the app records every
numeric-leaf defn's IR; `cgen/aot-build` compiles them all into ONE native
module (`gen-c-module`) and `write-manifest` persists `{sopath, [{ns name sym}]}`.
- **Deploy phase** (`:cgen-prebuilt`, NO cc): `cgen/load-aot` loads the prebuilt
`.so` (via the `native` builtin — no compiler) into a qname→cfunction map; the
backend's `:def` hook installs each as the var root with the same timing as the
JIT path, so callers direct-link to native code.
**Proven** (`spike/native/aot-demo.janet`, two processes): build with cc, then
deploy with `cc` removed from PATH → `count-point` is still native, mandelbrot =
3288753 at **12.4 ms** (full 18×). Test: `test/integration/cgen-aot-test.janet`.
This removes the runtime-toolchain dependency — the core of the deployment story.
What remains for a literal single binary: fuse the prebuilt `.so` + manifest into
the `jpm`-built executable (declare-native/static-lib link + an uberscript-style
source bundle), so it ships as one file instead of an exe + sidecar `.so`.
## Open questions for the implementation (next beads)
1. **IR→C for the numeric subset.** Translate jolt IR → C for proven-double