diff --git a/Makefile b/Makefile index 72c0b5e..9d34ba2 100644 --- a/Makefile +++ b/Makefile @@ -4,7 +4,7 @@ # build step. `make test` is the full gate. `make remint` rebuilds the seed after a # source change. -.PHONY: test ci values corpus unit smoke selfhost sci certify ffi transient remint +.PHONY: test ci values corpus unit smoke buildsmoke selfhost sci certify ffi transient remint # Full gate (dev machine). Includes the self-host byte-fixpoint, which only holds # on the same Chez that minted the seed. @@ -15,7 +15,7 @@ test: selfhost ci # lockfile) — it RUNS correctly on any Chez, but `selfhost` rebuilds it and a # different Chez version may emit byte-different (gensym/order) output, so the # byte-fixpoint is a dev-machine check, not a CI one (jolt-8479). -ci: values corpus unit smoke sci ffi transient certify +ci: values corpus unit smoke buildsmoke sci ffi transient certify @echo "OK: CI gates passed" # Self-host fixpoint: bootstrap.ss rebuild == checked-in seed. @@ -38,6 +38,10 @@ unit: smoke: @sh host/chez/smoke.sh +# `jolt build` produces a working standalone binary. +buildsmoke: + @sh host/chez/build-smoke.sh + # SCI conformance: load borkdude/sci's source through joltc (floor-gated). sci: @chez --script host/chez/run-sci.ss diff --git a/docs/rfc/0007-compilation-modes-and-binary-output.md b/docs/rfc/0007-compilation-modes-and-binary-output.md new file mode 100644 index 0000000..53ee2b3 --- /dev/null +++ b/docs/rfc/0007-compilation-modes-and-binary-output.md @@ -0,0 +1,186 @@ +# RFC 0007 — Compilation modes and binary output + +- **Status**: Draft. No code yet; this fixes the design before Phase 4 work + (beads `jolt-cf1q.5`) starts. +- **Champions**: jolt maintainers +- **Created**: 2026-06-22 + +## Summary + +Give jolt a `jolt build` command that emits a standalone executable, and a +three-mode model that trades dynamism for speed: + +- **dev** — open/indirect linking, redefinition works, no perf focus. What + `repl`/`-e`/`nrepl` already are. +- **release** (default for a built program) — direct-linked, closed-world, + per-namespace inference. Fast, still a recognizable Clojure runtime. +- **optimized** — whole-program inference, `fl*`/`fx*` typed emission, Chez + whole-program optimization. Fastest, sacrifices dynamic redefinition. + +All three already have their machinery in the tree — the inference and inline +passes were ported into `jolt-core/jolt/passes/`. What is missing is (a) a code +path that writes emitted Scheme to disk and AOT-compiles it instead of +eval'ing it in process, and (b) a switch that turns the dormant passes on. This +RFC specifies both. + +## Motivation + +The Janet host could produce binaries (`jolt uberscript` with dead-code +elimination, `jolt cgen-build` for a single native binary). The Chez rehost +dropped that machinery with the Janet host — it was Janet-specific (IR→C made +sense when the host was Janet). On Chez the natural target is Chez's own native +compiler, so the old emitters were deleted rather than re-pointed. + +The result today: `bin/joltc` only ever loads the checked-in seed and +compile-evals in process. `jolt.main/-main` dispatches `run / -M / -A / repl / +nrepl / task` and nothing else. There is no way to ship an app as a binary, and +the optimization passes are inert — `jolt.host/inline-enabled?` is a stub +returning `#f` (`host/chez/host-contract.ss:283`), so every call links +indirectly and nothing inlines. Jolt on Chez runs only in what this RFC calls +dev mode. + +The passes themselves survived intact: + +- `jolt/passes/types.clj` — structural collection-type inference (RFC 0005) + + success-type checking (RFC 0006). +- `jolt/passes/inline.clj` — inline + flatten-lets + scalar-replace, already + gated "direct-link only". +- `jolt/passes/fold.clj` — const-fold, including predicate folding. + +So this is not a port of lost code. It is wiring: a build front-end, a +file-emitting back-end path, and a mode switch over passes that already exist. + +## The three modes + +| Mode | Linking | Inference | Redefinition | Driver | +|---|---|---|---|---| +| **dev** | indirect (var-deref per call) | off | yes | `repl`, `-e`, `nrepl`, `run` of a file by default | +| **release** | direct, closed-world | per-namespace | no (closed world) | `jolt build` default | +| **optimized** | direct + whole-program | whole-program fixpoint, `fl*`/`fx*` | no | `jolt build --opt` / `-M`-style entry | + +The modes are points on one axis (how much the back end may assume is fixed), +not three code paths. Each mode is a setting of two independent knobs the passes +already understand: + +- **direct-link?** — may a call to a var compile to a direct procedure + reference instead of a `var-deref`? Enables inlining and call-site folding. + Opt-out is per-target: a `^:redef` or `^:dynamic` var always links indirect. +- **whole-program?** — does inference see the whole reachable program at once + (closed world), so a record param's callers in other namespaces are visible + and its field reads specialize? Without it, inference is per-namespace and a + cross-ns param de-specializes to `:any` (the cross-ns penalty documented in + the `cross-ns-param-penalty` memory; declared `^RecordType` hints are the + open-world escape hatch). + +``` +dev: direct-link? = false whole-program? = false +release: direct-link? = true whole-program? = false +optimized: direct-link? = true whole-program? = true +``` + +`fl*`/`fx*` typed emission (unchecked flonum/fixnum Scheme ops) rides on +optimized: only whole-program inference proves the types that make dropping the +numeric-tower dispatch sound. Release keeps the tower. + +## CLI surface + +``` +jolt build [-m NS | FILE] [-o OUT] [--opt] [--dev] +``` + +- Resolves `deps.edn` exactly as `run` does (reuse `jolt.deps`). +- Default mode is **release**. `--opt` selects optimized; `--dev` builds an + unoptimized binary (useful to ship a debuggable build, not for the REPL). +- `-o` names the output (default the entry ns / file stem). +- Output is a single executable: a Chez boot file plus the compiled program, + launched by a thin wrapper, or a fully linked image where the platform allows. + App libraries are baked in — no source roots needed at runtime. + +Env opt-outs for the build (mirrors the Janet knobs, now keyed off the mode +rather than the run): `JOLT_NO_DIRECT_LINK` forces open linking even in a build, +`JOLT_NO_WHOLE_PROGRAM` keeps direct-link but per-namespace, `JOLT_WHOLE_PROGRAM=1` +forces whole-program. These already name the two knobs above. + +## Emission pipeline + +The in-process spine today (`host/chez/compile-eval.ss`) is, per form: + +``` +source → read → analyze (→ IR) → emit (→ Scheme string) → (eval (read …)) +``` + +`jolt build` keeps everything up to `emit` and replaces the per-form `eval` with +accumulate-then-compile: + +1. **Assemble the program.** Starting from the entry ns's `-main`, load the + transitive `require` graph (the loader already does this) and collect every + reachable top-level form, in dependency order, with its compile namespace. +2. **Dead-code elimination.** Re-target the uberscript DCE idea: compute + reachability from `-main` plus non-prunable forms, drop dead `defn`/`defn-`. + Bail to keep-all on `resolve`/`ns-resolve`/`requiring-resolve`/`find-var`/ + `intern`/`eval`/`load-string` (anything that defeats static reachability); + keep and scan `defmethod`/`defrecord`/`extend` bodies so dispatch targets + stay live. +3. **Emit to a file.** Run `analyze → emit` for each surviving form under the + mode's knobs, concatenating the Scheme strings into one program source (the + core overlay prelude first, exactly as the seed image is built today). +4. **Compile.** Feed that source to Chez `compile-program` (release) or + `compile-whole-program` (optimized, which also lets Chez cross-module + inline), producing a compiled object, then link a boot file / wrapper into + the final executable. + +Steps 3–4 are the only genuinely new back-end code. Step 2 is a re-port of a +deleted pass. Steps before them already run on every `joltc` invocation. + +## Turning the passes on + +`inline-enabled?` is the existing gate. Today `host-contract.ss` hardwires it to +`#f`. Under this RFC the build sets it (and a parallel `whole-program?` flag) +from the chosen mode before compiling, so: + +- release: `inline-enabled?` → true, whole-program off. Per-ns inference and + inlining light up; `fl*`/`fx*` stays off. +- optimized: both on; the types pass runs its whole-program fixpoint and the + back end may emit unchecked numeric ops where a flonum/fixnum is proven. + +No new pass is required to reach release — it is the ported passes, ungated. + +## Staging + +1. **Spike (de-risk Chez AOT).** Emit a trivial whole program to disk and prove + `compile-program` + boot/static link yields a standalone binary that runs. + This is the only real unknown. +2. **`jolt build` release.** Front-end + file-emitting back-end path + flip + `inline-enabled?` from the mode. Gate against the bench/corpus suites; binary + output must pass the corpus a `run` passes. +3. **DCE.** Re-port the reachability pass; gate with a test like the old + `uberscript-dce` case. +4. **Optimized.** Whole-program flag, `compile-whole-program`, `fl*`/`fx*` + emission. Gate on the bench suite (ray tracer, binary-trees) for size and + speed vs the spike baseline. + +Each stage is TDD against the existing gates (`make test`, `make corpus`, the +`bench/` programs). Modes land behind the build command, so dev — the only mode +today — is unaffected until a stage proves out. + +## Open questions + +- **Static vs. boot-file linking.** A fully static Chez image is the smallest, + most portable artifact but the most work to link; a boot file plus a stub + launcher is the easy first cut. Spike decides which step 1 targets. +- **FFI in a built binary.** `jolt.ffi` loads native libraries at runtime; a + closed-world build still needs that to work. The build must bake the FFI + Clojure side and keep dynamic `dlopen` at run time. +- **Macro and `eval` at runtime.** Release/optimized are closed-world, but an + app that calls `eval`/`load-string` needs the compiler present. Either ship + the compiler image in the binary (larger) or reject those builds (the DCE + bail-out already detects the calls). + +## Prior art in this repo + +The optimization design these modes turn on is RFC 0004 (type hints), RFC 0005 +(structural inference), RFC 0006 (success checking). The linking model — direct +linking as a per-unit property, `^:redef`/`^:dynamic` as the only opt-out — and +the cross-ns specialization penalty are recorded in beads memories +(`jolt-linking-model`, `cross-ns-param-penalty`). Phase 4 (`jolt-cf1q.5`) is the +tracking issue. diff --git a/host/chez/build-smoke.sh b/host/chez/build-smoke.sh new file mode 100755 index 0000000..8cd75de --- /dev/null +++ b/host/chez/build-smoke.sh @@ -0,0 +1,32 @@ +#!/bin/sh +# build smoke: `jolt build` compiles a multi-namespace app (macro + cross-ns + +# clojure.string) into a standalone binary, which then runs with no jolt source +# or Chez install on the path — args reach -main, output matches. +root="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)" +cd "$root" + +app="$root/test/chez/build-app" +out="$(mktemp -d)/app-bin" +trap 'rm -rf "$(dirname "$out")"' EXIT + +echo "build smoke: compiling app.core -> $out" +if ! JOLT_PWD="$app" bin/joltc build -m app.core -o "$out" >/dev/null 2>&1; then + echo " FAIL: jolt build exited non-zero" + exit 1 +fi +[ -x "$out" ] || { echo " FAIL: no executable produced"; exit 1; } + +# Run from a neutral cwd with args; check the three output lines. +got="$(cd / && "$out" alpha bb ccc 2>&1)" +want='HELLO FROM A BUILT BINARY! +HELLO FROM A BUILT BINARY! +args: [alpha bb ccc] +sum: 10' +if [ "$got" = "$want" ]; then + echo "build smoke: passed" +else + echo " FAIL: binary output mismatch" + echo "--- want ---"; echo "$want" + echo "--- got ----"; echo "$got" + exit 1 +fi diff --git a/host/chez/build.ss b/host/chez/build.ss new file mode 100644 index 0000000..a3bacdb --- /dev/null +++ b/host/chez/build.ss @@ -0,0 +1,253 @@ +;; build.ss — `jolt build`: AOT-compile an app into a standalone executable. +;; +;; Loaded on demand by cli.ss when the command is `build`. Defines the host +;; primitive jolt.host/build-binary, which jolt.main's build command calls after +;; resolving the project's deps + source roots. +;; +;; The pipeline (Phase 4 stage 2): +;; 1. load the entry namespace — registers its macros/vars and follows requires, +;; recording the app namespaces in dependency order (loader's ns-loaded-hook). +;; 2. re-emit each app namespace to Scheme (the emit-image cross-compile path), +;; now that its macros are registered. +;; 3. textually inline the cli.ss runtime load sequence into one flat source, +;; append the app emission + a launcher that calls the entry's -main. +;; 4. compile-file -> make-boot-file -> embed the boot as C bytes -> cc-link +;; against libkernel.a into a single self-contained binary. +;; +;; emit-image.ss supplies the cross-compiler (ei-* helpers); it's loaded here so a +;; normal run never pays for it. + +(load "host/chez/emit-image.ss") + +;; --- shell helpers ---------------------------------------------------------- +;; Run a command, return its stdout as one trimmed string ("" on no output). +(define (bld-sh-capture cmd) + (let* ((p (process cmd)) (in (car p))) + (let loop ((acc '())) + (let ((l (get-line in))) + (if (eof-object? l) + (begin (close-port in) + (let ((s (apply string-append (reverse acc)))) + ;; trim a trailing newline-equivalent (we joined without them) + s)) + (loop (cons l acc))))))) + +(define (bld-system cmd) + (let ((rc (system cmd))) + (unless (zero? rc) + (error 'jolt-build (string-append "command failed (" (number->string rc) "): " cmd))))) + +(define (bld-contains? s sub) + (let ((ns (string-length s)) (nsub (string-length sub))) + (let loop ((i 0)) + (cond ((> (+ i nsub) ns) #f) + ((string=? (substring s i (+ i nsub)) sub) #t) + (else (loop (+ i 1))))))) + +;; --- toolchain discovery ---------------------------------------------------- +(define bld-machine (symbol->string (machine-type))) +(define bld-osx? (bld-contains? bld-machine "osx")) + +;; The Chez executable, for the isolated compile pass (see build-binary step 4). +(define bld-chez + (let ((p (bld-sh-capture "command -v chez || command -v scheme || command -v petite"))) + (if (> (string-length p) 0) p "chez"))) + +;; Chez version off (scheme-version) "Chez Scheme Version X.Y.Z" — last token. +(define bld-version + (let* ((s (scheme-version)) (n (string-length s))) + (let loop ((i n)) + (if (or (= i 0) (char=? (string-ref s (- i 1)) #\space)) + (substring s i n) + (loop (- i 1)))))) + +;; The csv/ dir holding scheme.h, libkernel.a, *.boot. Derived from +;; the chez executable's location; JOLT_CHEZ_CSV overrides. +(define bld-csv-dir + (let ((env (getenv "JOLT_CHEZ_CSV"))) + (or (and env (> (string-length env) 0) env) + (let* ((bindir (bld-sh-capture "dirname \"$(command -v chez || command -v scheme || command -v petite)\"")) + (cand (string-append bindir "/../lib/csv" bld-version "/" bld-machine))) + cand)))) + +(define (bld-check-toolchain) + (for-each + (lambda (f) + (let ((p (string-append bld-csv-dir "/" f))) + (unless (file-exists? p) + (error 'jolt-build (string-append "Chez build file missing: " p + "\nSet JOLT_CHEZ_CSV to the csv/ dir."))))) + '("scheme.h" "libkernel.a" "petite.boot" "scheme.boot"))) + +;; Link flags. macOS Homebrew layout for the kernel's lz4/zlib/ncurses deps. +(define (bld-link-libs) + (if bld-osx? + (let ((lz4 (bld-sh-capture "brew --prefix lz4 2>/dev/null"))) + (string-append + (if (> (string-length lz4) 0) (string-append "-L" lz4 "/lib ") "") + "-llz4 -lz -lncurses -framework Foundation -liconv -lm")) + ;; Best-effort Linux/other; untested here. + "-llz4 -lz -lncurses -ldl -lpthread -lm")) + +;; --- runtime manifest (mirrors host/chez/cli.ss's load order) --------------- +(define bld-runtime-manifest + (list + "(load \"host/chez/rt.ss\")" + "(set-chez-ns! \"clojure.core\")" + "(load \"host/chez/seed/prelude.ss\")" + "(load \"host/chez/post-prelude.ss\")" + "(set-chez-ns! \"user\")" + "(load \"host/chez/host-contract.ss\")" + "(load \"host/chez/seed/image.ss\")" + "(load \"host/chez/compile-eval.ss\")" + "(load \"host/chez/png.ss\")" + "(load \"host/chez/loader.ss\")" + "(load \"host/chez/ffi.ss\")" + "(set-source-roots! (list \"jolt-core\" \"stdlib\"))")) + +;; A single-line top-level `(load "PATH")` -> PATH, else #f. +(define (bld-load-path line) + (let ((s (let trim ((i 0)) + (if (and (< i (string-length line)) + (memv (string-ref line i) '(#\space #\tab))) + (trim (+ i 1)) + (substring line i (string-length line)))))) + (and (>= (string-length s) 7) + (string=? (substring s 0 6) "(load ") + (let* ((q1 (let scan ((i 6)) (if (char=? (string-ref s i) #\") i (scan (+ i 1))))) + (q2 (let scan ((i (+ q1 1))) (if (char=? (string-ref s i) #\") i (scan (+ i 1)))))) + (substring s (+ q1 1) q2))))) + +(define (bld-file-lines path) + (call-with-input-file path + (lambda (p) + (let loop ((acc '())) + (let ((l (get-line p))) + (if (eof-object? l) (reverse acc) (loop (cons l acc)))))))) + +;; Emit one line to OUT, recursively inlining a `(load ...)` of a repo file. +(define (bld-inline-line line out depth) + (when (> depth 50) (error 'jolt-build "load nesting too deep")) + (let ((p (bld-load-path line))) + (if p + (for-each (lambda (l) (bld-inline-line l out (+ depth 1))) (bld-file-lines p)) + (begin (put-string out line) (put-string out "\n"))))) + +(define (bld-emit-runtime out) + (for-each (lambda (l) (bld-inline-line l out 0)) bld-runtime-manifest)) + +;; --- app emission ----------------------------------------------------------- +;; Re-emit one app namespace to a list of Scheme strings. Like emit-image's +;; ei-emit-ns but WITHOUT the silent (guard ...) wrapper — a form that fails to +;; emit must fail the build, not vanish. +(define (bld-emit-ns ns-name src) + (let loop ((forms (ei-read-all src)) (acc '())) + (if (null? forms) + (reverse acc) + (let ((f (car forms))) + (ce-scan-requires! f ns-name) + (cond + ((ei-ns-form? f) (loop (cdr forms) acc)) + ((ce-macro-form? f) + (let-values (((nm fn-form) (ce-defmacro->fn f))) + (let ((scm (let ((ctx (make-analyze-ctx ns-name))) + (jolt-ce-emit (jolt-ce-analyze ctx fn-form))))) + (loop (cdr forms) + (cons (string-append + "(def-var! " (ei-str-lit ns-name) " " (ei-str-lit nm) "\n " + scm ")\n(mark-macro! " + (ei-str-lit ns-name) " " (ei-str-lit nm) ")") + acc))))) + (else + (let* ((ctx (make-analyze-ctx ns-name)) + (scm (jolt-ce-emit (jolt-ce-analyze ctx f)))) + (loop (cdr forms) (cons scm acc))))))))) + +;; --- the build -------------------------------------------------------------- +;; entry-ns: the app's main namespace (a string). out-path: the binary to write. +;; mode: "dev" | "release" | "optimized" (recorded; optimization passes wired in a +;; later stage). Deps + source roots are already applied by the caller. +(define (build-binary entry-ns out-path mode) + (bld-check-toolchain) + ;; 1. record app namespaces in dependency order as they finish loading. + (let ((app-order '())) + (set-ns-loaded-hook! + (lambda (name file) (set! app-order (cons (cons name file) app-order)))) + (load-namespace entry-ns) + (set-ns-loaded-hook! (lambda (name file) #f)) + (let ((ordered (reverse app-order))) ; deps first, entry last + (when (null? ordered) + (error 'jolt-build (string-append "no source namespace loaded for " entry-ns + " — is it on the source roots?"))) + ;; 2. emit each app namespace. + (let* ((app-strs (apply append + (map (lambda (nf) (bld-emit-ns (car nf) (read-file-string (cdr nf)))) + ordered))) + (builddir (string-append out-path ".build")) + (flat-ss (string-append builddir "/flat.ss")) + (flat-so (string-append builddir "/flat.so")) + (boot (string-append builddir "/jolt.boot")) + (boot-h (string-append builddir "/boot_data.h")) + (main-c (string-append builddir "/main.c"))) + (bld-system (string-append "mkdir -p '" builddir "'")) + ;; 3. flat source = runtime + app + launcher. + (let ((out (open-output-file flat-ss 'replace))) + (bld-emit-runtime out) + (put-string out "\n;; === app ===\n") + (for-each (lambda (s) (put-string out s) (put-string out "\n")) app-strs) + ;; The launcher runs as Chez's scheme-start (so argv reaches -main — + ;; top-level boot forms run during heap build, before args are set), and + ;; suppresses the interactive greeting. + (put-string out "\n;; === launcher ===\n") + (put-string out (string-append + "(suppress-greeting #t)\n" + "(scheme-start\n" + " (lambda args\n" + " (let ((mainv (var-deref " (ei-str-lit entry-ns) " \"-main\")))\n" + " (apply jolt-invoke mainv args))\n" + " (exit 0)))\n")) + (close-port out)) + ;; 4. compile -> boot -> embed -> link. + ;; compile-file/make-boot-file run in a FRESH Chez, not this process: the + ;; loaded runtime shadows `error` (regex.ss, for irregex), which would + ;; otherwise bake a broken `error` reference into the boot. + (display (string-append "jolt build: compiling " entry-ns " (" mode " mode)\n")) + (let ((cs (string-append builddir "/compile.ss"))) + (let ((p (open-output-file cs 'replace))) + (put-string p + (string-append + "(import (chezscheme))\n" + "(compile-file " (ei-str-lit flat-ss) " " (ei-str-lit flat-so) ")\n" + "(make-boot-file " (ei-str-lit boot) " '()\n " + (ei-str-lit (string-append bld-csv-dir "/petite.boot")) "\n " + (ei-str-lit (string-append bld-csv-dir "/scheme.boot")) "\n " + (ei-str-lit flat-so) ")\n")) + (close-port p)) + (bld-system (string-append bld-chez " --script '" cs "'"))) + (bld-system (string-append "xxd -i '" boot "' > '" boot-h "'")) + ;; The xxd symbol is derived from the path; normalize to jolt_boot. + (bld-system (string-append + "sed -i.bak -E 's/unsigned char [A-Za-z0-9_]+\\[\\]/unsigned char jolt_boot[]/; " + "s/unsigned int [A-Za-z0-9_]+_len/unsigned int jolt_boot_len/' '" boot-h "'")) + (let ((mc (open-output-file main-c 'replace))) + (put-string mc + (string-append + "#include \"scheme.h\"\n#include \"boot_data.h\"\n" + "int main(int argc, char *argv[]) {\n" + " Sscheme_init(0);\n" + " Sregister_boot_file_bytes(\"jolt\", jolt_boot, jolt_boot_len);\n" + " Sbuild_heap(0, 0);\n" + " int status = Sscheme_start(argc, (const char **)argv);\n" + " Sscheme_deinit();\n return status;\n}\n")) + (close-port mc)) + (bld-system (string-append + "cc -O2 -I'" bld-csv-dir "' '" main-c "' '" bld-csv-dir "/libkernel.a' " + "-o '" out-path "' " (bld-link-libs))) + (display (string-append "jolt build: wrote " out-path "\n")))))) + +(def-var! "jolt.host" "build-binary" + (lambda (entry out mode) + (build-binary (jolt-str-render-one entry) + (jolt-str-render-one out) + (jolt-str-render-one mode)) + jolt-nil)) diff --git a/host/chez/cli.ss b/host/chez/cli.ss index 349acfd..9f597cb 100644 --- a/host/chez/cli.ss +++ b/host/chez/cli.ss @@ -66,6 +66,11 @@ (display result) (newline)))) ;; otherwise dispatch the argv through jolt.main/-main (else + ;; `build` AOT-compiles an app to a standalone binary — load the build + ;; driver (the cross-compiler emitter) on demand so a normal run never pays + ;; for it. It defines jolt.host/build-binary, which jolt.main's build cmd calls. + (when (and (pair? cli-args) (string=? (car cli-args) "build")) + (load "host/chez/build.ss")) (load-namespace "jolt.main") (let ((mainv (var-deref "jolt.main" "-main"))) (apply jolt-invoke mainv cli-args))))) diff --git a/host/chez/loader.ss b/host/chez/loader.ss index 973e0cf..a0087b9 100644 --- a/host/chez/loader.ss +++ b/host/chez/loader.ss @@ -53,6 +53,23 @@ (vector-for-each (lambda (c) (hashtable-set! loaded-ns (var-cell-ns c) #t)) (hashtable-values var-table)) +;; Does `name` already have vars in the var-table? A namespace baked into the +;; image after the snapshot above — an AOT'd app namespace in a `jolt build` +;; binary — exists in memory with no source file; a later `require` of it must +;; no-op rather than hunt the (absent) source. +(define (ns-has-vars? name) + (let ((found #f)) + (vector-for-each + (lambda (c) (when (and (not found) (string=? (var-cell-ns c) name)) (set! found #t))) + (hashtable-values var-table)) + found)) + +;; Called after a file-backed namespace finishes loading, with (name file). The +;; build driver sets this to record app namespaces in dependency order for AOT +;; emission; a no-op for normal runs. +(define ns-loaded-hook (lambda (name file) #f)) +(define (set-ns-loaded-hook! f) (set! ns-loaded-hook f)) + ;; Read every form from a file and compile+eval it in turn. The first form is ;; normally (ns …), which expands to (in-ns …) and switches the current ns, so ;; later forms compile in that namespace — (chez-current-ns) is re-read each step. @@ -80,17 +97,22 @@ ;; restored afterward, since loading the file switched it. (define (load-namespace name) (unless (hashtable-ref loaded-ns name #f) - (hashtable-set! loaded-ns name #t) (let ((file (find-ns-file name))) - (if (not file) - (begin - (hashtable-delete! loaded-ns name) - (error #f (string-append "Could not locate " (ns-name->rel name) - ".clj (or .cljc) on the source roots") name)) - (let ((saved (chez-current-ns))) - (load-jolt-file file) - ;; restore the current ns (thread-local); *ns* reads derive from it. - (set-chez-ns! saved)))))) + (cond + (file + (hashtable-set! loaded-ns name #t) ; mark before load so a cycle terminates + (let ((saved (chez-current-ns))) + (load-jolt-file file) + ;; restore the current ns (thread-local); *ns* reads derive from it. + (set-chez-ns! saved)) + (ns-loaded-hook name file)) + ;; No source file but the namespace exists in memory (AOT'd into a built + ;; binary): it's already defined — mark loaded and move on. + ((ns-has-vars? name) + (hashtable-set! loaded-ns name #t)) + (else + (error #f (string-append "Could not locate " (ns-name->rel name) + ".clj (or .cljc) on the source roots") name)))))) ;; load-file: load an explicit path (a `run FILE`), in the current ns. (define (jolt-load-file path) diff --git a/jolt-core/jolt/main.clj b/jolt-core/jolt/main.clj index 94c2626..40e674a 100644 --- a/jolt-core/jolt/main.clj +++ b/jolt-core/jolt/main.clj @@ -112,6 +112,28 @@ (map? task) (do (apply-project! resolved) (apply-main-opts (:main-opts task) more)) :else (throw (ex-info (str "bad task " name) {}))))) +;; build [-m NS | FILE] [-o OUT] [--opt | --dev] — AOT-compile the app into a +;; standalone executable. Resolves deps + roots like `run`, then hands the entry +;; namespace to the host build driver (jolt.host/build-binary, defined by +;; build.ss). Default mode is release; --opt selects optimized, --dev unoptimized. +(defn- cmd-build [more] + (apply-project! (deps/resolve-project (project-dir))) + (let [opts (loop [a more, entry nil, out nil] + (cond + (empty? a) {:entry entry :out out} + (= "-m" (first a)) (recur (drop 2 a) (second a) out) + (= "-o" (first a)) (recur (drop 2 a) entry (second a)) + (str/starts-with? (first a) "-") (recur (rest a) entry out) + :else (recur (rest a) (or entry (first a)) out))) + entry (:entry opts) + mode (cond (some #{"--opt"} more) "optimized" + (some #{"--dev"} more) "dev" + :else "release")] + (when (nil? entry) + (throw (ex-info "build needs an entry: -m NS" {}))) + (let [out (or (:out opts) (first (str/split entry #"\.")))] + (jolt.host/build-binary entry out mode)))) + (defn- nrepl [more] ;; resolve the project (deps on the roots, native libs loaded), then start the ;; nREPL server so an editor can connect and (require '[some.lib]) live. A @@ -128,6 +150,7 @@ (println "usage: jolt [args]") (println " run -m NS [args] resolve deps.edn, load NS, call its -main") (println " run FILE load a Clojure file") + (println " build -m NS [-o OUT] [--opt|--dev] compile a standalone binary") (println " -M:alias [args] run the alias's :main-opts") (println " -A:alias [args] add the alias's paths/deps") (println " repl start a line REPL") @@ -146,4 +169,5 @@ (str/starts-with? cmd "-M") (cmd-M cmd more) (str/starts-with? cmd "-A") (cmd-A cmd more) (= cmd "-m") (cmd-run (cons "-m" more)) + (= cmd "build") (cmd-build more) :else (run-task cmd more)))) diff --git a/test/chez/build-app/deps.edn b/test/chez/build-app/deps.edn new file mode 100644 index 0000000..ccd9a31 --- /dev/null +++ b/test/chez/build-app/deps.edn @@ -0,0 +1 @@ +{:paths ["src"]} diff --git a/test/chez/build-app/src/app/core.clj b/test/chez/build-app/src/app/core.clj new file mode 100644 index 0000000..4fd95a0 --- /dev/null +++ b/test/chez/build-app/src/app/core.clj @@ -0,0 +1,7 @@ +(ns app.core + (:require [app.util :as util])) + +(defn -main [& args] + (util/twice (println (util/shout "hello from a built binary"))) + (println "args:" (vec args)) + (println "sum:" (reduce + (map count args)))) diff --git a/test/chez/build-app/src/app/util.clj b/test/chez/build-app/src/app/util.clj new file mode 100644 index 0000000..c4f7429 --- /dev/null +++ b/test/chez/build-app/src/app/util.clj @@ -0,0 +1,8 @@ +(ns app.util + (:require [clojure.string :as str])) + +(defn shout [s] + (str/upper-case (str s "!"))) + +(defmacro twice [x] + `(do ~x ~x))