A (defmethod m …) where m is :refer'd from another ns passed the bare symbol to
defmethod-setup, which resolved it in the current ns and created a shadow multifn
— the method never reached the real one. Resolve an unqualified name through the
refer table (then current ns) so it lands on the referred multifn.
The AOT build strips the ns form, so the refer table is empty in a binary; emit
chez-register-refer!/-refer-all! per app ns alongside the existing alias
registrations. build-app's fixture gains a defmethod on a referred multifn.
The source loader sets the current ns and registers :as aliases per file. The
build flattened every app namespace into one image with no such markers, so all
app forms ran under the last-set ns ("user"). Two breakages followed, both only
in a built binary:
- defmulti/defmethod resolve their target var through chez-current-ns, so they
registered the multifn under "user" while compiled var-derefs used the baked
ns — the multifn the app saw was uninitialized ("not a fn nil" on dispatch).
- a quoted alias-qualified symbol (a (defmethod ig/foo …) on an aliased multifn)
resolves its ns through chez-resolve-alias, but the stripped (ns …) form left
the alias table empty, so it landed in ns "ig".
bld-ns-prelude now emits (set-chez-ns! ns) plus chez-register-alias! for each
ns's :as aliases before that ns's forms, in both the normal and tree-shake emit
paths. The build-app fixture gains a :default multimethod and an aliased cross-ns
defmethod so buildsmoke covers both across all build modes.
--tree-shake now shakes the clojure.core/stdlib prelude in the same reachability
graph as the app + libraries — only core fns actually reached from -main ship.
dce-blob-records reads prelude.ss with Chez `read`, unwraps each
(guard ... (def-var! "ns" "name" V)) and builds the core->core call graph from the
(var-deref/jolt-var "ns" "name") refs in V — the real emitted edges, no
re-analysis. bld-shake-all merges core + app records into one BFS; roots are -main,
side-effecting forms, and the clojure.core fns the runtime .ss shims reference by
name (enumerated in dce-runtime-core-roots). The shaken core is spliced where the
prelude.ss blob was, in original (tier) order, so load-time deps are preserved.
Bail (reachable runtime resolve) keeps the prelude whole.
Soundness follows Stalin's rule (any reference, value or call, keeps a def live):
dce-collect-refs counts :var and :the-var; non-def registration forms are always
kept (covers protocol/multimethod dispatch). Validated by make shakesmoke: the four
deps.edn git-lib apps build byte-identical output shaken vs not.
Wins (binary): build-app 9.84MB -> 8.12MB (dropped 403/457 core defs); malli-app
10.0MB -> 8.1MB; markdown 9.9 -> 8.3; commonmark 9.8 -> 8.1 — all output-identical.
build-smoke asserts an unused core fn (group-by) is dropped; full make test green.
An AOT-compiled app only needs the analyzer/back end at runtime to compile from
source — eval / load-string / load-file. Macros are expanded at build time and a
require of a baked namespace no-ops, so a closed app that never compiles at runtime
carries the compiler image (~0.8MB) as dead weight.
Under --tree-shake, when reachability is trustworthy (no bail) and no reachable code
references eval/load-string/load-file/load-reader/load, omit host/chez/seed/image.ss
+ compile-eval.ss from the runtime manifest. bld-tree-shake returns the flag
alongside the shaken forms; bld-emit-runtime filters the manifest.
Measured: build-app 9.84MB -> 9.05MB, still runs. Safety verified: an app that evals
keeps the compiler (eval is a bail + compile ref) and eval works at runtime.
build-smoke asserts the compiler is gone in the no-eval app; full make test green.
`jolt build --tree-shake` (or deps.edn :jolt/build {:tree-shake true}) does
reachability DCE over the re-emitted app + library namespaces: keep -main, every
side-effecting (non-def) top-level form, and every def reachable from those; drop
the rest. A macro (expanded at AOT, never called at runtime) is prunable too.
Sound: bails (keeps everything) if REACHABLE code resolves vars by name at runtime
(eval/resolve/ns-resolve/requiring-resolve/find-var/intern/load-string/...), which a
static call graph can't follow. Unreached eval-using library code is simply shaken
away and never triggers the bail. clojure.core and the compiler image stay baked
(prelude + image blobs), so only re-emitted namespaces are shaken for now.
The reachability machinery is in emit-image.ss (records: keep?/fqn/refs/str via
reduce-ir-children) + build.ss (BFS + bail check). build-smoke covers it (drops the
unreachable `twice` macro, output unchanged). Opt-in; default builds are untouched.
full make test green.
Scope note: this shakes the re-emitted app/lib code only. Measurement shows jolt's
compiled code is ~5.8MB of a ~9.8MB binary, dominated by the clojure.core prelude
(~1.5-2MB) and the compiler image (~0.8MB) — both baked blobs this pass doesn't
touch. Those (shake-core, drop-compiler-when-no-eval) are the larger footprint wins,
filed as follow-ups.
Release builds can legitimately want runtime dynamism (redefinition, eval,
load-string), so closed-world direct-linking shouldn't be forced on them. Gate it
behind an explicit --direct-link flag (or deps.edn :jolt/build {:direct-link
true}); off by default in every mode, including release and --opt.
build-binary takes an explicit direct-link? arg instead of deriving it from the
mode. build-smoke now covers the --direct-link path and asserts the cross-ns call
actually lowers to a jv$ binding; default release stays dynamically linked.
A built binary dropped its deps.edn :jolt/native declarations and its
resource roots, so an FFI+resources app (ring-app) failed at runtime:
sockets/sqlite gave 'no entry for socket' and io/resource returned nil.
The buildsmoke fixture is pure compute, so neither path was exercised.
The launcher now loads required + :process native libs before the app's
top-level forms (a library's defcfn resolves its foreign-procedure symbols
at top-level eval during startup, so the libs must be loaded first);
optional libs load in the scheme-start launcher, where a missing lib is
caught rather than aborting the heap build.
deps.edn :jolt/build {:embed [dirs]} bakes those dirs' files into the heap
(register-embedded-resource! at heap build), so io/resource serves them with
no files on disk. Non-embedded resources resolve at runtime against JOLT_PWD,
and io/file reads (e.g. config.edn) stay external.
build-binary now takes the encoded natives, embed dirs, and project paths
from cmd-build; deps/resolve-project surfaces them. Buildsmoke fixture grows
an embedded resource + a :process native to cover both paths.
The standalone-binary build needs Chez's kernel dev files (libkernel.a,
scheme.h) and a C compiler. A distro chezscheme package ships neither, so the
gate failed on CI (apt installs petite+scheme only). Preflight for the csv dir
and cc and skip cleanly when they're missing — same pattern as certify skipping
without Clojure. Where the toolchain exists (dev machines), it runs as before;
the discovered csv dir is pinned via JOLT_CHEZ_CSV so the build uses exactly it.
Wire the optimization gate to build mode. inline-enabled? (which gates the
inference + flatten-lets + scalar-replace passes in jolt.passes/run-passes) was
hardwired off, so those passes had never run on Chez at all. host-contract now
exposes a settable hc-optimize? flag; `jolt build --opt` flips it on during app
emission.
Kept off for the default release build for now: the passes are sound by design
(RFC 0005/0006) but unexercised on Chez, so release stays on the proven
var-deref codegen until they're validated against the corpus. --opt is the
opt-in fast path. buildsmoke checks both modes produce the same result.
This does not yet deliver direct call binding — the backend has no direct-link
emission path (every :var call still routes through jolt-invoke/var-deref) and
the inline-ir host stash is still a stub. Those are the remaining stage-4 levers.
Restores the standalone-binary capability the Janet host had. `bin/joltc build
-m NS -o OUT` AOT-compiles an app into a single self-contained executable — the
whole runtime, clojure.core, stdlib and compiler embedded, no Chez install or
jolt source needed at runtime.
Pipeline (host/chez/build.ss, host primitive jolt.host/build-binary driven by
jolt.main's build command): resolve deps, load the entry namespace recording the
app namespaces in dependency order, re-emit each to Scheme, textually inline the
cli.ss runtime load sequence into one flat source + the app + a launcher, then
compile-file -> make-boot-file -> embed the boot as C bytes -> cc-link against
libkernel.a.
Two non-obvious bits: the compile pass runs in a fresh Chez, not the loaded
runtime (regex.ss shadows top-level `error`, which otherwise bakes a broken
reference into the boot); and the launcher installs scheme-start rather than
running -main at top level, since boot top-level forms execute during heap build
before argv is set, so args only reach -main through scheme-start.
Loader: a require of an in-memory namespace with no source file now no-ops, so
AOT'd app namespaces satisfy require in a built binary.
Mode flags (--opt/--dev, default release) are plumbed; the optimization passes
they gate come in a later stage. RFC 0007 has the design. Gated by `make
buildsmoke`.
