Static-link :jolt/native C libraries into built binaries by default

A :jolt/native spec can now carry a :static archive; `jolt build` links it
into the executable, so the app calls the C code with no shared object on the
target. --dynamic (or :jolt/build {:dynamic-natives true}) keeps the old
runtime load-shared-object behavior; a spec with no :static is unchanged.

The cc link force-loads the archive (-force_load on macOS, --whole-archive on
Linux) and exports the executable's symbols (-rdynamic on Linux) so the baked-in
symbols resolve via (load-shared-object #f) + foreign-procedure at startup. Build
step 1 evaluates the app's foreign-procedure forms in-process, so a static
archive is preloaded there as a throwaway shared object to resolve them.

The distributed self-contained joltc has no external cc/Chez but must build these
apps, so it now bundles the Chez kernel (libkernel.a + scheme.h) and the launcher
source and re-links a custom stub with the archives baked in — needing only a
system cc, no Chez. run/repl skip static-only specs (nothing to load); keep a
:darwin/:linux candidate to use such a lib interpreted.

Adds static-native-smoke (cc path) to ci and a static phase to the joltc
self-build smoke (distributed path).
This commit is contained in:
Yogthos 2026-07-01 09:52:00 -04:00
parent a2e99fff45
commit d79ad6dc6a
7 changed files with 366 additions and 33 deletions

View file

@ -4,7 +4,7 @@
# build step. `make test` is the full gate. `make remint` rebuilds the seed after a # build step. `make test` is the full gate. `make remint` rebuilds the seed after a
# source change. # source change.
.PHONY: test ci values corpus unit smoke buildsmoke selfhost sci certify ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline shakesmoke remint joltc joltc-release joltc-debug joltcsmoke .PHONY: test ci values corpus unit smoke buildsmoke staticnativesmoke selfhost sci certify ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline shakesmoke remint joltc joltc-release joltc-debug joltcsmoke
# Full gate (dev machine). Includes the self-host byte-fixpoint, which only holds # Full gate (dev machine). Includes the self-host byte-fixpoint, which only holds
# on the same Chez that minted the seed. # on the same Chez that minted the seed.
@ -15,7 +15,7 @@ test: selfhost ci
# lockfile) — it RUNS correctly on any Chez, but `selfhost` rebuilds it and a # lockfile) — it RUNS correctly on any Chez, but `selfhost` rebuilds it and a
# different Chez version may emit byte-different (gensym/order) output, so the # different Chez version may emit byte-different (gensym/order) output, so the
# byte-fixpoint is a dev-machine check, not a CI one (jolt-8479). # byte-fixpoint is a dev-machine check, not a CI one (jolt-8479).
ci: values corpus unit smoke buildsmoke sci ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline certify ci: values corpus unit smoke buildsmoke staticnativesmoke sci ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline certify
@echo "OK: CI gates passed" @echo "OK: CI gates passed"
# Self-host fixpoint: bootstrap.ss rebuild == checked-in seed. # Self-host fixpoint: bootstrap.ss rebuild == checked-in seed.
@ -42,6 +42,11 @@ smoke:
buildsmoke: buildsmoke:
@sh host/chez/build-smoke.sh @sh host/chez/build-smoke.sh
# `jolt build` cc-links a :jolt/native :static archive into the binary (the
# default), and --dynamic keeps the runtime load-shared-object path.
staticnativesmoke:
@sh host/chez/static-native-smoke.sh
# Build joltc as a self-contained native binary into target/<profile>/joltc. The # Build joltc as a self-contained native binary into target/<profile>/joltc. The
# binary bundles the runtime, compiler, jolt-core + stdlib source, the Chez boots, # binary bundles the runtime, compiler, jolt-core + stdlib source, the Chez boots,
# and a launcher stub, so it runs AND compiles jolt apps with no Chez or cc on the # and a launcher stub, so it runs AND compiles jolt apps with no Chez or cc on the

View file

@ -72,11 +72,41 @@ bindings resolve. Each entry is a map — `{:name "sqlite3" :darwin
the running process's own symbols, e.g. libc sockets, no external file). A the running process's own symbols, e.g. libc sockets, no external file). A
project inherits its dependencies' `:jolt/native`. project inherits its dependencies' `:jolt/native`.
### Static vs dynamic linking
When you `joltc build`, a native lib is **statically linked** into the binary by
default if the spec carries a `:static` archive — so the executable calls the C
code with no shared object present at runtime. Add `:static` alongside the runtime
candidates:
```clojure
{:name "sqlite3"
:static {:archive "/opt/homebrew/lib/libsqlite3.a"} ; or {:lib "sqlite3" :libdir "/usr/lib"}
:darwin ["libsqlite3.0.dylib"] ; still used by `run`/`repl` and by --dynamic
:linux ["libsqlite3.so.0"]}
```
`:static {:archive PATH}` force-loads the whole `.a` and is the reliable
cross-platform form. `:static {:lib NAME :libdir DIR}` links `-lNAME` (with a
`-Bstatic` preference on Linux); on macOS, which has no `-Bstatic`, prefer the
archive form. A spec with no `:static` (or a build passed `--dynamic`, or
`:jolt/build {:dynamic-natives true}`) keeps the old behavior — the shared object
is loaded at startup via `load-shared-object`.
Static linking needs a C compiler (`cc`) on `PATH` at build time (plus the C libs
the Chez kernel links — lz4, zlib, ncurses). The distributed `joltc` bundles the
Chez kernel, so it re-links the launcher stub with the archive baked in — no
external Chez, just `cc`. Without a `cc`, a `:static` lib fails with a message
pointing you to install one or pass `--dynamic`. Keep a `:darwin`/`:linux`
candidate on any `:static` spec so `run`/`repl` (which have no static binary) can
still load it.
## Standalone binaries ## Standalone binaries
`joltc build -m NS` compiles the app and every library into one executable (the `joltc build -m NS` compiles the app and every library into one executable (the
runtime + compiler are baked in). It loads the resolved `:jolt/native` libs at runtime + compiler are baked in). Resolved `:jolt/native` libs are statically
startup, so an FFI app — sockets, SQLite — runs with no jolt or Chez on the path. linked in (or loaded at startup — see [Native libraries](#native-libraries)), so
an FFI app — sockets, SQLite — runs with no jolt or Chez on the path.
Output goes under the project's `target/`, cargo-style: `target/release/<project>` Output goes under the project's `target/`, cargo-style: `target/release/<project>`
by default and with `--opt`, `target/debug/<project>` with `--dev` (the by default and with `--opt`, `target/debug/<project>` with `--dev` (the

View file

@ -101,7 +101,10 @@
(register-embedded-bytes! (car spec) bv))) (register-embedded-bytes! (car spec) bv)))
'((\"csv/petite.boot\" \"jolt_petite_boot\" \"jolt_petite_boot_len\") '((\"csv/petite.boot\" \"jolt_petite_boot\" \"jolt_petite_boot_len\")
(\"csv/scheme.boot\" \"jolt_scheme_boot\" \"jolt_scheme_boot_len\") (\"csv/scheme.boot\" \"jolt_scheme_boot\" \"jolt_scheme_boot_len\")
(\"stub/launcher\" \"jolt_stub\" \"jolt_stub_len\"))))) (\"stub/launcher\" \"jolt_stub\" \"jolt_stub_len\")
(\"csv/scheme.h\" \"jolt_scheme_h\" \"jolt_scheme_h_len\")
(\"csv/libkernel.a\" \"jolt_libkernel_a\" \"jolt_libkernel_a_len\")
(\"stub/launcher.c\" \"jolt_launcher_c\" \"jolt_launcher_c_len\")))))
(suppress-greeting #t) (suppress-greeting #t)
(scheme-start (scheme-start
@ -182,6 +185,13 @@
(jb-c-array (string-append bld-csv-dir "/petite.boot") (string-append jb-build "/petite_data.h") "jolt_petite_boot") (jb-c-array (string-append bld-csv-dir "/petite.boot") (string-append jb-build "/petite_data.h") "jolt_petite_boot")
(jb-c-array (string-append bld-csv-dir "/scheme.boot") (string-append jb-build "/scheme_data.h") "jolt_scheme_boot") (jb-c-array (string-append bld-csv-dir "/scheme.boot") (string-append jb-build "/scheme_data.h") "jolt_scheme_boot")
(jb-c-array jb-stub (string-append jb-build "/stub_data.h") "jolt_stub") (jb-c-array jb-stub (string-append jb-build "/stub_data.h") "jolt_stub")
;; Also bundle the Chez kernel (libkernel.a + scheme.h) and the launcher source,
;; so a `build` with :static native libs can re-link a custom stub with those
;; archives baked in — the appended-stub path can't add object code to a prebuilt
;; stub, so it relinks (build.ss bld-relink-stub). Needs the system cc at build.
(jb-c-array (string-append bld-csv-dir "/scheme.h") (string-append jb-build "/schemeh_data.h") "jolt_scheme_h")
(jb-c-array (string-append bld-csv-dir "/libkernel.a") (string-append jb-build "/libkernel_data.h") "jolt_libkernel_a")
(jb-c-array "host/chez/stub/launcher.c" (string-append jb-build "/launcherc_data.h") "jolt_launcher_c")
(define jb-main-c (string-append jb-build "/main.c")) (define jb-main-c (string-append jb-build "/main.c"))
(let ((mc (open-output-file jb-main-c 'replace))) (let ((mc (open-output-file jb-main-c 'replace)))
@ -192,6 +202,9 @@
"#include \"petite_data.h\"\n" "#include \"petite_data.h\"\n"
"#include \"scheme_data.h\"\n" "#include \"scheme_data.h\"\n"
"#include \"stub_data.h\"\n" "#include \"stub_data.h\"\n"
"#include \"schemeh_data.h\"\n"
"#include \"libkernel_data.h\"\n"
"#include \"launcherc_data.h\"\n"
"int main(int argc, char *argv[]) {\n" "int main(int argc, char *argv[]) {\n"
" Sscheme_init(0);\n" " Sscheme_init(0);\n"
" Sregister_boot_file_bytes(\"jolt\", jolt_boot, jolt_boot_len);\n" " Sregister_boot_file_bytes(\"jolt\", jolt_boot, jolt_boot_len);\n"

View file

@ -80,6 +80,9 @@
(cand (string-append bindir "/../lib/csv" bld-version "/" bld-machine))) (cand (string-append bindir "/../lib/csv" bld-version "/" bld-machine)))
cand)))) cand))))
(define (bld-have-cc?)
(> (string-length (bld-sh-capture "command -v cc")) 0))
(define (bld-check-toolchain) (define (bld-check-toolchain)
(for-each (for-each
(lambda (f) (lambda (f)
@ -271,21 +274,24 @@
(define (bld-strs x) (map jolt-str-render-one (seq->list x))) (define (bld-strs x) (map jolt-str-render-one (seq->list x)))
;; Emit native-library loads. `natives` is the encoded jolt seq jolt.main/ ;; Emit native-library loads. `natives` is the encoded jolt seq jolt.main/
;; encode-natives produced: each entry is ["process"] | ["req" cand…] | ["opt" cand…]. ;; encode-natives produced: each entry is ["process"] | ["static" form…] |
;; `which` selects 'required (process + req) or 'optional. Required + process loads ;; ["req" cand…] | ["opt" cand…]. `which` selects 'required (process + static +
;; are emitted before the app forms (the app's defcfn foreign-procedures resolve ;; req) or 'optional. Required loads are emitted before the app forms (the app's
;; their symbols at top-level eval during startup, so the libs must be loaded ;; defcfn foreign-procedures resolve their symbols at top-level eval during
;; first); a load-shared-object failure there is fatal — correct for a required ;; startup, so the libs must be loaded first); a load-shared-object failure there
;; lib. Optional loads run in the scheme-start launcher, where guard catches a ;; is fatal — correct for a required lib. A "static" lib is cc-linked into the
;; missing lib (an optional lib's namespace is only present when the app requires ;; binary (see bld-native-link-flags), so its symbols are already in the process:
;; it, so its foreign-procedures aren't among the baked top-level forms). ;; it loads them the same way a "process" lib does. Optional loads run in the
;; scheme-start launcher, where guard catches a missing lib (an optional lib's
;; namespace is only present when the app requires it, so its foreign-procedures
;; aren't among the baked top-level forms).
(define (bld-emit-natives out natives which) (define (bld-emit-natives out natives which)
(for-each (for-each
(lambda (entry) (lambda (entry)
(let* ((parts (bld-strs entry)) (kind (car parts)) (cands (cdr parts)) (let* ((parts (bld-strs entry)) (kind (car parts)) (cands (cdr parts))
(cand-lits (fold-left (lambda (s c) (string-append s (ei-str-lit c) " ")) "" cands))) (cand-lits (fold-left (lambda (s c) (string-append s (ei-str-lit c) " ")) "" cands)))
(cond (cond
((and (eq? which 'required) (string=? kind "process")) ((and (eq? which 'required) (or (string=? kind "process") (string=? kind "static")))
(put-string out "(jolt-build-load-native '() #f #t)\n")) (put-string out "(jolt-build-load-native '() #f #t)\n"))
((and (eq? which 'required) (string=? kind "req")) ((and (eq? which 'required) (string=? kind "req"))
(put-string out (string-append "(jolt-build-load-native (list " cand-lits ") #f #f)\n"))) (put-string out (string-append "(jolt-build-load-native (list " cand-lits ") #f #f)\n")))
@ -293,6 +299,66 @@
(put-string out (string-append "(jolt-build-load-native (list " cand-lits ") #t #f)\n")))))) (put-string out (string-append "(jolt-build-load-native (list " cand-lits ") #t #f)\n"))))))
(seq->list natives))) (seq->list natives)))
;; The cc link fragment for the "static" natives: each archive must be FORCE-loaded
;; (the linker would otherwise drop an archive member main.c never references) and,
;; on Linux, the executable's symbols exported into the dynamic table so the
;; startup (load-shared-object #f) + foreign-procedure can resolve them (-rdynamic,
;; added by build-with-cc when this fragment is non-empty). Returns "" when no lib
;; is statically linked. Entry forms: ["static" "archive" path] | ["static" "lib"
;; name libdir].
(define (bld-native-link-flags natives)
(fold-left
(lambda (acc entry)
(let ((parts (bld-strs entry)))
(if (string=? (car parts) "static")
(string-append acc " " (bld-one-static-link (cdr parts)))
acc)))
"" (seq->list natives)))
;; A statically-linked native is only in the OUTPUT binary, but build step 1
;; evaluates the app's `foreign-procedure` forms in THIS process (to register its
;; macros/vars), and Chez resolves a foreign entry eagerly. So make the archive's
;; symbols resolvable here: build a throwaway shared object from it (force-loading
;; every member) and load it. The output binary still cc-links the static archive;
;; this temp .so is build-time only. Only the "archive" form is preloaded — the
;; "lib" form names a system library the OS loader already finds by soname.
(define (bld-preload-static-natives! natives builddir)
(let ((n 0))
(for-each
(lambda (entry)
(let ((parts (bld-strs entry)))
(when (and (string=? (car parts) "static") (string=? (cadr parts) "archive"))
(let* ((archive (caddr parts))
(so (string-append builddir "/native-" (number->string n)
(if bld-osx? ".dylib" ".so"))))
(set! n (+ n 1))
(bld-system
(if bld-osx?
(string-append "cc -dynamiclib -undefined dynamic_lookup -Wl,-all_load '"
archive "' -o '" so "'")
(string-append "cc -shared -Wl,--whole-archive '" archive
"' -Wl,--no-whole-archive -Wl,--unresolved-symbols=ignore-all -o '" so "'")))
(load-shared-object so)))))
(seq->list natives))))
(define (bld-one-static-link form)
(let ((kind (car form)))
(cond
((string=? kind "archive")
(let ((path (cadr form)))
(if bld-osx?
(string-append "-Wl,-force_load," path)
(string-append "-Wl,--whole-archive " path " -Wl,--no-whole-archive"))))
((string=? kind "lib")
(let* ((lib (cadr form)) (dir (caddr form))
(L (if (> (string-length dir) 0) (string-append "-L" dir " ") "")))
;; -Bstatic forces the .a over a .so of the same -l name (GNU ld). macOS's
;; ld64 has no -Bstatic; there an :archive path is the reliable form.
(if bld-osx?
(string-append L "-l" lib)
(string-append L "-Wl,-Bstatic -l" lib " -Wl,-Bdynamic"))))
(else ""))))
;; Walk an embed root recursively; return (resource-name . abspath) pairs, where ;; Walk an embed root recursively; return (resource-name . abspath) pairs, where
;; resource-name is the "/"-joined path under the root (what io/resource is asked for). ;; resource-name is the "/"-joined path under the root (what io/resource is asked for).
(define (bld-walk-files root rel acc) (define (bld-walk-files root rel acc)
@ -338,6 +404,19 @@
;; The self-contained path (jolt-embedded-bytes "stub/launcher") needs no csv ;; The self-contained path (jolt-embedded-bytes "stub/launcher") needs no csv
;; kernel files, no Chez, no cc — only the legacy cc path does. ;; kernel files, no Chez, no cc — only the legacy cc path does.
(unless (jolt-embedded-bytes "stub/launcher") (bld-check-toolchain)) (unless (jolt-embedded-bytes "stub/launcher") (bld-check-toolchain))
(when (> (string-length (bld-native-link-flags natives)) 0)
;; :static natives are cc-linked into the binary, so a C compiler must be on
;; PATH — the self-contained joltc bundles the Chez kernel (libkernel.a +
;; scheme.h) and relinks a custom stub (see build-self-contained), but still
;; needs the system cc for that link. Fail early (before the app's foreign-
;; procedure forms eval below) with an actionable message.
(unless (bld-have-cc?)
(error 'jolt-build
"static native linking needs a C compiler (cc) on PATH; install one, or pass --dynamic to load the library at runtime."))
;; Preload static archives' symbols into this process so step 1's foreign-
;; procedure evals resolve; the .build dir must exist first.
(bld-mkdir-p (string-append out-path ".build"))
(bld-preload-static-natives! natives (string-append out-path ".build")))
;; 1. record app namespaces in dependency order as they finish loading. ;; 1. record app namespaces in dependency order as they finish loading.
(let ((app-order '())) (let ((app-order '()))
(set-ns-loaded-hook! (set-ns-loaded-hook!
@ -460,8 +539,10 @@
;; make-boot-file, then xxd the boot into a C array and cc-link against ;; make-boot-file, then xxd the boot into a C array and cc-link against
;; libkernel.a. Kept so `make buildsmoke` still exercises the cc path. ;; libkernel.a. Kept so `make buildsmoke` still exercises the cc path.
(if (jolt-embedded-bytes "stub/launcher") (if (jolt-embedded-bytes "stub/launcher")
(build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot) (build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot
(build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c))))))) (bld-native-link-flags natives))
(build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c
(bld-native-link-flags natives))))))))
;; --- self-contained link (in-process compile + append the boot to the stub) --- ;; --- self-contained link (in-process compile + append the boot to the stub) ---
;; compile-file runs with a FRESH scheme-environment as the interaction ;; compile-file runs with a FRESH scheme-environment as the interaction
@ -470,7 +551,7 @@
;; runs) must bind to the kernel primitive — a clean env gives exactly that, the ;; runs) must bind to the kernel primitive — a clean env gives exactly that, the
;; same as the legacy path compiling in a fresh Chez process. Without it the boot ;; same as the legacy path compiling in a fresh Chez process. Without it the boot
;; dies at startup with "variable error is not bound". ;; dies at startup with "variable error is not bound".
(define (build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot) (define (build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot native-link)
(let ((petite (string-append builddir "/petite.boot")) (let ((petite (string-append builddir "/petite.boot"))
(scheme (string-append builddir "/scheme.boot"))) (scheme (string-append builddir "/scheme.boot")))
(jolt-spill-embedded! "csv/petite.boot" petite) (jolt-spill-embedded! "csv/petite.boot" petite)
@ -479,8 +560,14 @@
(parameterize ((interaction-environment (copy-environment (scheme-environment)))) (parameterize ((interaction-environment (copy-environment (scheme-environment))))
(compile-file flat-ss flat-so) (compile-file flat-ss flat-so)
(make-boot-file boot '() petite scheme flat-so)) (make-boot-file boot '() petite scheme flat-so))
;; The stub is the native launcher the boot is appended to. With no :static
;; natives it's the prebuilt one bundled in joltc (no cc needed); with :static
;; natives it's re-linked here from the bundled kernel + launcher source so the
;; archives are baked in and their symbols resolve in the running binary.
(if (> (string-length native-link) 0)
(bld-relink-stub builddir native-link out-path)
(jolt-spill-embedded! "stub/launcher" out-path))
;; link: stub bytes ++ boot ++ frame, then make it executable. ;; link: stub bytes ++ boot ++ frame, then make it executable.
(jolt-spill-embedded! "stub/launcher" out-path)
(jolt-append-payload! out-path (read-file-bytes boot)) (jolt-append-payload! out-path (read-file-bytes boot))
(jolt-chmod-755 out-path) (jolt-chmod-755 out-path)
(display (string-append "jolt build: wrote " out-path "\n")) (display (string-append "jolt build: wrote " out-path "\n"))
@ -489,8 +576,27 @@
"jolt build: note — on macOS this binary is unsigned; to share it,\n" "jolt build: note — on macOS this binary is unsigned; to share it,\n"
" `xattr -d com.apple.quarantine " out-path "` on the target, or sign it.\n"))))) " `xattr -d com.apple.quarantine " out-path "` on the target, or sign it.\n")))))
;; Re-link the launcher stub with the app's static native archives baked in, to
;; OUT-PATH. The self-contained joltc bundles the Chez kernel (libkernel.a),
;; header, and launcher source; spill them and drive the system cc — the same link
;; build-joltc.ss ran once at joltc-build time, plus the force-load archive flags
;; (native-link) and, on Linux, -rdynamic so the baked-in symbols stay dlsym-
;; visible for (load-shared-object #f) + foreign-procedure at startup.
(define (bld-relink-stub builddir native-link out-path)
(let ((h (string-append builddir "/scheme.h"))
(lk (string-append builddir "/libkernel.a"))
(lc (string-append builddir "/launcher.c")))
(jolt-spill-embedded! "csv/scheme.h" h)
(jolt-spill-embedded! "csv/libkernel.a" lk)
(jolt-spill-embedded! "stub/launcher.c" lc)
(display "jolt build: relinking launcher stub with static native libraries\n")
(bld-system (string-append
"cc -O2 " (if bld-osx? "" "-rdynamic ")
"-I'" builddir "' '" lc "' '" lk "' -o '" out-path "' "
(bld-link-libs) native-link))))
;; --- legacy cc link (dev bin/joltc): fresh Chez compile + xxd + cc ------------ ;; --- legacy cc link (dev bin/joltc): fresh Chez compile + xxd + cc ------------
(define (build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c) (define (build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c native-link)
(display (string-append "jolt build: compiling " entry-ns " (" mode " mode)\n")) (display (string-append "jolt build: compiling " entry-ns " (" mode " mode)\n"))
(let ((cs (string-append builddir "/compile.ss"))) (let ((cs (string-append builddir "/compile.ss")))
(let ((p (open-output-file cs 'replace))) (let ((p (open-output-file cs 'replace)))
@ -520,9 +626,13 @@
" int status = Sscheme_start(argc, (const char **)argv);\n" " int status = Sscheme_start(argc, (const char **)argv);\n"
" Sscheme_deinit();\n return status;\n}\n")) " Sscheme_deinit();\n return status;\n}\n"))
(close-port mc)) (close-port mc))
;; -rdynamic (Linux) exports the executable's symbols into the dynamic table so
;; a statically-linked native lib's symbols resolve via (load-shared-object #f)
;; at startup. macOS keeps unstripped executable symbols dlsym-visible already.
(bld-system (string-append (bld-system (string-append
"cc -O2 -I'" bld-csv-dir "' '" main-c "' '" bld-csv-dir "/libkernel.a' " "cc -O2 " (if (and (not bld-osx?) (> (string-length native-link) 0)) "-rdynamic " "")
"-o '" out-path "' " (bld-link-libs))) "-I'" bld-csv-dir "' '" main-c "' '" bld-csv-dir "/libkernel.a' "
"-o '" out-path "' " (bld-link-libs) native-link))
(display (string-append "jolt build: wrote " out-path "\n"))) (display (string-append "jolt build: wrote " out-path "\n")))
(def-var! "jolt.host" "build-binary" (def-var! "jolt.host" "build-binary"

View file

@ -73,4 +73,36 @@ if [ "$got" != "$want" ]; then
echo "--- got ----"; echo "$got" echo "--- got ----"; echo "$got"
exit 1 exit 1
fi fi
echo "joltc self-build smoke: passed (joltc runs + builds a working app with no external toolchain)"
# 5. Static native linking through the distributed joltc: it bundles the Chez
# kernel, so with the system cc (but still no external Chez) it re-links a stub
# that bakes a :jolt/native :static archive into the app. The app then calls the
# C function with the archive removed from disk. Uses the normal PATH so cc — and
# the kernel's link deps (lz4/…) — are found, but Chez stays out of the build.
napp="$(mktemp -d)/native-app"
mkdir -p "$napp/src/app"
printf 'int jolt_static_answer(void){return 42;}\n' > "$napp/greet.c"
cc -c "$napp/greet.c" -o "$napp/greet.o" && ar rcs "$napp/libgreet.a" "$napp/greet.o"
cat > "$napp/src/app/core.clj" <<'EOF'
(ns app.core (:require [jolt.ffi :as ffi]))
(ffi/defcfn answer "jolt_static_answer" [] :int)
(defn -main [& _] (println "answer:" (answer)))
EOF
cat > "$napp/deps.edn" <<EOF
{:paths ["src"]
:jolt/native [{:name "greet" :static {:archive "$napp/libgreet.a"}}]}
EOF
nout="$napp/app"
echo "joltc self-build smoke: static-linking a native lib via the binary (no external Chez)"
if ! JOLT_PWD="$napp" "$joltc" build -m app.core -o "$nout" >/dev/null 2>&1; then
echo " FAIL: static native build via distributed joltc exited non-zero"
rm -rf "$(dirname "$napp")"; exit 1
fi
rm -f "$napp/libgreet.a" "$napp/greet.o" # nothing to load at runtime
got_n="$(cd / && "$nout" 2>&1)"
rm -rf "$(dirname "$napp")"
if [ "$got_n" != "answer: 42" ]; then
echo " FAIL: static-linked app (via distributed joltc) output mismatch"
echo "--- got ----"; echo "$got_n"; exit 1
fi
echo "joltc self-build smoke: passed (joltc runs + builds a working app with no external toolchain, incl. static native linking)"

View file

@ -0,0 +1,115 @@
#!/bin/sh
# static-native smoke: a project's :jolt/native lib with a :static archive is
# LINKED INTO the built binary (the default), so the binary calls the C function
# with no shared object on disk at runtime. --dynamic keeps the old behavior —
# load a shared object at runtime.
root="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)"
cd "$root"
# Preflight: needs cc (to build the test libs AND to cc-link the app) + Chez's
# kernel dev files, same as build-smoke. Skip otherwise (CI on a distro package).
csv="$JOLT_CHEZ_CSV"
if [ -z "$csv" ]; then
chez_bin="$(command -v chez || command -v scheme || command -v petite || true)"
if [ -n "$chez_bin" ]; then
base="$(cd "$(dirname "$chez_bin")/.." 2>/dev/null && pwd)"
for d in "$base"/lib/csv*/*/; do
[ -f "${d}libkernel.a" ] && csv="${d%/}" && break
done
fi
fi
if ! command -v cc >/dev/null 2>&1 || [ -z "$csv" ] || [ ! -f "$csv/scheme.h" ] || [ ! -f "$csv/libkernel.a" ]; then
echo "static-native smoke: skipped (Chez kernel dev files or C compiler not available)"
exit 0
fi
export JOLT_CHEZ_CSV="$csv"
case "$(uname -s)" in
Darwin) plat=":darwin"; soext="dylib"; shared="-dynamiclib" ;;
*) plat=":linux"; soext="so"; shared="-shared" ;;
esac
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
app="$work/app"
mkdir -p "$app/src/app"
# 1. a trivial C library, built BOTH as a static archive and a shared object.
cat > "$work/greet.c" <<'EOF'
int jolt_static_answer(void) { return 42; }
EOF
cc -c "$work/greet.c" -o "$work/greet.o"
ar rcs "$work/libgreet.a" "$work/greet.o"
cc $shared "$work/greet.c" -o "$work/libgreet.$soext"
# 2. an app that binds that symbol via FFI.
cat > "$app/src/app/core.clj" <<'EOF'
(ns app.core
(:require [jolt.ffi :as ffi]))
(ffi/defcfn answer "jolt_static_answer" [] :int)
(defn -main [& _]
(println "answer:" (answer)))
EOF
out="$work/app-bin"
# --- default: static link ---------------------------------------------------
# A static-only spec (no runtime candidate): the build resolves the symbol by
# preloading the archive, and the binary links it in — nothing to load at runtime.
cat > "$app/deps.edn" <<EOF
{:paths ["src"]
:jolt/native [{:name "greet" :static {:archive "$work/libgreet.a"}}]}
EOF
echo "static-native smoke: building (default: static link)"
if ! JOLT_PWD="$app" bin/joltc build -m app.core -o "$out" >"$work/build.log" 2>&1; then
echo " FAIL: jolt build (static) exited non-zero"; cat "$work/build.log"; exit 1
fi
[ -x "$out" ] || { echo " FAIL: no executable produced"; exit 1; }
# A static lib emits a process-symbol load (its archive is in-process), not a
# dlopen of the shared object.
if ! grep -q "jolt-build-load-native '() #f #t" "$out.build/flat.ss"; then
echo " FAIL: static native did not emit a process-symbol load"; exit 1
fi
if grep -q "libgreet.$soext" "$out.build/flat.ss"; then
echo " FAIL: static native baked a runtime shared-object load"; exit 1
fi
# Remove BOTH libs: a static-linked symbol lives in the binary, nothing to load.
rm -f "$work/libgreet.a" "$work/libgreet.$soext" "$work/greet.o"
got="$(cd / && "$out" 2>&1)"
if [ "$got" != "answer: 42" ]; then
echo " FAIL: static-linked binary output mismatch"
echo "--- want ---"; echo "answer: 42"; echo "--- got ----"; echo "$got"; exit 1
fi
# --- --dynamic: runtime load ------------------------------------------------
# Rebuild the shared object (static phase deleted it) and give the spec a runtime
# candidate; --dynamic loads it at startup instead of linking the archive.
cc $shared "$work/greet.c" -o "$work/libgreet.$soext"
cat > "$app/deps.edn" <<EOF
{:paths ["src"]
:jolt/native [{:name "greet"
:static {:archive "$work/libgreet.a"}
$plat ["$work/libgreet.$soext"]}]}
EOF
echo "static-native smoke: building (--dynamic: runtime load)"
if ! JOLT_PWD="$app" bin/joltc build -m app.core -o "$out" --dynamic >"$work/build.log" 2>&1; then
echo " FAIL: jolt build --dynamic exited non-zero"; cat "$work/build.log"; exit 1
fi
# --dynamic loads the shared object at runtime.
if ! grep -q "libgreet.$soext" "$out.build/flat.ss"; then
echo " FAIL: --dynamic did not emit a runtime shared-object load"; exit 1
fi
got="$(cd / && "$out" 2>&1)"
if [ "$got" != "answer: 42" ]; then
echo " FAIL: --dynamic binary output mismatch (shared object present)"
echo "--- got ----"; echo "$got"; exit 1
fi
# With the shared object gone, a --dynamic binary must FAIL — proving the symbol
# was loaded at runtime, not baked in.
rm -f "$work/libgreet.$soext"
rc=0; { (cd / && exec "$out"); } >/dev/null 2>&1 || rc=$?
if [ "$rc" -eq 0 ]; then
echo " FAIL: --dynamic binary still ran with its shared object removed"; exit 1
fi
echo "static-native smoke: passed (static default + --dynamic runtime load)"

View file

@ -28,7 +28,11 @@
(let [c (get spec plat) (let [c (get spec plat)
cands (if (string? c) [c] (vec c)) cands (if (string? c) [c] (vec c))
hit (some #(when (jolt.ffi/loaded? %) %) cands)] hit (some #(when (jolt.ffi/loaded? %) %) cands)]
(when (and (nil? hit) (not (:optional spec))) ;; A :static spec has no runtime shared object (it's linked into a
;; built binary), so an interpreted `run`/`repl` has nothing to load —
;; skip it rather than fail. Its foreign calls only resolve in a static
;; build; document a dynamic candidate too to use it under `run`.
(when (and (nil? hit) (not (:optional spec)) (not (:static spec)))
(throw (ex-info (str "required native library " (throw (ex-info (str "required native library "
(or (:name spec) (first cands) "?") (or (:name spec) (first cands) "?")
" not found — tried " (pr-str cands) " for " (name plat)) " not found — tried " (pr-str cands) " for " (name plat))
@ -174,18 +178,38 @@
;; --direct-link (or deps.edn :jolt/build {:direct-link true}) opts into closed-world ;; --direct-link (or deps.edn :jolt/build {:direct-link true}) opts into closed-world
;; direct-linking: app->app calls bind directly, giving up runtime redefinition of ;; direct-linking: app->app calls bind directly, giving up runtime redefinition of
;; those vars and eval/load-string. Off by default — release stays dynamically linked. ;; those vars and eval/load-string. Off by default — release stays dynamically linked.
;; The static-link description of a :jolt/native spec for this platform, or nil.
;; :static may be flat ({:archive "…"} / {:lib "z" :libdir "…"}) or per-platform
;; ({:darwin {…} :linux {…}}). Returns a vector build.ss reads and wraps in the
;; platform's force-load flags: ["archive" abspath] or ["lib" name libdir].
(defn- static-link-spec [spec plat]
(when-let [s (:static spec)]
(let [p (get s plat)
s (if (map? p) p s)]
(cond
(:archive s) ["archive" (:archive s)]
(:lib s) ["lib" (:lib s) (or (:libdir s) "")]
:else nil))))
;; Encode a deps.edn :jolt/native spec for the build launcher, resolving the ;; Encode a deps.edn :jolt/native spec for the build launcher, resolving the
;; current platform's candidate list now (the binary runs on this OS). Each entry ;; current platform's candidate list now (the binary runs on this OS). Each entry
;; becomes a vector the launcher (build.ss) reads: ["process"] for the running ;; becomes a vector the launcher (build.ss) reads:
;; binary's own symbols, else ["req"|"opt" cand…] to try in turn. ;; ["process"] — the running binary's own symbols (libc)
(defn- encode-natives [natives] ;; ["static" form …] — the lib's archive, cc-linked into the binary; its
;; symbols load from the process (default when :static
;; is present and --dynamic wasn't passed)
;; ["req"|"opt" cand…] — load a shared object at runtime, trying each in turn
;; dynamic? forces the runtime path for every lib (the --dynamic build flag).
(defn- encode-natives [natives dynamic?]
(let [plat (current-platform)] (let [plat (current-platform)]
(vec (for [spec natives] (vec (for [spec natives]
(if (:process spec) (let [static (and (not dynamic?) (static-link-spec spec plat))]
["process"] (cond
(let [c (get spec plat) (:process spec) ["process"]
cands (if (string? c) [c] (vec c))] static (into ["static"] static)
(into [(if (:optional spec) "opt" "req")] cands))))))) :else (let [c (get spec plat)
cands (if (string? c) [c] (vec c))]
(into [(if (:optional spec) "opt" "req")] cands))))))))
(defn- cmd-build [more] (defn- cmd-build [more]
(let [{:keys [project-paths embed-dirs build] :as resolved} (let [{:keys [project-paths embed-dirs build] :as resolved}
@ -219,7 +243,11 @@
(nil? o) (str pdir "/target/" (if (= mode "dev") "debug" "release") "/" proj) (nil? o) (str pdir "/target/" (if (= mode "dev") "debug" "release") "/" proj)
(str/starts-with? o "/") o (str/starts-with? o "/") o
:else (str pdir "/" o))) :else (str pdir "/" o)))
natives (encode-natives (:natives resolved)) ;; :jolt/native libs with a :static archive are cc-linked into the
;; binary by default; --dynamic (or deps.edn :jolt/build {:dynamic-natives
;; true}) keeps the old behavior — load a shared object at runtime.
dynamic-natives? (boolean (or (some #{"--dynamic"} more) (:dynamic-natives build)))
natives (encode-natives (:natives resolved) dynamic-natives?)
;; closed-world direct-linking is opt-in: the --direct-link flag or a ;; closed-world direct-linking is opt-in: the --direct-link flag or a
;; deps.edn :jolt/build {:direct-link true}. Off otherwise. ;; deps.edn :jolt/build {:direct-link true}. Off otherwise.
direct-link? (boolean (or (some #{"--direct-link"} more) (:direct-link build))) direct-link? (boolean (or (some #{"--direct-link"} more) (:direct-link build)))
@ -269,7 +297,7 @@
(println "usage: jolt <command> [args]") (println "usage: jolt <command> [args]")
(println " run -m NS [args] resolve deps.edn, load NS, call its -main") (println " run -m NS [args] resolve deps.edn, load NS, call its -main")
(println " run FILE load a Clojure file") (println " run FILE load a Clojure file")
(println " build -m NS [-o OUT] [--opt|--dev] [--direct-link] [--tree-shake] compile a standalone binary") (println " build -m NS [-o OUT] [--opt|--dev] [--direct-link] [--tree-shake] [--dynamic] compile a standalone binary")
(println " -M:alias [args] run the alias's :main-opts") (println " -M:alias [args] run the alias's :main-opts")
(println " -A:alias [args] add the alias's paths/deps") (println " -A:alias [args] add the alias's paths/deps")
(println " repl start a line REPL") (println " repl start a line REPL")