Static-link :jolt/native C libraries into built binaries by default
A :jolt/native spec can now carry a :static archive; `jolt build` links it
into the executable, so the app calls the C code with no shared object on the
target. --dynamic (or :jolt/build {:dynamic-natives true}) keeps the old
runtime load-shared-object behavior; a spec with no :static is unchanged.
The cc link force-loads the archive (-force_load on macOS, --whole-archive on
Linux) and exports the executable's symbols (-rdynamic on Linux) so the baked-in
symbols resolve via (load-shared-object #f) + foreign-procedure at startup. Build
step 1 evaluates the app's foreign-procedure forms in-process, so a static
archive is preloaded there as a throwaway shared object to resolve them.
The distributed self-contained joltc has no external cc/Chez but must build these
apps, so it now bundles the Chez kernel (libkernel.a + scheme.h) and the launcher
source and re-links a custom stub with the archives baked in — needing only a
system cc, no Chez. run/repl skip static-only specs (nothing to load); keep a
:darwin/:linux candidate to use such a lib interpreted.
Adds static-native-smoke (cc path) to ci and a static phase to the joltc
self-build smoke (distributed path).
This commit is contained in:
Yogthos
parent
a2e99fff45
commit
d79ad6dc6a
7 changed files with 366 additions and 33 deletions
9
Makefile
9
Makefile
|
|
@ -4,7 +4,7 @@
|
|||
# build step. `make test` is the full gate. `make remint` rebuilds the seed after a
|
||||
# source change.
|
||||
|
||||
.PHONY: test ci values corpus unit smoke buildsmoke selfhost sci certify ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline shakesmoke remint joltc joltc-release joltc-debug joltcsmoke
|
||||
.PHONY: test ci values corpus unit smoke buildsmoke staticnativesmoke selfhost sci certify ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline shakesmoke remint joltc joltc-release joltc-debug joltcsmoke
|
||||
|
||||
# Full gate (dev machine). Includes the self-host byte-fixpoint, which only holds
|
||||
# on the same Chez that minted the seed.
|
||||
|
|
@ -15,7 +15,7 @@ test: selfhost ci
|
|||
# lockfile) — it RUNS correctly on any Chez, but `selfhost` rebuilds it and a
|
||||
# different Chez version may emit byte-different (gensym/order) output, so the
|
||||
# byte-fixpoint is a dev-machine check, not a CI one (jolt-8479).
|
||||
ci: values corpus unit smoke buildsmoke sci ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline certify
|
||||
ci: values corpus unit smoke buildsmoke staticnativesmoke sci ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline certify
|
||||
@echo "OK: CI gates passed"
|
||||
|
||||
# Self-host fixpoint: bootstrap.ss rebuild == checked-in seed.
|
||||
|
|
@ -42,6 +42,11 @@ smoke:
|
|||
buildsmoke:
|
||||
@sh host/chez/build-smoke.sh
|
||||
|
||||
# `jolt build` cc-links a :jolt/native :static archive into the binary (the
|
||||
# default), and --dynamic keeps the runtime load-shared-object path.
|
||||
staticnativesmoke:
|
||||
@sh host/chez/static-native-smoke.sh
|
||||
|
||||
# Build joltc as a self-contained native binary into target/<profile>/joltc. The
|
||||
# binary bundles the runtime, compiler, jolt-core + stdlib source, the Chez boots,
|
||||
# and a launcher stub, so it runs AND compiles jolt apps with no Chez or cc on the
|
||||
|
|
|
|||
|
|
@ -72,11 +72,41 @@ bindings resolve. Each entry is a map — `{:name "sqlite3" :darwin
|
|||
the running process's own symbols, e.g. libc sockets, no external file). A
|
||||
project inherits its dependencies' `:jolt/native`.
|
||||
|
||||
### Static vs dynamic linking
|
||||
|
||||
When you `joltc build`, a native lib is **statically linked** into the binary by
|
||||
default if the spec carries a `:static` archive — so the executable calls the C
|
||||
code with no shared object present at runtime. Add `:static` alongside the runtime
|
||||
candidates:
|
||||
|
||||
```clojure
|
||||
{:name "sqlite3"
|
||||
:static {:archive "/opt/homebrew/lib/libsqlite3.a"} ; or {:lib "sqlite3" :libdir "/usr/lib"}
|
||||
:darwin ["libsqlite3.0.dylib"] ; still used by `run`/`repl` and by --dynamic
|
||||
:linux ["libsqlite3.so.0"]}
|
||||
```
|
||||
|
||||
`:static {:archive PATH}` force-loads the whole `.a` and is the reliable
|
||||
cross-platform form. `:static {:lib NAME :libdir DIR}` links `-lNAME` (with a
|
||||
`-Bstatic` preference on Linux); on macOS, which has no `-Bstatic`, prefer the
|
||||
archive form. A spec with no `:static` (or a build passed `--dynamic`, or
|
||||
`:jolt/build {:dynamic-natives true}`) keeps the old behavior — the shared object
|
||||
is loaded at startup via `load-shared-object`.
|
||||
|
||||
Static linking needs a C compiler (`cc`) on `PATH` at build time (plus the C libs
|
||||
the Chez kernel links — lz4, zlib, ncurses). The distributed `joltc` bundles the
|
||||
Chez kernel, so it re-links the launcher stub with the archive baked in — no
|
||||
external Chez, just `cc`. Without a `cc`, a `:static` lib fails with a message
|
||||
pointing you to install one or pass `--dynamic`. Keep a `:darwin`/`:linux`
|
||||
candidate on any `:static` spec so `run`/`repl` (which have no static binary) can
|
||||
still load it.
|
||||
|
||||
## Standalone binaries
|
||||
|
||||
`joltc build -m NS` compiles the app and every library into one executable (the
|
||||
runtime + compiler are baked in). It loads the resolved `:jolt/native` libs at
|
||||
startup, so an FFI app — sockets, SQLite — runs with no jolt or Chez on the path.
|
||||
runtime + compiler are baked in). Resolved `:jolt/native` libs are statically
|
||||
linked in (or loaded at startup — see [Native libraries](#native-libraries)), so
|
||||
an FFI app — sockets, SQLite — runs with no jolt or Chez on the path.
|
||||
|
||||
Output goes under the project's `target/`, cargo-style: `target/release/<project>`
|
||||
by default and with `--opt`, `target/debug/<project>` with `--dev` (the
|
||||
|
|
|
|||
|
|
@ -101,7 +101,10 @@
|
|||
(register-embedded-bytes! (car spec) bv)))
|
||||
'((\"csv/petite.boot\" \"jolt_petite_boot\" \"jolt_petite_boot_len\")
|
||||
(\"csv/scheme.boot\" \"jolt_scheme_boot\" \"jolt_scheme_boot_len\")
|
||||
(\"stub/launcher\" \"jolt_stub\" \"jolt_stub_len\")))))
|
||||
(\"stub/launcher\" \"jolt_stub\" \"jolt_stub_len\")
|
||||
(\"csv/scheme.h\" \"jolt_scheme_h\" \"jolt_scheme_h_len\")
|
||||
(\"csv/libkernel.a\" \"jolt_libkernel_a\" \"jolt_libkernel_a_len\")
|
||||
(\"stub/launcher.c\" \"jolt_launcher_c\" \"jolt_launcher_c_len\")))))
|
||||
|
||||
(suppress-greeting #t)
|
||||
(scheme-start
|
||||
|
|
@ -182,6 +185,13 @@
|
|||
(jb-c-array (string-append bld-csv-dir "/petite.boot") (string-append jb-build "/petite_data.h") "jolt_petite_boot")
|
||||
(jb-c-array (string-append bld-csv-dir "/scheme.boot") (string-append jb-build "/scheme_data.h") "jolt_scheme_boot")
|
||||
(jb-c-array jb-stub (string-append jb-build "/stub_data.h") "jolt_stub")
|
||||
;; Also bundle the Chez kernel (libkernel.a + scheme.h) and the launcher source,
|
||||
;; so a `build` with :static native libs can re-link a custom stub with those
|
||||
;; archives baked in — the appended-stub path can't add object code to a prebuilt
|
||||
;; stub, so it relinks (build.ss bld-relink-stub). Needs the system cc at build.
|
||||
(jb-c-array (string-append bld-csv-dir "/scheme.h") (string-append jb-build "/schemeh_data.h") "jolt_scheme_h")
|
||||
(jb-c-array (string-append bld-csv-dir "/libkernel.a") (string-append jb-build "/libkernel_data.h") "jolt_libkernel_a")
|
||||
(jb-c-array "host/chez/stub/launcher.c" (string-append jb-build "/launcherc_data.h") "jolt_launcher_c")
|
||||
|
||||
(define jb-main-c (string-append jb-build "/main.c"))
|
||||
(let ((mc (open-output-file jb-main-c 'replace)))
|
||||
|
|
@ -192,6 +202,9 @@
|
|||
"#include \"petite_data.h\"\n"
|
||||
"#include \"scheme_data.h\"\n"
|
||||
"#include \"stub_data.h\"\n"
|
||||
"#include \"schemeh_data.h\"\n"
|
||||
"#include \"libkernel_data.h\"\n"
|
||||
"#include \"launcherc_data.h\"\n"
|
||||
"int main(int argc, char *argv[]) {\n"
|
||||
" Sscheme_init(0);\n"
|
||||
" Sregister_boot_file_bytes(\"jolt\", jolt_boot, jolt_boot_len);\n"
|
||||
|
|
|
|||
|
|
@ -80,6 +80,9 @@
|
|||
(cand (string-append bindir "/../lib/csv" bld-version "/" bld-machine)))
|
||||
cand))))
|
||||
|
||||
(define (bld-have-cc?)
|
||||
(> (string-length (bld-sh-capture "command -v cc")) 0))
|
||||
|
||||
(define (bld-check-toolchain)
|
||||
(for-each
|
||||
(lambda (f)
|
||||
|
|
@ -271,21 +274,24 @@
|
|||
(define (bld-strs x) (map jolt-str-render-one (seq->list x)))
|
||||
|
||||
;; Emit native-library loads. `natives` is the encoded jolt seq jolt.main/
|
||||
;; encode-natives produced: each entry is ["process"] | ["req" cand…] | ["opt" cand…].
|
||||
;; `which` selects 'required (process + req) or 'optional. Required + process loads
|
||||
;; are emitted before the app forms (the app's defcfn foreign-procedures resolve
|
||||
;; their symbols at top-level eval during startup, so the libs must be loaded
|
||||
;; first); a load-shared-object failure there is fatal — correct for a required
|
||||
;; lib. Optional loads run in the scheme-start launcher, where guard catches a
|
||||
;; missing lib (an optional lib's namespace is only present when the app requires
|
||||
;; it, so its foreign-procedures aren't among the baked top-level forms).
|
||||
;; encode-natives produced: each entry is ["process"] | ["static" form…] |
|
||||
;; ["req" cand…] | ["opt" cand…]. `which` selects 'required (process + static +
|
||||
;; req) or 'optional. Required loads are emitted before the app forms (the app's
|
||||
;; defcfn foreign-procedures resolve their symbols at top-level eval during
|
||||
;; startup, so the libs must be loaded first); a load-shared-object failure there
|
||||
;; is fatal — correct for a required lib. A "static" lib is cc-linked into the
|
||||
;; binary (see bld-native-link-flags), so its symbols are already in the process:
|
||||
;; it loads them the same way a "process" lib does. Optional loads run in the
|
||||
;; scheme-start launcher, where guard catches a missing lib (an optional lib's
|
||||
;; namespace is only present when the app requires it, so its foreign-procedures
|
||||
;; aren't among the baked top-level forms).
|
||||
(define (bld-emit-natives out natives which)
|
||||
(for-each
|
||||
(lambda (entry)
|
||||
(let* ((parts (bld-strs entry)) (kind (car parts)) (cands (cdr parts))
|
||||
(cand-lits (fold-left (lambda (s c) (string-append s (ei-str-lit c) " ")) "" cands)))
|
||||
(cond
|
||||
((and (eq? which 'required) (string=? kind "process"))
|
||||
((and (eq? which 'required) (or (string=? kind "process") (string=? kind "static")))
|
||||
(put-string out "(jolt-build-load-native '() #f #t)\n"))
|
||||
((and (eq? which 'required) (string=? kind "req"))
|
||||
(put-string out (string-append "(jolt-build-load-native (list " cand-lits ") #f #f)\n")))
|
||||
|
|
@ -293,6 +299,66 @@
|
|||
(put-string out (string-append "(jolt-build-load-native (list " cand-lits ") #t #f)\n"))))))
|
||||
(seq->list natives)))
|
||||
|
||||
;; The cc link fragment for the "static" natives: each archive must be FORCE-loaded
|
||||
;; (the linker would otherwise drop an archive member main.c never references) and,
|
||||
;; on Linux, the executable's symbols exported into the dynamic table so the
|
||||
;; startup (load-shared-object #f) + foreign-procedure can resolve them (-rdynamic,
|
||||
;; added by build-with-cc when this fragment is non-empty). Returns "" when no lib
|
||||
;; is statically linked. Entry forms: ["static" "archive" path] | ["static" "lib"
|
||||
;; name libdir].
|
||||
(define (bld-native-link-flags natives)
|
||||
(fold-left
|
||||
(lambda (acc entry)
|
||||
(let ((parts (bld-strs entry)))
|
||||
(if (string=? (car parts) "static")
|
||||
(string-append acc " " (bld-one-static-link (cdr parts)))
|
||||
acc)))
|
||||
"" (seq->list natives)))
|
||||
|
||||
;; A statically-linked native is only in the OUTPUT binary, but build step 1
|
||||
;; evaluates the app's `foreign-procedure` forms in THIS process (to register its
|
||||
;; macros/vars), and Chez resolves a foreign entry eagerly. So make the archive's
|
||||
;; symbols resolvable here: build a throwaway shared object from it (force-loading
|
||||
;; every member) and load it. The output binary still cc-links the static archive;
|
||||
;; this temp .so is build-time only. Only the "archive" form is preloaded — the
|
||||
;; "lib" form names a system library the OS loader already finds by soname.
|
||||
(define (bld-preload-static-natives! natives builddir)
|
||||
(let ((n 0))
|
||||
(for-each
|
||||
(lambda (entry)
|
||||
(let ((parts (bld-strs entry)))
|
||||
(when (and (string=? (car parts) "static") (string=? (cadr parts) "archive"))
|
||||
(let* ((archive (caddr parts))
|
||||
(so (string-append builddir "/native-" (number->string n)
|
||||
(if bld-osx? ".dylib" ".so"))))
|
||||
(set! n (+ n 1))
|
||||
(bld-system
|
||||
(if bld-osx?
|
||||
(string-append "cc -dynamiclib -undefined dynamic_lookup -Wl,-all_load '"
|
||||
archive "' -o '" so "'")
|
||||
(string-append "cc -shared -Wl,--whole-archive '" archive
|
||||
"' -Wl,--no-whole-archive -Wl,--unresolved-symbols=ignore-all -o '" so "'")))
|
||||
(load-shared-object so)))))
|
||||
(seq->list natives))))
|
||||
|
||||
(define (bld-one-static-link form)
|
||||
(let ((kind (car form)))
|
||||
(cond
|
||||
((string=? kind "archive")
|
||||
(let ((path (cadr form)))
|
||||
(if bld-osx?
|
||||
(string-append "-Wl,-force_load," path)
|
||||
(string-append "-Wl,--whole-archive " path " -Wl,--no-whole-archive"))))
|
||||
((string=? kind "lib")
|
||||
(let* ((lib (cadr form)) (dir (caddr form))
|
||||
(L (if (> (string-length dir) 0) (string-append "-L" dir " ") "")))
|
||||
;; -Bstatic forces the .a over a .so of the same -l name (GNU ld). macOS's
|
||||
;; ld64 has no -Bstatic; there an :archive path is the reliable form.
|
||||
(if bld-osx?
|
||||
(string-append L "-l" lib)
|
||||
(string-append L "-Wl,-Bstatic -l" lib " -Wl,-Bdynamic"))))
|
||||
(else ""))))
|
||||
|
||||
;; Walk an embed root recursively; return (resource-name . abspath) pairs, where
|
||||
;; resource-name is the "/"-joined path under the root (what io/resource is asked for).
|
||||
(define (bld-walk-files root rel acc)
|
||||
|
|
@ -338,6 +404,19 @@
|
|||
;; The self-contained path (jolt-embedded-bytes "stub/launcher") needs no csv
|
||||
;; kernel files, no Chez, no cc — only the legacy cc path does.
|
||||
(unless (jolt-embedded-bytes "stub/launcher") (bld-check-toolchain))
|
||||
(when (> (string-length (bld-native-link-flags natives)) 0)
|
||||
;; :static natives are cc-linked into the binary, so a C compiler must be on
|
||||
;; PATH — the self-contained joltc bundles the Chez kernel (libkernel.a +
|
||||
;; scheme.h) and relinks a custom stub (see build-self-contained), but still
|
||||
;; needs the system cc for that link. Fail early (before the app's foreign-
|
||||
;; procedure forms eval below) with an actionable message.
|
||||
(unless (bld-have-cc?)
|
||||
(error 'jolt-build
|
||||
"static native linking needs a C compiler (cc) on PATH; install one, or pass --dynamic to load the library at runtime."))
|
||||
;; Preload static archives' symbols into this process so step 1's foreign-
|
||||
;; procedure evals resolve; the .build dir must exist first.
|
||||
(bld-mkdir-p (string-append out-path ".build"))
|
||||
(bld-preload-static-natives! natives (string-append out-path ".build")))
|
||||
;; 1. record app namespaces in dependency order as they finish loading.
|
||||
(let ((app-order '()))
|
||||
(set-ns-loaded-hook!
|
||||
|
|
@ -460,8 +539,10 @@
|
|||
;; make-boot-file, then xxd the boot into a C array and cc-link against
|
||||
;; libkernel.a. Kept so `make buildsmoke` still exercises the cc path.
|
||||
(if (jolt-embedded-bytes "stub/launcher")
|
||||
(build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot)
|
||||
(build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c)))))))
|
||||
(build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot
|
||||
(bld-native-link-flags natives))
|
||||
(build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c
|
||||
(bld-native-link-flags natives))))))))
|
||||
|
||||
;; --- self-contained link (in-process compile + append the boot to the stub) ---
|
||||
;; compile-file runs with a FRESH scheme-environment as the interaction
|
||||
|
|
@ -470,7 +551,7 @@
|
|||
;; runs) must bind to the kernel primitive — a clean env gives exactly that, the
|
||||
;; same as the legacy path compiling in a fresh Chez process. Without it the boot
|
||||
;; dies at startup with "variable error is not bound".
|
||||
(define (build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot)
|
||||
(define (build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot native-link)
|
||||
(let ((petite (string-append builddir "/petite.boot"))
|
||||
(scheme (string-append builddir "/scheme.boot")))
|
||||
(jolt-spill-embedded! "csv/petite.boot" petite)
|
||||
|
|
@ -479,8 +560,14 @@
|
|||
(parameterize ((interaction-environment (copy-environment (scheme-environment))))
|
||||
(compile-file flat-ss flat-so)
|
||||
(make-boot-file boot '() petite scheme flat-so))
|
||||
;; The stub is the native launcher the boot is appended to. With no :static
|
||||
;; natives it's the prebuilt one bundled in joltc (no cc needed); with :static
|
||||
;; natives it's re-linked here from the bundled kernel + launcher source so the
|
||||
;; archives are baked in and their symbols resolve in the running binary.
|
||||
(if (> (string-length native-link) 0)
|
||||
(bld-relink-stub builddir native-link out-path)
|
||||
(jolt-spill-embedded! "stub/launcher" out-path))
|
||||
;; link: stub bytes ++ boot ++ frame, then make it executable.
|
||||
(jolt-spill-embedded! "stub/launcher" out-path)
|
||||
(jolt-append-payload! out-path (read-file-bytes boot))
|
||||
(jolt-chmod-755 out-path)
|
||||
(display (string-append "jolt build: wrote " out-path "\n"))
|
||||
|
|
@ -489,8 +576,27 @@
|
|||
"jolt build: note — on macOS this binary is unsigned; to share it,\n"
|
||||
" `xattr -d com.apple.quarantine " out-path "` on the target, or sign it.\n")))))
|
||||
|
||||
;; Re-link the launcher stub with the app's static native archives baked in, to
|
||||
;; OUT-PATH. The self-contained joltc bundles the Chez kernel (libkernel.a),
|
||||
;; header, and launcher source; spill them and drive the system cc — the same link
|
||||
;; build-joltc.ss ran once at joltc-build time, plus the force-load archive flags
|
||||
;; (native-link) and, on Linux, -rdynamic so the baked-in symbols stay dlsym-
|
||||
;; visible for (load-shared-object #f) + foreign-procedure at startup.
|
||||
(define (bld-relink-stub builddir native-link out-path)
|
||||
(let ((h (string-append builddir "/scheme.h"))
|
||||
(lk (string-append builddir "/libkernel.a"))
|
||||
(lc (string-append builddir "/launcher.c")))
|
||||
(jolt-spill-embedded! "csv/scheme.h" h)
|
||||
(jolt-spill-embedded! "csv/libkernel.a" lk)
|
||||
(jolt-spill-embedded! "stub/launcher.c" lc)
|
||||
(display "jolt build: relinking launcher stub with static native libraries\n")
|
||||
(bld-system (string-append
|
||||
"cc -O2 " (if bld-osx? "" "-rdynamic ")
|
||||
"-I'" builddir "' '" lc "' '" lk "' -o '" out-path "' "
|
||||
(bld-link-libs) native-link))))
|
||||
|
||||
;; --- legacy cc link (dev bin/joltc): fresh Chez compile + xxd + cc ------------
|
||||
(define (build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c)
|
||||
(define (build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c native-link)
|
||||
(display (string-append "jolt build: compiling " entry-ns " (" mode " mode)\n"))
|
||||
(let ((cs (string-append builddir "/compile.ss")))
|
||||
(let ((p (open-output-file cs 'replace)))
|
||||
|
|
@ -520,9 +626,13 @@
|
|||
" int status = Sscheme_start(argc, (const char **)argv);\n"
|
||||
" Sscheme_deinit();\n return status;\n}\n"))
|
||||
(close-port mc))
|
||||
;; -rdynamic (Linux) exports the executable's symbols into the dynamic table so
|
||||
;; a statically-linked native lib's symbols resolve via (load-shared-object #f)
|
||||
;; at startup. macOS keeps unstripped executable symbols dlsym-visible already.
|
||||
(bld-system (string-append
|
||||
"cc -O2 -I'" bld-csv-dir "' '" main-c "' '" bld-csv-dir "/libkernel.a' "
|
||||
"-o '" out-path "' " (bld-link-libs)))
|
||||
"cc -O2 " (if (and (not bld-osx?) (> (string-length native-link) 0)) "-rdynamic " "")
|
||||
"-I'" bld-csv-dir "' '" main-c "' '" bld-csv-dir "/libkernel.a' "
|
||||
"-o '" out-path "' " (bld-link-libs) native-link))
|
||||
(display (string-append "jolt build: wrote " out-path "\n")))
|
||||
|
||||
(def-var! "jolt.host" "build-binary"
|
||||
|
|
|
|||
|
|
@ -73,4 +73,36 @@ if [ "$got" != "$want" ]; then
|
|||
echo "--- got ----"; echo "$got"
|
||||
exit 1
|
||||
fi
|
||||
echo "joltc self-build smoke: passed (joltc runs + builds a working app with no external toolchain)"
|
||||
|
||||
# 5. Static native linking through the distributed joltc: it bundles the Chez
|
||||
# kernel, so with the system cc (but still no external Chez) it re-links a stub
|
||||
# that bakes a :jolt/native :static archive into the app. The app then calls the
|
||||
# C function with the archive removed from disk. Uses the normal PATH so cc — and
|
||||
# the kernel's link deps (lz4/…) — are found, but Chez stays out of the build.
|
||||
napp="$(mktemp -d)/native-app"
|
||||
mkdir -p "$napp/src/app"
|
||||
printf 'int jolt_static_answer(void){return 42;}\n' > "$napp/greet.c"
|
||||
cc -c "$napp/greet.c" -o "$napp/greet.o" && ar rcs "$napp/libgreet.a" "$napp/greet.o"
|
||||
cat > "$napp/src/app/core.clj" <<'EOF'
|
||||
(ns app.core (:require [jolt.ffi :as ffi]))
|
||||
(ffi/defcfn answer "jolt_static_answer" [] :int)
|
||||
(defn -main [& _] (println "answer:" (answer)))
|
||||
EOF
|
||||
cat > "$napp/deps.edn" <<EOF
|
||||
{:paths ["src"]
|
||||
:jolt/native [{:name "greet" :static {:archive "$napp/libgreet.a"}}]}
|
||||
EOF
|
||||
nout="$napp/app"
|
||||
echo "joltc self-build smoke: static-linking a native lib via the binary (no external Chez)"
|
||||
if ! JOLT_PWD="$napp" "$joltc" build -m app.core -o "$nout" >/dev/null 2>&1; then
|
||||
echo " FAIL: static native build via distributed joltc exited non-zero"
|
||||
rm -rf "$(dirname "$napp")"; exit 1
|
||||
fi
|
||||
rm -f "$napp/libgreet.a" "$napp/greet.o" # nothing to load at runtime
|
||||
got_n="$(cd / && "$nout" 2>&1)"
|
||||
rm -rf "$(dirname "$napp")"
|
||||
if [ "$got_n" != "answer: 42" ]; then
|
||||
echo " FAIL: static-linked app (via distributed joltc) output mismatch"
|
||||
echo "--- got ----"; echo "$got_n"; exit 1
|
||||
fi
|
||||
echo "joltc self-build smoke: passed (joltc runs + builds a working app with no external toolchain, incl. static native linking)"
|
||||
|
|
|
|||
115
host/chez/static-native-smoke.sh
Normal file
115
host/chez/static-native-smoke.sh
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
#!/bin/sh
|
||||
# static-native smoke: a project's :jolt/native lib with a :static archive is
|
||||
# LINKED INTO the built binary (the default), so the binary calls the C function
|
||||
# with no shared object on disk at runtime. --dynamic keeps the old behavior —
|
||||
# load a shared object at runtime.
|
||||
root="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)"
|
||||
cd "$root"
|
||||
|
||||
# Preflight: needs cc (to build the test libs AND to cc-link the app) + Chez's
|
||||
# kernel dev files, same as build-smoke. Skip otherwise (CI on a distro package).
|
||||
csv="$JOLT_CHEZ_CSV"
|
||||
if [ -z "$csv" ]; then
|
||||
chez_bin="$(command -v chez || command -v scheme || command -v petite || true)"
|
||||
if [ -n "$chez_bin" ]; then
|
||||
base="$(cd "$(dirname "$chez_bin")/.." 2>/dev/null && pwd)"
|
||||
for d in "$base"/lib/csv*/*/; do
|
||||
[ -f "${d}libkernel.a" ] && csv="${d%/}" && break
|
||||
done
|
||||
fi
|
||||
fi
|
||||
if ! command -v cc >/dev/null 2>&1 || [ -z "$csv" ] || [ ! -f "$csv/scheme.h" ] || [ ! -f "$csv/libkernel.a" ]; then
|
||||
echo "static-native smoke: skipped (Chez kernel dev files or C compiler not available)"
|
||||
exit 0
|
||||
fi
|
||||
export JOLT_CHEZ_CSV="$csv"
|
||||
|
||||
case "$(uname -s)" in
|
||||
Darwin) plat=":darwin"; soext="dylib"; shared="-dynamiclib" ;;
|
||||
*) plat=":linux"; soext="so"; shared="-shared" ;;
|
||||
esac
|
||||
|
||||
work="$(mktemp -d)"
|
||||
trap 'rm -rf "$work"' EXIT
|
||||
app="$work/app"
|
||||
mkdir -p "$app/src/app"
|
||||
|
||||
# 1. a trivial C library, built BOTH as a static archive and a shared object.
|
||||
cat > "$work/greet.c" <<'EOF'
|
||||
int jolt_static_answer(void) { return 42; }
|
||||
EOF
|
||||
cc -c "$work/greet.c" -o "$work/greet.o"
|
||||
ar rcs "$work/libgreet.a" "$work/greet.o"
|
||||
cc $shared "$work/greet.c" -o "$work/libgreet.$soext"
|
||||
|
||||
# 2. an app that binds that symbol via FFI.
|
||||
cat > "$app/src/app/core.clj" <<'EOF'
|
||||
(ns app.core
|
||||
(:require [jolt.ffi :as ffi]))
|
||||
(ffi/defcfn answer "jolt_static_answer" [] :int)
|
||||
(defn -main [& _]
|
||||
(println "answer:" (answer)))
|
||||
EOF
|
||||
|
||||
out="$work/app-bin"
|
||||
|
||||
# --- default: static link ---------------------------------------------------
|
||||
# A static-only spec (no runtime candidate): the build resolves the symbol by
|
||||
# preloading the archive, and the binary links it in — nothing to load at runtime.
|
||||
cat > "$app/deps.edn" <<EOF
|
||||
{:paths ["src"]
|
||||
:jolt/native [{:name "greet" :static {:archive "$work/libgreet.a"}}]}
|
||||
EOF
|
||||
echo "static-native smoke: building (default: static link)"
|
||||
if ! JOLT_PWD="$app" bin/joltc build -m app.core -o "$out" >"$work/build.log" 2>&1; then
|
||||
echo " FAIL: jolt build (static) exited non-zero"; cat "$work/build.log"; exit 1
|
||||
fi
|
||||
[ -x "$out" ] || { echo " FAIL: no executable produced"; exit 1; }
|
||||
# A static lib emits a process-symbol load (its archive is in-process), not a
|
||||
# dlopen of the shared object.
|
||||
if ! grep -q "jolt-build-load-native '() #f #t" "$out.build/flat.ss"; then
|
||||
echo " FAIL: static native did not emit a process-symbol load"; exit 1
|
||||
fi
|
||||
if grep -q "libgreet.$soext" "$out.build/flat.ss"; then
|
||||
echo " FAIL: static native baked a runtime shared-object load"; exit 1
|
||||
fi
|
||||
# Remove BOTH libs: a static-linked symbol lives in the binary, nothing to load.
|
||||
rm -f "$work/libgreet.a" "$work/libgreet.$soext" "$work/greet.o"
|
||||
got="$(cd / && "$out" 2>&1)"
|
||||
if [ "$got" != "answer: 42" ]; then
|
||||
echo " FAIL: static-linked binary output mismatch"
|
||||
echo "--- want ---"; echo "answer: 42"; echo "--- got ----"; echo "$got"; exit 1
|
||||
fi
|
||||
|
||||
# --- --dynamic: runtime load ------------------------------------------------
|
||||
# Rebuild the shared object (static phase deleted it) and give the spec a runtime
|
||||
# candidate; --dynamic loads it at startup instead of linking the archive.
|
||||
cc $shared "$work/greet.c" -o "$work/libgreet.$soext"
|
||||
cat > "$app/deps.edn" <<EOF
|
||||
{:paths ["src"]
|
||||
:jolt/native [{:name "greet"
|
||||
:static {:archive "$work/libgreet.a"}
|
||||
$plat ["$work/libgreet.$soext"]}]}
|
||||
EOF
|
||||
echo "static-native smoke: building (--dynamic: runtime load)"
|
||||
if ! JOLT_PWD="$app" bin/joltc build -m app.core -o "$out" --dynamic >"$work/build.log" 2>&1; then
|
||||
echo " FAIL: jolt build --dynamic exited non-zero"; cat "$work/build.log"; exit 1
|
||||
fi
|
||||
# --dynamic loads the shared object at runtime.
|
||||
if ! grep -q "libgreet.$soext" "$out.build/flat.ss"; then
|
||||
echo " FAIL: --dynamic did not emit a runtime shared-object load"; exit 1
|
||||
fi
|
||||
got="$(cd / && "$out" 2>&1)"
|
||||
if [ "$got" != "answer: 42" ]; then
|
||||
echo " FAIL: --dynamic binary output mismatch (shared object present)"
|
||||
echo "--- got ----"; echo "$got"; exit 1
|
||||
fi
|
||||
# With the shared object gone, a --dynamic binary must FAIL — proving the symbol
|
||||
# was loaded at runtime, not baked in.
|
||||
rm -f "$work/libgreet.$soext"
|
||||
rc=0; { (cd / && exec "$out"); } >/dev/null 2>&1 || rc=$?
|
||||
if [ "$rc" -eq 0 ]; then
|
||||
echo " FAIL: --dynamic binary still ran with its shared object removed"; exit 1
|
||||
fi
|
||||
|
||||
echo "static-native smoke: passed (static default + --dynamic runtime load)"
|
||||
|
|
@ -28,7 +28,11 @@
|
|||
(let [c (get spec plat)
|
||||
cands (if (string? c) [c] (vec c))
|
||||
hit (some #(when (jolt.ffi/loaded? %) %) cands)]
|
||||
(when (and (nil? hit) (not (:optional spec)))
|
||||
;; A :static spec has no runtime shared object (it's linked into a
|
||||
;; built binary), so an interpreted `run`/`repl` has nothing to load —
|
||||
;; skip it rather than fail. Its foreign calls only resolve in a static
|
||||
;; build; document a dynamic candidate too to use it under `run`.
|
||||
(when (and (nil? hit) (not (:optional spec)) (not (:static spec)))
|
||||
(throw (ex-info (str "required native library "
|
||||
(or (:name spec) (first cands) "?")
|
||||
" not found — tried " (pr-str cands) " for " (name plat))
|
||||
|
|
@ -174,18 +178,38 @@
|
|||
;; --direct-link (or deps.edn :jolt/build {:direct-link true}) opts into closed-world
|
||||
;; direct-linking: app->app calls bind directly, giving up runtime redefinition of
|
||||
;; those vars and eval/load-string. Off by default — release stays dynamically linked.
|
||||
;; The static-link description of a :jolt/native spec for this platform, or nil.
|
||||
;; :static may be flat ({:archive "…"} / {:lib "z" :libdir "…"}) or per-platform
|
||||
;; ({:darwin {…} :linux {…}}). Returns a vector build.ss reads and wraps in the
|
||||
;; platform's force-load flags: ["archive" abspath] or ["lib" name libdir].
|
||||
(defn- static-link-spec [spec plat]
|
||||
(when-let [s (:static spec)]
|
||||
(let [p (get s plat)
|
||||
s (if (map? p) p s)]
|
||||
(cond
|
||||
(:archive s) ["archive" (:archive s)]
|
||||
(:lib s) ["lib" (:lib s) (or (:libdir s) "")]
|
||||
:else nil))))
|
||||
|
||||
;; Encode a deps.edn :jolt/native spec for the build launcher, resolving the
|
||||
;; current platform's candidate list now (the binary runs on this OS). Each entry
|
||||
;; becomes a vector the launcher (build.ss) reads: ["process"] for the running
|
||||
;; binary's own symbols, else ["req"|"opt" cand…] to try in turn.
|
||||
(defn- encode-natives [natives]
|
||||
;; becomes a vector the launcher (build.ss) reads:
|
||||
;; ["process"] — the running binary's own symbols (libc)
|
||||
;; ["static" form …] — the lib's archive, cc-linked into the binary; its
|
||||
;; symbols load from the process (default when :static
|
||||
;; is present and --dynamic wasn't passed)
|
||||
;; ["req"|"opt" cand…] — load a shared object at runtime, trying each in turn
|
||||
;; dynamic? forces the runtime path for every lib (the --dynamic build flag).
|
||||
(defn- encode-natives [natives dynamic?]
|
||||
(let [plat (current-platform)]
|
||||
(vec (for [spec natives]
|
||||
(if (:process spec)
|
||||
["process"]
|
||||
(let [c (get spec plat)
|
||||
cands (if (string? c) [c] (vec c))]
|
||||
(into [(if (:optional spec) "opt" "req")] cands)))))))
|
||||
(let [static (and (not dynamic?) (static-link-spec spec plat))]
|
||||
(cond
|
||||
(:process spec) ["process"]
|
||||
static (into ["static"] static)
|
||||
:else (let [c (get spec plat)
|
||||
cands (if (string? c) [c] (vec c))]
|
||||
(into [(if (:optional spec) "opt" "req")] cands))))))))
|
||||
|
||||
(defn- cmd-build [more]
|
||||
(let [{:keys [project-paths embed-dirs build] :as resolved}
|
||||
|
|
@ -219,7 +243,11 @@
|
|||
(nil? o) (str pdir "/target/" (if (= mode "dev") "debug" "release") "/" proj)
|
||||
(str/starts-with? o "/") o
|
||||
:else (str pdir "/" o)))
|
||||
natives (encode-natives (:natives resolved))
|
||||
;; :jolt/native libs with a :static archive are cc-linked into the
|
||||
;; binary by default; --dynamic (or deps.edn :jolt/build {:dynamic-natives
|
||||
;; true}) keeps the old behavior — load a shared object at runtime.
|
||||
dynamic-natives? (boolean (or (some #{"--dynamic"} more) (:dynamic-natives build)))
|
||||
natives (encode-natives (:natives resolved) dynamic-natives?)
|
||||
;; closed-world direct-linking is opt-in: the --direct-link flag or a
|
||||
;; deps.edn :jolt/build {:direct-link true}. Off otherwise.
|
||||
direct-link? (boolean (or (some #{"--direct-link"} more) (:direct-link build)))
|
||||
|
|
@ -269,7 +297,7 @@
|
|||
(println "usage: jolt <command> [args]")
|
||||
(println " run -m NS [args] resolve deps.edn, load NS, call its -main")
|
||||
(println " run FILE load a Clojure file")
|
||||
(println " build -m NS [-o OUT] [--opt|--dev] [--direct-link] [--tree-shake] compile a standalone binary")
|
||||
(println " build -m NS [-o OUT] [--opt|--dev] [--direct-link] [--tree-shake] [--dynamic] compile a standalone binary")
|
||||
(println " -M:alias [args] run the alias's :main-opts")
|
||||
(println " -A:alias [args] add the alias's paths/deps")
|
||||
(println " repl start a line REPL")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue