Add a Clojure FFI so libraries can bind native code (jolt.ffi)

A jolt library can now bind its own native dependencies and expose a Clojure API
over them — no jolt built-in required. This is the foundation for moving the
http-client / db / adapter functionality out of the host and into real libraries.

- jolt.ffi/foreign-fn (sugar: defcfn) is a compiler special form: a compile-time
  -typed C signature lowers to a real Chez foreign-procedure (analyzer :ffi-fn ->
  backend foreign-procedure), so calls are typed and marshaled, not eval'd.
- host/chez/ffi.ss provides the rest under jolt.ffi: load-library, alloc/free,
  read/write/sizeof, ptr<->string, null/null?. Loaded after the loader snapshot
  so a library's (require '[jolt.ffi]) still loads the macro side.
- Types: int/uint/long/ulong/int64/uint64/size_t/ssize_t/iptr/uptr/double/float/
  pointer/string/void/uint8/char.

Validated end to end: a pure-Clojure file binds libc (getpid/strlen/abs) and
libsqlite3 (open/prepare/step/column/finalize over out-param pointers) and runs a
query. Gate test test/chez/ffi-binding-test.ss (make ffi); selfhost holds.
This commit is contained in:
Yogthos 2026-06-22 10:59:51 -04:00
parent ccf93c896a
commit 537cb360b4
8 changed files with 282 additions and 62 deletions

View file

@ -393,6 +393,20 @@
(defn- analyze-ctor [ctx class args env]
(host-new class (mapv #(analyze ctx % env) args)))
;; jolt.ffi/__cfn (jolt-ffi): the low-level foreign-function form a jolt library
;; uses (via the jolt.ffi/foreign-fn macro) to bind native code. Shape:
;; (jolt.ffi/__cfn "c_symbol" [:argtype ...] :rettype)
;; The C symbol is a string literal and the types are literal keywords, read here
;; at compile time; the Chez back end lowers it to a real `foreign-procedure`
;; (typed marshaling, no runtime eval). A leaf IR node.
(defn- analyze-ffi-fn [ctx items env]
(when (not= 4 (count items))
(throw (str "jolt.ffi/foreign-fn expects (foreign-fn \"sym\" [argtypes] rettype)")))
{:op :ffi-fn
:csym (nth items 1)
:argtypes (mapv name (form-vec-items (nth items 2)))
:rettype (name (nth items 3))})
;; The `.` special form: `(. target member arg*)` — member access / method call.
;; A symbol member whose name starts with "-" is a field read; otherwise it is a
;; method (call with the trailing args). Both lower to a :host-call carrying the
@ -476,6 +490,11 @@
;; read -> macroexpand -> analyze. A local shadows both.
(and (form-sym? head) (not shadowed) (form-macro? ctx head))
(analyze ctx (form-expand-1 ctx form) env)
;; jolt.ffi/__cfn — the foreign-function special form (always emitted
;; fully-qualified by the jolt.ffi/foreign-fn macro, so aliases resolve).
(and (form-sym? head) (= "jolt.ffi" (form-sym-ns head))
(= "__cfn" (form-sym-name head)))
(analyze-ffi-fn ctx items env)
;; special-form heads are NOT shadowable (unlike macros): a local named
;; `if` does not change the meaning of (if …) in operator position, per
;; spec §3 and the reference. No (not shadowed) guard here.

View file

@ -279,6 +279,23 @@
body (binding [*recur-target* label] (emit (:body node)))]
(str "(let* (" seq-bs ") (let " label " (" rebinds ") " body "))")))
;; jolt.ffi/__cfn -> a Chez foreign-procedure (jolt-ffi). The C symbol + types are
;; compile-time literals from the analyzer, so this emits a real typed binding;
;; the resulting Scheme procedure is callable like any jolt fn. The library must
;; have loaded the shared object (jolt.ffi/load-library) before this def runs.
(def ^:private ffi-types
{"int" "int" "uint" "unsigned-int" "long" "long" "ulong" "unsigned-long"
"int64" "integer-64" "uint64" "unsigned-64" "size_t" "size_t" "ssize_t" "ssize_t"
"iptr" "iptr" "uptr" "uptr" "double" "double" "float" "float"
"pointer" "void*" "void*" "void*" "string" "string" "void" "void"
"uint8" "unsigned-8" "u8" "unsigned-8" "byte" "unsigned-8" "char" "char"})
(defn- ffi-type->chez [t]
(or (ffi-types t) (throw (ex-info (str "jolt.ffi: unknown foreign type :" t) {}))))
(defn- emit-ffi-fn [node]
(str "(foreign-procedure " (chez-str-lit (:csym node))
" (" (str/join " " (map ffi-type->chez (:argtypes node))) ") "
(ffi-type->chez (:rettype node)) ")"))
(defn- emit-recur [node]
(when-not *recur-target* (throw (ex-info "emit: recur outside a loop/fn target" {})))
(let [arg-nodes (:args node)]
@ -491,6 +508,7 @@
:let (emit-let node)
:loop (emit-loop node)
:recur (emit-recur node)
:ffi-fn (emit-ffi-fn node)
:fn (emit-fn node)
;; (def name) with no init (declare): reserve the cell. A def with non-empty
;; reader metadata lowers to def-var-with-meta! (ported in a later increment).