Adds two IR passes to jolt.passes that run when a unit opts into
direct-linking (JOLT_DIRECT_LINK=1, off by default). The inline pass splices
small direct-linked fns at their call sites, copy-propagating trivial args so
that scalar replacement can then see map literals across the call boundary.
Scalar replacement is AOT escape analysis: a map allocation whose only use is
constant-keyword lookup is dropped and each (:k m) is replaced with the value
at :k, both for a literal lookup subject and for a non-escaping let-bound map.
Inlining and scalar replacement iterate to a capped fixpoint, since inlining
exposes literals that scalar replacement then collapses.
The back end stashes the body IR of each single-fixed-arity defn on its var
cell (inline-stash!), and the portable pass reads it through two new jolt.host
contract fns (inline-enabled?, inline-ir). Inlining is gated on :inline?, which
is off for all of init so core and the self-hosted compiler compile exactly as
before (const-fold only); api/init and main re-read JOLT_DIRECT_LINK so the
flag works both for a freshly built context and for the build-time-baked one in
the shipped binary.
Only inline-safe targets are spliced: a single fixed arity, no recur/loop/fn/
try crossing the boundary, within a size budget, a closed body (no free locals
beyond the params, so a self-recursive fn's name reference can't dangle), and
not ^:redef / ^:dynamic. Bodies are fully alpha-renamed so no spliced name can
collide with a caller local.
On the ray tracer this is 15.3s -> 13.0s (1.18x). The ceiling is honest: that
workload's cost is dominated by lookups on maps that genuinely escape (rays,
hits, materials) and by dynamic dispatch (the reduce closure, the :scatter fn),
which escape analysis cannot remove. On allocation-bound code where the
temporaries are local it is far larger: a vec3 reflect+dot loop goes 9.3s ->
0.38s (25x), with the loop body reduced to pure arithmetic.
Verified: full jpm test passes (inline off, no regression); conformance 335/335
in all three modes and the clojure-test-suite both pass with inline on; new
inline-sra-test pins the transform and its semantics.
jolt-p3c: Clojure evaluates map-literal entries left to right, but the
reader represented map forms as bare janet structs, so entries ran in
hash order. The reader now carries [k v ...] source order out-of-band —
on a struct PROTOTYPE (keys/kvs/length ignore protos, so macros that
get/keys literal map forms see no change; jolt-equal? was already
structural) and as a plain field on the phm rep (nil key/value). The
analyzer (form-map-pairs), the interpreter's map eval, both
syntax-quote walks, and core-sqmap (the lowered `{...} builder — the
array-map case, where Clojure also preserves insertion order) all honor
it, so the order survives macroexpansion in both modes.
jolt-507 root-caused: the parked inline put a LOCAL in janet call-head
position for the first time, and janet resolves head symbols against
the macro table before lexical upvalues — clojure.core/repeat's
self-name local expanded as janet's (repeat n & body) macro, compiling
the self-call into a countdown loop returning nil. Everything in the
issue (interpose, interleave) traced to that one name collision. The
emitter now rebinds local callees to reserved _fp$ symbols (argument
positions never consult the macro table), and the inline — direct
calls for function locals, jolt-call only for IFn-collection
leftovers — lands. Spec rows pin locals named repeat/seq/with called
in head position.
Gate green, suite 4718 steady, bench even with main.
jank's ray tracer benchmark (examples/ray-tracer) drops from 165.6s to
15.8s per render (10.5x) — from 118x JVM Clojure to 11x. The changes
mirror the optimizations in jank's June 2026 post, adapted to the
janet backend (jolt-4vr, jolt-h79):
- (:kw m) emits an inline lookup instead of variadic jolt-call ->
core-get's predicate chain. The guard is (get m :jolt/type): janet
compiles get to an opcode (~17ns) where a struct? cfunction call
costs ~85ns/lookup. :jolt/type is reserved (the reader rejects it in
map literals) and every table rep that must not be raw-indexed
carries it — phm tables now tagged too — so tagged values route to
core-get and everything else gets janet get, which matches core-get
for keyword keys on structs/records/nil/arrays. 929ns -> 90ns.
- {:k v ...} literals with scalar const keys emit let-bound values, an
`and` truthiness test (pure branch opcodes), and a native (struct ...)
call instead of variadic build-map-literal + runtime kv re-scan. nil
or false values fall back to build-map-literal, which keeps Clojure's
nil-entry semantics via the phm rep. 890ns -> 246ns.
- native-op additions: min/max (janet's are variadic with the same
numeric semantics), nil?/some? lowered to janet's fastfun = / not=
against nil, and not.
- clojure.math (Clojure 1.11) installed as a namespace whose vars hold
janet's math natives directly, so calls direct-link. Math/sqrt-style
interop stays in the frozen interpret-only punt set (~5us/call); this
is the compiled route (~30ns).
- reduce over pvec/tuple/array iterates indexed in place — it was
copying the whole pvec into a fresh array on every reduce call — and
stops at `reduced` instead of scanning the tail.
- interpreter coll-lookup gains the sorted-coll arm: (:k (sorted-map ..))
was nil in interpret mode (compiled mode had it right).
map-fastpath-spec pins keyword-invoke/map-literal semantics (16+15
rows incl. nil-value maps, records, sorted, vectors-of-internals) and
clojure.math (10 rows). Two pre-existing bugs found and filed while
writing it: map literals evaluate entries in reader-hash order
(jolt-p3c), and an attempted local-callee call inline that breaks
overlay lazy self-recursion is parked with a repro (jolt-507).
Gate green, conformance 335/335 x3, suite 4718 steady, core bench even
with main back-to-back.
Syntax errors were positionless ('Unterminated list', no idea where).
Now they use Clojure's shape:
Error: Syntax error reading source at (src/app/syn.clj:3:8): Unmatched delimiter: ]
at src/app/top.clj:1
The reader's 15 error sites raise a {:jolt/reader-error :msg :pos}
struct carrying the byte offset (every site already had pos in scope).
The parse entry points convert offset -> line:col on demand and
re-raise the formatted message: parse-string against its own string
(no file), parse-all-positioned against the full source with the
file threaded in from the loaders — rebasing slice-relative offsets
onto the original source so positions stay absolute. No per-token
cost; nothing is tracked until an error actually happens.
Unmatched-delimiter messages match Clojure's 'Unmatched delimiter: )'
wording. cli-test rows assert positions for unterminated string/list,
unmatched delimiter through a require (composing with round 4's
'while loading' chain), and bad ## tokens. Gate green, suite 4718
steady, bench within noise.
A failing top-level form now reports where it lives:
Error: Cannot add 1 and "boom" — + expects numbers
at /path/src/app/broken.clj:3
while loading /path/src/app/mid.clj
while loading /path/src/app/top.clj
The reader has no per-form positions (round 5), but the loaders know
exactly which slice of source each form came from: parse-all-positioned
(reader) counts newlines around parse-next and returns [form line]
pairs; load-ns, load-string and load-ns-source evaluate through a
positioned loop that on error stashes the innermost form's {:file
:line} on the env and appends each file unwound through to a loading
chain. report-error prints both, suppressing the synthetic <eval>
strings the CLI feeds itself (the require/apply one-liners).
load-string takes an optional file arg; run-file passes the script
path so script errors name the script. cli-test rows cover the
3-requires-deep case, script files, and that one-line -e output stays
clean. Gate green, suite 4718 steady, bench within noise.
A typo'd symbol used to auto-intern an unbound var and die later as
'Cannot call nil as a function' with no hint which symbol. Now:
$ jolt -e '(undefined-fn 1)'
Error: Unable to resolve symbol: undefined-fn in this context
The analyzer's :unresolved fallthrough now punts to the interpreter
(whose resolver raises the message above when the form runs) instead of
emitting a var-ref that interned the var. A punt rather than a hard
throw because runtime-interning forms (defmulti's setup) legitimately
reference the var they're about to create from a nested do.
Pulling that thread surfaced three real bugs the leniency was masking:
- h-resolve-global resolved unqualified symbols against ctx-current-ns,
which during analysis is jolt.analyzer — so user-ns vars NEVER
resolved through it; the lenient arm happened to emit the right ns.
Now resolves against the compile ns like the qualified branch.
- Top-level (do ...) wasn't split: Clojure compiles and EVALS each
child in sequence so earlier children's runtime effects (defmulti's
intern) are visible while later children compile. eval-toplevel now
splits.
- The stdlib itself had forward references the auto-intern hid:
10-seq's transducers used vreset!/vswap! from 20-coll (moved to
10-seq); in 20-coll qualified-ident?/realized?/list*/underive
referenced defs declared later in the file (reordered); sorted? and
partition-all are genuinely later-tier and got (declare ...).
Test rows updated where they encoded the old leniency: ir-passes'
dead-branch row (unresolved in a dead branch is an error, as in
Clojure), compile-mode's ctx-isolation row (other ctx now errors
instead of reading nil), cli rows assert the new message. Gate green,
conformance 335/335 x3, suite 4718 steady, bench within noise.
Before: (+ 1 "a") printed 'could not find method :+ for 1 or :r+ for "a"'
followed by three janet frames pointing at jolt internals. After:
Error: Cannot add 1 and "a" — + expects numbers
at app.deep/level3
Round 1 — compiled fns carry their Clojure identity:
- The analyzer's recur target (which doubles as the compiled janet fn's
name) is now ns/fn-name (_r$app.deep/level3--N), so janet stack traces
name the user's fns; defn passes the self-name through to fn.
- eval-toplevel re-raises with propagate instead of protect+error — the
failing fiber's stack was being discarded, which is why every trace began
at eval-toplevel.
- require/maybe-require-ns route loaded namespaces through the loader's
compile-or-interpret eval-toplevel via a ctx hook (the evaluator can't
import the loader). Previously REQUIRED namespaces always ran interpreted:
slower, and their fns were anonymous in traces.
Round 2 — report-error presents for users (rephrase-inspired):
- The full trace text is stashed at the innermost eval-toplevel boundary
(janet's debug/stacktrace walks the fiber propagation chain; debug/stack
cannot), then filtered: _r$ frames demangled to ns/fn-name, jolt-internal
and [eval] frames dropped. JOLT_DEBUG=1 restores the raw janet trace.
- Message rewrites: janet arithmetic dispatch -> 'Cannot add X and Y — +
expects numbers'; compiled arity -> Clojure's 'Wrong number of args (N)
passed to: ns/fn'; nil-call gets an undefined-symbol hint (round 3 will
fix resolution properly).
6 cli-test rows assert the exact user-visible output. Gate green, suite
4718 steady, bench within noise.
Follow-ups for running reitit alongside default-feature libraries in one app:
- __reader-features / __reader-features-set! exposed to Clojure so a namespace
can load a clj-targeted lib (reitit, under :clj) WITHOUT forcing the whole
process to :clj — set features, require, restore. Honeysql/selmer/ring stay
on the default set they were validated under. (jolt vector args coerced to a
janet array — janet map over a pvec iterates keys otherwise.)
- clojure.template added to the stdlib (verbatim, pure Clojure over
clojure.walk) — honeysql's :clj branch requires it.
- java.util.Locale registered as a qualified alias (+ ROOT) for selmer's :clj
Locale/US use.
Everything reitit-core needs to load unmodified from git under :clj features:
- The baked binary now re-reads JOLT_FEATURES at startup (like JOLT_PATH).
reader-features-set! runs at module load = BUILD time for a binary, so a
process opting into :clj (to read a lib's :clj branches) was ignored, and
unmatched #?(...) forms silently spliced to nothing — defn of a fn with an
empty arglist, hence the cryptic index errors.
- (get s i) indexes a string and returns the char, as in Clojure (nth did;
get returned nil). reitit's path parser is (get path i)-based — without
this every route read as static.
- Class-shim registration exposed to Clojure: __register-class-statics! /
__register-class-methods! / __register-class-ctor!, so a library can mirror
a Java class jolt doesn't ship (the reitit.Trie mirror lives in jolt-lang/
router on top of these).
- Java surface reitit's :clj branches call: .getMessage (on exceptions and
strings) and a small universal object-method set, .intern, java.util.HashMap
(a mutable map wrapper). Plus defprotocol already took keyword options.
Gate green; clojure-test-suite 4715 -> 4718 (the get-on-strings fix).
Clojure's defprotocol takes an optional docstring and leading keyword
options (:extend-via-metadata true) before the signatures; jolt's macro
fed the option keyword to (first sig). honeysql declares its InlineValue
protocol exactly that way — with the fix, all four honeysql namespaces
load unmodified from git and the formatter produces correct sqlvecs for
selects/inserts/updates/deletes/joins/:inline. Listed in libraries.md.
A native-executable build bakes the jolt ctx, and env-reading libraries
(config.core/load-env) snapshot the ENTIRE build environment into it — jpm
marshals that into the binary. GitHub push protection caught real API
tokens inside an example's build output this way.
With JOLT_BAKE_ENV_ALLOWLIST set (comma-separated names — a project's
build.sh exports it for the bake), System/getenv serves only the listed
variables: single-name reads of unlisted vars return nil and the full
snapshot is filtered. Unset — every normal runtime — reads stay live and
unfiltered, so a baked binary that re-reads env at startup (config.core/
reload-env) sees the real runtime environment.
Verified A/B on ring-app: a planted token appears in the unscrubbed binary
(strings | grep: 1 hit) and not in the scrubbed one (0), which still serves.
Direct janet.os/environ bridge calls remain unfiltered host access, as
documented.
Running a checkout's build/jolt-deps by path failed with ENOENT unless
build/ was also on PATH: exec-jolt spawned a bare "jolt". Resolution is
now $JOLT_BIN, then the jolt sitting beside this binary (argv[0]'s
directory — the pair is built together), then PATH.
The vendored spork/http is gone — jpm owns janet packages. In its place:
- The janet.* bridge autoloads jpm-installed modules on first reference:
janet.spork.http/server requires spork/http from the module path and
caches its bindings (failures are negatively cached). Works for any
module, in every mode, including inside net/server connection fibers.
- deps.edn grows a :jpm/module coordinate: jolt-deps verifies the module is
importable at resolve time, optionally running `jpm install` on the
:jpm/install package once when it isn't, and otherwise fails with the
install hint. Contributes no source roots. ring-app declares spork/http
this way.
Docs: README's interop section, docs/tools-deps.md (:jpm/module reference),
and the ring-app README (including the jpm-version caveat for spork HEAD's
.janet native sources, which older jpm rejects).
(^bytes [b])-style return hints reach fn as a (with-meta [b] {:tag ...})
form; unhint sheds the wrapper through the rebuild path so the clause
representation never changes. The host-interop hint rows exercise it.
Class names evaluate to their canonical class-name STRINGS (the same values
class returns), so a (defmulti m (comp class :body)) matches (defmethod m
String ...) — ring.util.request dispatches exactly this way. Constructor
sugar and the new special form resolve the actual ctor from the registry
when given a token; dispatch-only names (InputStream, File, ISeq, ...) are
interned for defmethod position. nil is a legal multimethod dispatch value
now (sentinel-keyed: janet tables drop nil keys) — ring keys body-string's
no-body case on it.
spork/http is VENDORED (vendor/spork/http.janet, MIT) and baked into the
image, reaching the jolt layer as janet.spork.http/* through the janet.*
bridge — whose lookup is now a chain (runtime fiber env, module env, the
vendored registry), which also fixes janet/* resolution inside net/server
connection fibers (they carry a foreign env). jolt.http is rewritten over
the spork client (its old net/request never existed; also fixes its own
get shadowing clojure.core/get).
Also: slurp accepts opts and DRAINS reader shims (ring middleware slurps
request bodies), clojure.string/replace takes fn replacements with Clojure's
match-or-groups argument, .indexOf int needles are char codes, .getBytes on
the String surface, and ^bytes-style return hints on param vectors parse
(the fn macro unwraps the with-meta form).
Suite steady at 4715/5348; conformance x3 green; deps-conformance medley +
cuerdas green (the stuartsierra/dependency failure predates this change).
The interop surface ring.util.codec needs (registered through the javatime
shim registries): URLEncoder/URLDecoder (www-form-urlencoded in pure janet),
Charset/forName, Base64 encoder/decoder, Integer/valueOf with radix +
parseInt, StringTokenizer, clojure.lang.MapEntry (a 2-tuple), a String ctor
from bytes, .getBytes on the String surface, and a java.lang.Number method
surface (byteValue and friends).
Protocol fixes: extend-protocol on java.util.Map/Set/List now dispatches
(maps — phm/struct/sorted/records — never produced host tags and fell to
Object), lazy seqs gained their ISeq tags, and a nil extension arm works
(group-by-head and extend-type both choked on the nil head). reduce
dispatches to a reified clojure.lang.IReduceInit's own reduce method, which
is how ring-codec tokenizes.
jolt-deps learns :deps/root (tools.deps monorepo subdirectory checkouts —
ring-core lives inside ring-clojure/ring). spork/http, when jpm-installed,
reaches the jolt layer as janet.spork.http/* through the janet.* bridge
(soft: nothing requires it unless used).
The protocol fixes alone let 29 more clojure-test-suite assertions execute:
5319 -> 5348 run, 4706 -> 4715 pass.
The first per-type migration print-method unlocked: uuid, regex, transient,
and channel rendering move from host pr-render branches to io-tier
defmethods (exact same output). The renderer's tagged fallthrough now
dispatches ANY remaining :jolt/* value through the print-method hook before
the raw pairs view — so every tagged type is user-overridable, atoms
included ((defmethod print-method :jolt/atom ...) fires nested), and future
per-type migrations are pure overlay additions.
Hot types (numbers, strings, symbols, collections) stay native, and inst/
namespace/var stay host for now — their formatters (rfc3339, display names)
live there anyway. A transient's :kind is read with jolt.host/ref-get: get
on a transient is the dispatched collection lookup (same trap as sorted
colls).
Before the hook is wired (init-time error messages) tagged values fall
through to the pairs view — bootstrap rendering only.
print-method/print-dup are now multimethods in the io tier with Clojure's
exact dispatch ((:type meta) keyword, else type — core.clj 3693). On jolt the
dispatch value for a record is its quoted full-name symbol, since class names
aren't values here.
Records used to pr-str as the raw janet table; the renderer's record branch
now prints Clojure's #ns.Type{:k v} syntax, and first consults a callback the
api wires up after the overlay loads — so a user defmethod on a record type
fires everywhere: top level, nested in collections, through pr/prn/pr-str.
Builtin overrides (a :number method) fire only on direct print-method calls;
pr keeps the native fast path (documented divergence).
java.io.Writer arrives as a shim beside the StringReader/StringBuilder ones:
a :jolt/writer tagged value with write/append/flush/toString, a StringWriter
ctor, and a sink variant the renderer callback uses.
Two latent host bugs fixed on the way: the interpreted syntax-quote splice
blew up on ~@nil (an interpreted macro's empty & rest binds nil — first tier
user of defmulti found it; d-realize now treats nil as the empty seq), and
(print-method x nil) now throws like the JVM instead of returning nil.
10 spec rows; bench dead even (sandwich run); greeter green on a fresh
binary.
Batch 2 of the post-shrink sweep, all pure compositions or documented stubs:
enumeration-seq/iterator-seq (seq), promise/deliver (an atom — deref of an
undelivered promise stays nil, single-threaded host), bean, uri?,
special-symbol? (an evaluated set of quoted symbols — a QUOTED set literal
stays an unevaluated reader form on jolt, which the first version tripped
over), print-method/print-dup (inert until jolt-g1r), and the whole proxy
surface (mappings/call-with-super/init/update pass through, the constructive
half throws). The seed loses 16 defns and bindings; nothing kept.
Three canonical-conformance fixes from the post-shrink batch:
- bit-and/bit-or/bit-xor/bit-and-not get Clojure's variadic arities as
20-coll shells folding the binary host ops (now __bit-* seams). 2-arg call
sites still compile to the native janet op via the backend's native-ops
table. The passes.clj constant-fold table now names the seams — the public
fns are overlay and don't exist when the compiler loads (this briefly broke
every compile-mode init).
- core-set? recognizes the :jolt/sorted-set representation (jolt-dpn):
(set? (sorted-set 1)) was false, and ifn? on sorted sets inherited the bug.
- (if) / (if test) / (if test then else extra) throw in both the analyzer
and the interpreter — spec 03-special-forms X1, now marked verified.
Suite 4704 -> 4706; bench and the greeter example benchmark are flat.
Fixed arities now throw Clojure's ArityException shape — 'Wrong number of
args (N) passed to: name' — on any count mismatch; variadic arities on fewer
than their fixed params. The compiled path already enforced fixed arities via
janet's native fn check and multi-arity dispatch; this adds the check to the
interpreter's single-arity closures (the oracle was silently dropping extra
args and giving a raw tuple-index error for missing ones) and guards the
compiled single-variadic wrapper's minimum. Messages carry the fn name when
there is one. 16 spec rows; the update.cljc suite row flipped green (4703 ->
4704).
Enforcement exposed that seq-to-map-for-destructuring had drifted: the spec
row called the 1-arity fn with two args, and the body silently dropped a
trailing unpaired element. Replaced with the canonical Clojure 1.11 version
(even pairs build the map, a single trailing element passes through — so
(f {:b 2}) kwargs calls work — and an unpaired key throws).
Also: transients RFC notes tuple support from the seed-shrink rounds.
Round 6 of the seed shrink (the printer round, scoped by the perf wall). The
five wrappers move to 20-coll over two new host seams: __write (push a string
to *out*) and __pr-str1 (render one value readably). The renderer itself
stays in the seed — it's representation-coupled (pvec/phm/phs/sorted
internals) and shared with the hot str, and rendering through overlay calls
would pay the per-element call cost everywhere big values get printed.
print-method as a real multimethod is follow-up work.
The new spec rows caught a renderer bug: string bodies were never escaped, so
(pr-str "a\"b") didn't round-trip through the reader. pr-render now
escapes quote/backslash/control chars per Clojure.
Round 5 of the seed shrink. transduce is the canonical 5-liner over reduce
(which already honors reduced and steps lazy seqs); eduction composes with
comp and stays eager into a vector (documented divergence, as before);
td-comp — eduction's last caller — is deleted from the seed. transient
accepts tuples now (reader vectors / map entries), so (into [] (first {:a 1}))
keeps working everywhere a vector does.
into was moved, benched, and moved back: the overlay call layers cost the
into-vec suite ~11% back-to-back (536 vs 480ms), the same per-call wall that
sent even?/odd? home in round 4. A transient conj! fast path didn't pay for
itself either (jolt call overhead dominates, not the per-element conj). The
seed keeps core-into + its private transduce machinery; the binding count
still drops by three.
Round 4 of the seed shrink. zero?, pos?, every? move to the syntax tier
(empty? and the analyzer use them — raw def+fn* per the file constraint);
char? joins the tagged-value predicates in 20-coll. coll? stays seed: host
set? doesn't cover sorted sets (filed jolt-dpn) and the tag check from the
overlay would hit the sorted-coll get trap. pos? guards number? explicitly —
the staged recompile emits bare > as the native janet op, which orders
strings (zero? gets the same guard; spec rows lock both plus neg?).
The canonical every? seq-walks its coll, which exposed that rest/next over
sets, phms, struct maps and sorted colls fell into core-rest's indexed
fall-through and walked the wrapper table's INTERNAL fields — (next #{1 2})
was (nil nil), (clojure.set/subset?) broke. core-rest now seqs those
representations (branches placed AFTER the hot vector/lazy paths; the first
ordering cost seq-pipe 4x). Suite rises 4700 -> 4703; baseline 4660 -> 4695.
even?/odd? are back in the seed after the bench A/B: (filter even? ...) pays
an extra call layer per element through the overlay (seq-pipe 262 -> 1100ms).
They join the perf-wall list with the lazy hot fns.
Round 3 of the seed shrink. To the overlay: identity, constantly, neg?,
even?, odd? (20-coll, ahead of their first in-tier uses), not= and unreduced
(00-syntax — the kernel and seq tiers use them), ==, ensure-reduced,
halt-when, parse-boolean, parse-uuid, newline, seque, array-seq, to-array-2d,
and the masking unchecked-byte/short/char/float/double coercions. parse-uuid
validates via re-matches over a new __make-uuid host binding (overlay source
can't write :jolt/type map literals). memfn moves to 30-macros as a working
macro over the .method call sugar instead of a fn that throws.
Behavior fixes toward Clojure, each with spec rows: == now throws on
non-numbers instead of comparing them, and halt-when is the canonical
::halt-map version (the halt value replaces the whole reduction result, no
double completion). list? and map-entry? stay in the seed — both are
representation-coupled (plist/tuple checks).
clojure-test-suite goes 4701 -> 4700: update.cljc expects
(update {:k 1} :k identity 1 2 3 4) to throw an arity error, and jolt fns
don't enforce fixed arity anywhere (pre-existing, language-wide — the seed's
Janet identity threw natively). Filed as jolt-6xn; fixing it should flip
several suite rows at once.
