types.clj held the inferencer, the success-type checker, and the driver in one
716-line namespace. Move the self-contained checker into jolt.passes.types.check:
the error-domain predicates (not-number?/not-seqable?/not-callable?), the op
tables, type-name, check-invoke, and the user-fn registry. These are pure over
inferred types and the run's env cells, with no inference, so a check-rule edit
can no longer perturb the inferencer.
The infer-coupled probes stay in types.clj — isolated-diag-count and
check-user-call re-run inference, so moving them would make check depend on the
inferencer and reintroduce the cycle. Verbatim move; new ns wired into
ei-compiler-ns-files; seed re-minted to the byte-fixpoint.
The stderr seam wrote to current-error-port without flushing, so a process that
never returns from -main (a server loop) left its log lines in a buffer that only
drained at exit — they never appeared. Flush each write, like the JVM's
auto-flushing System.err.
The source loader sets the current ns and registers :as aliases per file. The
build flattened every app namespace into one image with no such markers, so all
app forms ran under the last-set ns ("user"). Two breakages followed, both only
in a built binary:
- defmulti/defmethod resolve their target var through chez-current-ns, so they
registered the multifn under "user" while compiled var-derefs used the baked
ns — the multifn the app saw was uninitialized ("not a fn nil" on dispatch).
- a quoted alias-qualified symbol (a (defmethod ig/foo …) on an aliased multifn)
resolves its ns through chez-resolve-alias, but the stripped (ns …) form left
the alias table empty, so it landed in ns "ig".
bld-ns-prelude now emits (set-chez-ns! ns) plus chez-register-alias! for each
ns's :as aliases before that ns's forms, in both the normal and tree-shake emit
paths. The build-app fixture gains a :default multimethod and an aliased cross-ns
defmethod so buildsmoke covers both across all build modes.
walk treated a record as a plain map (record? implies map?), rebuilding it via
(into (empty form) ...) which yields a bare map and drops the type. Add a record
branch before the map branch that conj-es the walked entries back onto the
original, matching JVM clojure.walk's IRecord case. Type-dispatched walks need it
— integrant resolves #ig/ref by detecting its Ref record while postwalking the
config, so without this every ref silently fails to resolve.
clojure.walk is baked into the prelude, so the seed is re-minted. Corpus gains
five JVM-certified rows for record type/instance? survival through pre/postwalk.
infer's :invoke case was ~120 lines of cond arms hand-coding eight call
patterns, all destructured positionally with (nth r 0)/(nth r 1) on the
[type node'] tuples infer returns. Split each pattern into a named helper
(infer-pred-fold/-kw-lookup/-get-lookup/-reduce-hof/-seq-hof/-conj-into/-call)
behind an infer-invoke dispatcher that keeps the cond guards verbatim, and add
ty/nd accessors for the tuple so a silent transposition can't hide.
The accessors are applied only to genuine infer results (the new helpers and
infer-fn-seeded); the :map/:let/:loop arms interleave non-infer pairs
(binding tuples, accumulator pairs) with infer results, so those keep nth.
Pure restructuring — the guards, order, and bodies are unchanged; seed
re-minted to the byte-fixpoint, gate green, 0 new corpus divergences.
The inliner and the type inferencer each recognized (:k m) and (get m :k)
lookups with their own copy of the callee tests — the get-callee check was
duplicated verbatim across both. Lift kw-callee?/get-callee? into
jolt.passes.fold (alongside scalar-const?) and call them from both passes so
the head recognition can't drift.
Only the head predicates move. The deliberate differences stay: the inliner
still accepts any scalar key in the get-form (its scalar-replacement targets
can be string/number-keyed maps) while the inferencer requires keyword keys
for struct field typing, and the inferencer keeps its two arms separate so each
rebuilds args for its form. The backend's value-as-fn ifn-kind is left alone.
Two small clarity moves from the review:
- dynamic-vars.ss -> dynamic-var-defaults.ss. It holds the default VALUES of a few
core dynamic vars (*clojure-version*, *assert*, …); the near-identical dyn-binding.ss
holds the binding-stack machinery. The names were easy to confuse.
- Pull the ~60-line %-format engine out of the natives-misc.ss grab-bag into
natives-format.ss (its ->long/pad-left/fmt-float helpers were local to it).
rt.ss loads + MODULES.md updated. Runtime .ss, no re-mint; make test green, format +
the dynamic vars verified.
natives-parity.ss was a grab-bag (its own header called it 'parity'). Dissolve it
into homes that say what they hold:
- cat -> natives-transduce.ss (was natives-xform.ss, renamed for the
whole transducer surface: volatiles + cat + transduce/sequence)
- transient? -> transients.ss
- rseq -> natives-seq.ss
- hash family -> natives-misc.ss (the public hash API over jolt-hash)
The remainder — the #?() feature set, the reader-conditional + re-matcher tagged-map
ctors, and macroexpand — is a coherent reader/macro runtime-support unit, kept as
natives-reader.ss (np- prefix -> nr-). rt.ss loads + two comment refs updated.
Runtime .ss, no re-mint. make test green; hash/transient?/rseq/cat/macroexpand and
#?() reader features all verified.
The trio split on a fine axis (registry core / statics / object classes) but the
names didn't say so — 'static-statics'/'static-objects' and headers that read
'Continues X'. Rename:
host-static-statics.ss -> host-static-methods.ss (Class/member statics + fields)
host-static-objects.ss -> host-static-classes.ss (instantiable object classes)
host-static.ss stays the registry core. Headers rewritten to state each file's role
and what it covers instead of chaining. rt.ss loads + the one comment reference +
MODULES.md updated. No code moved; runtime .ss, make test green.
Under --direct-link a top-level def binds jv$<fqn> and app->app calls bound directly
to it. emit-invoke raw-applied that binding for any var callee, but only a fn-valued
def is a Scheme procedure: (def cfg {...}) then (cfg :a) emitted (jv$cfg :a), applying
a pmap -> "attempt to apply non-procedure". Maps/sets/keywords are invokable in Clojure
via jolt-invoke, which the indirect path used, so this only bit closed-world builds.
Track which direct-linked vars hold fn literals (direct-link-fns, registered at the def
site when the init op is :fn) and only raw-apply those. A non-fn callee falls through to
the jolt-invoke branch, which still uses the direct jv$ binding as the invoke target —
so the var-deref is still skipped, just not the dispatch.
Seed source: re-minted. Regression in directlink-test.ss (jolt-cw1o).
The printer's two entry points (jolt-pr-str in rt.ss, jolt-pr-readable in printing.ss)
get register-pr-str-arm! / register-pr-readable-arm!, plus register-pr-arm! for the
types whose str and readable forms match (bigdec/inst/uuid/tagged/record/ns/var). The
normalize arms (sorted, lazy-seq, queue) and the uri readable arm register per-printer.
Also folds in the hash (dyn-binding var-cell), class (io uri/uuid/file), and get
(transients) arms missed earlier.
natives-array's get stays a case-lambda wrapper on purpose: its 2-arg path errors on
an out-of-bounds index while the 3-arg path returns the default, an arity distinction
the (coll k d) registry collapses — left as-is to preserve behaviour.
Completes jolt-lmot: all six dispatchers (hash/class/get/=/pr-str/pr-readable) off the
set!-rebind chains. make test green, 0 new corpus divergences; pr-str/str of inst,
uuid, bigdec, sorted-map, record-with-lazyseq, queue all verified.
jolt-get (4 sites: host-table/natives-misc/inst-time/records) and jolt=2 (6 sites:
records/vars/inst-time/natives-misc/bigdec/host-table) move off the set!-rebind
chains to register-get-arm! / register-eq-arm!. get's case-lambda becomes a stable
2/3-arg entry over a 3-arg dispatch; the equality arm pred is (a b) since either arg
may carry the type, and the host-table sorted arm normalizes-then-re-dispatches.
Behaviour-preserving (runtime .ss): var/inst/bigdec/record/uuid equality, record!=map,
sorted-map=plain-map, and all the get cases verified; make test green, 0 new corpus
divergences. Four of six dispatchers done; the printer (pr-str/pr-readable) remains.
Replace the set!-capture-and-rebind chains extending jolt-hash (3 sites: inst-time/
host-table/records) and jolt-class (3 sites: bigdec/natives-queue/host-table) with a
register-hash-arm! / register-class-arm! registry (the pattern already used by
register-str-render!). The base dispatcher walks its arms — disjoint types, so order
is immaterial — then falls to the base cases. The entry is stable, so the per-site
(def-var! "clojure.core" "class" …) re-points are gone.
Behaviour-preserving (runtime .ss, no re-mint): jinst hash, bigdec/queue/record
class, record hash, and a sorted-map hashing as its plain map all verified; make test
green, 0 new corpus divergences. First two of six dispatchers; get/=/pr-str follow.
set-optimize!/set-direct-link! are process-global flags in the back end, set then
reset around the emit. A strict-form build error (failing forms error the build by
design) skipped the reset, leaving the compiler in optimize/direct-link mode for any
later in-process caller. dynamic-wind guarantees the revert. Behaviour unchanged on
the success path; both --tree-shake and --opt --direct-link build + run identically.
Tree-shaking was split across emit-image.ss (the dce-* helpers + record producer)
and build.ss (bld-shake-all + the manifest splice), with the DCE record accessed by
raw (vector-ref r 0..3) in ~10 places and the manifest splice/drop driven by
substring-matching (load "…prelude.ss").
- New host/chez/dce.ss owns the whole DCE concern: a named record API
(dce-rec/-keep?/-fqn/-refs/-str — no more positional vector indexing), the ref
extraction + ref-set constants, dce-blob-records, and dce-shake decomposed into
dce-build-graph / dce-reachable / dce-bail-scan / dce-partition (was one 50-line
bld-shake-all doing five jobs with shared mutable state).
- emit-image.ss keeps only ei-emit-ns-records (it drives the ei-* compiler) and uses
the dce-rec constructor.
- The runtime manifest is now tagged ('prelude/'image/'compile-eval); bld-emit-runtime
dispatches on the tag instead of substring-matching file paths, so the core-splice
and compiler-drop can't silently break on a rename/reorder.
Behaviour-preserving (runtime .ss, no re-mint): build-app shakes identically
(56/460, 8.12MB), make test green, make shakesmoke green (4 git-lib apps
byte-identical, same sizes).
When --tree-shake keeps everything because reachable code resolves vars at runtime,
list the offending def -> bail-ref pairs (up to 6) instead of just saying it
skipped. Makes it actionable: e.g. ring-app shows
clojure.tools.logging/call-str -> ns-resolve and selmer.filters/generate-json ->
resolve, so you know which library (not your code) blocks shaking.
--tree-shake now shakes the clojure.core/stdlib prelude in the same reachability
graph as the app + libraries — only core fns actually reached from -main ship.
dce-blob-records reads prelude.ss with Chez `read`, unwraps each
(guard ... (def-var! "ns" "name" V)) and builds the core->core call graph from the
(var-deref/jolt-var "ns" "name") refs in V — the real emitted edges, no
re-analysis. bld-shake-all merges core + app records into one BFS; roots are -main,
side-effecting forms, and the clojure.core fns the runtime .ss shims reference by
name (enumerated in dce-runtime-core-roots). The shaken core is spliced where the
prelude.ss blob was, in original (tier) order, so load-time deps are preserved.
Bail (reachable runtime resolve) keeps the prelude whole.
Soundness follows Stalin's rule (any reference, value or call, keeps a def live):
dce-collect-refs counts :var and :the-var; non-def registration forms are always
kept (covers protocol/multimethod dispatch). Validated by make shakesmoke: the four
deps.edn git-lib apps build byte-identical output shaken vs not.
Wins (binary): build-app 9.84MB -> 8.12MB (dropped 403/457 core defs); malli-app
10.0MB -> 8.1MB; markdown 9.9 -> 8.3; commonmark 9.8 -> 8.1 — all output-identical.
build-smoke asserts an unused core fn (group-by) is dropped; full make test green.
Two things, both from studying Stalin's closed-world DCE:
1. Soundness fix: dce-collect-refs now counts a :the-var (#'x / (var x)) reference,
not just a :var. Stalin's rule is that ANY reference — value position, not only a
direct call — keeps the target live; a var referenced as #'x would otherwise be
wrongly dropped. (My :var collection already covered value-position refs; this
closes the the-var hole.)
2. host/chez/tree-shake-smoke.sh (make shakesmoke): builds example apps default vs
--tree-shake and requires identical output — the real risk is shaking a binary
that pulled libraries via deps.edn. Covers markdown/malli/commonmark/hiccup
(git-lib apps). All produce byte-identical output shaken vs not, and drop
~0.8-1MB (malli 10.0MB -> 9.0MB) from the compiler-drop. Slow; not in the default
gate. Skips without the examples repo.
An AOT-compiled app only needs the analyzer/back end at runtime to compile from
source — eval / load-string / load-file. Macros are expanded at build time and a
require of a baked namespace no-ops, so a closed app that never compiles at runtime
carries the compiler image (~0.8MB) as dead weight.
Under --tree-shake, when reachability is trustworthy (no bail) and no reachable code
references eval/load-string/load-file/load-reader/load, omit host/chez/seed/image.ss
+ compile-eval.ss from the runtime manifest. bld-tree-shake returns the flag
alongside the shaken forms; bld-emit-runtime filters the manifest.
Measured: build-app 9.84MB -> 9.05MB, still runs. Safety verified: an app that evals
keeps the compiler (eval is a bail + compile ref) and eval works at runtime.
build-smoke asserts the compiler is gone in the no-eval app; full make test green.
`jolt build --tree-shake` (or deps.edn :jolt/build {:tree-shake true}) does
reachability DCE over the re-emitted app + library namespaces: keep -main, every
side-effecting (non-def) top-level form, and every def reachable from those; drop
the rest. A macro (expanded at AOT, never called at runtime) is prunable too.
Sound: bails (keeps everything) if REACHABLE code resolves vars by name at runtime
(eval/resolve/ns-resolve/requiring-resolve/find-var/intern/load-string/...), which a
static call graph can't follow. Unreached eval-using library code is simply shaken
away and never triggers the bail. clojure.core and the compiler image stay baked
(prelude + image blobs), so only re-emitted namespaces are shaken for now.
The reachability machinery is in emit-image.ss (records: keep?/fqn/refs/str via
reduce-ir-children) + build.ss (BFS + bail check). build-smoke covers it (drops the
unreachable `twice` macro, output unchanged). Opt-in; default builds are untouched.
full make test green.
Scope note: this shakes the re-emitted app/lib code only. Measurement shows jolt's
compiled code is ~5.8MB of a ~9.8MB binary, dominated by the clojure.core prelude
(~1.5-2MB) and the compiler image (~0.8MB) — both baked blobs this pass doesn't
touch. Those (shake-core, drop-compiler-when-no-eval) are the larger footprint wins,
filed as follow-ups.
A loop var with an integer-literal init now types :long (fx ops) when every recur
arg in its slot is an increment-style step — the var unchanged, inc/dec, or (+/-
var <int-literal>). So (loop [i 0] (recur (inc i))) gets fx1+/fx<? without a hint,
matching how Clojure treats a primitive-long loop counter.
Soundness: only increment steps qualify. A multiplicative or large-growth
accumulator like (recur (* acc 2)) is never seeded, so it stays generic and keeps
arbitrary precision — a bignum-producing loop (e.g. a factorial) is unaffected.
counter-step? gates this; the existing fixpoint demotes anything inconsistent.
test/chez/numeric-test.ss 44/44 (incl. a factorial loop staying bignum-exact while
its counter is fx); full make test green, 0 new corpus divergences.
jolt.passes.inline was fully written but dormant — it fetched bodies via the
inline-ir host hook, which was a stub returning nil. Wire it up: run-passes stashes
each inline-eligible defn (single fixed arity) as its form is optimized, and
inline-ir hands the body back at call sites under --opt.
The catch was the ^double/^long coercion: an inlined fn drops its param-entry and
return coercion, so (work 3 4) on a ^double fn would return 25 instead of 25.0. New
:coerce IR node carries the coercion inside the spliced body — the inline pass wraps
a hinted param's arg and the return in :coerce, the back end lowers it
(exact->inexact / jolt->fx), and jolt.passes.numeric reads its :kind. So an inlined
call matches the called one and the body's fl*/fx* fast path still fires.
Only under --opt (closed world); the seed mint and -e don't inline, so selfhost and
the corpus are unaffected. test/chez/inline-test.ss 12/12 (make inline); full make
test green, 0 new corpus divergences.
Bench (hot loop, body is a ^double helper call): direct-link 500ms -> --opt
(inlined) 184ms = 2.7x, by eliminating the call + coercion wrappers and letting Chez
fuse the fl-ops unboxed. ~26x over the default dispatched build.
A ^double/^long return hint on a fn's name now (a) coerces the fn's value on the
way out — exact->inexact / jolt->fx, like a JVM primitive return — and (b) types a
call to it, so an accumulator over the result specializes:
(defn ^double work [^double x ^double y] (+ (* x x) (* y y)))
(loop [acc 0.0] (recur (+ acc (work a b)))) ; (+ acc (work ..)) -> fl+
The analyzer pushes the name's numeric tag onto each arity (:ret-nhint) for the
back-end coercion, and resolve-global surfaces the callee's declared return
(:num-ret, read from var meta) onto the :var node so jolt.passes.numeric types the
call. defn carries the name hint through.
This unblocks the accumulator-over-fn-result pattern that round 2 had to demote.
The win is bounded by call overhead in an open/dispatched build (~1.15x on a hot
loop whose body is a helper call); it compounds with direct-linking and, later,
inlining. A numeric return hint is a contract, like ^long — redefining the var to
return another type in an open build breaks it.
Not yet: per-arity arglist return hints, (defn f (^double [x] ..)). Gate:
test/chez/numeric-test.ss 39/39; full make test green, 0 new corpus divergences.
loop-kinds only typed :double accumulators; a ^long-seeded loop var (e.g.
(loop [acc start] ...) with a ^long start) stayed generic even though it's sound
to fx-type — :long only ever comes from an explicit hint, and a ^long value is
already coerced to a fixnum at fn entry. Keep the init's kind (:double or :long)
through the fixpoint, demoting only on a recur-arg mismatch.
Integer-literal-init loop vars (a bare (loop [i 0] ...)) still stay generic by
design: :long is never seeded from a literal, so a bignum-producing loop keeps
arbitrary precision.
A loop binding whose init is double and whose every recur arg stays double (a
bounded monotone fixpoint) is typed :double, so its arithmetic — and the recur
args feeding it — emit fl-ops. Chez can then keep the accumulator unboxed in a
float register across the loop.
Integer loop vars stay untyped: a bare integer init never seeds :long (same rule
as round 1), so a bignum-producing loop keeps arbitrary precision rather than
overflowing a fixnum. recur-kinds walks only tail position (if/do-ret/let-body),
stopping at nested loop/fn so a loop sees only its own recur.
A/B on a loop-carried double accumulator: 735ms generic -> 500ms typed (1.47x),
closing the gap to the JVM from ~3.3x to ~2.2x. The integer counter stays generic,
which is most of the residual.
A ^double/^long param hint (or a float literal) now drives Chez flonum/fixnum
ops instead of generic arithmetic — JVM-style primitive hints, available in every
build and at -e (not gated on direct-linking or whole-program inference).
New pass jolt.passes.numeric: a local forward type-flow seeded from ^double/^long
fn-param hints (analyzer attaches :nhints per arity) and float literals,
propagated through let inits / arithmetic / if / do. It tags an arithmetic invoke
:num-kind :double|:long when every operand is that kind (an integer literal is a
wildcard, coerced to a flonum in a double op). The back end lowers a tagged node
to fl+/fl-/fl*/fl//fl<?/... or fx+/fx*/fx1+/fxquotient/... (unchecked-add etc.
join the fixnum path; == too). Runs last in run-passes, both branches.
Soundness: :long is seeded ONLY from an explicit ^long hint, never a bare integer
literal, so un-hinted integer code keeps jolt's arbitrary-precision numbers — no
fixnum-overflow surprise, no corpus divergence. :double comes from ^double hints
and float literals (flonum arithmetic is always flonum, matching the generic
result). A ^long hint is a promise the value is a fixnum: fx+ raises on overflow,
like a JVM fixed-width long.
Numeric-hinted params coerce at fn entry (exact->inexact / jolt->fx), the way the
JVM coerces a primitive parameter — so the body's fl*/fx* ops can rely on the
type even when a caller passes an exact int (e.g. Chez's (* 0 1.0) => exact 0).
Round 1 specializes hinted straight-line / fn-body arithmetic. fl-ops are ~4x
generic in a tight Chez loop, but realizing that on loop-carried accumulators
needs loop-var typing — round 2. Sound foundation, gated by test/chez/numeric-test.ss.
Release builds can legitimately want runtime dynamism (redefinition, eval,
load-string), so closed-world direct-linking shouldn't be forced on them. Gate it
behind an explicit --direct-link flag (or deps.edn :jolt/build {:direct-link
true}); off by default in every mode, including release and --opt.
build-binary takes an explicit direct-link? arg instead of deriving it from the
mode. build-smoke now covers the --direct-link path and asserts the cross-ns call
actually lowers to a jv$ binding; default release stays dynamically linked.
A release/optimized `jolt build` is a closed world: every app def is final, so
an app->app call can bind to the def's Scheme binding directly instead of going
through (jolt-invoke (var-deref ns name)).
The emitter gains a direct-link mode (off for the seed mint, runtime -e/repl, and
dev builds). With it on, a top-level app def also emits a binding jv$<ns>$<name>
that def-var! aliases; an app->app call or value-ref to a name already emitted in
the unit lowers to that binding, skipping both the var-table lookup and the
generic IFn dispatch. ^:dynamic/^:redef defs and nested defs (a defonce's inner
def) opt out and stay indirect. Off direct-link mode, emit-top-form is exactly
emit, so the seed and runtime eval are byte-unchanged (selfhost holds).
build.ss turns it on for release + optimized; the defined-set accumulates across
the dependency-ordered namespaces so a dep's defs are linkable by the time the
entry that calls them is emitted. App->core calls stay indirect for now (core is
the baked seed); that's a later stage.
~1.74x on a hot cross-namespace call loop (26.5s -> 15.2s).
A built binary dropped its deps.edn :jolt/native declarations and its
resource roots, so an FFI+resources app (ring-app) failed at runtime:
sockets/sqlite gave 'no entry for socket' and io/resource returned nil.
The buildsmoke fixture is pure compute, so neither path was exercised.
The launcher now loads required + :process native libs before the app's
top-level forms (a library's defcfn resolves its foreign-procedure symbols
at top-level eval during startup, so the libs must be loaded first);
optional libs load in the scheme-start launcher, where a missing lib is
caught rather than aborting the heap build.
deps.edn :jolt/build {:embed [dirs]} bakes those dirs' files into the heap
(register-embedded-resource! at heap build), so io/resource serves them with
no files on disk. Non-embedded resources resolve at runtime against JOLT_PWD,
and io/file reads (e.g. config.edn) stay external.
build-binary now takes the encoded natives, embed dirs, and project paths
from cmd-build; deps/resolve-project surfaces them. Buildsmoke fixture grows
an embedded resource + a :process native to cover both paths.
jolt could call C (foreign-fn -> foreign-procedure) but C could not call back
into jolt, which GTK signals (and any callback-taking C API) require. Add the
inverse: jolt.ffi/foreign-callable wraps a jolt fn as a C-callable function
pointer, mirroring the foreign-fn pipeline.
A new jolt.ffi/__ccallable special form carries the fn as a child expression
(analyzed + walked by the passes; ir.clj gains an :ffi-callable arm in both
child walks) plus literal arg/ret type keywords. The back end lowers it to a
locked Chez foreign-callable and returns its entry-point address as a jolt
pointer; host/chez/ffi.ss registers the code object so the collector keeps it,
and free-callable unlocks it. :collect-safe emits the convention that
reactivates the thread on entry, for callbacks fired while it is parked in a
:blocking call (a GTK main loop).
Test: ffi-binding-test.ss sorts an int array through libc qsort with a jolt
comparator (C -> jolt -> C). Re-minted seed.
Adds 3 cases for the opt-path checker (set-check-mode! -> run-inference ->
take-diags!): diagnostics drain once, the buffer empties on re-drain, and
check-mode off produces none. This is the public checker entry the other 23
cases didn't touch. Runtime-only.
types.clj drove inference through ~14 module-level atoms; the infer walk was
non-reentrant and depended on hidden set-*! install order. Thread one immutable
env (mk-env) through infer instead: it snapshots the installed config
(rtenv/vtypes/record-shapes/protocol-methods/map-shapes?) and carries the
per-run flags and accumulator/guard cells (diags/calls/checking-set/diag-memo).
A fresh env per run makes the pass re-entrant — isolated-diag-count's probe now
runs under a sub-env with its own diags cell instead of save/restoring a shared
atom.
Only state whose lifecycle spans separate API calls stays module-level: a
config-box the set-*! API writes, the escapes/user-sig sweep registries, and a
bridge holding the last checking run's diags for take-diags!. record-type-from-
entry/field-type-from-tag now take the shapes map directly rather than reading a
global.
jolt-ogib.10. Behavior pinned by the new infer gate (23 cases) plus selfhost +
buildsmoke. Re-minted seed.
The corpus/unit gates compile through run-passes' const-fold-only branch, so
the type-inference walk runs only under jolt build --opt — buildsmoke hit one
trivial app and checked stdout. run-infer.ss drives the pass directly: analyze
a source string, then call check-form / infer-body / the set-*! registries and
assert diagnostic counts and collected calls/escapes. Wired into make ci.
Gives the inference pass real behavior coverage so refactoring its internal
state is gate-validatable. jolt-ogib.10 groundwork.
jolt-str-render-one and instance-check were each extended by a chain of
set!-wrapping closures spread across ~10 and ~5 host files, so the real
behavior of either was scattered and load-order-dependent. Give each a
registry the base file owns: converters.ss/records-interop.ss define the
registry plus a register-* helper, and each extending file registers one arm
instead of capturing %prev and set!-ing the global.
str-render arms are type-disjoint; instance-check arms run newest-first (the
old outermost-wins order) and may return 'pass to defer. The string-token ->
symbol normalization the natives-array arm did for every inner arm moves to
the dispatcher head; array tokens stay strings for that arm to decide.
jolt-ogib.14. Runtime-only shims, no re-mint.
The lattice-split commit staged its seed before make remint ran, so image.ss
lagged the source. Commit the correct re-minted image (gensym renumbering only).
types.clj was 852 lines mixing the pure structural-type algebra with the
inference engine, checker, and driver. Move the lattice — scalar/struct/vec/set/
union types, join-t, depth-cap, shape, and the numeric/vector return-fn sets —
into jolt.passes.types.lattice (no inference state, no requires). types.clj
requires it; the engine is now ~720 lines. Compiled into the image before
jolt.passes.types. Re-minted seed differs only by gensym label renumbering.
records.ss mixed the record/protocol/reify model with JVM exception emulation.
Move the contiguous ex-info accessors, the exception supertype hierarchy,
instance-check, case-string, and the instance-check def-var into records-interop.ss,
loaded right after records.ss. Those are only called at runtime, so the relocated
forms resolve fine; records.ss is now the value model and protocol dispatch.
Runtime shim — no seed change.
The 929-line interop registry split three ways: host-static.ss keeps the
registries, the jhost record, the emit entry points and coercion helpers;
host-static-statics.ss holds the java.lang/util static-method registrations;
host-static-objects.ss holds the host object classes (ArrayList, HashMap, the
reader/writer/tokenizer shims, ctors, URL codecs) and the instance? hook. rt.ss
loads the three in order. Runtime shim — no seed change.
The 1123-line collection tier is the largest source file. Cut it at two existing
section banners into 20-coll (predicates, printing, hierarchies, pure-over-core
leaves), 21-coll (rand/sort seams, the test runner, fn combinators), and 22-coll
(canonical Clojure ports, transduce/into, JVM-shape stubs). No macros in this tier,
so order is the only constraint; the emit-image manifest lists the three in
sequence. Re-minted seed is identical apart from gensym label renumbering.
check-user-call rebuilt the all-:any env once per parameter (O(params^2)) and
re-inferred a callee body at every call site. Build the env once and memoize each
probe by [key i argtype] (and the baseline by [:base key]), cleared per form in
check-form. The global type-env is stable within a form's check and the probe's
calls/escapes side effects aren't read there, so a skipped repeat is observably
identical. (The inline-side re-walk the audit flagged is moot: hc-inline-ir is a
no-op on Chez, so try-inline never reaches body-size/body-closed?.)
ei-emit-ns (emit-image) and bld-emit-ns (build) were near-verbatim copies that had
drifted: the minter guard-wraps and skips failing forms, the build is strict, and
since the passes were wired the build also runs run-passes. Fold both into
ei-emit-ns* with optimize?/guard? flags; ei-emit-ns and bld-emit-ns become one-line
callers. Output is byte-identical (selfhost fixpoint and build smoke stay green).
- str-join delegates to str-join-strs (it only adds the per-element render).
- loader: extract resolve-on-roots; find-ns-file and load both use it.
- NumberFormat registers its short + FQ names from one shared member list.
- inst-time's private floor-div/floor-mod renamed inst-floor-* so they don't read
as the math.ss reals version.
Left the fold/inline/types pure-fn sets and the keyword/symbol ns builders alone:
those are file-local and semantically distinct (e.g. the intern key uses NUL, not
"/"), so merging them would be wrong.
- take-last / drop-last return seqs, not vectors: take-last wraps in seq; drop-last
is the JVM (map (fn [x _] x) coll (drop n coll)) form (lazy, () when empty).
- cycle is lazy ((lazy-seq (concat coll (cycle coll)))) so it no longer counts its
argument and terminates on a lazy/infinite input.
- fold's foldable-call catch uses :default, matching the rest of jolt-core and
also catching a raw host condition from a folding primitive.
- alts! rejects non-channel ports with a clear error (put specs / :default are
unsupported) instead of crashing inside ac-poll!.
- Misc: drop the unreachable second getCause clause; jolt-nth on a string raises
'nth "index out of bounds" like the vector branch; name the inline fixpoint cap;
bld-sh-capture rejoins lines with newlines; clarify a couple of comments.
map-ir-children single-sourced the child layout for rewrite passes; the read-only
analyses each re-enumerated ops by hand. Add a fold companion, reduce-ir-children,
and rebuild body-size, pure?, and body-closed? on it (each reduces to a leaf value
+ the special ops it actually needs). local-escapes? stays an explicit walk — its
default is conservatively true and it inspects node shape beyond child purity, so
folding an unhandled op over its children would be unsound for scalar replacement.
analyze-special inlined def (~35 lines) and set! while try/letfn/fn* were already
helpers. Pull both out and move field-head? above analyze-special so its set! arm
and analyze-list reach it without a forward reference — the file's "only analyze
is forward-declared" invariant holds again. Pure code motion.
The fold/inline/types passes and the jolt.passes façade were baked into neither
seed half and never invoked: compile-eval and build went analyze -> emit directly,
and `jolt build --opt` flipped an optimize flag that nothing consumed.
- Compile the passes into the image (emit-image manifest): fold, inline, types,
then the jolt.passes façade, after jolt.ir.
- compile-eval and build.ss now run jolt.passes/run-passes between analyze and
emit. Off the direct-link path it is a pure const-fold; `jolt build --opt`
turns on inline + flatten + scalar-replace + type inference (it sets
hc-optimize?, which inline-enabled? reads).
- The seed minter (emit-image) stays analyze -> emit, so the seed is built
un-optimized and the self-host fixpoint is unaffected.
build-smoke already exercised --opt; it now actually optimizes and still matches
the release binary's output. Corpus floor and the fixpoint are green.
Round 1 (correctness + dead code):
- Fix duplicate java.util.HashMap registration in host-static.ss: the alist
impl shadowed the hashtable ctor while leaving the hashtable methods bound,
so .keySet/.values/.remove/.clear crashed. Drop the alist version.
- Delete jolt-core/jolt/reader.clj: a 463-line dead duplicate reader, never
required or compiled (the live reader is host/chez/reader.ss) and drifted.
- Remove dead defs: ir/rt + :rt op + unused ir/op; the Janet branch in
clojure.edn/drain-reader; a shadowed first clojure.string/trim-newline;
io.ss jolt-char-array + the reader def-var (both shadowed by natives-array);
concurrency.ss jolt-future-done?*; compile-eval.ss jolt-analyze-emit.
Round 2 (perf + determinism):
- emit-quoted-map-value / quoted sets now emit sorted by emitted text instead
of host-hash order, which isn't stable across Chez versions (jolt-8479).
- jolt-into folds through a transient, so into/vec/mapv/filterv onto a vector
are O(n) instead of O(n^2).
- deps resolve-deps walks its queue with an index cursor (was subvec-per-pop).
- async channel and agent action queues use amortized-O(1) FIFOs; ArrayList is
backed by a growable vector (O(1) add/get) instead of a list.
Thread/yield was a no-op and Thread/interrupted always returned false. Now:
- yield calls libc sched_yield (resolved once via the process symbols), so a
spin loop relinquishes the CPU. Falls back to a zero-length park if the symbol
can't be resolved.
- each OS thread carries an interrupt flag (a box, thread-local). currentThread
returns a handle wrapping the calling thread's flag, so .interrupt from another
thread sets the target's flag. .isInterrupted reads without clearing; the static
Thread/interrupted reads and clears — JVM semantics.
Consolidates the Thread surface: currentThread + the instance methods live in
io.ss (where the handle and its classloader are built), the flag box + yield +
the interrupted static in host-static.ss. Unit cases cover yield, the read/clear
split, and a cross-thread interrupt over a future.
The apt chezscheme package ships petite+scheme only — no kernel dev files — so
the standalone-binary gate skipped on CI, leaving the whole jolt build pipeline
and the --opt inference passes uncovered on Linux. Build Chez v10.4.1 from
source (cached) to get libkernel.a + scheme.h, install the libs the kernel links
against, and set the Linux link flags. buildsmoke now runs for real in CI.
The standalone-binary build needs Chez's kernel dev files (libkernel.a,
scheme.h) and a C compiler. A distro chezscheme package ships neither, so the
gate failed on CI (apt installs petite+scheme only). Preflight for the csv dir
and cc and skip cleanly when they're missing — same pattern as certify skipping
without Clojure. Where the toolchain exists (dev machines), it runs as before;
the discovered csv dir is pinned via JOLT_CHEZ_CSV so the build uses exactly it.
