The targeted-specialization work (jolt-ffn) concluded that the constant-factor
gap vs JVM is structural, not per-form: three targeted passes (field-read,
inline cache, ctor descriptor-bake) all came back flat. mandelbrot (pure
compute) is ~15x off JVM and that's the floor — Janet bytecode VM + mark-sweep
GC + indirect calls.
This doc hands off the successor epic (jolt-5vsp): the foundational levers
(native codegen, GC-pressure reduction, deeper devirt+inline) and, importantly,
the spike to run first — localize the 15x floor by comparing jolt-compiled vs
hand-written-Janet vs JVM mandelbrot before committing to any big lever. Also
records what not to repeat.
Co-authored-by: Yogthos <yogthos@gmail.com>
A protocol method reads its fields through the generic guarded keyword lookup
because the method's `this` param is untyped. defrecord now hints `this` with
the record type, the per-form inference seeds ^Record-hinted params (the
:fn branch previously typed all params :any — only the whole-program path
seeded phints), and run-passes feeds the inference the record shapes. So a
hinted param's field reads bare-index instead of going through the :jolt/type
tag guard.
This needed a with-meta fix: (with-meta sym ..) returned a proto'd table, so
symbol? was false and the macro-attached hint broke fn destructuring. Symbols
now carry metadata in-place in their struct (matching how the reader attaches
^hint), keeping symbol? true, as in Clojure.
Modest on dispatch (~3-5%): the field read is a small fraction of a dispatch;
the machinery (record-tag + protocol lookup + wrapper) dominates, which is the
inline-cache target (jolt-ez5h). But it's a correctness fix and lets any
^Record-hinted code — not just methods — drop the field-read guard per-form,
not only under whole-program.
Co-authored-by: Yogthos <yogthos@gmail.com>
The ray tracer is compute-bound and the three existing benches only cover
alloc / megamorphic-dispatch / collections. Add three axes the epic needs to
judge itself holistically:
- mono-dispatch: monomorphic protocol dispatch. Its jolt/JVM ratio (~110x) is
*worse* than megamorphic (~76x) — the JVM inline-caches a runtime-monomorphic
call site to near-free while jolt does a full registry dispatch (devirt only
fires on statically-proven receivers). Points at the call-site inline cache.
- mandelbrot: pure float compute, no alloc/dispatch. The floor at ~15x — native
arith already gets close to the JVM.
- fib: recursion, call + integer-arith overhead.
run.sh gains JVM=1, which runs each bench on JVM Clojure too and prints the
jolt/JVM ratio. collections sized up now that the map is a HAMT (jolt-684u).
README documents the axes and the current scorecard.
Co-authored-by: Yogthos <yogthos@gmail.com>
run-tests.janet runs the same file set as `jpm test` across a pool of worker
processes (one `janet FILE` each, ev-based). The full gate goes from ~790s
serial to ~98s here (8x), and more on CI where the heavy files don't thrash on
swap. CI and the docs point at it; `jpm test` still works serially.
Three things dominated the wall:
- Nine integration tests cold-built a compile ctx (~8s each); switch them to
api/init-cached so they share the prebuilt image. The cache key already
fingerprints the ctx-shaping env vars, so the direct-link ones share one DL
image and the rest share the plain one.
- core-bench's main ran on every gate (~35s of benchmark loops that assert
nothing); gate it behind JOLT_BENCH=1.
- cli-test spawned `janet src/jolt/main.janet` ~20 times at ~8s cold each
(340s under parallel load, and it was the whole wall); prefer build/jolt
(~20ms baked ctx) when present, fall back to from-source for an unbuilt tree.
type-check-test stays on cold init: a snapshot-loaded ctx loses the success
checker's op/msg detail (jolt-vley). jolt-pria tracks caching from-source
startup generally, which would let cli-test drop the build/jolt preference.
Co-authored-by: Yogthos <yogthos@gmail.com>
* Bind *command-line-args* after the deps-image cache swap (jolt-4mui)
Under whole-program (deps-image cache active), `jolt -m NS ARG` dropped ARG:
run-main set *command-line-args* on the current ctx, but a cache HIT then
replaced ctx with the saved image (via `set ctx cached`), whose *command-line-
args* was whatever got baked when the image was saved. The stale binding won at
`(apply NS/-main *command-line-args*)`, so -main ran with the wrong (usually
default) args — silently, for any optimized -m program.
Move set-command-line-args to AFTER the cache swap so it binds on the final ctx.
Repro/regression in deps-cache-args-test.janet: first run builds the image
(arg "first"), second run (cache hit) must echo "second", not the baked "first".
* docs: RFC 0003 — phm is a HAMT, sorted colls a red-black tree
The transients RFC described phm as "bucket-based copy-on-write" and mused about
"if it ever becomes a HAMT" — it is one now (jolt-684u), and sorted maps/sets are
a red-black tree (jolt-0hbr). Update the deviation/future-work notes accordingly.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Sorted collections were a sorted VECTOR — insert-at = (into (conj (subvec es
0 i) x) (subvec es i)) is O(n) per assoc with a large constant, so building was
O(n^2): 2000 entries took 55.6s.
Replace the rep with a red-black tree (assoc/dissoc/get/contains O(log n)),
ported from the ClojureScript PersistentTreeMap (cljs.core: tree-map-add /
balance-left / balance-right / tree-map-append / balance-*-del). This tier (25)
loads before 30-macros so deftype isn't available; a node is a plain vector
[color k v left right] and cljs's BlackNode/RedNode methods become functions —
the algorithm is unchanged. A sorted-set stores elements as keys with a nil
value; its ops project the key.
The seed read the old :entries vector directly for equality/printing; route
those through a new :entries op that materializes ascending from the tree
(core_types/sorted-entries-arr + main.janet's printer).
2000 sorted-map assocs: 55.6s -> 0.98s (57x); now O(log n) (per-op cost flat
from n=2000 to 10000). Correctness in test/integration/sorted-rbtree-test.janet
(shuffled insert ordering, delete rebalancing, custom comparator, comparator
lookup, subseq, count); sorted specs + full gate green. (key/val on sorted
entries stays a pre-existing gap — entries are pvecs not host tuples; jolt-jk23.)
Co-authored-by: Yogthos <yogthos@gmail.com>
* Add benchmark suite for alloc/dispatch/collection workloads (jolt-1r86)
The ray tracer is float-compute-bound (devirt, alloc removal, type-proving all
measured flat on it), so it can't validate the optimization passes. Add a small
cross-language suite (AWFY + CLBG style, portable Clojure) isolating the axes it
misses:
binary-trees allocation / GC pressure (escaping short-lived records)
dispatch megamorphic protocol dispatch (~1M dispatches/s; WP can't devirt)
collections persistent map/vector churn
bench/run.sh runs them; bench/README.md maps each to the pass it exercises.
collections immediately surfaced jolt-684u: the persistent hash map is O(n) per
assoc (flat copy-on-write bucket array, not a HAMT) — n=4000 assocs take 50s.
Invisible to the ray tracer (no maps).
* Persistent hash map: HAMT instead of O(n) copy-on-write (jolt-684u)
The map was a flat bucket array whose assoc copied the whole array every insert
(O(n)/assoc, O(n^2) to build). Compounding it, small maps are Janet structs that
only promoted to phm for collection keys — never for size — so a scalar-key map
stayed an O(n)-copy struct forever. Building a 4000-entry map took ~50s.
Two fixes, following ClojureScript's design:
- phm.janet is now a HAMT (hash array mapped trie): BitmapIndexedNode /
ArrayNode / HashCollisionNode, 32-way, 5 hash bits per level, structural
sharing — assoc/dissoc/get are O(log32 n). Translated from cljs.core, adapted
to Janet's 32-bit bit-ops (the hash is carried unsigned, the level index is
extracted with arithmetic, and bits are tested with band against 1<<i since
brushift rejects negative bitmaps). The public phm-* API and the value shape
(:jolt/type :jolt/phm, :cnt) are unchanged; transients are a separate rep and
untouched.
- core_coll promotes a struct map to a phm past 8 entries (not only for
collection keys), mirroring cljs PersistentArrayMap -> PersistentHashMap, so
incremental building isn't O(n^2).
20000 raw assocs: 7.1s -> 0.105s. The collections benchmark: 16.7s -> 0.2s.
Correctness covered by test/unit/phm-hamt-test.janet (oracle vs a Janet table,
nil keys, dissoc, a real hash-collision pair, and a sub-linear-assoc guard);
full gate green.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
The ray tracer is float-compute-bound (devirt, alloc removal, type-proving all
measured flat on it), so it can't validate the optimization passes. Add a small
cross-language suite (AWFY + CLBG style, portable Clojure) isolating the axes it
misses:
binary-trees allocation / GC pressure (escaping short-lived records)
dispatch megamorphic protocol dispatch (~1M dispatches/s; WP can't devirt)
collections persistent map/vector churn
bench/run.sh runs them; bench/README.md maps each to the pass it exercises.
collections immediately surfaced jolt-684u: the persistent hash map is O(n) per
assoc (flat copy-on-write bucket array, not a HAMT) — n=4000 assocs take 50s.
Invisible to the ray tracer (no maps).
Co-authored-by: Yogthos <yogthos@gmail.com>
A ^Record param hint was applied only at the final re-emit (reinfer-def), not
during the inter-procedural fixpoint. So a hinted param with no callers stayed
:any while inference ran, and a field read off it (e.g. (:origin ^Ray r)) never
told a non-inlined callee that its arg is a Vec3 — the callee's params stayed
unproven and its field reads kept the dynamic guard.
Seed declared hints as a param-type floor in the fixpoint: phint-seed (passes/
types) resolves an arity's :phints to positional record types via the
record-shapes registry, and infer-unit! initializes each fn's fresh param slots
from them instead of nil. A fixed declared type can't poison the least-fixpoint
the way an early-iteration :any would, and a hinted param now propagates its
(and its field reads') types to its callees during inference.
Scope: this closes the hinted-propagation gap. It does NOT help the ray tracer,
which uses zero ^-hinted params (only hinted fields) — its remaining type gap is
unhinted record-param inference on recursive/non-inlined hot fns, and per the
jolt-15jq A/B it's allocation-bound regardless (jolt-8flj). Tracked on the bead.
Co-authored-by: Yogthos <yogthos@gmail.com>
Dependency resolution now lives in the `jolt` CLI itself instead of a separate
jolt-deps executable. `jolt` resolves a deps.edn into JOLT_PATH/JOLT_APP_PATHS
in-process and dispatches the deps subcommands:
jolt -M:alias [args] run the alias :main-opts
jolt -A:alias CMD run CMD with the alias paths
jolt run FILE resolve, then run FILE
jolt path | tasks | task NAME
A deps.edn in the working dir is auto-resolved for the runnable commands
(repl/-m/-e/nrepl-server/FILE), so e.g. `jolt -M:nrepl` (or plain
`jolt nrepl-server`) starts an nREPL with the project and its deps loaded.
The runtime core stays deps-agnostic — it only reads JOLT_PATH. The resolver
(deps.janet) is reached only from the CLI entry and loads jpm lazily, so a run
with no deps.edn never touches it and an app baked from its own jolt/api entry
never links it. resolve-deps-argv only resolves on an explicit deps command or
when a deps.edn is present; help/version never do.
jolt-deps stays as a thin deprecation shim that forwards to `jolt`, so existing
scripts keep working. Docs (README, CLAUDE.md, building-and-deps, tools-deps)
and the help text updated.
Co-authored-by: Yogthos <yogthos@gmail.com>
scalar-replace already folds non-escaping const-key map literals
((:k {:k a ..}) -> a, and drops a let-bound map that doesn't escape).
Extend the same fold to record constructors: a (->Rec a b c) is a
positional struct whose declared field order lives in the record-shapes
registry, so a field read on a non-escaping ctor folds to the matching
positional arg and the allocation disappears.
Direct form (:field (->Rec ..)) and the let-bound form both handled,
threaded through run-passes via a per-unit shape registry (new
jolt.host/record-shapes accessor). Soundness: ctor args must be pure
(duplicated/discarded like map vals), arg count must equal the field
count, and only declared-field reads fold — a record answers the virtual
:jolt/deftype key with its type tag and any other key with nil, neither
of which is a positional arg, so those keep the allocation. pure? now
treats a record ctor of pure args as pure, so nested records (a Ray
holding a Vec3) fold bottom-up.
Allocation-bound microbench (non-escaping record built + field-read in a
hot loop): 69.6s -> 2.4s, landing on the no-record arithmetic baseline.
The ray tracer is unchanged — its vec3 results escape (returned/stored
each op), so they genuinely allocate; that's a separate problem.
Co-authored-by: Yogthos <yogthos@gmail.com>
cap truncates a deep type's field VALUES to :any so the inter-procedural
fixpoint stays finite, but it rebuilt the struct via mk-struct and dropped the
record :type tag along the way. The tag is identity — independent of field
depth — so a record stored in a deep container (a Sphere in a world vector, a
material on a hit) degraded to a plain struct, and devirtualization (jolt-41m)
and record? folding silently stopped firing on it.
Preserve :type alongside :shape when capping. Verified: a protocol call on a
record read out of a vector now devirtualizes (the call node gets :devirt-type,
which needs the receiver's record type). Sound — the tag stays accurate; only
field values below the depth cap are truncated.
No measurable wall-clock change on its own (jolt's protocol dispatch is already
cheap), but it restores the record fast path / devirt / record?-folding on
records-in-containers, and unblocks downstream work that keys off record types.
Co-authored-by: Yogthos <yogthos@gmail.com>
A bundle is closed-world — everything it needs is inlined and nothing is
required afterward — so a user defn unreachable from the entry's reference
graph can be dropped. The bundler now computes reachability from main-ns/-main
plus every non-prunable form and drops dead defn/defn- by exact source span
(formatting and reader macros in the surviving code are untouched).
Conservative and sound: only plain defn/defn- are prunable; a defn is kept if
its bare or ns-qualified name appears in any kept form, the closure runs to a
fixpoint, and any use of resolve/ns-resolve/requiring-resolve/find-var/intern/
eval/load-string disables pruning entirely. A parse failure on any file also
falls back to verbatim bundling, so the command stays as robust as a plain
concatenation. defmethod/defrecord/extend bodies are non-prunable and scanned,
so a fn reached only via dynamic dispatch stays live.
New reader/parse-all-spans returns [form start end] byte offsets so the drop
is a verbatim slice, not a re-print.
30-fn library used by a 3-fn entry: bundle 1114 -> 437 bytes (61% smaller, 27
dead fns dropped), output byte-identical.
Co-authored-by: Yogthos <yogthos@gmail.com>
When the collection-type inference proves an argument's type, number?/
string?/keyword?/record?/nil?/some? fold to a compile-time boolean. A
const-fold now runs after inference so a folded predicate propagates and
collapses any if it gates to the taken branch.
Sound by construction: only a provable answer folds, and only when the
argument is side-effect-free (a const or local) so dropping its evaluation
is a no-op. Unknown types (:any/:truthy) and impure args keep the call.
vector?/set?/map? are left out — the :vec tag conflates a real vector with
a range/seq, so vector? could be wrong.
50M-iter loop, same shape isolated with a carry-only control: number? call+
branch 5080ms, predicate folded 1365ms — matching the 1417ms control floor,
so the 3.7x is entirely the eliminated call+branch.
Co-authored-by: Yogthos <yogthos@gmail.com>
* Reader: #() params survive syntax-quote (auto-gensym names)
#(...) named its synthesized params with bare gensyms, so a #() written inside a
syntax-quote had its params qualified to the current ns by sq-symbol — and a
qualified symbol isn't a valid fn param. hiccup's compiler emits
`(let [sb# ..] (iterate! #(.append sb# %) ..)), which broke with "Unable to
resolve symbol: ns/_NNNN".
Name the params with a trailing # (auto-gensym suffix, like Clojure's p1__N#) so
syntax-quote maps them consistently and leaves them unqualified. Harmless outside
a backtick (just a regular symbol name).
* interop: String/valueOf static + String is a CharSequence
Two interop gaps surfaced bringing up hiccup and malli:
- String/valueOf(Object): hiccup's compiler stringifies attribute values with
(String/valueOf (or arg "")). Added the static — "null" for nil, else core-str.
- (instance? CharSequence s) returned false for a string; String implements
CharSequence, and malli's :re validator gates on it before matching, so :re
schemas always failed. instance-check now answers true for strings.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Under JOLT_OPTIMIZE a -m program run inferred + specialized EVERY loaded
namespace, including every transitive dependency. On a dep-heavy app that's
prohibitive: malli-app cold-started in ~2m10s (hundreds of dep namespaces, each
run through the per-form inline + inference passes).
The closed world a whole-program pass reasons over is the APP, not its
libraries. jolt-deps now passes the project's own source roots (its deps.edn
:paths) to the runtime as JOLT_APP_PATHS. A namespace loaded from an app root
gets full optimization (and joins the one whole-program fixpoint); a dependency
namespace compiles at default cost — :inline? off for its load, so the per-form
optimize passes don't run over library code — staying direct-linked but
generically typed (the open-world default). With no app roots declared (a bare
program run, or jolt without jolt-deps) everything counts as app, so behavior is
unchanged.
malli-app JOLT_OPTIMIZE cold start: 2m10s -> 4.5s. Compute-heavy programs whose
hot code is their own namespaces (the typed ray tracer) are unaffected — their
code is app code and still fully optimized (9s/frame render). Applied at runtime
in main for the same baked-at-build-time reason as JOLT_PATH; added to the
ctx-image cache key. Help text corrected: optimization is opt-in, not default.
Co-authored-by: Yogthos <yogthos@gmail.com>
A deftype field tagged ^:unsynchronized-mutable / ^:volatile-mutable is set!-able,
but under direct-link immutable records are shape-rec tuples, so set! errored
("Can't set! field on non-deftype: tuple").
A deftype with any mutable field now opts out of the shape-rec layout and uses
the existing :jolt/deftype table form regardless of :shapes? — set! already
mutates that form and field reads route through the tagged-table path. Such a
type is also not registered as a shape, so the inference never emits a bare-index
read against the table. Immutable deftypes/records keep the fast shape-rec.
deftype extracts per-field mutability from the field metadata and passes it to
make-deftype-ctor, which picks the representation at ctor-build time.
Co-authored-by: Yogthos <yogthos@gmail.com>
* Don't direct-link a var redefined earlier in the same unit (jolt-wf4)
defrecord/deftype expands to (do (def R (make-deftype-ctor ...)) (def ->R R) ...),
so the ->R alias references R within one compiled unit. Under direct-link a var
ref embeds the cell's root as a compile-time constant, but on a redefine R's old
root is still in place when that unit compiles — the (def R new) sibling hasn't
run yet — so ->R sealed to the stale pre-redef ctor. (defrecord R [x a])
(defrecord R [a x]) (:a (->R 10 20)) read the old [x a] layout and returned 20.
Track the vars a unit (re)defines and force their later in-unit references to the
live indirect deref. The cell is registered only after its own init is emitted,
so a recursive self-reference inside the init still direct-links (it runs after
the def completes); only sibling references after the def go indirect.
* Emit Janet's `in` as a value so a user local can't shadow it (jolt-fjb1)
The back end emits `in` to deref var cells ((in cell :root)) and index
shape-recs. It emitted the bare symbol, so a user local named `in` shadowed
Janet's builtin in the surrounding scope and the generated cell-deref called the
user's value as a function — "<table> called with 2 arguments, possibly expected
1". malli's explainer binds [value in acc], so m/explain hit this on every
schema (m/validate was unaffected — its path doesn't bind `in`).
Embed `in`'s function value at the emit sites (as jolt-call/core-get already
are); a value in head position can't be shadowed. Fixes m/explain on malli
(loaded with JOLT_FEATURES=clj so its .cljc reader-conditionals resolve).
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
eval-form treated only a reader LIST (a Janet array) as a call; a runtime-built
list — a plist or lazy-seq from cons/concat/list or ~@ (list?/seq? true but
array? false) — fell through to self-eval. So (eval (cons '+ '(1 2))) returned
the list as data instead of 3, and a macro whose output contained such a subform
left it unevaluated. Add a plist?/lazy-seq? branch that coerces to the element
array via d-realize and dispatches through eval-list; an empty list self-evals.
The analyzer already punts these forms to the interpreter (analyze's :else ->
uncompilable -> interpreter fallback), so this one interpreter branch fixes the
correctness bug across the eval and macro-expansion paths; compiling them
directly (vs punting) would be a separate perf change. Verified: conformance
355/355, syntax-quote ~@ splice, list values unchanged.
A deftype with (Object (toString [_] s)) had its toString ignored: the generic
object-methods "toString" fired in dispatch-member before the record's own
method (the record isn't a tagged shim, so that guard passed), and str rendered
the #Type{...} data repr instead of routing through toString.
- dispatch-member: a record's own method (instance/reified/protocol) now wins
over the generic object-methods table — so .toString/.equals/.hashCode on a
record use the record's definitions; plain records still reach object-methods.
- str: add a late-bound record-tostring-cb (wired per-ctx by
install-print-method-cb!, mirroring print-method-cb) that str-render-one
consults for records — a deftype with a custom toString renders via it, plain
records keep the data repr. pr-str is unchanged.
Needed by hiccup's RawString. Adds deftype-tostring-spec (.toString + str +
concatenation + a regression guard that record-less-toString keeps its repr).
walk only handled vector?/map? and fell through :else for everything else, so
postwalk over a quoted list (a plist) never touched its elements —
postwalk-replace with symbol keys silently no-op'd, which broke
clojure.template/apply-template (found during reitit work). Add list? (rebuild as
a list) and seq? (map over it) branches after the vector/map ones so concrete
collections stay authoritative. Adds walk-spec covering list/seq walking plus a
vector/keywordize regression guard and the apply-template trigger.
The 813-line host_interop.janet (java.lang statics, java.time, the 458-line
java.io/util/net/sql/text install-io!, and collection interop all in one file)
becomes a 19-line aggregator over src/jolt/interop/:
- java_base.janet — java.lang statics (Math/Thread/System/Long) + the java.time
shim + the shared chr/pad2/formatter coercion helpers
- host_io.janet — java.io/util/net/sql/text shims; imports java_base for the
shared helpers it reuses (it constructs java.time values and
formats dates)
- collections.janet— the late-bound .iterator/.nth/.count/.seq interop hooks
host_interop.janet just loads the three and runs install!/install-io!/
install-collections! in order. chr/pad2/formatter become public (they cross the
java_base->host_io boundary now). The registry machinery (class-statics/
tagged-methods/register-*!) stays in the evaluator, which loads first — moving it
out touches the hot dot-dispatch and is left as a separate step.
Pure move, no behavior change. Also fixes a cache footgun the subdir exposed:
source-fingerprint walked src/jolt/ with a non-recursive (os/dir), so an edit to
an interop/ file would not have busted the ctx image cache (stale-image bug) —
made it recurse, keyed by repo-relative path.
Full gate green: host-interop-spec (130+ rows across every JDK area), ctx-image
cold/warm, conformance x3, suite >=4695/88, fixpoint.
init-cached (core image) and the deps-image (main.janet) each hand-wrote the same
fork->slurp->validate-ctx?->rewire load and the snapshot->tmp->atomic-rename save.
Extract load-ctx-image / save-ctx-image (in api.janet, beside snapshot/fork): the
two callers now differ only in the validity predicate they pass — none for the
core image (its source fingerprint is already in the path), a deps-manifest mtime
check for the deps image. The per-process print-method-cb rewiring an image
restore must replay is threaded as a callback so the helpers don't depend on core.
Kept in api.janet rather than a new ctx_image.janet module: Janet's `use` doesn't
transitively re-export, and snapshot/fork already live in api and are consumed via
(use ./api) by main and the test harnesses — a separate module would force every
caller to import it directly. (load-image/save-image collide with Janet builtins,
hence the ctx- prefix.)
Full gate green; ctx-image-test cold/warm + deps tests pass.
core-count/core-seq/core-conj each walked a chain of (and (table? x) (= :jolt/X
(get x :jolt/type))) predicates — re-fetching the type tag per predicate and, for
conj, a 14-deep nested if. Replace with a single (case (get coll :jolt/type) ...)
per op: the type is read once and the arm calls the concrete op directly. Host
values (tuple/array/nil) and tuple-based shape-recs carry no :jolt/type and stay
in a per-op fallback cond (shape-rec? kept before tuple? — shape-recs ARE tuples).
Factor the two map-conj paths (phm vs struct) into one conj-into-map parameterized
by the assoc fn.
Behavior-preserving: the :jolt/type tags are disjoint, so the case is an exact
re-expression of the predicate chain; the fallbacks reproduce the originals
(incl. count erroring on a raw host table, which only seq ever handled).
This is the perf-neutral realization of the planned collection vtable. A shared
:jolt/type->{ops} table was tried first and REGRESSED core-bench ~2-4% (table
lookup + indirect call on the hot path); the inline case instead runs ~1.7%
FASTER than main (4864ms vs ~4950ms baseline) since one type-get + a jump beats
the sequential predicate chain. Full gate green (suite >=4695/88, fixpoint).
analyze-try assoc'd :catch-sym/:catch-body/:finally nil-when-absent, so a try
with no catch (or no finally) carried a nil-valued key — which makes the node a
phm in jolt's map representation and forces the back end to densify it
(norm-node) before reading :op. That's the map-nil-representation trap Phase 2
already cleaned up for def/fn/arity nodes. Add those keys only when the clause is
present, matching the arity :rest discipline; a try node stays a fast struct.
Behavior-invisible: emit-try reads each key with a nil-safe (node :k) and gates
on it, so an absent key and a present-nil key are indistinguishable to every
consumer. Adds ir-try-shape-test asserting the node shape across all four
try/catch/finally combinations plus end-to-end eval.
Note on scope: the plan's "delete the defensive norm-node calls" is NOT done — it
can't be. {:op :const :val nil} (e.g. (def x nil)) and nil map keys are
inherently phm, so the emit-dispatch norm-node guards a real case, not a
present-or-absent artifact. This PR removes a source of gratuitous phm nodes
rather than the densification itself. Full gate green.
Six bottom-up IR rewrites (const-fold, inline-node, subst, flatten-lets,
subst-lookup, scalar-replace) each hand-listed every op's child positions —
~250 lines of identical "recurse children, rebuild" arms that had to be kept in
sync whenever an op was added. Extract one map-ir-children into ir.clj that
knows each op's child layout; each walk keeps only its genuine specials
(const-fold's invoke/if, inline-node's invoke, subst's local/let alpha-rename,
scalar-replace's invoke/let folds) and delegates the rest.
The combinator is total over the op set, so the walks are now total too: a
couple soundly gain coverage they previously skipped (const-fold now folds
inside :try; subst-lookup now recurses :def inits, which fixes a latent dangling
ref where a dropped const-key-map binding was referenced inside a def). These
are sound — all six are result-preserving optimizations — and 3-mode conformance
+ fixpoint confirm identical program behavior.
map-ir-children is shape-preserving for :try (recurses :catch-body/:finally only
when present, never assoc's nil) so it can't turn a struct node into a phm.
Written with cond/get only, matching the passes' tier, so no new load-order dep.
Predicates (body-closed?/pure?/local-escapes?), the type-threading infer, and the
Janet backend emit stay as-is: their conservative :else defaults / [type node]
threading / host language don't fit a node-rebuilding combinator.
Adds ir-passes-test coverage for folding reaching fn/loop/try bodies. Full gate
green (conformance x3, suite >=4695/88, fixpoint stage1==2==3, inline-sra + devirt).
eval-dot copy-pasted its entire dispatch chain across the (. obj method args...)
and (. obj member) forms — string/number/object/tagged-shim lookup duplicated,
hand-synced on every interop change. Extract one dispatch-member that takes the
evaluated args plus a has-args flag. The shared head (string/number/object/
tagged) is single-sourced; the genuinely divergent tails (call form: record →
native field → coll-interop(args); bare form: zero-arg coll-interop → field /
zero-arg method) stay branched on has-args. The guards that differed between the
arms (object-methods checks table? only; tagged dispatch checks table-or-struct;
bare-form tagged dispatch requires the member present) are preserved verbatim,
keyed off has-args, so behavior is identical.
Adds a "dot dispatch arms" spec locking the divergent cases: zero-arg vs
with-arg coll-interop, record/deftype zero-arg vs with-args methods, -field
access. Full gate green.
5d: document the seed↔overlay boundary and add a drift check. core fns split
across a Janet seed (core-X, registered in core-bindings) and a Clojure overlay;
five names (char?/sorted?/sorted-map?/sorted-set?/transduce) carry a defn in
both, with the overlay copy authoritative and the seed copy internal-only. The
into-vs-transduce home asymmetry was undocumented. Adds docs/seed-overlay-registry.md,
SEED-TWIN: comments at the five seed sites, and a build-time drift check
(test/unit/seed-overlay-registry-test.janet) that recomputes the twin set from
source and fails if it diverges or a twin leaks into core-bindings.
5e: rep↔API pointer comments in pv/plist/phm/phs/lazyseq (representation lives
here; Clojure-facing ops dispatch in core_coll/core_types) and back-pointers in
core_coll. No behavior change — comments, docs, one source-analysis test.
Full gate green (suite ≥4695 pass / ≥88 clean files), drift check passes.
main.janet held ~45 lines of env-knob policy (open-mode / direct-link / optimize
/ shapes / whole-program gates) that couldn't be unit-tested without the CLI, and
two disk-image caches (api/init-cached, main/deps-image) each hand-built a
POSITIONAL "%q|%q|..." key that silently misaligned if a ctx-shaping knob was
added in only one place.
config.janet now owns:
ctx-shaping-env-vars the canonical list of env vars that shape the built ctx
ctx-cache-key a labeled key (name=value) over a prefix + every shaping
var, so adding a knob updates BOTH cache keys at once and
can't positionally alias two different builds
resolve-run-mode [open-mode? main-entry?] -> the ctx env knob map
main shrinks to: compute open-mode?/main-entry? from argv, call resolve-run-mode,
install the knobs. Both deps-image-path (main) and image-cache-path (api) build
their keys via ctx-cache-key. New test/unit/config-test.janet locks in the
run-mode cases and asserts every ctx-shaping env var participates in the key.
Scope: this is 4a + the 4b cache-key footgun fix. The optional 4b cleanup
(folding the load/save image dance + aot marshal helpers into one ctx_image
module) is left for a follow-up — it's lower value and higher blast radius.
No behavior change (cache keys now key on a superset of env vars, so at worst a
one-time cold rebuild). Gate green: conformance 355x3, clojure-test-suite 4718
pass (>= 4695 baseline), config-test, full jpm test exit 0.
Two small structural dedups from the plan's Phase 3c.
reader.janet: read-list/read-vector/read-set each had a copy of the same
read-loop (skip whitespace, stop at close char, drop #_ discards, splice #?@).
The skip/splice logic had drifted between them once. Hoisted into one
read-delimited [s pos close err] -> [items end]; the three readers now just wrap
its result. read-map keeps its own loop (its key/value pairing needs a different
value-slot scan).
phm.janet: phm-bucket-find/contains?/assoc/dissoc and phm-get each open-coded the
same stride-2 key scan. Extracted bucket-index-of [bucket k] -> index|nil; all
five now share it.
No behavior change. Gate green: conformance 355x3, clojure-test-suite 4718 pass
(>= 4695 baseline), full jpm test exit 0.
types.janet held five concerns under one generic name. Split into sibling
modules along its existing section boundaries:
types_symbols characters + symbol helpers
types_var Var
types_ns Namespace
types_ctx Context (+ inst/uuid values)
types_protocols protocol/type registry + shape-records
types.janet is now a pure aggregator: it loads the clusters in dependency order
and re-exports their defs (import :prefix "" :export true), so every consumer
keeps its single (use ./types) unchanged.
Order-preserving (statically verified: zero backward references, zero
cross-cluster private helpers — the cleanest of the seed splits). No behavior
change.
Gate green: conformance 355x3, clojure-test-suite 4718 pass (>= 4695 baseline),
full jpm test exit 0.
evaluator.janet was a 2597-line file with a 680-line eval-list. Split into
cluster modules behind a re-export aggregator (same pattern as core):
eval_base forward vars, syntax-quote, ns-loading, registries, jolt-invoke
eval_resolve symbol/var resolution, params, destructuring, class lookup
eval_runtime protocols, multimethods, deftype/reify, install-stateful-fns!
eval_special the special forms (eval-list dispatch)
evaluator.janet stays the module every consumer imports: it loads the clusters
in dependency order and re-exports their defs (import :prefix "" :export true),
so the five (use ./evaluator) consumers are unchanged. It still owns the
eval-form entry that ties resolution + special forms + map/coll evaluation.
In eval_special, the giant eval-list match is exploded: each multi-line arm is
now a named (defn eval-<form> [ctx bindings form] ...) — eval-def, eval-fn*,
eval-let*, eval-loop*, eval-try, eval-set!, eval-dot, etc. — and eval-list is a
thin dispatch table over them. "where is try handled" is now `grep eval-try`.
Order-preserving (statically verified: no symbol used before its cluster loads;
zero backward refs). 27 helpers shared across clusters are now public so `use`
shares them. The two near-duplicate .method dot blocks are NOT merged here — that
is a behavior-sensitive dedup tracked separately (jolt-eos3); this PR is pure
moves + the mechanical eval-list explosion, no behavior change.
Gate green: conformance 355x3, clojure-test-suite 4718 pass (>= 4695 baseline),
full jpm test exit 0.
core.janet was a 3013-line grab-bag. Split into six order-preserving cluster
modules:
core_types vector helpers, predicates, math, comparison, equality
core_coll collections, transducers, seqs, HOFs, constructors
core_print string + pr-str/str rendering
core_io I/O, files, JDBC, compare, type
core_refs arrays, bit ops, coercions, hash, atoms/refs
core_extra additional clojure.core fns, transients, hashing
core.janet stays the module everyone imports: it loads the clusters in
dependency order and re-exports each one's defs (import :prefix "" :export true),
so every consumer keeps its single (use ./core) with no change. The aggregator
also owns core-bindings and init-core!, which reference fns from every cluster.
The split preserves definition order exactly (verified: no symbol is used in a
cluster that loads before its definition), so seed load-order semantics are
unchanged. Three private helpers used across clusters (map-entries-of,
map-assoc1, str-render-one) are now public so `use` shares them; the five
forward vars (canon-key/jolt-equal?/pr-render/core-compare/print-method-cb) each
stay within their owning cluster.
No behavior change. Gate green: conformance 355x3, clojure-test-suite 4718 pass
(>= 4695 baseline), full jpm test exit 0.
passes.clj was a 1486-line grab-bag mixing three weakly-coupled concerns. Split
along the clusters the review mapped (only run-passes + the dirty flag were
shared):
jolt.passes.fold const-fold + the shared scalar-const? predicate (base)
jolt.passes.inline inline + flatten-lets + scalar-replace
jolt.passes.types collection-type inference + success checker + driver API
jolt.passes façade: run-passes + :refer re-exports of the driver fns
the back end looks up by name
scalar-const? was used by both the inline pass and the inference walk, so it
moves to fold (the base layer) and both refer it. The check-mode state stays
private to jolt.passes.types behind a new run-inference fn; run-passes calls it.
build-compiler! loads the three in dependency order before the façade, mirroring
the existing jolt.ir -> jolt.analyzer bootstrap. No behavior change. Also fixed
the stale ns docstring that listed four passes and omitted the type system.
Gate green: conformance 355x3, clojure-test-suite 4718 pass (>= 4695 baseline),
full jpm test exit 0.
