A deftype with (Object (toString [_] s)) had its toString ignored: the generic
object-methods "toString" fired in dispatch-member before the record's own
method (the record isn't a tagged shim, so that guard passed), and str rendered
the #Type{...} data repr instead of routing through toString.
- dispatch-member: a record's own method (instance/reified/protocol) now wins
over the generic object-methods table — so .toString/.equals/.hashCode on a
record use the record's definitions; plain records still reach object-methods.
- str: add a late-bound record-tostring-cb (wired per-ctx by
install-print-method-cb!, mirroring print-method-cb) that str-render-one
consults for records — a deftype with a custom toString renders via it, plain
records keep the data repr. pr-str is unchanged.
Needed by hiccup's RawString. Adds deftype-tostring-spec (.toString + str +
concatenation + a regression guard that record-less-toString keeps its repr).
walk only handled vector?/map? and fell through :else for everything else, so
postwalk over a quoted list (a plist) never touched its elements —
postwalk-replace with symbol keys silently no-op'd, which broke
clojure.template/apply-template (found during reitit work). Add list? (rebuild as
a list) and seq? (map over it) branches after the vector/map ones so concrete
collections stay authoritative. Adds walk-spec covering list/seq walking plus a
vector/keywordize regression guard and the apply-template trigger.
The 813-line host_interop.janet (java.lang statics, java.time, the 458-line
java.io/util/net/sql/text install-io!, and collection interop all in one file)
becomes a 19-line aggregator over src/jolt/interop/:
- java_base.janet — java.lang statics (Math/Thread/System/Long) + the java.time
shim + the shared chr/pad2/formatter coercion helpers
- host_io.janet — java.io/util/net/sql/text shims; imports java_base for the
shared helpers it reuses (it constructs java.time values and
formats dates)
- collections.janet— the late-bound .iterator/.nth/.count/.seq interop hooks
host_interop.janet just loads the three and runs install!/install-io!/
install-collections! in order. chr/pad2/formatter become public (they cross the
java_base->host_io boundary now). The registry machinery (class-statics/
tagged-methods/register-*!) stays in the evaluator, which loads first — moving it
out touches the hot dot-dispatch and is left as a separate step.
Pure move, no behavior change. Also fixes a cache footgun the subdir exposed:
source-fingerprint walked src/jolt/ with a non-recursive (os/dir), so an edit to
an interop/ file would not have busted the ctx image cache (stale-image bug) —
made it recurse, keyed by repo-relative path.
Full gate green: host-interop-spec (130+ rows across every JDK area), ctx-image
cold/warm, conformance x3, suite >=4695/88, fixpoint.
init-cached (core image) and the deps-image (main.janet) each hand-wrote the same
fork->slurp->validate-ctx?->rewire load and the snapshot->tmp->atomic-rename save.
Extract load-ctx-image / save-ctx-image (in api.janet, beside snapshot/fork): the
two callers now differ only in the validity predicate they pass — none for the
core image (its source fingerprint is already in the path), a deps-manifest mtime
check for the deps image. The per-process print-method-cb rewiring an image
restore must replay is threaded as a callback so the helpers don't depend on core.
Kept in api.janet rather than a new ctx_image.janet module: Janet's `use` doesn't
transitively re-export, and snapshot/fork already live in api and are consumed via
(use ./api) by main and the test harnesses — a separate module would force every
caller to import it directly. (load-image/save-image collide with Janet builtins,
hence the ctx- prefix.)
Full gate green; ctx-image-test cold/warm + deps tests pass.
core-count/core-seq/core-conj each walked a chain of (and (table? x) (= :jolt/X
(get x :jolt/type))) predicates — re-fetching the type tag per predicate and, for
conj, a 14-deep nested if. Replace with a single (case (get coll :jolt/type) ...)
per op: the type is read once and the arm calls the concrete op directly. Host
values (tuple/array/nil) and tuple-based shape-recs carry no :jolt/type and stay
in a per-op fallback cond (shape-rec? kept before tuple? — shape-recs ARE tuples).
Factor the two map-conj paths (phm vs struct) into one conj-into-map parameterized
by the assoc fn.
Behavior-preserving: the :jolt/type tags are disjoint, so the case is an exact
re-expression of the predicate chain; the fallbacks reproduce the originals
(incl. count erroring on a raw host table, which only seq ever handled).
This is the perf-neutral realization of the planned collection vtable. A shared
:jolt/type->{ops} table was tried first and REGRESSED core-bench ~2-4% (table
lookup + indirect call on the hot path); the inline case instead runs ~1.7%
FASTER than main (4864ms vs ~4950ms baseline) since one type-get + a jump beats
the sequential predicate chain. Full gate green (suite >=4695/88, fixpoint).
analyze-try assoc'd :catch-sym/:catch-body/:finally nil-when-absent, so a try
with no catch (or no finally) carried a nil-valued key — which makes the node a
phm in jolt's map representation and forces the back end to densify it
(norm-node) before reading :op. That's the map-nil-representation trap Phase 2
already cleaned up for def/fn/arity nodes. Add those keys only when the clause is
present, matching the arity :rest discipline; a try node stays a fast struct.
Behavior-invisible: emit-try reads each key with a nil-safe (node :k) and gates
on it, so an absent key and a present-nil key are indistinguishable to every
consumer. Adds ir-try-shape-test asserting the node shape across all four
try/catch/finally combinations plus end-to-end eval.
Note on scope: the plan's "delete the defensive norm-node calls" is NOT done — it
can't be. {:op :const :val nil} (e.g. (def x nil)) and nil map keys are
inherently phm, so the emit-dispatch norm-node guards a real case, not a
present-or-absent artifact. This PR removes a source of gratuitous phm nodes
rather than the densification itself. Full gate green.
Six bottom-up IR rewrites (const-fold, inline-node, subst, flatten-lets,
subst-lookup, scalar-replace) each hand-listed every op's child positions —
~250 lines of identical "recurse children, rebuild" arms that had to be kept in
sync whenever an op was added. Extract one map-ir-children into ir.clj that
knows each op's child layout; each walk keeps only its genuine specials
(const-fold's invoke/if, inline-node's invoke, subst's local/let alpha-rename,
scalar-replace's invoke/let folds) and delegates the rest.
The combinator is total over the op set, so the walks are now total too: a
couple soundly gain coverage they previously skipped (const-fold now folds
inside :try; subst-lookup now recurses :def inits, which fixes a latent dangling
ref where a dropped const-key-map binding was referenced inside a def). These
are sound — all six are result-preserving optimizations — and 3-mode conformance
+ fixpoint confirm identical program behavior.
map-ir-children is shape-preserving for :try (recurses :catch-body/:finally only
when present, never assoc's nil) so it can't turn a struct node into a phm.
Written with cond/get only, matching the passes' tier, so no new load-order dep.
Predicates (body-closed?/pure?/local-escapes?), the type-threading infer, and the
Janet backend emit stay as-is: their conservative :else defaults / [type node]
threading / host language don't fit a node-rebuilding combinator.
Adds ir-passes-test coverage for folding reaching fn/loop/try bodies. Full gate
green (conformance x3, suite >=4695/88, fixpoint stage1==2==3, inline-sra + devirt).
eval-dot copy-pasted its entire dispatch chain across the (. obj method args...)
and (. obj member) forms — string/number/object/tagged-shim lookup duplicated,
hand-synced on every interop change. Extract one dispatch-member that takes the
evaluated args plus a has-args flag. The shared head (string/number/object/
tagged) is single-sourced; the genuinely divergent tails (call form: record →
native field → coll-interop(args); bare form: zero-arg coll-interop → field /
zero-arg method) stay branched on has-args. The guards that differed between the
arms (object-methods checks table? only; tagged dispatch checks table-or-struct;
bare-form tagged dispatch requires the member present) are preserved verbatim,
keyed off has-args, so behavior is identical.
Adds a "dot dispatch arms" spec locking the divergent cases: zero-arg vs
with-arg coll-interop, record/deftype zero-arg vs with-args methods, -field
access. Full gate green.
5d: document the seed↔overlay boundary and add a drift check. core fns split
across a Janet seed (core-X, registered in core-bindings) and a Clojure overlay;
five names (char?/sorted?/sorted-map?/sorted-set?/transduce) carry a defn in
both, with the overlay copy authoritative and the seed copy internal-only. The
into-vs-transduce home asymmetry was undocumented. Adds docs/seed-overlay-registry.md,
SEED-TWIN: comments at the five seed sites, and a build-time drift check
(test/unit/seed-overlay-registry-test.janet) that recomputes the twin set from
source and fails if it diverges or a twin leaks into core-bindings.
5e: rep↔API pointer comments in pv/plist/phm/phs/lazyseq (representation lives
here; Clojure-facing ops dispatch in core_coll/core_types) and back-pointers in
core_coll. No behavior change — comments, docs, one source-analysis test.
Full gate green (suite ≥4695 pass / ≥88 clean files), drift check passes.
main.janet held ~45 lines of env-knob policy (open-mode / direct-link / optimize
/ shapes / whole-program gates) that couldn't be unit-tested without the CLI, and
two disk-image caches (api/init-cached, main/deps-image) each hand-built a
POSITIONAL "%q|%q|..." key that silently misaligned if a ctx-shaping knob was
added in only one place.
config.janet now owns:
ctx-shaping-env-vars the canonical list of env vars that shape the built ctx
ctx-cache-key a labeled key (name=value) over a prefix + every shaping
var, so adding a knob updates BOTH cache keys at once and
can't positionally alias two different builds
resolve-run-mode [open-mode? main-entry?] -> the ctx env knob map
main shrinks to: compute open-mode?/main-entry? from argv, call resolve-run-mode,
install the knobs. Both deps-image-path (main) and image-cache-path (api) build
their keys via ctx-cache-key. New test/unit/config-test.janet locks in the
run-mode cases and asserts every ctx-shaping env var participates in the key.
Scope: this is 4a + the 4b cache-key footgun fix. The optional 4b cleanup
(folding the load/save image dance + aot marshal helpers into one ctx_image
module) is left for a follow-up — it's lower value and higher blast radius.
No behavior change (cache keys now key on a superset of env vars, so at worst a
one-time cold rebuild). Gate green: conformance 355x3, clojure-test-suite 4718
pass (>= 4695 baseline), config-test, full jpm test exit 0.
Two small structural dedups from the plan's Phase 3c.
reader.janet: read-list/read-vector/read-set each had a copy of the same
read-loop (skip whitespace, stop at close char, drop #_ discards, splice #?@).
The skip/splice logic had drifted between them once. Hoisted into one
read-delimited [s pos close err] -> [items end]; the three readers now just wrap
its result. read-map keeps its own loop (its key/value pairing needs a different
value-slot scan).
phm.janet: phm-bucket-find/contains?/assoc/dissoc and phm-get each open-coded the
same stride-2 key scan. Extracted bucket-index-of [bucket k] -> index|nil; all
five now share it.
No behavior change. Gate green: conformance 355x3, clojure-test-suite 4718 pass
(>= 4695 baseline), full jpm test exit 0.
types.janet held five concerns under one generic name. Split into sibling
modules along its existing section boundaries:
types_symbols characters + symbol helpers
types_var Var
types_ns Namespace
types_ctx Context (+ inst/uuid values)
types_protocols protocol/type registry + shape-records
types.janet is now a pure aggregator: it loads the clusters in dependency order
and re-exports their defs (import :prefix "" :export true), so every consumer
keeps its single (use ./types) unchanged.
Order-preserving (statically verified: zero backward references, zero
cross-cluster private helpers — the cleanest of the seed splits). No behavior
change.
Gate green: conformance 355x3, clojure-test-suite 4718 pass (>= 4695 baseline),
full jpm test exit 0.
evaluator.janet was a 2597-line file with a 680-line eval-list. Split into
cluster modules behind a re-export aggregator (same pattern as core):
eval_base forward vars, syntax-quote, ns-loading, registries, jolt-invoke
eval_resolve symbol/var resolution, params, destructuring, class lookup
eval_runtime protocols, multimethods, deftype/reify, install-stateful-fns!
eval_special the special forms (eval-list dispatch)
evaluator.janet stays the module every consumer imports: it loads the clusters
in dependency order and re-exports their defs (import :prefix "" :export true),
so the five (use ./evaluator) consumers are unchanged. It still owns the
eval-form entry that ties resolution + special forms + map/coll evaluation.
In eval_special, the giant eval-list match is exploded: each multi-line arm is
now a named (defn eval-<form> [ctx bindings form] ...) — eval-def, eval-fn*,
eval-let*, eval-loop*, eval-try, eval-set!, eval-dot, etc. — and eval-list is a
thin dispatch table over them. "where is try handled" is now `grep eval-try`.
Order-preserving (statically verified: no symbol used before its cluster loads;
zero backward refs). 27 helpers shared across clusters are now public so `use`
shares them. The two near-duplicate .method dot blocks are NOT merged here — that
is a behavior-sensitive dedup tracked separately (jolt-eos3); this PR is pure
moves + the mechanical eval-list explosion, no behavior change.
Gate green: conformance 355x3, clojure-test-suite 4718 pass (>= 4695 baseline),
full jpm test exit 0.
core.janet was a 3013-line grab-bag. Split into six order-preserving cluster
modules:
core_types vector helpers, predicates, math, comparison, equality
core_coll collections, transducers, seqs, HOFs, constructors
core_print string + pr-str/str rendering
core_io I/O, files, JDBC, compare, type
core_refs arrays, bit ops, coercions, hash, atoms/refs
core_extra additional clojure.core fns, transients, hashing
core.janet stays the module everyone imports: it loads the clusters in
dependency order and re-exports each one's defs (import :prefix "" :export true),
so every consumer keeps its single (use ./core) with no change. The aggregator
also owns core-bindings and init-core!, which reference fns from every cluster.
The split preserves definition order exactly (verified: no symbol is used in a
cluster that loads before its definition), so seed load-order semantics are
unchanged. Three private helpers used across clusters (map-entries-of,
map-assoc1, str-render-one) are now public so `use` shares them; the five
forward vars (canon-key/jolt-equal?/pr-render/core-compare/print-method-cb) each
stay within their owning cluster.
No behavior change. Gate green: conformance 355x3, clojure-test-suite 4718 pass
(>= 4695 baseline), full jpm test exit 0.
passes.clj was a 1486-line grab-bag mixing three weakly-coupled concerns. Split
along the clusters the review mapped (only run-passes + the dirty flag were
shared):
jolt.passes.fold const-fold + the shared scalar-const? predicate (base)
jolt.passes.inline inline + flatten-lets + scalar-replace
jolt.passes.types collection-type inference + success checker + driver API
jolt.passes façade: run-passes + :refer re-exports of the driver fns
the back end looks up by name
scalar-const? was used by both the inline pass and the inference walk, so it
moves to fold (the base layer) and both refer it. The check-mode state stays
private to jolt.passes.types behind a new run-inference fn; run-passes calls it.
build-compiler! loads the three in dependency order before the façade, mirroring
the existing jolt.ir -> jolt.analyzer bootstrap. No behavior change. Also fixed
the stale ns docstring that listed four passes and omitted the type system.
Gate green: conformance 355x3, clojure-test-suite 4718 pass (>= 4695 baseline),
full jpm test exit 0.
Completes the phm.janet decomposition (jolt-bvek): after lazyseq left, the set
follows. phm.janet is now purely the PersistentHashMap; phs.janet is the thin
set layer over it (members are keys -> true), and (use ./phm) for the builders.
Importers using set?/phs-* via (use ./phm) add (use ./phs); backend's emitted
set literal head changes phm/make-phs -> phs/make-phs. Behaviour unchanged
(sets verified interpreted + compiled; full gate green).
phm.janet held the PersistentHashMap, the PersistentHashSet, AND the LazySeq
primitives — a lazy sequence has nothing to do with hash maps; both were just
tagged tables, which is why they shared a file (jolt-bvek). An agent looking for
lazy-seq realization would never grep phm.janet.
Move the LazySeq section (lazy-seq?/make-lazy-seq/realize-ls/ls-first/ls-rest/
ls-rest-cached/ls-seq/ls-count/lazy-cons) to a new self-contained lazyseq.janet
(janet builtins only, no jolt deps). Importers that used the fns through
(use ./phm) add (use ./lazyseq); host_interop's one phm/lazy-seq? becomes
lazy-seq?. Behaviour unchanged (covered by test/unit/lazy-seq-test.janet + the
full gate). phs split is a follow-up.
Co-authored-by: Yogthos <yogthos@gmail.com>
Cheap-version consolidation of the host-interop sprawl (jolt-jx5l). The JVM
class/method shims were scattered across javatime.janet, evaluator.janet,
api.janet and core.janet; the recent hiccup/markdown/malli fixes landed ad hoc.
- Rename javatime.janet -> host_interop.janet. The name described ~20% of its
contents (java.io/util/net/sql/lang all lived there); it is now the one home
for host shims, and greppable.
- Move the four hardcoded static tables (Math/Thread/System/Long) out of the
evaluator and register them through the generic class-statics registry. The
special-case dispatch in resolve-sym is deleted — one mechanism, not two.
- Move the collection-interop wiring (set-coll-realizer!, set-coll-interop!,
malli's LazilyPersistentVector/PersistentArrayMap statics) from api.janet into
host_interop's install-collections!. api's 40 lines of wiring become a single
(import ./host_interop).
No behavior change. Full per-package src/jolt/interop/ split is the follow-up;
string/number/object method tables and core's File/JDBC ctors stay put for now
(they're coupled to the dot-dispatch / collection layer). Full gate green.
Co-authored-by: Yogthos <yogthos@gmail.com>
* Protocol/interop fixes to run metosin/malli
Bringing up malli (schema validation) surfaced a batch of protocol and host-interop
gaps. m/validate now works across the schema vocabulary (predicates, :map incl.
nested/optional, :vector, :tuple, :enum, :maybe, :and, bounded int/string).
- extend-type and reify now accept MULTIPLE protocols in one form (each bare
symbol switches the current protocol). reify records every protocol it
implements, so instance?/satisfies? recognise all of them.
- Protocol method params support destructuring: reify/extend-type/deftype/
defrecord emit (fn ...) (which desugars patterns) instead of raw fn*.
- instance? of a PROTOCOL works like satisfies? for reify/record instances,
matching short names across qualified/bare protocol references.
- @x reads as the qualified clojure.core/deref, so it still derefs where a ns
excludes and rebinds deref (malli does). Updated reader-test + the reader
spec/grammar (S11, deref rule).
- Java collection interop on jolt collections: .nth/.count/.valAt/.get/.seq/
.containsKey route to the clojure.core equivalent (1-arg and 0-arg paths).
- java.util.HashMap capacity/load-factor constructors + .putAll.
- A class used as a value resolves to its instances' type, so Pattern -> the
regex type (malli keys class-schemas by it).
- Shims for malli's load path: LazilyPersistentVector/createOwning and
PersistentArrayMap/createWithCheck statics.
m/explain not yet working (jolt-fjb1). Full gate green.
* satisfies? recognizes reify, consistent with instance?
A reify's protocol methods are instance-local, so they aren't in the global type
registry that type-satisfies? consults — satisfies? returned false for a reify
even when it implemented the protocol. Check the protocols the reify records on
itself (the same :jolt/protocols list instance? uses), matching short names like
instance? does. Covers single- and multi-protocol reify.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
* Add architecture refactor plan
Synthesizes a six-part architectural review into phased, gate-validated cleanup
work. Targets LLM-maintainability: one home per feature, no god-files, explicit
checked contracts, no copy-paste dispatch. No code changes yet — the plan only.
* Refactor phase 0: dead code + isolated bugs
Pure cleanup ahead of the structural phases (docs/architecture-refactor-plan.md).
No behavior change except the two bug fixes, which are covered by a regression row.
Dead code (all verified zero-reference or overridden):
- core-resolve / core-satisfies? / core-type->str seed stubs + bindings —
resolve and satisfies? are interned by install-stateful-fns! (the seed copies
were shadowed); type->str was an inert SCI stub with no callers.
- find defined twice in 20-coll.clj; the dead copy returned a plain vector
(wrong — the live def at :787 returns a real map-entry) with a comment that
contradicted it.
- mark-hint (passes.clj), phs-to-struct (phm), shape-vals / ns-imports-fn
(types) — unreferenced.
- redundant local pad2 in javatime (module-level one already in scope).
Bugs:
- File.toURL stored :url but every :jolt/url method reads :spec, so a URL from
(.toURL file) returned nil from all its methods. Now stores :spec (+ spec row).
- pl-rest had a no-op (if (plist? r) r r); collapsed to r.
- :map-shapes? was missing from the deps-image cache key — two runs differing
only in map-shapes could reuse each other's image.
Also dropped read-quote's unused pos param. Full gate green.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Bringing up yogthos/markdown-clj surfaced a batch of Clojure-conformance gaps:
- clojure.java.io/writer returned nil for a Writer/StringWriter (it only
handled paths); now passes a Writer or file handle through, like reader does.
- StringWriter had no :close field, so with-open errored closing it.
- java.io.Reader had no .readLine method (only the :read-line-fn used by
line-seq); markdown's main loop calls .readLine directly.
- Writer.write(int) wrote the int's digits instead of the char for that code.
StringBuilder.append(int) keeps Java semantics (the digits) — the two differ,
so the char-code path is local to the writer, not shared render-piece.
- drop-while over a string errored in array/slice; it now char-seqs the string
like take-while/remove already do.
- re-seq returned an empty seq instead of nil on no match, so
(if-let [m (re-seq ...)] ...) always took the truthy branch — an infinite loop
in markdown's thaw-string.
- The #() reader didn't scan % inside map {} or set #{} literals, so
#(identity {:text %}) compiled as a 0-arg fn.
re-seq-nil and the #() map/set scan are general bugs, not markdown-specific.
Co-authored-by: Yogthos <yogthos@gmail.com>
Some Clojure libraries loop with the Java Iterator protocol — e.g. hiccup's
iterate! does (let [it (.iterator coll)] (while (.hasNext it) (f (.next it)))).
jolt had no Iterator, so (.iterator coll) returned nil and the loop did nothing
(silently dropping content). Add an (.iterator coll) object-method that returns a
:jolt/iterator over any seqable, with hasNext/next; the collection is materialized
via core's realize-for-iteration (late-bound through the evaluator since core
loads after it, wired in api). Found while bringing up weavejester/hiccup.
Co-authored-by: Yogthos <yogthos@gmail.com>
Two things made a program run catastrophically slow to start (ring-app: 111s),
which is backwards — direct-linking should make startup FASTER (Clojure: smaller
classes, faster startup; Stalin: whole-program analysis is strictly ahead-of-
time, zero startup cost). We were doing the opposite: paying compile-time work at
every startup.
1. Decouple inference from direct-linking. Direct-linking (cheap compile-time
call resolution) was bundled with the whole inference/specialization pipeline
(inlining + scalar replacement + structural types + the per-ns re-emit
fixpoint) — the expensive part. Now a program run direct-links + uses records
by default (fast, faster calls) but the inference is opt-in via JOLT_OPTIMIZE
(or an explicit JOLT_DIRECT_LINK / the native build). ring-app: 111s -> 6s.
2. Compile dependencies once. The baked core loads in ~10ms, but a program still
re-compiled every dependency namespace (reitit, ring, selmer, honeysql, …)
from source on every run — the whole 6s. Snapshot the ctx after the require
chain to a per-project image (the same marshal/fork the core image uses),
keyed on version + entry + source roots + flags, with a per-file mtime
manifest so any edit invalidates it. First run compiles + caches; later runs
fork the image (~10ms) and skip compilation. ring-app: 6s -> ~1s warm,
competitive with the JVM. JOLT_NO_DEPS_CACHE disables.
Full gate green (all -m integration tests now exercise the cache); deps-image
invalidation verified (touching a source recompiles).
Under direct-linking a record is a Janet tuple (its shape-rec), and core-vector?
just delegated to jvec? which is true for any tuple — so (vector? a-record) and
(sequential? a-record) returned true. That broke map-destructuring of a record:
the destructure coerce treats a sequential source as & {:keys} kwargs and does
(apply hash-map x), so destructuring a record fed its entries to make-phm as a
flat kv-list and corrupted. Surfaced as reitit's router crashing on a wildcard
route ('expected integer key for tuple in range [0,5), got 5') whenever the new
direct-link default was on; minimal repro is (let [{:keys [a]} (->R ...)] ...).
Fix: core-vector? excludes shape-recs, matching Clojure (a record is not a
vector or sequential). jvec? is unchanged for internal representation dispatch.
Regression cases added to record-declared-shape-test.
Running a program is a closed world — every namespace is required, then it runs
to completion — so make it direct-link by default (inlining, record shapes, the
inference's specialization), and for a -m/-M entry auto-enable the whole-program
cross-namespace inference pass. A decomposed multi-namespace program was ~3.7x
slower than the same code in one namespace purely because per-namespace
inference can't see a caller in a not-yet-loaded namespace; this closes that for
the common case with no flags and no hints.
Interactive modes (repl, -e, nrepl-server) stay indirect/open — they have to let
you redefine vars, which direct-linking seals against. Opt-outs:
JOLT_NO_DIRECT_LINK forces the open path even for a program run (hot-reload,
runtime redefinition); JOLT_NO_WHOLE_PROGRAM keeps direct-linking but per-ns;
JOLT_DIRECT_LINK / JOLT_WHOLE_PROGRAM still force-on. Namespaces required inside
-main (after the batch pass) fall back to per-ns inference.
The success checker (RFC 0006) rides on the inference for free, but a casual
program run shouldn't spam type warnings just because it now direct-links, so its
default-on is suppressed when direct-linking was auto-enabled (:direct-link-auto?);
an explicit JOLT_DIRECT_LINK or JOLT_TYPE_CHECK still turns it on. whole-program-
test and devirt-test opt their per-ns baseline out of the new auto-default.
Docs: RFC 0005 gains 'Compilation modes and defaults' + 'Cross-namespace
inference'; RFC 0004 documents cross-ns/param hints; self-hosting-compiler and
--help updated. Full gate green.
A ^RecordType hint only resolved against the current namespace's ctor key, so a
hint naming a record defined in another namespace degraded to :any. That made a
decomposed multi-namespace program much slower than the monolith: per-namespace
inference can't see a record param's callers in other namespaces, and the
declared hint that could have typed it was dropped.
Resolution now works cross-namespace, for both record FIELD hints (defrecord)
and fn PARAM hints, in both spellings — ^Vec3 where the type is referred and
^v/Vec3 where the namespace is aliased:
- reader keeps a tag's namespace qualifier (^t/Ray -> "t/Ray", was "Ray").
- make-deftype-ctor-impl indexes each ctor closure by value; record-hint-ctor-key
resolves a hint name against the COMPILE ns (referred names live there; aliases
resolve through it) and maps the type var's root back to its home ctor key.
Using the ctor value, not the var's :ns, is what makes :refer work — :refer
re-interns a fresh var whose :ns is the referring ns.
- the analyzer captures record param hints as arity :phints [name ctor-key];
reinfer-def seeds those param types, so a record param is typed even with no
inferred caller — the open-world / cross-ns case.
Effect on the multi-namespace ray tracer: per-ns compile 30.4s -> 7.9s with
param hints, matching whole-program (8.1s) and the single-ns monolith (8.3s).
cross-ns-hints-test covers field + param hints, refer + as, and the reader tag.
direct-var? now treats a cfunction root the same as a function root, so a
call/ref to a native fn (clojure.math/sqrt et al.) embeds the value instead of
a per-call cell deref. This was the hot indirection in the ray tracer — sqrt
runs every bounce — and it applies in every direct-link build, not just
whole-program.
const-link? is new and whole-program-only: in a closed world every non-dynamic
var has a stable root, so embed it as a constant (quoted unless it's already
callable) rather than reading the cell each reference. Covers what direct-var?
can't — ^:redef vars (reloading is off under the flag), data defs, and record
type/ctor roots. Dynamic vars stay indirect; a nil (not-yet-defined) root stays
indirect and the whole-program re-emit picks it up once the root is in place.
Measured on the records ray tracer: hot-path indirect refs (sqrt + data vars)
gone; the only indirect refs left are cold defrecord self-references. whole-
program-test now also checks a ^:redef fn and a data def so the per-ns vs
whole-program comparison guards const-link soundness.
A record field can carry a type hint — ^Vec3 (a defined record type) or ^:num —
and the inference now resolves it so reading the field back yields that exact type
instead of :any. A Vec3 stored in a Ray field reads out as Vec3, so the vec ops on
field-read values prove their reads (bare-index). This is Stalin's per-slot type
sets, but DECLARED rather than inferred: the exact shape is known up front.
- deftype captures each field's :tag / :num metadata (was stripped) and passes it
to make-deftype-ctor; the ctor registers per-field tags, resolving a record-type
hint to its ctor-key (same-ns) so the inference can look it up directly.
- call-ret-type builds a record's struct type with field types resolved from the
hints, recursing into nested record types (depth-bounded for self/cyclic types).
Measured: a nested-record read loop (:r (:origin ray)) runs 1.3s with ^Vec3 hints
vs 7.1s without — 5.5x. This is the lever the ray tracer needed (vecs flow through
container fields); records without it read back as :any and stay unproven.
A protocol method call compiles to (protocol-dispatch proto method this rest) — a
runtime registry walk (type-tag -> proto -> method) on every call, ~19x a direct
call. When the inference proves the receiver (arg 0) is a known record type, the
call now resolves to a DIRECT method call at compile time, skipping the registry.
- defprotocol registers each method's var-key 'ns/method' -> [proto method] (a
ctx-capturing register-protocol-methods! emitted into the do-block); infer-unit!
feeds it to the inference via a box (like record-shapes).
- the record-ctor return type carries :type (the record tag) so the inference
knows the receiver type; the :else invoke case annotates a protocol call whose
arg0 has a known :type with :devirt-{type,proto,method}.
- emit-invoke resolves the impl via find-protocol-method at emit time and emits a
direct call to the embedded impl fn value. Unknown/polymorphic receivers (no
proven :type) fall back to the dispatch path unchanged.
Measured: removes the dispatch overhead (14.7s -> 9.3s on a 10M-call loop); the
remaining cost is the method body itself (non-inlined, unproven reads) — inlining
the resolved method is the follow-up (jolt-t6r) toward direct-call speed.
Sound under the closed-world assumption direct-linking already makes (the impl is
resolved + embedded at compile time). Adds devirt-test (subprocess: dispatched ==
devirtualized across polymorphic dispatch, unknown-receiver fallback, and
heterogeneous collections). Stalin's compile-call/callee-environment is the model.
JOLT_WHOLE_PROGRAM (requires direct-linking) defers the per-namespace inference
and runs ONE fixpoint over every user unit at once, so param types propagate
across namespace boundaries — a non-inlined fn's record params get proven from
its callers in another unit, which the per-ns pass can't see. Sound only under
the closed-world assumption (no later eval/redefinition) the flag asserts; slow,
memory-heavy builds are the documented trade-off (the reason it's opt-in).
infer-unit! now takes one ns-name OR a list; infer-program! gathers all recorded
user namespaces and runs the existing fixpoint over the union (re-emit was already
ns-agnostic — keyed by var-key, callee-first). The evaluator defers + records each
unit under the flag; run-main triggers infer-program! after all requires, before
-main. Off by default — per-ns behaviour unchanged.
Measured: a recursive (non-inlined) cross-ns record reader runs 1.66x faster
(8.9s -> 5.3s) — params proven -> bare-index reads. NOTE: small accessor fns are
INLINED cross-ns and records carry GLOBAL declared shapes, so most record reads
are already proven without this pass; the win is for non-inlined hot fns, and it's
the foundation for future whole-program work (devirtualization, unboxing).
Adds whole-program-test (subprocess soundness: per-ns and whole-program produce
identical results on a cross-ns record program).
