General fixes shaken out running clojure.core.cache (66 -> 198 of its assertions):
- Map destructuring applied an :or default only for :keys/:strs/:syms, not a
direct {x :x} binding — so {x :x :or {x 9}} (and the & {…} kwargs form) ignored
the default. Apply it for the direct binding too.
- fn didn't implement :pre/:post: a leading conditions map was evaluated as a body
literal (so % was unbound and (.q %) blew up). Recognize it and assert pre
before the body, bind % to the result, assert post, return %.
- (.q inst) on a deftype field with no matching method reads the field, like the
JVM (was "No method q").
- A deftype implementing the clojure.lang collection interfaces now dispatches
dissoc (without), contains? (containsKey), peek/pop (IPersistentStack), and
keys/vals (via its Seqable seq) through its methods — they were field-only, so
core.cache's caches and data.priority-map didn't behave as maps.
JVM-certified corpus rows for each. make test + shakesmoke green.
Finishes core.match — its full test suite (115/115) now passes, including the
two patterns the earlier work left out:
- Regex-literal patterns. A #"…" now reads as a regex VALUE (Clojure parity: the
reader constructs the Pattern, so a macro receives a regex, not jolt's tagged
form), and the analyzer compiles a regex value to the same :regex IR leaf via
its source. emit-quoted handles a quoted regex; a regex value carries the
java.util.regex.Pattern host tag so extend-protocol/instance? dispatch on it.
- Primitive-array patterns. A ^Type hint's :tag is now the SYMBOL (e.g. `ints`),
matching the JVM, so core.match's array-tag lookup engages the array
specialization (alength/aget). jolt's :tag consumers already tolerate a symbol
(hc-cell-num-ret normalizes; tag->nkind/def-meta handle both).
Also: a library-conformance directive in CLAUDE.md, and the supported-libraries
list (docs + site) simplified to one-line entries — a listed library is assumed
to work fully, so no tallies or feature enumerations. core.match + transit-jolt
added to the list.
Seed change (reader/backend/30-macros) -> re-minted; the rest runtime. JVM-
certified corpus rows; the stale `symbol hint -> :tag` divergence is dropped from
the allowlist (jolt now matches the JVM). make test + shakesmoke green.
Running clojure.core.match (a macro-heavy library that builds its compiler out
of deftypes implementing clojure.lang interfaces) shook out a cluster of general
gaps. Its own suite goes from not-loading to 111/115 assertions.
- deftype/defrecord implementing a clojure.lang collection interface now drives
the core fns: Indexed -> nth, Counted -> count, Associative -> assoc, ILookup
-> get/valAt (non-field keys only, so a method's own field bindings don't
recurse), ISeq -> seq/first/rest, IPersistentCollection -> conj, IFn -> the
value is callable. A jrec is still a map of fields by default; the interface
method wins when declared.
- Multi-arity inline methods are grouped into one fn (a type with (nth [_ i]) and
(nth [_ i x]) kept only the last before). Built as data, not a nested
syntax-quote, so a `(= ~ocr ~l) method body keeps its unquotes.
- instance?/satisfies? recognize a protocol a type implements, including a MARKER
protocol with no methods (core.match's IPseudoPattern) — deftype/defrecord now
record protocol satisfaction even with zero methods. Added ILookup/Indexed/
Counted to the instance? taxonomy for the built-in collections.
- Syntax-quote: a fully-qualified class name (clojure.lang.ILookup) stays bare
instead of being namespace-qualified; (unquote x) is detected in a lazy seq
(a macro that builds its template with map, e.g. deftype's rewrite-set).
- clojure.set union/intersection/difference are variadic (& sets) + union 0-arity.
- java map view methods: keySet/values/entrySet/size/isEmpty.
- deprecated java.util.Date getters (getYear/getMonth/...) + the multi-arg
(Date. year-1900 month0 date hrs min) constructor.
Seed change (deftype/defrecord macros + clojure.set) -> re-minted; the rest are
runtime. 11 JVM-certified corpus rows; make test + shakesmoke green.
Four general gaps, shaken out by loading clojure.spec.alpha:
- Special forms were shadowable by a same-named macro. analyze-list
macroexpanded before checking special forms, so a ns that redefs def/and/or
(spec excludes them via :refer-clojure :exclude) made a bare def resolve to
the macro instead of the special form, breaking every defn after. Now a head
in the special-form set is never macroexpanded, matching the reference
macroexpand1 isSpecial check.
- reify dropped all but the last arity of a multi-arity protocol method (spec
reifies (specize* [s]) and (specize* [s _])). The macro keyed methods by name
and overwrote; now it groups arities into one multi-arity fn.
- reify instances did not implement IObj: with-meta threw and (instance?
clojure.lang.IObj r) was false. Every Clojure reify carries metadata. with-meta
now copies the reify to a fresh identity (shared method table) and keys its
meta; instance? IObj/IMeta is true for any reify. This was the registry bug —
spec's with-name returned nil for specs, so get-spec missed.
- (set! (. Class field) val) was rejected. spec toggles
clojure.lang.RT/checkSpecAsserts this way; the analyzer now lowers it to a
jolt.host/set-static-field! call over a mutable-statics table, and a plain
Class/field read consults that table.
Also: .name/.getName on a Namespace and .ns/.sym on a Var (spec's ns-qualify /
->sym). analyzer + reify are seed sources (re-minted). spec.alpha now does
valid?/conform/cat/keys/explain-str/check-asserts. tick.alpha.interval-test still
needs time-literals data readers (separate).
Duration (ISO PT.. toString, between, full arithmetic), Period (between with
borrow, P.. toString, normalized), full Month/DayOfWeek enums (named constants,
print as their name — fixes the Phase-1 raw-jhost print), Year, YearMonth
(2020-02 toString, leap, atDay/atEndOfMonth), ChronoUnit (between/getDuration)
and ChronoField. The temporal machinery on the Phase-1 types now works with
ChronoUnit/ChronoField: (.plus t n DAYS), (.until t1 t2 unit), (.get/.getLong
t field), (.with t field v), (.isSupported ..), (.truncatedTo ..).
Analyzer: (. Class method args) with a class target lowers to a static call
(Class/method args) instead of mis-dispatching as an instance call on the arg
— matches JVM; needed by cljc.java-time.year. Seed re-minted; selfhost holds.
The Phase-2 cljc.java-time namespaces load; tick.core advances to a Phase-3
zone gap. 10 corpus rows certified vs JVM. make test + shakesmoke green, 0 new
divergences, data.json stays 138/139.
Replace the 33-line pprint shim with a column-aware pretty printer and a
Common Lisp cl-format engine, ported from the ClojureScript implementation
(no STM — atom-backed fields) and adapted to JVM-Clojure interop. Provides
pprint/write/write-out/with-pprint-dispatch/formatter-out/cl-format and
simple/code dispatch.
core print routes column-aware into an active pretty-writer via a __write
hook (suppressed inside with-out-str captures); PrintWriter host class
forwards into the wrapped writer. Re-mint: pprint is baked into the seed.
Unblocks clojure.data.json/pprint (its pretty-printing test passes).
Directives ~R/~P/~C/~F/~E/~G/~$/~(~) not ported (unused by the targets).
make test + shakesmoke green, 0 new divergences, selfhost holds.
A slash-free dotted symbol with a Capitalized final segment (java.util.Map,
clojure.lang.Named, java.time.Instant) now self-evaluates to its name string
instead of resolving to nil — jolt models a class as its name, so a library
can extend a protocol to, or instance?-check, a host class jolt has no shim
for. hc-resolve-global classifies these as :class; the analyzer emits a const.
extends? now matches when either the query or the registered tag is a dotted
suffix of the other, so (extends? P java.util.Collection) finds the impl
extend registered under the canonical short tag.
Add DateTimeFormatter/ISO_INSTANT (UTC, trailing Z).
These unblock loading clojure.data.json, which dispatches JSONWriter on
java.util.Map/Collection/CharSequence/Instant and defaults a formatter to
ISO_INSTANT.
read-string/read now return real sets for #{...} literals (top-level and
nested) instead of the reader's {:jolt/type :jolt/set} form — the data
seams convert set forms to sets (recursing, preserving metadata and source
map-key order); clojure.edn already did this. The compiler keeps reading
via the raw reader, so set literals in code stay forms the analyzer lowers.
format %x now emits lowercase hex (Chez number->string is uppercase); %X
unchanged.
extend and extends? handle a nil target type (host tag "nil"), matching
extend-type — protocols can be extended to nil via the function form, not
just the macro.
Found porting transit/data.json and shaking out aero.
conj/assoc/dissoc/disj/pop/into and empty now thread the receiver's
metadata onto the result, matching Clojure (each op constructs a new
collection with meta() carried forward; coll.empty() is
EMPTY.withMeta(meta())). The metadata side-table is now weak so meta on
intermediate collections is reclaimed with them, and empty-list-t carries
an (unused) field so a metadata-bearing () is a distinct identity from the
shared singleton instead of leaking meta onto every ().
Unblocks metadata-driven walks (aero/integrant): (into (empty form) ...)
now preserves a vector/map/set's metadata, so a postwalk whose outer fn
reads (meta x) sees it.
The reader lowered ^meta on a vector/map/set literal to a runtime
(with-meta form meta) list, so read-string/edn of data with metadata
returned the form and lost the metadata. Attach it to the value instead,
as Clojure does; the analyzer re-emits (with-meta coll meta) for a
meta-carrying collection literal in code, so a literal still carries its
metadata at runtime and ^Type/^long arglist hints (consumed by
analyze-arity directly) are unaffected.
Also: pr honors *print-meta*, and clojure.walk/clojure.edn re-attach
metadata to the collections they rebuild (matches Clojure; a
metadata-driven config lib like aero relies on it).
types.clj held the inferencer, the success-type checker, and the driver in one
716-line namespace. Move the self-contained checker into jolt.passes.types.check:
the error-domain predicates (not-number?/not-seqable?/not-callable?), the op
tables, type-name, check-invoke, and the user-fn registry. These are pure over
inferred types and the run's env cells, with no inference, so a check-rule edit
can no longer perturb the inferencer.
The infer-coupled probes stay in types.clj — isolated-diag-count and
check-user-call re-run inference, so moving them would make check depend on the
inferencer and reintroduce the cycle. Verbatim move; new ns wired into
ei-compiler-ns-files; seed re-minted to the byte-fixpoint.
walk treated a record as a plain map (record? implies map?), rebuilding it via
(into (empty form) ...) which yields a bare map and drops the type. Add a record
branch before the map branch that conj-es the walked entries back onto the
original, matching JVM clojure.walk's IRecord case. Type-dispatched walks need it
— integrant resolves #ig/ref by detecting its Ref record while postwalking the
config, so without this every ref silently fails to resolve.
clojure.walk is baked into the prelude, so the seed is re-minted. Corpus gains
five JVM-certified rows for record type/instance? survival through pre/postwalk.
infer's :invoke case was ~120 lines of cond arms hand-coding eight call
patterns, all destructured positionally with (nth r 0)/(nth r 1) on the
[type node'] tuples infer returns. Split each pattern into a named helper
(infer-pred-fold/-kw-lookup/-get-lookup/-reduce-hof/-seq-hof/-conj-into/-call)
behind an infer-invoke dispatcher that keeps the cond guards verbatim, and add
ty/nd accessors for the tuple so a silent transposition can't hide.
The accessors are applied only to genuine infer results (the new helpers and
infer-fn-seeded); the :map/:let/:loop arms interleave non-infer pairs
(binding tuples, accumulator pairs) with infer results, so those keep nth.
Pure restructuring — the guards, order, and bodies are unchanged; seed
re-minted to the byte-fixpoint, gate green, 0 new corpus divergences.
The inliner and the type inferencer each recognized (:k m) and (get m :k)
lookups with their own copy of the callee tests — the get-callee check was
duplicated verbatim across both. Lift kw-callee?/get-callee? into
jolt.passes.fold (alongside scalar-const?) and call them from both passes so
the head recognition can't drift.
Only the head predicates move. The deliberate differences stay: the inliner
still accepts any scalar key in the get-form (its scalar-replacement targets
can be string/number-keyed maps) while the inferencer requires keyword keys
for struct field typing, and the inferencer keeps its two arms separate so each
rebuilds args for its form. The backend's value-as-fn ifn-kind is left alone.
Under --direct-link a top-level def binds jv$<fqn> and app->app calls bound directly
to it. emit-invoke raw-applied that binding for any var callee, but only a fn-valued
def is a Scheme procedure: (def cfg {...}) then (cfg :a) emitted (jv$cfg :a), applying
a pmap -> "attempt to apply non-procedure". Maps/sets/keywords are invokable in Clojure
via jolt-invoke, which the indirect path used, so this only bit closed-world builds.
Track which direct-linked vars hold fn literals (direct-link-fns, registered at the def
site when the init op is :fn) and only raw-apply those. A non-fn callee falls through to
the jolt-invoke branch, which still uses the direct jv$ binding as the invoke target —
so the var-deref is still skipped, just not the dispatch.
Seed source: re-minted. Regression in directlink-test.ss (jolt-cw1o).
A loop var with an integer-literal init now types :long (fx ops) when every recur
arg in its slot is an increment-style step — the var unchanged, inc/dec, or (+/-
var <int-literal>). So (loop [i 0] (recur (inc i))) gets fx1+/fx<? without a hint,
matching how Clojure treats a primitive-long loop counter.
Soundness: only increment steps qualify. A multiplicative or large-growth
accumulator like (recur (* acc 2)) is never seeded, so it stays generic and keeps
arbitrary precision — a bignum-producing loop (e.g. a factorial) is unaffected.
counter-step? gates this; the existing fixpoint demotes anything inconsistent.
test/chez/numeric-test.ss 44/44 (incl. a factorial loop staying bignum-exact while
its counter is fx); full make test green, 0 new corpus divergences.
jolt.passes.inline was fully written but dormant — it fetched bodies via the
inline-ir host hook, which was a stub returning nil. Wire it up: run-passes stashes
each inline-eligible defn (single fixed arity) as its form is optimized, and
inline-ir hands the body back at call sites under --opt.
The catch was the ^double/^long coercion: an inlined fn drops its param-entry and
return coercion, so (work 3 4) on a ^double fn would return 25 instead of 25.0. New
:coerce IR node carries the coercion inside the spliced body — the inline pass wraps
a hinted param's arg and the return in :coerce, the back end lowers it
(exact->inexact / jolt->fx), and jolt.passes.numeric reads its :kind. So an inlined
call matches the called one and the body's fl*/fx* fast path still fires.
Only under --opt (closed world); the seed mint and -e don't inline, so selfhost and
the corpus are unaffected. test/chez/inline-test.ss 12/12 (make inline); full make
test green, 0 new corpus divergences.
Bench (hot loop, body is a ^double helper call): direct-link 500ms -> --opt
(inlined) 184ms = 2.7x, by eliminating the call + coercion wrappers and letting Chez
fuse the fl-ops unboxed. ~26x over the default dispatched build.
A ^double/^long return hint on a fn's name now (a) coerces the fn's value on the
way out — exact->inexact / jolt->fx, like a JVM primitive return — and (b) types a
call to it, so an accumulator over the result specializes:
(defn ^double work [^double x ^double y] (+ (* x x) (* y y)))
(loop [acc 0.0] (recur (+ acc (work a b)))) ; (+ acc (work ..)) -> fl+
The analyzer pushes the name's numeric tag onto each arity (:ret-nhint) for the
back-end coercion, and resolve-global surfaces the callee's declared return
(:num-ret, read from var meta) onto the :var node so jolt.passes.numeric types the
call. defn carries the name hint through.
This unblocks the accumulator-over-fn-result pattern that round 2 had to demote.
The win is bounded by call overhead in an open/dispatched build (~1.15x on a hot
loop whose body is a helper call); it compounds with direct-linking and, later,
inlining. A numeric return hint is a contract, like ^long — redefining the var to
return another type in an open build breaks it.
Not yet: per-arity arglist return hints, (defn f (^double [x] ..)). Gate:
test/chez/numeric-test.ss 39/39; full make test green, 0 new corpus divergences.
loop-kinds only typed :double accumulators; a ^long-seeded loop var (e.g.
(loop [acc start] ...) with a ^long start) stayed generic even though it's sound
to fx-type — :long only ever comes from an explicit hint, and a ^long value is
already coerced to a fixnum at fn entry. Keep the init's kind (:double or :long)
through the fixpoint, demoting only on a recur-arg mismatch.
Integer-literal-init loop vars (a bare (loop [i 0] ...)) still stay generic by
design: :long is never seeded from a literal, so a bignum-producing loop keeps
arbitrary precision.
A loop binding whose init is double and whose every recur arg stays double (a
bounded monotone fixpoint) is typed :double, so its arithmetic — and the recur
args feeding it — emit fl-ops. Chez can then keep the accumulator unboxed in a
float register across the loop.
Integer loop vars stay untyped: a bare integer init never seeds :long (same rule
as round 1), so a bignum-producing loop keeps arbitrary precision rather than
overflowing a fixnum. recur-kinds walks only tail position (if/do-ret/let-body),
stopping at nested loop/fn so a loop sees only its own recur.
A/B on a loop-carried double accumulator: 735ms generic -> 500ms typed (1.47x),
closing the gap to the JVM from ~3.3x to ~2.2x. The integer counter stays generic,
which is most of the residual.
A ^double/^long param hint (or a float literal) now drives Chez flonum/fixnum
ops instead of generic arithmetic — JVM-style primitive hints, available in every
build and at -e (not gated on direct-linking or whole-program inference).
New pass jolt.passes.numeric: a local forward type-flow seeded from ^double/^long
fn-param hints (analyzer attaches :nhints per arity) and float literals,
propagated through let inits / arithmetic / if / do. It tags an arithmetic invoke
:num-kind :double|:long when every operand is that kind (an integer literal is a
wildcard, coerced to a flonum in a double op). The back end lowers a tagged node
to fl+/fl-/fl*/fl//fl<?/... or fx+/fx*/fx1+/fxquotient/... (unchecked-add etc.
join the fixnum path; == too). Runs last in run-passes, both branches.
Soundness: :long is seeded ONLY from an explicit ^long hint, never a bare integer
literal, so un-hinted integer code keeps jolt's arbitrary-precision numbers — no
fixnum-overflow surprise, no corpus divergence. :double comes from ^double hints
and float literals (flonum arithmetic is always flonum, matching the generic
result). A ^long hint is a promise the value is a fixnum: fx+ raises on overflow,
like a JVM fixed-width long.
Numeric-hinted params coerce at fn entry (exact->inexact / jolt->fx), the way the
JVM coerces a primitive parameter — so the body's fl*/fx* ops can rely on the
type even when a caller passes an exact int (e.g. Chez's (* 0 1.0) => exact 0).
Round 1 specializes hinted straight-line / fn-body arithmetic. fl-ops are ~4x
generic in a tight Chez loop, but realizing that on loop-carried accumulators
needs loop-var typing — round 2. Sound foundation, gated by test/chez/numeric-test.ss.
A release/optimized `jolt build` is a closed world: every app def is final, so
an app->app call can bind to the def's Scheme binding directly instead of going
through (jolt-invoke (var-deref ns name)).
The emitter gains a direct-link mode (off for the seed mint, runtime -e/repl, and
dev builds). With it on, a top-level app def also emits a binding jv$<ns>$<name>
that def-var! aliases; an app->app call or value-ref to a name already emitted in
the unit lowers to that binding, skipping both the var-table lookup and the
generic IFn dispatch. ^:dynamic/^:redef defs and nested defs (a defonce's inner
def) opt out and stay indirect. Off direct-link mode, emit-top-form is exactly
emit, so the seed and runtime eval are byte-unchanged (selfhost holds).
build.ss turns it on for release + optimized; the defined-set accumulates across
the dependency-ordered namespaces so a dep's defs are linkable by the time the
entry that calls them is emitted. App->core calls stay indirect for now (core is
the baked seed); that's a later stage.
~1.74x on a hot cross-namespace call loop (26.5s -> 15.2s).
jolt could call C (foreign-fn -> foreign-procedure) but C could not call back
into jolt, which GTK signals (and any callback-taking C API) require. Add the
inverse: jolt.ffi/foreign-callable wraps a jolt fn as a C-callable function
pointer, mirroring the foreign-fn pipeline.
A new jolt.ffi/__ccallable special form carries the fn as a child expression
(analyzed + walked by the passes; ir.clj gains an :ffi-callable arm in both
child walks) plus literal arg/ret type keywords. The back end lowers it to a
locked Chez foreign-callable and returns its entry-point address as a jolt
pointer; host/chez/ffi.ss registers the code object so the collector keeps it,
and free-callable unlocks it. :collect-safe emits the convention that
reactivates the thread on entry, for callbacks fired while it is parked in a
:blocking call (a GTK main loop).
Test: ffi-binding-test.ss sorts an int array through libc qsort with a jolt
comparator (C -> jolt -> C). Re-minted seed.
types.clj drove inference through ~14 module-level atoms; the infer walk was
non-reentrant and depended on hidden set-*! install order. Thread one immutable
env (mk-env) through infer instead: it snapshots the installed config
(rtenv/vtypes/record-shapes/protocol-methods/map-shapes?) and carries the
per-run flags and accumulator/guard cells (diags/calls/checking-set/diag-memo).
A fresh env per run makes the pass re-entrant — isolated-diag-count's probe now
runs under a sub-env with its own diags cell instead of save/restoring a shared
atom.
Only state whose lifecycle spans separate API calls stays module-level: a
config-box the set-*! API writes, the escapes/user-sig sweep registries, and a
bridge holding the last checking run's diags for take-diags!. record-type-from-
entry/field-type-from-tag now take the shapes map directly rather than reading a
global.
jolt-ogib.10. Behavior pinned by the new infer gate (23 cases) plus selfhost +
buildsmoke. Re-minted seed.
The lattice-split commit staged its seed before make remint ran, so image.ss
lagged the source. Commit the correct re-minted image (gensym renumbering only).
check-user-call rebuilt the all-:any env once per parameter (O(params^2)) and
re-inferred a callee body at every call site. Build the env once and memoize each
probe by [key i argtype] (and the baseline by [:base key]), cleared per form in
check-form. The global type-env is stable within a form's check and the probe's
calls/escapes side effects aren't read there, so a skipped repeat is observably
identical. (The inline-side re-walk the audit flagged is moot: hc-inline-ir is a
no-op on Chez, so try-inline never reaches body-size/body-closed?.)
- take-last / drop-last return seqs, not vectors: take-last wraps in seq; drop-last
is the JVM (map (fn [x _] x) coll (drop n coll)) form (lazy, () when empty).
- cycle is lazy ((lazy-seq (concat coll (cycle coll)))) so it no longer counts its
argument and terminates on a lazy/infinite input.
- fold's foldable-call catch uses :default, matching the rest of jolt-core and
also catching a raw host condition from a folding primitive.
- alts! rejects non-channel ports with a clear error (put specs / :default are
unsupported) instead of crashing inside ac-poll!.
- Misc: drop the unreachable second getCause clause; jolt-nth on a string raises
'nth "index out of bounds" like the vector branch; name the inline fixpoint cap;
bld-sh-capture rejoins lines with newlines; clarify a couple of comments.
map-ir-children single-sourced the child layout for rewrite passes; the read-only
analyses each re-enumerated ops by hand. Add a fold companion, reduce-ir-children,
and rebuild body-size, pure?, and body-closed? on it (each reduces to a leaf value
+ the special ops it actually needs). local-escapes? stays an explicit walk — its
default is conservatively true and it inspects node shape beyond child purity, so
folding an unhandled op over its children would be unsound for scalar replacement.
analyze-special inlined def (~35 lines) and set! while try/letfn/fn* were already
helpers. Pull both out and move field-head? above analyze-special so its set! arm
and analyze-list reach it without a forward reference — the file's "only analyze
is forward-declared" invariant holds again. Pure code motion.
The fold/inline/types passes and the jolt.passes façade were baked into neither
seed half and never invoked: compile-eval and build went analyze -> emit directly,
and `jolt build --opt` flipped an optimize flag that nothing consumed.
- Compile the passes into the image (emit-image manifest): fold, inline, types,
then the jolt.passes façade, after jolt.ir.
- compile-eval and build.ss now run jolt.passes/run-passes between analyze and
emit. Off the direct-link path it is a pure const-fold; `jolt build --opt`
turns on inline + flatten + scalar-replace + type inference (it sets
hc-optimize?, which inline-enabled? reads).
- The seed minter (emit-image) stays analyze -> emit, so the seed is built
un-optimized and the self-host fixpoint is unaffected.
build-smoke already exercised --opt; it now actually optimizes and still matches
the release binary's output. Corpus floor and the fixpoint are green.
Round 1 (correctness + dead code):
- Fix duplicate java.util.HashMap registration in host-static.ss: the alist
impl shadowed the hashtable ctor while leaving the hashtable methods bound,
so .keySet/.values/.remove/.clear crashed. Drop the alist version.
- Delete jolt-core/jolt/reader.clj: a 463-line dead duplicate reader, never
required or compiled (the live reader is host/chez/reader.ss) and drifted.
- Remove dead defs: ir/rt + :rt op + unused ir/op; the Janet branch in
clojure.edn/drain-reader; a shadowed first clojure.string/trim-newline;
io.ss jolt-char-array + the reader def-var (both shadowed by natives-array);
concurrency.ss jolt-future-done?*; compile-eval.ss jolt-analyze-emit.
Round 2 (perf + determinism):
- emit-quoted-map-value / quoted sets now emit sorted by emitted text instead
of host-hash order, which isn't stable across Chez versions (jolt-8479).
- jolt-into folds through a transient, so into/vec/mapv/filterv onto a vector
are O(n) instead of O(n^2).
- deps resolve-deps walks its queue with an index cursor (was subvec-per-pop).
- async channel and agent action queues use amortized-O(1) FIFOs; ArrayList is
backed by a growable vector (O(1) add/get) instead of a list.
Shake-out from the conformance-library sweep. Host-side fixes (runtime .ss,
no re-mint) plus one analyzer change (re-minted):
- Exception fidelity: ex-info and host-constructed throwables (RuntimeException.
etc.) now carry their JVM class, so (class e), instance? across the exception
hierarchy, .getMessage, and clojure.test thrown?/thrown-with-msg? all work.
- .getBytes returns a seqable/countable byte-array and honors UTF-16/UTF-32;
String. decodes them. ->bytevector accepts byte-arrays (Base64).
- Universal .getClass / .toString / .indexOf / .lastIndexOf on any value/seq.
- record? uses the host jrec? predicate (the old (get x :jolt/deftype) crashed
on a sorted-map by invoking its comparator).
- extend-protocol to abstract host types (clojure.lang.Fn/IFn/APersistentVector,
java.net.URI) dispatches.
- New host classes: clojure.lang.PersistentQueue, java.util.ArrayList,
java.net.URI, java.io.File / java.util.UUID ctors, Double/Float ctors+statics,
regex instance? Pattern, System/setProperty.
- *assert* / *print-readably* are real settable/bindable vars.
- (symbol "ns/name") splits the namespace at the last slash.
- letfn fn params desugar destructuring (analyzer; re-minted).
unit.edn gains exinfo/hostobj/queue/hostctor/destructure regression rows.
A library binding a blocking native call (accept/recv/connect/...) needs it
emitted __collect_safe so the thread deactivates for the call and doesn't pin
the stop-the-world collector. foreign-fn / defcfn take an optional trailing
:blocking; the backend emits (foreign-procedure __collect_safe ...). Needed for
the ring-janet-adapter socket-server port. ffi-binding-test asserts a thread
parked in a :blocking call doesn't block (collect).
A jolt library can now bind its own native dependencies and expose a Clojure API
over them — no jolt built-in required. This is the foundation for moving the
http-client / db / adapter functionality out of the host and into real libraries.
- jolt.ffi/foreign-fn (sugar: defcfn) is a compiler special form: a compile-time
-typed C signature lowers to a real Chez foreign-procedure (analyzer :ffi-fn ->
backend foreign-procedure), so calls are typed and marshaled, not eval'd.
- host/chez/ffi.ss provides the rest under jolt.ffi: load-library, alloc/free,
read/write/sizeof, ptr<->string, null/null?. Loaded after the loader snapshot
so a library's (require '[jolt.ffi]) still loads the macro side.
- Types: int/uint/long/ulong/int64/uint64/size_t/ssize_t/iptr/uptr/double/float/
pointer/string/void/uint8/char.
Validated end to end: a pure-Clojure file binds libc (getpid/strlen/abs) and
libsqlite3 (open/prepare/step/column/finalize over out-param pointers) and runs a
query. Gate test test/chez/ffi-binding-test.ss (make ffi); selfhost holds.
Shaking the ring-app example's real library stack out against jolt surfaced a
batch of divergences from JVM Clojure, the biggest being evaluation order.
backend_scheme: call and recur arguments were emitted as bare Scheme operands,
so Chez's unspecified (right-to-left) order won out. Clojure evaluates left to
right, which selmer's reader loop relies on: (recur (add-node ... rdr) (read-char
rdr)) consumed a char early and dropped the first chars of every {{tag}}. Bind
operands to fresh temps in a let* (only when two or more can have side effects,
so hot calls over locals/consts stay un-wrapped). emit-ordered already did this
for collection literals; generalize it.
host-contract: syntax-quote now resolves the alias part of a qualified symbol
(impl/foo -> clojure.tools.logging.impl/foo) instead of leaving it bare, which
limped along via short-name matching until two loaded namespaces (reitit.impl,
clojure.tools.logging.impl) shared the short name and it broke.
collections: key-hash masks with bitwise-and, not fxand — jolt-hash is set!-
decorated per type (records return their own hash) and Chez's equal-hash can be a
bignum, so a key's hash isn't always a fixnum.
seq: even?/odd? handle bignums (JVM accepts any integer; the fxand crashed).
records: Keyword/Symbol .sym/.getName/.toString (honeysql's :clj branch reads
(.sym k)); Throwable .getMessage/.toString over a Chez condition.
host-static: __register-class-ctor!/__register-class-statics! so a host shim
(reitit.trie-jolt) can mirror a Java class.
natives-str: String.intern returns the string.
sqlite: jdbc.core fetch/fetch-one kebab-case column keys (the jolt-lang/db
convention; created_at -> :created-at).
io: a relative io/file path resolves against JOLT_PWD (the user's cwd), not the
repo root the launcher cd'd to — matches JVM cwd semantics, so config.edn loads.
cli: render an uncaught jolt throw (ex-info message + ex-data, or a condition)
instead of Chez's opaque "non-condition value" dump.
Reader / loader:
- #?@ splicing reader conditionals now actually splice the matched collection's
items into the enclosing sequence; the splice flag was read but ignored, so a
binding vector like [a #?@(:clj [b (.foo b)])] lost its alignment.
- the file loader reads by position and skips a top-level form that reads as
nothing (a :cljs-only #?, a #_ discard, a trailing comment) instead of
treating it as EOF — which silently dropped the rest of a large .cljc file.
- jolt's reader feature set now includes :clj (was {:jolt :default}). jolt is a
Clojure/JVM-compatible host that emulates clojure.lang.* and java.* interop,
so it reads the :clj branch of a .cljc library, not :cljs. This also lets four
more reader-conditional corpus cases pass (floor 2726 -> 2730).
Backend:
- munge-name escapes ' (prime) -> _PRIME_; a Clojure symbol like f' otherwise
emitted a bare ' into Scheme, which is the quote reader macro and unbalanced
the output.
Host shims:
- clojure.java.io/writer (pass through a StringWriter, file-back a path) and a
readLine on the string reader, so line-seq over (io/reader …) works (markdown).
A better "unsupported destructuring pattern: <pat>" error message.
deftype fields tagged ^:unsynchronized-mutable / ^:volatile-mutable can now be
reassigned in place from a method, as on the JVM. A jrec stores fields as cons
cells, so a new jolt-set-field! mutates the pair with set-cdr!. The deftype macro
rewrites (set! mutable-field v) in a method body to (set! (.-field inst) v), and
the analyzer compiles a (set! (.-field obj) v) target to jolt-set-field! — so
both the rewritten symbol form and an explicit interop (set! (.-root this) v) go
through one path. Field reads remain a snapshot at method entry, which is correct
for the universal read-then-set pattern (a repeated set! of the same field in one
call would read the entry value).
Closes the set!-of-local SCI failures: SCI load 202 -> 205/218.
Found in a read/eval review: a local named like a special form wrongly took
over operator position. (let [if (fn ...)] (if true 1 2)) returned the fn, but
per spec section 3 (and the reference) special-form heads are not shadowable;
only macros are. Two fixes: drop the (not shadowed) guard on the special-form
branch of analyze-list (so an (if ...) head is always the special), and prefix
a local whose name is a Scheme keyword when emitting (so a value local legally
named if does not shadow the (if ...) the back end emits). Value-position
locals named if/or/case still work.
The analyzer checked special forms before expanding macros, the reverse of the
canonical read -> macroexpand -> analyze order (Clojure/CLJS analyze-seq). Move
macroexpansion to the front of analyze-list. Knock-on fixes:
- letfn was both a (broken) macro expanding to let* AND a primitive special
(analyze-letfn, proper letrec*). Macroexpand-first surfaced the macro, breaking
mutual recursion; remove the macro, keep letfn a primitive.
- defmacro is now compiled by the analyzer (a :set-var-style :defmacro node that
defs the expander fn via the fn macro — so destructuring arglists desugar — and
marks the var a macro), so a non-top-level (when … (defmacro …)) works. The
runtime spine's separate top-level defmacro interception is removed: one path.
SCI load 162 -> 202/218.
The analyzer punted set! as uncompilable. Add it as a special form: (set! sym
val) on a var emits jolt-var-set, which updates the innermost thread binding (or
the root when unbound), returning val. A local target (deftype mutable field) or
an interop (.-field) target stays uncompilable for now. Also defines
*warn-on-reflection* (false) so set! on it resolves. SCI load 186 -> 196/218.
Chez rejects duplicate lambda formals, so any (fn [_ _] ...) failed to compile
— including every macro expander, whose &form/&env slots both expand to _. The
analyzer now renames each earlier duplicate param to a fresh name (Clojure binds
the last occurrence, so the earlier ones are shadowed and unreferenceable). SCI
load 162 -> 186/218.
bigdec / 1.5M / 0.0M silently produced doubles. Add a jbigdec value type
{unscaled, scale} over Chez exact integers (host/chez/bigdec.ss): value =
unscaled * 10^-scale. An M-suffix literal reads to a :bigdec form that the back
end lowers to jolt-bigdec-from-string (same IR-leaf path as #inst/#uuid); bigdec
coerces a number/string. Equality is by value (1.0M = 1.00M true, 3M = 3 false),
str drops the M and pr keeps it, class is java.math.BigDecimal, decimal? is true.
Arithmetic contagion isn't modelled (out of scope). The old corpus cases passed
spuriously as doubles; they now exercise a genuine BigDecimal.
(def ^String tv ...) left (:tag (meta (var tv))) as the unresolved "String";
the JVM compiler resolves the hint to java.lang.String at def time. Add a
resolve-class-hint host seam (built from the existing class-token table) and
resolve a def's :tag through it in the analyzer. The reader path
(read-string "^String x") stays unresolved, matching the JVM (only the
compiler resolves). Closes ^Type-tag-on-var.
(map? *in*) was true because *in* was a plain map of read-line-fn/read-fn
closures; the JVM *in* is a java.io.Reader so map? is false. A defrecord
doesn't help (records are maps). Make the reader a reify over a new IReader
protocol — a non-map value — and route read/read-line/read+string/line-seq
through its -read-line/-read-form/-read+string methods instead of keyword
access. with-in-str's __string-reader and the stdin *in* both reify it.
Closes *in*-bound + *in*-is-bound.
A macro like (defmacro cur-ns [] `(str ~*ns*)) splices the live *ns* value
into its expansion, leaving an opaque jns object as a list element. The
analyzer had no way to carry a runtime value and threw uncompilable — the last
remaining corpus crash. Recognize a jns via the host contract (form-ns-value?)
and emit a :the-ns leaf that reconstructs it by name (intern-ns!) at the call
site, the same IR-leaf pattern as regex/inst/uuid. Closes unquote-*ns*-in-
template; corpus crash count -> 0.
A namespace fast path rather than a general constant pool: it's the only
embedded-value case in the corpus and the common real-world one (libs splice
~*ns*). A general pool can come later if other value types appear.
The . special form rejected a non-symbol member; a keyword member now lowers to
an invoke of the keyword on the target ((. {:value 41} :value) => 41, as on the
JVM). Added a form-keyword? contract seam. Corpus 2692->2693. Re-minted.
(.. x m ...) failed: the analyzer classified the .. head as a .method interop
call (method-head? matched any "."-prefixed name) and form-special?
(hc-interop-head?) also flagged it, so it never reached the macro check. Exclude
".." from both (the char after "." being "." means the threading macro, not
.method). Corpus 2690->2691. Re-minted.
jolt was all-flonum (one :number type, inherited from Janet whose only number
type is a double). The Chez runtime has a full numeric tower, so the zero-Janet
path now carries it = JVM Clojure semantics:
(/ 1 2) => 1/2 (exact Ratio, was 0.5)
(integer? 3) => true (integer? 3.0) => false (float? 3.0) => true
(ratio? (/ 1 2)) => true (= 3 3.0) => false (== 3 3.0) => true
(+ 1 2) => 3 (exact) (/ 1.0 2) => 0.5 (double)
jolt= was already exactness-aware (values.ss) and == is value-equality, so
=/== match the JVM split. The reader preserves exactness (integer literals exact,
a/b ratios exact rationals, decimals/exponents flonums); backend_scheme emit-const
renders exact ints/ratios and flonums faithfully; the value-position arithmetic,
count, int, compare, bit ops, parseLong, string .length/.indexOf, range,
timestamps, and array bytes return exact integers (= JVM int/long) instead of
coercing to flonum. double/parseDouble/clojure.math floor|ceil|signum stay double.
Only the zero-Janet path carries the tower (the Janet reader loses exactness into
a double before emit). The prelude/all-flonum path is unaffected for compiled code;
the runtime reader is shared, so a couple of all-flonum reader assertions become
value (==) assertions. ~16 numeric corpus cases now give the JVM tower value vs the
Janet-era :expected and are allowlisted as tower divergences (Chez == reference
JVM) pending the corpus flip to JVM (jolt-ecz0). No BigDecimal type (1M).
Re-minted. zero-janet 2682 (floor 2698->2682, the reclassified tower cases), 0 new
divergences; fixpoint 10/10, bootstrap 6/6, spine 35/35, cli 49/49; Janet gate 155
files 0 failed.
Two Chez reader bugs, both JVM-parity gaps:
inc'/+'/foo' (trailing apostrophe) were mis-read as a symbol followed by a
quote macro, because the reader treated ' as a terminator. In Clojure ' is a
NON-terminating macro char (constituent after the first char). Since the seed
is minted on Chez, (def inc' inc) became (def inc 'inc), clobbering inc's var
cell with its own symbol -- so (var-get (var inc)) returned the symbol, not the
fn. Drop ' from the token terminator set; a leading ' still quotes.
^bytes [b] / ^String [x y] return-type hints: the Chez reader lowered ^meta on
a collection to a (with-meta vec meta) form, but emitted a QUALIFIED
clojure.core/with-meta while the Janet reader emits a bare with-meta -- so the
fn/defn macros' unwrap logic (matching the bare head) slipped past it and choked
on a non-vector arglist. Emit bare with-meta to match Janet, and unwrap a
(with-meta <vec> _) arglist in analyze-fn as a backstop.
Re-minted the seed. zero-janet 2699, prelude 2652, Janet gate 155/0, fixpoint
10/10, bootstrap 6/6, all 0 new divergences.
The corpus certifier (test/conformance) flagged four cases where jolt's
hand-written :expected matched a real defect rather than Clojure. Fixed in the
jolt-core overlay, corrected the spec :expected, re-certified against JVM Clojure:
- ex-message: returns nil for a non-throwable (dropped the lenient string branch);
still returns the message for ex-info. (jolt-l8e8)
- munge: preserves the argument's type — a symbol munges to a symbol, not a string.
(jolt-hc35)
- print: (print nil) emits "nil", not "" (top-level nil guard; str yields "").
(jolt-pqio)
- bounded-count: uses the counted? fast path (full count), else counts up to n via
seq — was (min n (count coll)), wrong for counted colls. Added an uncounted-coll
spec case. (jolt-2507)
Removed the 4 :bug entries from known-divergences.edn (now certified), regenerated
corpus + profile, re-minted the Chez bootstrap seed (clojure.core changed). Gates:
Janet 155/0, JVM certify clean, both Chez corpus gates 2534 (floors raised),
bootstrap 6/6, fixpoint intact.
