jolt.host/record-ctor-key and record-type? were stubs returning nil since the
rehost, so phint-of always produced nil — a ^Record param hint (^Sphere s) was
silently dead. The inference only ever typed a record param when its callers
passed a concrete ctor (whole-program), so a fn called with an untyped value (a
vector element, a value threaded through recursion) read its fields generically.
Resolve the tag against the record registry (records.ss): chez-find-ctor-key maps
a ^Type tag to the ctor-key "ns/->Name" the inference seeds with, preferring the
compile ns and falling back to any registered record of that simple name (cross-ns
/ imported). record-type? / record-ctor-key call it. Runtime .ss — no re-mint (the
analyzer reaches record-ctor-key through the host var).
With this, a ^Sphere/^Ray-hinted hit-sphere in the ray tracer types its params, so
its 48 generic jolt-get field reads + boxed arithmetic become bare-index reads +
fl-ops: ray tracer 38.4s -> 23.9s (~1.6x), make test / shakesmoke green, selfhost
holds, 0 new divergences. A wrong hint stays safe — jrec-field-at falls back to
jolt-get for a non-record.
run-fieldnum.ss: a ^V-hinted param (no inferable caller) bare-indexes + unboxes,
with no generic jolt-get left.
Co-authored-by: Yogthos <yogthos@gmail.com>
A record field tagged ^double now reads back as a flonum and feeds the numeric
pass, so hintless arithmetic over those fields lowers to fl-ops — the leaf-numeric
analog of the ^Vec3 nested-field hints. Combined with the whole-program :double
param inference, a vec3-dot over a ^double-fielded record unboxes end to end with
no per-fn hints.
records.ss: a ^double field tag passes through resolution, and the ctor (and a
mutable-field set!) coerce a ^double field to a flonum — JVM primitive-field
parity (jolt returned an exact 1, not 1.0, before), and what makes reading the
field back as :double sound for an fl-op.
types.clj: field-type-from-tag maps "double" -> :double, and a keyword/get lookup
whose result is :double annotates the node :num-read :double. numeric.clj reads
that annotation and classifies the field read as a :double operand, so the
enclosing arithmetic specializes — the read itself keeps its jrec-field-at/jolt-get
emit.
run-fieldnum.ss gate: ctor coercion (int field -> flonum), field-field arithmetic
emitting fl*/fl+, and an untagged field staying generic. make test / shakesmoke
green, selfhost holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
The devirtualized protocol call emitted find-protocol-method on the inferred
record tag, but a record can satisfy a protocol via an Object/host-tag default
rather than a direct impl — find-protocol-method on its own tag misses that,
while protocol-resolve walks to the default. So a record relying on
(extend-protocol P Object ...) resolved under ordinary dispatch but applied #f
under devirt and crashed. Closed-world opt builds only; the gate previously
covered just direct inline/extend-type impls so it shipped green.
Emit devirt-resolve, which tries the static tag and falls back to
protocol-resolve on a miss — same fast path, correct regardless of how the
record satisfies the protocol. Mirrors jrec-field-at falling back to jolt-get.
The receiver binds to one temp so it feeds the resolve and the application
without double-evaluating a side-effecting arg 0.
Also widen the whole-program fixpoint to :any on hitting the iteration cap: a
non-converged pre-fixpoint is more specific than the least fixpoint, so seeding
it would be unsound. Not reached in practice (~2 passes); a defensive floor.
run-devirt.ss gains an Object-default case. make test / shakesmoke green,
selfhost holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
When the inference types a keyword-lookup receiver as a record — it carries the
field-order :shape and :hint :struct from the whole-program fixpoint — the back
end reads the field by its declared slot via jrec-field-at instead of jolt-get.
That skips the jolt-get case-lambda, the dispatch fn, and the field-key
hashtable lookup, leaving a jrec? check + a static-index vector-ref.
jrec-field-at falls back to jolt-get when the receiver isn't the expected record
(a map downgraded by dissoc, or a value the inference mistyped), so it stays
correct if the static type is wrong. Only the no-default form takes the bare
path (a declared field is always present).
Sound only for non-nil records: a self-recursive param that can be nil (e.g.
binary-trees check-tree, whose untagged child is nil at leaves) types :any and
keeps jolt-get — the whole-program fixpoint demotes it. The target is non-nil
record params, like a Vec3 dot product (~5% there; boxed-flonum arithmetic
dominates the rest, a separate numeric lever).
run-fieldread.ss gate: emitted form uses jrec-field-at at the right slot and
matches jolt-get for each declared field; a non-field key and a default-arg form
keep the generic path. make test / shakesmoke green, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
Re-derive each app fn's param types from its call sites under --opt, so a
record type flows across fn boundaries: a ctor's return reaches a callee
param, and a typed vector's element reaches a HOF closure's param. The back
end can then bare-index field reads and devirtualize protocol calls at those
sites (it reads the resulting :hint/:devirt annotations; consuming them is
separate work).
This rebuilds the inter-procedural driver the Janet host had — the API
(infer-body/reinfer-def) survived the rehost but nothing drove it, and the
record-shapes/protocol-methods registries were empty stubs.
- records.ss: populate record-shapes (ctor key -> fields/tags/type, resolving
nested record field tags) and protocol-methods (method var -> [proto method])
registries at deftype/defprotocol load time; jolt.host accessors materialize
them.
- passes/types.clj: wp-infer! runs a closed-world fixpoint joining call-site
arg types into callee params; reinfer-def re-seeds each def at emit. Self-
recursive calls and fn-level recur are collected so a recursive fn's params
are constrained by its recursion, not just external callers — else a param
the recursion widens (e.g. binary-trees check-tree, whose untagged child can
be nil) would be unsoundly typed non-nil. A fn used in value position keeps
:any params (callers unknown). Megamorphic sites join to :any.
- build.ss: analyze all app forms and run the fixpoint before per-form emit.
- run-wp.ss: gate (cross-fn propagation, escape soundness, self-recursion).
make test / shakesmoke green, 0 new divergences, selfhost holds.
Co-authored-by: Yogthos <yogthos@gmail.com>
defprotocol emitted one variadic (fn [this & rest] (protocol-dispatch P m this
(list->cseq rest))) per method, so every protocol call — even a no-extra-arg one
like (area s) — consed a rest list, wrapped it in a cseq, var-deref'd
protocol-dispatch, and jolt-invoke'd it (consing again). On mono-dispatch that was
2.07GB of allocation, ~65% of the benchmark.
Emit one fixed-arity clause per declared arglist instead. The 1/2/3-param arities
call positional protocol-dispatch{1,2,3}, which resolve the impl (by record tag,
reify method, or host-tag extension — factored into protocol-resolve) and apply it
directly; no rest-list, no seq round-trip. The dispatchN entry points are in the
native-op table so the shim calls bind straight to the records.ss procedures
rather than var-deref. 4+ params fall back to the variadic protocol-dispatch.
mono-dispatch 1.5s/2.07GB -> 0.69s/280MB; dispatch 26x -> 12.2x, mono-dispatch
111x -> 51x vs JVM. 5 new corpus rows pin multi-arity methods, host-type args,
and protocol-method-as-value against JVM Clojure.
Co-authored-by: Yogthos <yogthos@gmail.com>
Records were a jrec holding an alist of (kw . val) conses: ~113B/node, built
fresh per construction, field reads a list scan. Replace that with a shared
per-type descriptor (tag + field keywords + an eq?-keyed keyword->index table)
plus a flat per-instance value vector and an extension map for any non-field
keys assoc'd on (jolt-nil when there are none). Construction now allocates one
vector instead of a cons chain and a field read is an index lookup. binary-trees
construction allocation drops 2.085GB -> 1.19GB.
That alone barely moved binary-trees wall-time: profiling showed the read loop,
not allocation, dominates, and the read loop's own allocation came from (nil? l)
lowering to (jolt-invoke (var-deref "clojure.core" "nil?") l), which conses its
args every call. Add nil?/some? to the backend native-op table so they inline to
jolt-nil?/jolt-some? (and drop the truthy wrapper, like the other predicates).
check-tree's read loop goes from 1.476GB allocated to zero; binary-trees 18.9x
-> 9.7x vs JVM. The remaining gap is the field-read dispatch chain (jolt-c3mw).
Two JVM divergences fixed along the way, both certified:
- dissoc of a declared field downgrades a record to a plain map (was kept as a
record); an extension key still drops cleanly.
- map->R keeps extension keys (was dropping anything outside the declared basis).
16 new corpus rows pin assoc/dissoc/count/keys/seq/=/hash/extension-field
behavior against JVM Clojure.
Co-authored-by: Yogthos <yogthos@gmail.com>
* Make the benchmark harness build optimized binaries on Chez
bench/run.sh was Janet-era: it invoked a 'jolt' binary and set
JOLT_DIRECT_LINK/JOLT_WHOLE_PROGRAM, none of which exist on Chez, where
'joltc run -m' runs fully unoptimized (direct-link and inline default off). So
the suite was measuring jolt's unoptimized path.
run.sh now compiles each benchmark to an optimized AOT binary (joltc build
--direct-link --opt) and times it against JVM Clojure on the same portable
source, auto-detecting the Chez kernel dev files like build-smoke.sh. Adds
bench/deps.edn so joltc resolves the namespaces, NO_JVM to skip the reference.
mandelbrot.clj dropped its jolt.png require so the JVM reference can run it; the
picture demo moved to mandelbrot_png.clj (jolt-only). README scorecard refreshed
with current Chez numbers and the two-regime read (compute ~8-10x substrate floor;
dispatch/alloc ~120-330x architectural gaps the passes don't touch). Stale
'jolt -m' header lines point at bench/run.sh.
* Emit direct self-calls for named-fn self-recursion
A self-recursive call to a named fn compiled to (jolt-invoke fib ...) instead of
a direct (fib ...): emit-invoke handled a :local callee only when it was NOT a
known proc, so a :local that IS in *known-procs* (the letrec-bound self-name) fell
through to the :else jolt-invoke branch. Now a :local known proc emits a direct
Scheme call — no jolt-invoke, no per-call arg-list consing; case-lambda handles
arity.
fib 30: 63.3ms -> 4.7ms (faster than JVM Clojure's 7.1ms; was 9x slower). The win
is on every self-recursive non-loop fn, including the compiler's own. No semantic
change — selfhost holds, make test green, shakesmoke/buildsmoke byte-identical.
Re-mint (backend is seed). Corpus rows pin self-recursion across fixed/multi/
variadic arities.
* Intern no-ns keywords without per-call allocation
(keyword #f name) built a fresh combined-key string (string-append) on every
call just to do the intern-table lookup — ~80 bytes of garbage per (:kw x), map
literal, keyword arg, etc. A no-ns keyword now interns in a table keyed by the
name string directly, so a lookup of an already-interned keyword is one
hashtable-ref with no allocation. The ns table keeps the combined key; both share
the keyword-t khash (equal-hash of the combined key) so hash values are unchanged.
Small time win on its own (the field-read dispatch dominates hot record code —
see jolt-unx4) but removes per-call keyword allocation everywhere. Runtime .ss,
no re-mint; identity/=/hash unchanged, make test green.
* Fast record field reads: single eq? scan, skip the get-arm walk
(:field rec) / (get rec :field) lowers to (jolt-get rec kw), which walked the
get-arm list to reach the jrec arm, then did jrec-has? + jrec-lookup — TWO linear
scans, each comparing keys through the generic jolt=2 equality dispatcher. Field
keys are interned keywords, so:
- jrec-key=? compares a keyword query by eq? (jolt=2 only for non-keyword keys),
- jrec-ref does ONE scan (vs has?+lookup) and runs a deftype's ILookup valAt only
when the field is genuinely absent (present-nil still returns nil, not default),
- jolt-get-dispatch checks jrec? first, skipping the get-arm walk for the hottest
get target. jrec-lookup/jrec-has? (used by =, contains?, etc.) get the fast
compare too.
binary-trees 135x->18.9x, dispatch 121x->26.4x, mono-dispatch 327x->108x vs JVM.
Runtime .ss (collections.ss + records.ss), no re-mint; make test + shakesmoke +
buildsmoke green, record get/assoc/keys/=/count semantics unchanged.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Running cognitect aws-api's pure test namespaces (signing/shapes/protocols/
util/retry/endpoints) surfaced general gaps:
- extend-protocol/extend-type accept a computed class type, e.g.
(Class/forName "[B") for the byte-array class — the byte-array idiom data.json
and aws-api use. The macro grouping handled only symbol/nil heads (it crashed on
a list type); type->name resolves a Class value via .getName; a byte-array
dispatches on the "[B" host tag.
- java.nio.ByteBuffer over a jolt byte-array (wrap/allocate/get/put/array/
remaining/position/limit/duplicate/flip), plus extend-protocol to it.
- java.util.Arrays (equals/copyOf/copyOfRange/fill) and java.util.Random
(nextBytes/nextInt/…).
- java.net.URI/create and clojure.lang.RT/baseLoader statics.
- clojure.core.async/promise-chan (deliver-once, peek-don't-pop).
- a failed java.time parse throws DateTimeParseException (typed), so
(catch DateTimeParseException …) matches it instead of leaking an untyped
condition.
The XML side lives in the jolt-lang/xml library (libxml2 over jolt.ffi); ByteBuffer
stays in core as a generic java.nio primitive.
Gate: make test green (corpus +6 JVM-certified rows, 0 NEW divergence; unit
553/553; SCI 211).
Further clojure.core.cache fixes (198 -> 257 of its assertions):
- delay: a throwing body re-ran on every force and never became realized?. Run it
once like Clojure's Delay — cache the exception, mark realized, re-throw the same
on each deref. Fixes value-fn memoization / cache-stampede protection.
- deftype/defrecord: a method name appearing in two protocols with different
arities (data.priority-map's seq is in IPersistentMap [this] AND Sorted
[this asc]) registered per-protocol and shadowed; merge clauses by name across
all protocols into one multi-arity fn.
- empty?/peek/pop (IPersistentStack) dispatch through a deftype's methods; (= a-
deftype other) uses its equiv method (so caches compare to their backing map);
seq handles a host iterator (iterator-seq over .iterator).
- pop of an empty PersistentQueue returns it, like the JVM (was an error).
JVM-certified corpus rows. make test + shakesmoke green.
General fixes shaken out running clojure.core.cache (66 -> 198 of its assertions):
- Map destructuring applied an :or default only for :keys/:strs/:syms, not a
direct {x :x} binding — so {x :x :or {x 9}} (and the & {…} kwargs form) ignored
the default. Apply it for the direct binding too.
- fn didn't implement :pre/:post: a leading conditions map was evaluated as a body
literal (so % was unbound and (.q %) blew up). Recognize it and assert pre
before the body, bind % to the result, assert post, return %.
- (.q inst) on a deftype field with no matching method reads the field, like the
JVM (was "No method q").
- A deftype implementing the clojure.lang collection interfaces now dispatches
dissoc (without), contains? (containsKey), peek/pop (IPersistentStack), and
keys/vals (via its Seqable seq) through its methods — they were field-only, so
core.cache's caches and data.priority-map didn't behave as maps.
JVM-certified corpus rows for each. make test + shakesmoke green.
Finishes core.match — its full test suite (115/115) now passes, including the
two patterns the earlier work left out:
- Regex-literal patterns. A #"…" now reads as a regex VALUE (Clojure parity: the
reader constructs the Pattern, so a macro receives a regex, not jolt's tagged
form), and the analyzer compiles a regex value to the same :regex IR leaf via
its source. emit-quoted handles a quoted regex; a regex value carries the
java.util.regex.Pattern host tag so extend-protocol/instance? dispatch on it.
- Primitive-array patterns. A ^Type hint's :tag is now the SYMBOL (e.g. `ints`),
matching the JVM, so core.match's array-tag lookup engages the array
specialization (alength/aget). jolt's :tag consumers already tolerate a symbol
(hc-cell-num-ret normalizes; tag->nkind/def-meta handle both).
Also: a library-conformance directive in CLAUDE.md, and the supported-libraries
list (docs + site) simplified to one-line entries — a listed library is assumed
to work fully, so no tallies or feature enumerations. core.match + transit-jolt
added to the list.
Seed change (reader/backend/30-macros) -> re-minted; the rest runtime. JVM-
certified corpus rows; the stale `symbol hint -> :tag` divergence is dropped from
the allowlist (jolt now matches the JVM). make test + shakesmoke green.
Running clojure.core.match (a macro-heavy library that builds its compiler out
of deftypes implementing clojure.lang interfaces) shook out a cluster of general
gaps. Its own suite goes from not-loading to 111/115 assertions.
- deftype/defrecord implementing a clojure.lang collection interface now drives
the core fns: Indexed -> nth, Counted -> count, Associative -> assoc, ILookup
-> get/valAt (non-field keys only, so a method's own field bindings don't
recurse), ISeq -> seq/first/rest, IPersistentCollection -> conj, IFn -> the
value is callable. A jrec is still a map of fields by default; the interface
method wins when declared.
- Multi-arity inline methods are grouped into one fn (a type with (nth [_ i]) and
(nth [_ i x]) kept only the last before). Built as data, not a nested
syntax-quote, so a `(= ~ocr ~l) method body keeps its unquotes.
- instance?/satisfies? recognize a protocol a type implements, including a MARKER
protocol with no methods (core.match's IPseudoPattern) — deftype/defrecord now
record protocol satisfaction even with zero methods. Added ILookup/Indexed/
Counted to the instance? taxonomy for the built-in collections.
- Syntax-quote: a fully-qualified class name (clojure.lang.ILookup) stays bare
instead of being namespace-qualified; (unquote x) is detected in a lazy seq
(a macro that builds its template with map, e.g. deftype's rewrite-set).
- clojure.set union/intersection/difference are variadic (& sets) + union 0-arity.
- java map view methods: keySet/values/entrySet/size/isEmpty.
- deprecated java.util.Date getters (getYear/getMonth/...) + the multi-arg
(Date. year-1900 month0 date hrs min) constructor.
Seed change (deftype/defrecord macros + clojure.set) -> re-minted; the rest are
runtime. 11 JVM-certified corpus rows; make test + shakesmoke green.
Shaking out tick's api and alpha.interval suites (api 353->359, interval
0->103 passing) cleared a set of general gaps:
- Named-zone DST. Zones resolved to a fixed representative offset, so
America/New_York in August read -05:00 not -04:00. Add US/EU DST rules
(compact transition-date math) and make instant<->zoned, the zone rules'
getOffset, and the zoned equality arm DST-aware.
- Nanosecond zoned/offset times. Instant is nanos but atZone/atOffset/
toInstant/withZoneSameInstant and Instant/parse went through epoch-ms,
truncating sub-ms. Route them through nanos.
- Locale month/day names. A formatter dropped its Locale; carry it and add
French names so MMM under Locale/FRENCH renders "mai".
- Callable records. A defrecord implementing clojure.lang.IFn (tick's
GeneralRelation) is now invokable: jolt-invoke dispatches to its inline
invoke method. Also give collections the Iterable host tag so a protocol
extended to Iterable matches vectors/seqs.
- Imported class short names. (:import [java.time ZonedDateTime]) then
(. ZonedDateTime parse s) resolved to nil; an otherwise-unresolved bare
Capitalized name that's a registered host class now resolves as a class.
- Data readers. A project's data_readers.{clj,cljc} is loaded into
*data-readers* (reader namespaces required eagerly); registered #tag
literals in source rewrite to (reader-fn 'form). clojure.core/read-string
now applies #inst/#uuid/#"regex" and *data-readers* like Clojure.
- Duration/between accepts zoned/offset date-times.
All runtime shims, no re-mint. docs/libraries.md: tick full pass + aero.
Four general gaps, shaken out by loading clojure.spec.alpha:
- Special forms were shadowable by a same-named macro. analyze-list
macroexpanded before checking special forms, so a ns that redefs def/and/or
(spec excludes them via :refer-clojure :exclude) made a bare def resolve to
the macro instead of the special form, breaking every defn after. Now a head
in the special-form set is never macroexpanded, matching the reference
macroexpand1 isSpecial check.
- reify dropped all but the last arity of a multi-arity protocol method (spec
reifies (specize* [s]) and (specize* [s _])). The macro keyed methods by name
and overwrote; now it groups arities into one multi-arity fn.
- reify instances did not implement IObj: with-meta threw and (instance?
clojure.lang.IObj r) was false. Every Clojure reify carries metadata. with-meta
now copies the reify to a fresh identity (shared method table) and keys its
meta; instance? IObj/IMeta is true for any reify. This was the registry bug —
spec's with-name returned nil for specs, so get-spec missed.
- (set! (. Class field) val) was rejected. spec toggles
clojure.lang.RT/checkSpecAsserts this way; the analyzer now lowers it to a
jolt.host/set-static-field! call over a mutable-statics table, and a plain
Class/field read consults that table.
Also: .name/.getName on a Namespace and .ns/.sym on a Var (spec's ns-qualify /
->sym). analyzer + reify are seed sources (re-minted). spec.alpha now does
valid?/conform/cat/keys/explain-str/check-asserts. tick.alpha.interval-test still
needs time-literals data readers (separate).
ZoneOffset/ZoneId (SHORT_IDS, fixed-offset + UTC + system; named zones via a
fixed-offset table), ZonedDateTime/OffsetDateTime/OffsetTime, Clock (fixed/
system, with now [clock] arity), and DateTimeFormatter integration (ofPattern
+ ISO_* constants, .format/.parse over the rich java.time values via the
inst-time.ss pattern engine). systemDefault resolves to UTC to keep the
#inst atZone/toInstant round-trip machine-tz-independent.
tick.core + tick.protocols + tick.locale-en-us load; tick's api_test runs
31 tests / 352 pass / 7 fail / 0 error. The 7 are host gaps: named-zone DST
(no tzdb), French locale month names (no locale DB), nanosecond Instant.
General fixes surfaced by tick: :ns/keys map destructuring ({:tick/keys [..]})
in 00-syntax.clj (re-minted), and extend-protocol to java.time classes
(records.ss host-type-set). 12 corpus rows certified vs JVM. make test +
shakesmoke green, selfhost holds, 0 new divergences, data.json stays 138/139.
Duration (ISO PT.. toString, between, full arithmetic), Period (between with
borrow, P.. toString, normalized), full Month/DayOfWeek enums (named constants,
print as their name — fixes the Phase-1 raw-jhost print), Year, YearMonth
(2020-02 toString, leap, atDay/atEndOfMonth), ChronoUnit (between/getDuration)
and ChronoField. The temporal machinery on the Phase-1 types now works with
ChronoUnit/ChronoField: (.plus t n DAYS), (.until t1 t2 unit), (.get/.getLong
t field), (.with t field v), (.isSupported ..), (.truncatedTo ..).
Analyzer: (. Class method args) with a class target lowers to a static call
(Class/method args) instead of mis-dispatching as an instance call on the arg
— matches JVM; needed by cljc.java-time.year. Seed re-minted; selfhost holds.
The Phase-2 cljc.java-time namespaces load; tick.core advances to a Phase-3
zone gap. 10 corpus rows certified vs JVM. make test + shakesmoke green, 0 new
divergences, data.json stays 138/139.
Core java.time value types as jolt host objects backed by the inst-time.ss
calendar engine (days-from-civil/civil-from-days/inst-fields/format-ms), in a
new host/chez/java-time.ss. tz-free reps: LocalDate=epoch-day,
LocalTime=nano-of-day, LocalDateTime=(epoch-day,nano-of-day); Instant reuses
the ms-based instant jhost (ms granularity; nano is a documented gap). Each
type registers statics (of/parse/now/MIN/MAX/...), instance methods
(plus/minus/with/get/isBefore/until/toString/...), =/hash, compare,
ISO-8601 print, instance?, and value-host-tags for protocol dispatch.
Reconciles the old ms-based local-date/local-dt stubs into the rich types
(LocalDateTime now prints ISO; .toLocalDate/.atZone paths preserved). The
four cljc.java-time namespaces (local-date/local-time/local-date-time/instant)
load. Deep temporal-field/unit machinery (range/get-long/with-field/until/
adjust-into) stubbed for Phase 2.
12 corpus rows certified vs JVM 1.12.5. make test + shakesmoke green, 0 new
divergences, data.json stays 138/139, selfhost holds.
java.util.Calendar (a UTC calendar over epoch-ms): getInstance, the int
field constants, set/get/setTime/getTime/getTimeInMillis. java.time pieces
for a zoned round-trip: DateTimeFormatter/ISO_ZONED_DATE_TIME, formatter
.withZone/.parse, LocalDateTime/parse (with formatter), Instant/from,
.toLocalDate/.atStartOfDay. java.sql.Date is now a distinct class (its own
host value + dispatch tags) so a protocol extended to both java.util.Date
and java.sql.Date routes a sql.Date to its own impl. All UTC-consistent.
data.json print-sql-date + print-time-supports-format pass (suite 137/139).
Shaking out clojure.data.json's own test suite (now 134/137):
- Reader.read(char[],off,len) bulk read + PushbackReader.unread(char[],off,len)
on the string/pushback reader jhosts; instance? java.io.PushbackReader/
Reader/StringReader (data.json re-wraps a reader unless it's already a
PushbackReader, so this is load-bearing for repeated reads).
- number protocol dispatch by actual type: a flonum is Double (not Long),
exact ratio is Ratio, exact integer is Long — value-host-tags split.
- Integer/toHexString|toOctalString|toBinaryString|toString; .isNaN/.isInfinite
as instance methods on numbers.
- EOFException ctor/class; .isArray on a class-name string.
- dispatch tags for the uuid/bigdec/inst host values so a protocol extended to
java.util.UUID / java.math.BigDecimal / java.util.Date / java.time.Instant
reaches its impl; canonical-host-tag strips java.math./java.time.
- instant/zoned/local time values compare = by epoch-ms (two parsed Instants).
- java.time.Instant/parse, java.sql.Date ctor + valueOf, TimeZone/getDefault,
DateTimeFormatter/ISO_INSTANT.
All runtime .ss (no re-mint). 9 corpus rows certified vs JVM; make test +
shakesmoke green, 0 new divergences.
clojure.test reported a host-condition crash with a blank message (ex-message is
nil for non-ex-info conditions); fall back to jolt.host/condition-message.
condition-message itself left a Chez format-template message (open-input-file's
'failed for ~a: ~(~a~)') unformatted; apply its irritants.
The printer's two entry points (jolt-pr-str in rt.ss, jolt-pr-readable in printing.ss)
get register-pr-str-arm! / register-pr-readable-arm!, plus register-pr-arm! for the
types whose str and readable forms match (bigdec/inst/uuid/tagged/record/ns/var). The
normalize arms (sorted, lazy-seq, queue) and the uri readable arm register per-printer.
Also folds in the hash (dyn-binding var-cell), class (io uri/uuid/file), and get
(transients) arms missed earlier.
natives-array's get stays a case-lambda wrapper on purpose: its 2-arg path errors on
an out-of-bounds index while the 3-arg path returns the default, an arity distinction
the (coll k d) registry collapses — left as-is to preserve behaviour.
Completes jolt-lmot: all six dispatchers (hash/class/get/=/pr-str/pr-readable) off the
set!-rebind chains. make test green, 0 new corpus divergences; pr-str/str of inst,
uuid, bigdec, sorted-map, record-with-lazyseq, queue all verified.
jolt-get (4 sites: host-table/natives-misc/inst-time/records) and jolt=2 (6 sites:
records/vars/inst-time/natives-misc/bigdec/host-table) move off the set!-rebind
chains to register-get-arm! / register-eq-arm!. get's case-lambda becomes a stable
2/3-arg entry over a 3-arg dispatch; the equality arm pred is (a b) since either arg
may carry the type, and the host-table sorted arm normalizes-then-re-dispatches.
Behaviour-preserving (runtime .ss): var/inst/bigdec/record/uuid equality, record!=map,
sorted-map=plain-map, and all the get cases verified; make test green, 0 new corpus
divergences. Four of six dispatchers done; the printer (pr-str/pr-readable) remains.
Replace the set!-capture-and-rebind chains extending jolt-hash (3 sites: inst-time/
host-table/records) and jolt-class (3 sites: bigdec/natives-queue/host-table) with a
register-hash-arm! / register-class-arm! registry (the pattern already used by
register-str-render!). The base dispatcher walks its arms — disjoint types, so order
is immaterial — then falls to the base cases. The entry is stable, so the per-site
(def-var! "clojure.core" "class" …) re-points are gone.
Behaviour-preserving (runtime .ss, no re-mint): jinst hash, bigdec/queue/record
class, record hash, and a sorted-map hashing as its plain map all verified; make test
green, 0 new corpus divergences. First two of six dispatchers; get/=/pr-str follow.
jolt-str-render-one and instance-check were each extended by a chain of
set!-wrapping closures spread across ~10 and ~5 host files, so the real
behavior of either was scattered and load-order-dependent. Give each a
registry the base file owns: converters.ss/records-interop.ss define the
registry plus a register-* helper, and each extending file registers one arm
instead of capturing %prev and set!-ing the global.
str-render arms are type-disjoint; instance-check arms run newest-first (the
old outermost-wins order) and may return 'pass to defer. The string-token ->
symbol normalization the natives-array arm did for every inner arm moves to
the dispatcher head; array tokens stay strings for that arm to decide.
jolt-ogib.14. Runtime-only shims, no re-mint.
records.ss mixed the record/protocol/reify model with JVM exception emulation.
Move the contiguous ex-info accessors, the exception supertype hierarchy,
instance-check, case-string, and the instance-check def-var into records-interop.ss,
loaded right after records.ss. Those are only called at runtime, so the relocated
forms resolve fine; records.ss is now the value model and protocol dispatch.
Runtime shim — no seed change.
- take-last / drop-last return seqs, not vectors: take-last wraps in seq; drop-last
is the JVM (map (fn [x _] x) coll (drop n coll)) form (lazy, () when empty).
- cycle is lazy ((lazy-seq (concat coll (cycle coll)))) so it no longer counts its
argument and terminates on a lazy/infinite input.
- fold's foldable-call catch uses :default, matching the rest of jolt-core and
also catching a raw host condition from a folding primitive.
- alts! rejects non-channel ports with a clear error (put specs / :default are
unsupported) instead of crashing inside ac-poll!.
- Misc: drop the unreachable second getCause clause; jolt-nth on a string raises
'nth "index out of bounds" like the vector branch; name the inline fixpoint cap;
bld-sh-capture rejoins lines with newlines; clarify a couple of comments.
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.
Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.
Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.
Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
A reify that doesn't implement a given protocol method now dispatches to that
protocol's extended impls over the reify's host tags (e.g. an Object/default
extension) instead of erroring 'No reified method'. This is malli's pattern: it
reifies some protocols and relies on RegexSchema's default for the rest. A method
with neither a reify impl nor a default still errors.
Surfaced running the DB libraries (migratus) on the jolt db library:
- java.sql.SQLException .getNextException / .getStackTrace / .printStackTrace on
jolt throwables (conditions + ex-info) return nil/empty, so a library walking
the exception chain doesn't crash.
- java.net.URL ctor + .getProtocol (file/http), alongside the existing url shim.
- A generic java.lang.ClassLoader: getSystemClassLoader / a thread's
contextClassLoader resolve a named resource against the source roots (the same
model as clojure.java.io/resource) — file: URL or nil. Thread/currentThread.
These are generic host capabilities, not DB-specific.
The jolt-lang/db next.jdbc surface now runs migratus far enough to connect, build
the migrations table, and discover migrations; migratus's remaining dependency is
java.nio.file (FileSystems/Path/PathMatcher glob), a JVM filesystem API kept out
of core.
Shake-out from the conformance-library sweep. Host-side fixes (runtime .ss,
no re-mint) plus one analyzer change (re-minted):
- Exception fidelity: ex-info and host-constructed throwables (RuntimeException.
etc.) now carry their JVM class, so (class e), instance? across the exception
hierarchy, .getMessage, and clojure.test thrown?/thrown-with-msg? all work.
- .getBytes returns a seqable/countable byte-array and honors UTF-16/UTF-32;
String. decodes them. ->bytevector accepts byte-arrays (Base64).
- Universal .getClass / .toString / .indexOf / .lastIndexOf on any value/seq.
- record? uses the host jrec? predicate (the old (get x :jolt/deftype) crashed
on a sorted-map by invoking its comparator).
- extend-protocol to abstract host types (clojure.lang.Fn/IFn/APersistentVector,
java.net.URI) dispatches.
- New host classes: clojure.lang.PersistentQueue, java.util.ArrayList,
java.net.URI, java.io.File / java.util.UUID ctors, Double/Float ctors+statics,
regex instance? Pattern, System/setProperty.
- *assert* / *print-readably* are real settable/bindable vars.
- (symbol "ns/name") splits the namespace at the last slash.
- letfn fn params desugar destructuring (analyzer; re-minted).
unit.edn gains exinfo/hostobj/queue/hostctor/destructure regression rows.
The line REPL was broken (read-line called nil — the __stdin-read-line host seam
the clojure.core *in* reader drives was never implemented on Chez) and didn't
load the project, so (require '[some.lib]) failed. Now:
- __stdin-read-line reads a line from stdin (get-line); read-line / read / the
REPL work.
- repl resolves the project first (deps on the roots, native libs loaded), so
libraries are available — same context a run gets.
- jolt.nrepl: a jolt-native nREPL server (bencode over a loopback jolt.ffi
socket) speaking the real protocol — clone / describe / eval / load-file /
close, with stdout capture, :ns-scoped eval (in-ns; binding *ns* doesn't drive
load-string resolution here), and real error text. 'joltc nrepl [port]' applies
the project then serves; writes .nrepl-port. Editors (CIDER/Calva/Cursive)
connect and develop live; project libraries load in the session.
- ex-message returns nil for raw Chez conditions, so jolt.host/condition-message
exposes the condition text; the REPL and nREPL surface it instead of an opaque
#<compound condition>.
Why native, not real nREPL: nrepl.server is welded to java.util.concurrent
executors, two compiled Java helper classes, a DynamicClassLoader, Compiler
internals and a JVMTI agent — not faithfully shimmable. The wire protocol, which
is what clients depend on, is small and implemented directly.
Runtime .ss + jolt-core, no re-mint. Full gate green.
Shaking the ring-app example's real library stack out against jolt surfaced a
batch of divergences from JVM Clojure, the biggest being evaluation order.
backend_scheme: call and recur arguments were emitted as bare Scheme operands,
so Chez's unspecified (right-to-left) order won out. Clojure evaluates left to
right, which selmer's reader loop relies on: (recur (add-node ... rdr) (read-char
rdr)) consumed a char early and dropped the first chars of every {{tag}}. Bind
operands to fresh temps in a let* (only when two or more can have side effects,
so hot calls over locals/consts stay un-wrapped). emit-ordered already did this
for collection literals; generalize it.
host-contract: syntax-quote now resolves the alias part of a qualified symbol
(impl/foo -> clojure.tools.logging.impl/foo) instead of leaving it bare, which
limped along via short-name matching until two loaded namespaces (reitit.impl,
clojure.tools.logging.impl) shared the short name and it broke.
collections: key-hash masks with bitwise-and, not fxand — jolt-hash is set!-
decorated per type (records return their own hash) and Chez's equal-hash can be a
bignum, so a key's hash isn't always a fixnum.
seq: even?/odd? handle bignums (JVM accepts any integer; the fxand crashed).
records: Keyword/Symbol .sym/.getName/.toString (honeysql's :clj branch reads
(.sym k)); Throwable .getMessage/.toString over a Chez condition.
host-static: __register-class-ctor!/__register-class-statics! so a host shim
(reitit.trie-jolt) can mirror a Java class.
natives-str: String.intern returns the string.
sqlite: jdbc.core fetch/fetch-one kebab-case column keys (the jolt-lang/db
convention; created_at -> :created-at).
io: a relative io/file path resolves against JOLT_PWD (the user's cwd), not the
repo root the launcher cd'd to — matches JVM cwd semantics, so config.edn loads.
cli: render an uncaught jolt throw (ex-info message + ex-data, or a condition)
instead of Chez's opaque "non-condition value" dump.
Gaps surfaced loading ring-core / hiccup / config / tools.logging on Chez:
- clojure.java.io/resource — resolve a named resource against the loader's
source roots (no classpath), returning a slurp-able File.
- (Object.) constructor — a fresh distinct value (lock / unique sentinel).
- *out* / *err* as dynamic vars over a port-writer, so (binding [*out* *err*] …)
and #'*out* compile and run (tools.logging, selmer).
- :refer :all now registers a refer-all relation, so an unqualified var from a
(require '[ns :refer :all]) resolves at compile time — including #'var.
- java.lang.Character interop ((.toString \+) etc.).
- StringReader accepts a char[] ((StringReader. (char-array s))), not just a
String.
- the \p{L} translation stops just below the UTF-16 surrogate gap (\x{D7FF})
instead of \x{10FFFF} — a range across the surrogates made irregex's char-set
construction call integer->char on a surrogate and crash.
A mutable Map keyed by jolt values (jolt-hash/=) with put/get/putAll/containsKey/
size/remove/keySet/values/entrySet — enough for libraries that build a fast
lookup table (malli's fast-registry doto's a HashMap then .get's it). The
"No method M for value" dot-dispatch error now includes the value, which makes
a wrong-type interop target far easier to pin down.
Four runtime/reader gaps that blocked real libraries (hiccup, commonmark):
- reader: a type hint on a code form (^String (to-str x)) was lowered to a
runtime (with-meta (to-str x) {:tag String}), mis-applying the hint to the
call's RESULT and throwing when it's a string/number. A :tag hint on an
evaluated form is compile-time only in Clojure — attach it to the form
instead. Collection literals (^:foo [1 2 3]) still get runtime metadata.
- deftype: register the ctor globally by simple class name (like StringBuilder)
so (Name. ...) interop resolves ns-agnostically. host-new resolved the ctor
against the runtime current ns, which is the caller's, not the defining ns,
once a deftype is used across files.
- protocol dispatch: canonical-host-tag now strips the clojure.lang. prefix too
(clojure.lang.Keyword -> Keyword), and keywords/symbols carry the Named tag,
numbers a Ratio tag. An (extend-protocol P clojure.lang.Keyword ...) was
missing the dispatch, so e.g. hiccup rendered <:html> instead of <html>.
- regex: parse leading Java inline flags ((?s)/(?i)/(?m)) and pass the
equivalent irregex options (single-line/case-insensitive/multi-line); irregex
rejects the inline syntax. Adds a java.util.Iterator shim ((.iterator coll)/
.hasNext/.next) for the run!-style loop hiccup compiles.
deftype fields tagged ^:unsynchronized-mutable / ^:volatile-mutable can now be
reassigned in place from a method, as on the JVM. A jrec stores fields as cons
cells, so a new jolt-set-field! mutates the pair with set-cdr!. The deftype macro
rewrites (set! mutable-field v) in a method body to (set! (.-field inst) v), and
the analyzer compiles a (set! (.-field obj) v) target to jolt-set-field! — so
both the rewritten symbol form and an explicit interop (set! (.-root this) v) go
through one path. Field reads remain a snapshot at method entry, which is correct
for the universal read-then-set pattern (a repeated set! of the same field in one
call would read the entry value).
Closes the set!-of-local SCI failures: SCI load 202 -> 205/218.
deftype/defrecord inline protocol methods went through extend-type ->
register-method, so a record implementing a protocol inline showed up in
(extenders P) — the JVM only lists extend/extend-type/extend-protocol
registrations there (inline impls compile into the class). Add
register-inline-method: it registers for dispatch under the record tag but
skips the extender mark. The mark lives inside type-registry so the per-case
corpus prune restores it. Closes corpus lists-extended-type + seq-of-tags.
- (class x) returns per-type JVM class names (Long/Double/Ratio/Character/Atom),
not a blanket java.lang.Number.
- register fully-qualified class tokens (java.lang.Long, clojure.lang.Keyword,
clojure.lang.Atom, ...) that self-evaluate to their name, so (= (class 1)
java.lang.Long) and (instance? clojure.lang.Atom x) resolve.
- instance? recognizes Long/Double/Ratio/Character/Symbol/Atom/IFn built-ins.
Closes class number/string/keyword/name, instance? Atom, atom?. Corpus 2699->2705.
Rephrase comments that pointed at deleted Janet files (emit.janet, the seed
sources, 'the Janet back end punts ...') to present-tense descriptions of the
Chez behavior. Comment/docstring-only; the self-host fixpoint is unchanged
(comments don't affect the compiled seed).
Delete five files that were Janet-host shims with no Chez path: clojure.java.io
(provided natively by host/chez/io.ss), and jolt.{nrepl,png,interop,shell}
(the janet.* bridge, os/shell, janet.net — none exist on Chez).
jolt-cf1q.6
jolt was all-flonum (one :number type, inherited from Janet whose only number
type is a double). The Chez runtime has a full numeric tower, so the zero-Janet
path now carries it = JVM Clojure semantics:
(/ 1 2) => 1/2 (exact Ratio, was 0.5)
(integer? 3) => true (integer? 3.0) => false (float? 3.0) => true
(ratio? (/ 1 2)) => true (= 3 3.0) => false (== 3 3.0) => true
(+ 1 2) => 3 (exact) (/ 1.0 2) => 0.5 (double)
jolt= was already exactness-aware (values.ss) and == is value-equality, so
=/== match the JVM split. The reader preserves exactness (integer literals exact,
a/b ratios exact rationals, decimals/exponents flonums); backend_scheme emit-const
renders exact ints/ratios and flonums faithfully; the value-position arithmetic,
count, int, compare, bit ops, parseLong, string .length/.indexOf, range,
timestamps, and array bytes return exact integers (= JVM int/long) instead of
coercing to flonum. double/parseDouble/clojure.math floor|ceil|signum stay double.
Only the zero-Janet path carries the tower (the Janet reader loses exactness into
a double before emit). The prelude/all-flonum path is unaffected for compiled code;
the runtime reader is shared, so a couple of all-flonum reader assertions become
value (==) assertions. ~16 numeric corpus cases now give the JVM tower value vs the
Janet-era :expected and are allowlisted as tower divergences (Chez == reference
JVM) pending the corpus flip to JVM (jolt-ecz0). No BigDecimal type (1M).
Re-minted. zero-janet 2682 (floor 2698->2682, the reclassified tower cases), 0 new
divergences; fixpoint 10/10, bootstrap 6/6, spine 35/35, cli 49/49; Janet gate 155
files 0 failed.
Port the java.lang.String/CharSequence method surface to the Chez RT so
(.toUpperCase s), (.substring s a b), (.indexOf s x), the regex methods
(.matches/.replaceAll/.replaceFirst/.split), etc. run on a string target.
natives-str.ss holds jolt-string-method, ported from the seed's surface in
eval_resolve.janet: ASCII case mapping (byte-oriented like the seed), -1 on
indexOf miss, flonum numeric returns to match jolt's number model, Scheme
chars for charAt, and the regex methods over the irregex compiled via
jolt-re-pattern. record-method-dispatch gains a string? arm that falls
through to it (unsupported methods still throw).
Corpus prelude floor 2002 -> 2026 (+24), 0 new divergences. _str 27/27 vs
build/jolt; full jpm test + conformance x3 green.
The (. x m) dot-form (the . special form, distinct from .method sugar) and
the clojure.string namespace (needs prelude plumbing + Pattern) stay deferred.
Implement (type x) on the Chez RT (host/chez/natives-meta.ss). Mirrors the
seed's core-type: the :type metadata wins when present, a record yields its
ns-qualified class-name SYMBOL (user.TyR — no-ns sentinel #f so it = the
overlay's (symbol (str t))), everything else a host-taxonomy keyword
(:number/:string/:vector/:map/:set/:seq/:fn/…). Total by construction — a
non-record value falling through to a crash would read as a divergence, so the
cond covers every value type incl. the host wrappers (atom/volatile/regex/var/
transient/uuid -> :jolt/*, a :jolt/type-tagged map like ex-info -> its tag,
sorted-set -> :jolt/sorted-set, sorted-map -> :map) and a final :object.
Also pin sequential?/seq? on lazy seqs (test/chez/_seqpred.janet): the inc M
seq? re-def-var! fix already covers sequential? transitively (sequential? is
overlay and delegates to seq?), so no code change — the earlier "still broken"
note was wrong, it assumed sequential? was native like seq?.
Prelude corpus parity 2000 -> 2002 (the two type cases), floor raised, 0 new
divergences. Gate: _type 37/37 + _seqpred 22/22 (both vs build/jolt oracle),
emit-test 321/321, full jpm test, conformance 355x3.
The largest remaining crash bucket: defrecord/deftype/defprotocol/extend-type/
reify. make-deftype-ctor/make-protocol/protocol-dispatch/register-method/
satisfies?/extenders/instance-check/make-reified were ctx-capturing seed natives
resolving to jolt-nil.
records.ss adds a jrec type (tag + field alist), set!-extended into every
collection dispatcher (get/=/hash/count/keys/vals/seq/assoc/conj/contains?/map?/
pr-str/pr-readable/str) via the transients.ss capture-prev pattern, plus a
protocol registry (type-tag -> proto -> method -> fn) and dispatch over record
tags / host-type candidates. (get rec :jolt/deftype) returns the tag, so the
overlay record? works unchanged. A record equals another of the same type with
equal fields, is map?/coll? not vector?, prints #ns.Name{...}, and str uses a
custom Object toString impl when defined.
emit.janet :host-call now routes a non-shimmed method to record-method-dispatch
(was an emit-fail), so (.protoMethod record args) compiles; .-field access is
still analyzer-punted (deferred).
Prelude parity 1652 -> 1701, 0 new divergences. 4 print-method-multimethod cases
moved crash->allowlist (the printer doesn't consult a custom print-method yet).
jolt-jgoc.
