A macro body can now read &form (the call form) and &env (a map of the in-scope
local symbols), like Clojure. This is what core.logic's matche/defne use to tell
a pattern symbol that names an enclosing local from a fresh pattern var — so
locals-membero and the recursive checko in `matches` now compute correctly. The
suite reaches 535/2/0 (the last two are constraint reification ORDER, where the
constraint set is right but it is spliced from a set whose iteration order differs
from the JVM — a host set-ordering divergence, not a bug).
&form/&env are clojure.core dynamic vars bound around each expander call rather
than prepended params, so the macro calling convention is unchanged and the mint
stays consistent (the seed prelude is byte-identical; only the analyzer carries
the env into form-expand-1). macroexpand-1 passes an empty env.
corpus.edn: the ~@ unquote row is now a boolean compare (a bare clojure.core/
unquote-splicing symbol evaluates to an unbound var, not the symbol).
Two general fixes that clear core.logic's finite-domain -difference, safefd, and
the defne quoted-list patterns (form->ast), taking the suite to 532/5/0.
- instance? on a deftype matched a simple type name against the qualified tag by
raw string suffix, so "a.b.MultiIntervalFD" tested true for IntervalFD. The
suffix must land on a "." boundary. core.logic's fd dispatches on
(interval? x) = (instance? IntervalFD x), and a MultiIntervalFD wrongly counted
as an interval, so -difference/safefd computed the wrong set.
- the reader reads ~ / ~@ as clojure.core/unquote(-splicing), like the JVM reader,
instead of a bare unquote. Code that inspects quoted pattern/template data —
core.logic's defne checks (= f 'clojure.core/unquote) — now sees the symbol it
expects, so '(fn ~args . ~body) patterns compile. hc-head-is? accepts the
qualified head in syntax-quote lowering; the value-preserving change leaves the
minted seed byte-identical.
corpus.edn: 2 JVM-certified unquote rows. unit.edn: two reader rows updated to the
qualified unquote. make test + shakesmoke green, 0 new divergences, self-host holds.
value-host-tags returned only ("Object") for a record, so a protocol extended to
clojure.lang.IRecord / IPersistentMap / Associative / Seqable / … (and not to the
record's own type) dispatched to the Object default. core.logic extends IWalkTerm
to IRecord, and walking a record value hit Object's (walk-term [v f] (f v)) — which
re-enters walk* and loops forever (the test-53-lossy-records hang).
A defrecord now carries the map/record interface tags it genuinely satisfies. The
record's own type is still tried first (jrec-tag, before these tags), so a direct
extend to the record type wins, and record equality / map? / record? are unchanged.
A bare deftype stays opaque (its tag + Object; declared interfaces dispatch via its
inline methods). Runtime only, no re-mint.
A name a namespace excludes from clojure.core (:refer-clojure :exclude) is not
clojure.core/name even before the ns defines its own — syntax-quote must qualify
it to the current ns, like Clojure. refer-clojure was a no-op, so a syntax-quoted
excluded name (core.logic.fd's `==`, referenced by a constraint's -rator before fd
defines ==) resolved to clojure.core/==.
jolt-refer-clojure now records the :exclude set per ns; hc-sq-symbol consults it
before falling back to clojure.core. Fixes core.logic's fd constraint -rator names.
Runtime only, no re-mint.
Follow-on to the core.logic relational-engine work. These clear every crash in
core.logic's constraint-logic-programming and unifier layers (33 errors -> 0) and
most of the value mismatches; the suite goes 504 -> 523 passing assertions. All
are general gaps, not core.logic-specific.
- symbols intern their ns/name strings (JVM Symbol.intern .intern()s them): two
separately-read `?a` symbols now share one name-string object. core.logic's
non-unique lvars compare names by identity (via (str sym)), so without this a
term's lvar and a constraint's lvar built from different `?a` reads never matched
and constraints silently never fired.
- (str x) of a single arg returns its rendering directly instead of copying through
string-append, and a symbol stringifies to its (interned) name — JVM (str x) is
x.toString(). Needed for the identity comparison above.
- a clojure.core-qualified special form dispatches correctly: syntax-quote
namespace-qualifies a macro like letfn to clojure.core/letfn (matching Clojure,
where it's a macro), and the analyzer now maps that back to the special form
instead of treating it as an invoke of a nil var. core.logic's fnc/defnc emit
(clojure.core/letfn ...). Re-mint.
- (disj nil ...) is nil (JVM), instead of crashing in the set path — core.logic's
constraint store does (disj (get km v) id) where the get can be nil.
corpus.edn: 4 JVM-certified rows. make test + shakesmoke green, 0 new divergences,
self-host fixpoint holds.
Running clojure/core.logic's own suite surfaced a batch of general jolt gaps.
None are core.logic-specific; each is a language/host behavior that was wrong or
missing. With these, the core relational engine (unify, run/fresh/conde,
conso/membero/appendo, reification to _0/_1, lcons) runs; the remaining failures
are in core.logic's constraint-logic-programming and finite-domain layers
(tracked separately).
- analyzer: accept the list-member dot form (. target (method args)), sugar for
(. target method args). Re-mint.
- identical? is reference identity (eq?), not value equality. It was aliased to =,
which infinite-loops when a deftype's .equals short-circuits on (identical? this o)
(core.logic's Substitutions) and is wrong for distinct equal collections.
- jrecs use a deftype's declared hashCode/equals/equiv for map/set keying instead
of structural field comparison, so metadata-wrapped keys still match (core.logic
keys substitutions on lvar id, ignoring metadata).
- meta/with-meta dispatch to a deftype's clojure.lang.IObj meta/withMeta methods
when present, so metadata threaded through the type's own assoc/withMeta survives
(previously kept in an identity side-table the reconstructed instances didn't share).
- coll?/seqable? on a deftype require IPersistentCollection (cons) or ISeq (first);
ILookup(valAt)/Indexed(nth)/Counted(count)/Seqable(seq) alone no longer qualify,
matching the JVM.
- syntax-quote resolves a bare symbol to the compile ns's own def before
clojure.core, so a name the ns excluded and redefined (core.logic's == after
:refer-clojure :exclude) qualifies correctly in macro output.
- reader: record literals #ns.Type{...} / #ns.Type[...] expand to the map->/->
factory call.
- structmap API: defstruct/create-struct/struct-map/struct/accessor (map-backed,
insertion-ordered). Re-mint.
- .hashCode on strings/symbols (Java String.hashCode, Symbol Util.hashCombine);
Class.isInstance; java.util.Collection.contains over vector/list/set;
clojure.lang.RT/nextID and clojure.lang.Util hash/hasheq/equiv/identical statics.
corpus.edn: 8 JVM-certified rows. unit.edn: a Counted+Seqable deftype is coll?=false
(was a stale expectation encoding the old behavior).
jolt maps were HAMTs with hash iteration order; Clojure keeps small maps as
PersistentArrayMap (insertion order), converting to PersistentHashMap past a
threshold. Map literals, array-map, assoc, into/transient, merge, zipmap,
select-keys, update-keys/vals, frequencies and group-by now iterate in insertion
order for <=8 entries, matching the JVM. hash-map and >8-entry maps stay hash
order; sets stay hash order.
The pmap record gains an order field (the insertion-order key list, or #f once
hashed); the HAMT still backs the values so equality/hash/lookup are unchanged.
pmap-fold visits an array-mode map last-to-first so the runtime's cons-accumulate
idiom reconstructs insertion order without touching its many call sites, and
hash-mode output stays byte-identical; pmap-fold-fwd visits in order for the few
sites that build a value directly. Transient maps track insertion order and
promote to hash past max(8, source-count), matching TransientArrayMap.
The hash-map native-op retargets to a hash-order builder so (hash-map ...) stays
hash-ordered while {...} literals are ordered; syntax-quote builds maps via the
hash builder (Clojure expands `{...} to apply hash-map). The core overlay map
builders seed from {} instead of (hash-map) to keep order.
Threshold is 8 for any key (the keyword exception in newer Clojure isn't in
1.12.5). honeysql now passes 832/0/0; 19 JVM-certified corpus rows added.
defn- now adds :private to the var metadata (like Clojure), and ns-publics
filters those out while ns-interns/ns-map keep them — they were all the same
unfiltered scan before. A lib that introspects ns-publics (honeysql asserts
every public helper has a docstring, and that the clause set matches the public
helpers) saw the private defn- helpers and failed; now honeysql 636/8 -> 638/6
(the rest are map key-order).
jolt's case ops are codepoint-based and locale-independent, so the default
locale is a no-op token: getDefault/setDefault/forLanguageTag + ROOT/US/ENGLISH.
honeysql sets and restores the locale around formatting to assert output is
locale-stable (its Turkish-İ regression guard) — that test errored on the
missing Locale/setDefault static, now passes (honeysql 635/8/1 -> 636/8/0).
clojure.edn/read built the built-in #inst/#uuid eagerly (via read-string), so a
:readers override couldn't win and #inst applied to a non-string form (aero's
#inst ^:ref […]) threw. Read the raw form instead and let edn->value route every
tag through :readers then :default then the built-in — matching clojure.edn,
where a reader from opts wins. edn->value now also converts the (recursively
converted) metadata, since the raw path skips the read-string data seam. aero
suite: 59/0/0 (full pass). clojure.edn baked, re-minted.
meta-copy keyed metadata on the SAME cseq/lazyseq object (the else branch), so
(with-meta xs m) mutated the original list in place — Clojure's PersistentList
is immutable and withMeta returns a new list. (with-meta xs {:k xs}) thus built
a self-referential cycle (the list's metadata pointed at the list), which looped
*print-meta* printing forever — the root of aero's meta-preservation hang. Now
copies the cseq/lazyseq node like the other collections. aero suite completes:
58/0/1 (was hanging).
clojure.edn/read with :readers/:default recursed into vectors/maps/seqs but
not a constructed set, so a tagged literal in #{…} (aero's #ref in a set) kept
its raw form. edn->value now recurses into a set. clojure.edn is baked into the
seed, re-minted. Fixes aero #ref-in-set + falsey-user-return.
Clojure's reader reads a ^{…} map with the same read() as any value, so a set/
tagged literal in metadata is a value, not a form. jolt's data seam converted a
set-form to a set in the VALUE but left it as the tagged form inside the
METADATA, and dropped metadata on an empty list entirely (a wrong 'interned, =
Clojure' special case in rdr-attach-meta — Clojure's MetaReader withMetas () via
IObj). rdr-form->data now always converts + carries the (recursively converted)
metadata, whether or not the value structure changed; rdr-attach-meta no longer
skips (). Fixes aero's meta-preservation (set/map/vector/empty-list ds round-
trip). All runtime .ss (data seam), no re-mint.
Two gaps reitit-core surfaced (now 322/0/1 -> 327/0/0):
- A deftype/defrecord inline method with two _ params, (m [_ _] field), read
the field as nil: mk-clause bound fields off (get _ :field) where _ was the
first param, but the second _ shadowed it. Each _ param is now renamed to a
fresh symbol so the instance is unambiguous.
- A var did not dispatch to a protocol's clojure.lang.Var extension (reitit
extends Expand to Var for a #'handler route): value-host-tags gained a var arm
(Var/clojure.lang.Var/IDeref/IFn) and host-type-set gained Var/IDeref so the
extension keys under Var.
deftype/defrecord is a seed source, re-minted.
jolt had Long/Integer/Double class tokens but not Byte/Short/Float, and no
Byte/Short MIN_VALUE/MAX_VALUE/valueOf/parse* statics. clojure.test.check (a
malli dependency) references Byte/MIN_VALUE and Byte/MAX_VALUE. The values are
plain integers on jolt; the statics expose the JVM ranges (127/-128, 32767/
-32768).
A defrecord is Associative/ILookup/IPersistentMap/Seqable/Counted on the JVM,
so (.assoc r k v) / (.valAt r k) / (.without r k) / (.containsKey r k) /
(.cons r x) / (.count r) / (.seq r) / (.equiv r o) / (.entryAt r k) now work
via Java interop, delegating to the map fns when not overridden by a declared
method. reitit's impl calls (.assoc match k v) directly. A bare deftype uses
its own declared methods (record-only branch). reitit-core 58/1/18 -> 321/1/1
with the router lib's Trie shim.
#{...} reads as the tagged set-form for the analyzer, but a macro saw that
map instead of a set (set? false / map? true, unlike a vector). hc-expand-1
now converts a set-form argument to a real set before calling the expander, so
(set? arg)/conj/seq work — hiccup's compiler introspects a literal set this way
(str (html #{"<>"}) was empty, now #{"<>"}). Elements stay as
read; a deeply-nested set literal inside another form is left for the analyzer.
hiccup 382->383. Jolt-side unit guards (macro def+use in one form isn't
JVM-portable).
(defn name docstring? {:k v} arglists...) and the multi-arity name+attr-map
now merge the attr-map into the var metadata like Clojure — jolt was parsing
the map out of the body and discarding it. The metadata (the name's own ^{},
the attr-map, and the docstring as :doc) is attached to the def name symbol,
which analyze-def reads and evaluates. defn is in the earliest tier, so the
macro uses only conj/assoc/meta/with-meta (not merge/last). The rare trailing
attr-map (after the last arity) is not yet handled. Fixes hiccup's defelem
meta + honeysql docstring tests.
('sym coll) / ('sym coll default) now do (get coll 'sym ...), like keywords —
a symbol is IFn on the JVM. jolt threw "cannot be cast to clojure.lang.IFn".
Pre-existing gap (not a regression), surfaced by honeysql's :checking mode,
which does ('where dsl) to look up a clause. honeysql 623/13/8 -> 635/8/1.
Corpus rows added.
The deftype-is-not-a-map change (#245) gated map? on jrec-record?, so only a
defrecord was map?. But a deftype that implements clojure.lang.IPersistentMap is
map? on the JVM — clojure.core.cache's caches are exactly that, and its TTL
factory asserts (map? base) on an LRUCache passed as the base (its suite went
1314 -> 2 errors). map? now also covers a deftype whose without/dissoc method is
registered — the IPersistentMap-distinctive op a vector or set lacks. An opaque
deftype (RawString) stays non-map?; a defrecord stays both. Guards added to
unit.edn (jolt-side: a full IPersistentMap impl will not compile on the JVM
corpus oracle).
The rest = more() change made (rest coll) return a jolt-lazyseq, so the very
common (conj (rest xs) y) hit jolt-conj1's base case, which doesn't recognize a
lazyseq, and threw "conj: unsupported collection" (caught by core.match's
seq-pattern compiler). conj on a lazy-seq now prepends like conj on any seq.
The corpus had no row exercising a collection op on a rest-derived seq, so the
class slipped past the gate; add a seqs/lazy-seq-interop suite (conj/into/first/
count/nth/reduce/map/filter/apply/cons/=/empty?/seq over (rest …) and lazy-seq),
all JVM-certified.
- with-deeper-print only parameterizes the print depth when *print-level* is
set, so printing pays no parameterize on the common nil-default path.
- re-find (the matcher) and re-seq advance past a zero-width match relative to
the match's own start, not the search origin — a zero-width match found past
the origin (lookahead/boundary) no longer repeats. (re-seq #"a*" "aba") now
matches the JVM.
A library that throws a typed host exception (an http client raising
java.net.ConnectException) needs (class e), instance?, .getMessage and
ex-message to all reflect the named class. jolt-host-throwable already builds
that value; expose it as a jolt.host seam so libraries stop hand-rolling a
:jolt/ex-info table that carries only the class (its .getMessage/ex-message
return nil).
Six correctness fixes, each a general gap (not hiccup-specific):
- deftype is not a map. jolt treated every deftype instance as a map
(map?/record?/seqable over its fields); in Clojure only a defrecord is
map-like, a bare deftype is an opaque object. defrecord now marks its type;
map?/record?/coll?/seq/empty? gate on it, while a deftype implementing a
collection interface still dispatches through its methods.
- cross-ns extend-protocol on an imported deftype. register-method built the
type tag from the *calling* ns + bare name, so (extend-protocol P Raw …) in
one ns missed a Raw value defined in another. A simple-name index resolves
the bare name to the type's real tag (local ns still wins).
- str vs print. str of a collection is its readable form (nested strings
quoted: (str ["x"]) => ["x"]); print leaves them raw. jolt defined print
as str, conflating the two. Split via a __print1 seam.
- clojure.test thrown? now honors the exception hierarchy (instance?), so
(thrown? IllegalArgumentException …) matches an ArityException subclass.
- java.net.URI is value-equal (= and hash by string form).
- clojure.walk/macroexpand-all was missing; an unresolved qualified var made
the analyzer report "Unknown class walk".
deftype/defrecord + print are seed sources, re-minted. hiccup 365->381 of its
own suite; the rest are charset-encoding / var-meta niches.
jolt's seq layer realized one element ahead of Clojure, so a side-effecting
lazy seq ran its producer too eagerly. Four changes bring it in line:
- rest is Clojure's more(): it returns the tail without realizing it. An
unforced tail (vector / string / lazy-seq cell) comes back as a deferred
seq, so (rest (iterate f x)) does not call f. next still realizes one.
- iterate applies f lazily, inside the tail thunk, so (first (iterate f x))
is x with no call to f (clojure.lang.Iterate parity).
- take realizes exactly n: the last element terminates without touching the
rest, instead of forcing one more element of the source.
- an empty realized lazy seq is still a sequence value, printing "()" not
"nil" (a JVM LazySeq is never nil).
Also: the map transducer's step fn now takes multiple inputs
([result input & inputs]) so a multi-collection transduce applies f across
all of them. Fixes medley's join/window/sequence-padded laziness and
multi-input transducer tests (now 293/293). The rest change also fixed a
latent overrun in distinct/dedupe over a map's empty tail.
iterate is a seed source, re-minted.
Both printers (jolt-pr-str, jolt-pr-readable) now thread a print depth and
read the two limit vars. *print-length* truncates each collection to N
elements + "...", walking seqs lazily so an infinite seq prints under the
limit without realizing it. *print-level* renders a collection at depth >=
the level as "#". The reader consults *default-data-reader-fn* for an
unregistered #tag before falling back (tagged form on the data seam, throw
on the edn seam). All three interned with nil defaults.
A wrong-arity or non-seqable error that Chez raises carried no jolt exception
class, so (class e) was :object and (thrown? ArityException …) / (thrown?
IllegalArgumentException …) never matched. Classify these by message:
incorrect-number-of-arguments -> clojure.lang.ArityException, not-seqable ->
java.lang.IllegalArgumentException, with ArityException modeled as a subclass of
IllegalArgumentException. (class e) and instance? now match the JVM.
All java-layer (records-interop.ss classifies, host-class.ss reports the class +
the ArityException token). medley 281 -> 283 passing.
jolt-o9dc
Spec coverage dashboard had 6 missing-portable and 24 dynamic-var entries. The
portable ones are now implemented (missing-portable -> 0, dynamic-var -> 14):
- Stateful matcher: re-matcher now returns a real mutable Matcher; re-find over
it steps through matches and re-groups returns the last match's groups (was an
inert tagged map). Closes re-groups.
- letfn is interned as a clojure.core var so (resolve 'letfn) matches the JVM. It
stays a special form (the value is never invoked, not marked a macro).
- *1 *2 *3 *e interned (nil outside a REPL).
- Portable dynamic vars whose default already matches jolt's behaviour:
*read-eval* *print-dup* *print-namespace-maps* *flush-on-newline*
*compile-files* *math-context* *command-line-args* *file*.
The remaining 14 dynamic-var entries are host-internal (compile-path,
compiler-options, fn-loader, reader-resolver, repl, source-path, ...) or deferred
pending printer/reader support (*print-length* *print-level*
*default-data-reader-fn*). Corpus rows added for each closed gap; coverage.md
regenerated.
irregex rejected two patterns the JVM accepts, which blocked library loads:
- [\w-_] errored with bad char-set because a - after a shorthand class was
read as a range start. Java reads it as a literal hyphen. Preprocess the
pattern to escape such a dash.
- (X+)* errored with duplicate repetition because sre-repeater? recurses
through submatch, treating a quantified group like a dangling a**. Override
it to a bare leading * / + check, matching the JVM (which only rejects the
dangling case).
Both in regex.ss (runtime). Unblocks cuerdas (was load-fail, now 292 passing)
and aws-api config-test. Also documents the host/chez/java source-layering rule
in host-interop.md.
jolt-l8so
(type x) was jolt's internal taxonomy keyword (:string/:set/:jolt/inst), which
breaks any library dispatching a multimethod on [(type a) (type b)] against
java/clojure.lang classes (e.g. clojure.tools.logging.test's matchers). Make the
PUBLIC clojure.core/type Clojure's (or (:type meta) (class x)).
The taxonomy keyword stays the core model: natives-meta.ss keeps jolt-type and
exposes it as __type-tag, which print-method/print-dup dispatch on (so #uuid/#regex/
records still print). The JVM mapping lives in the java host layer — host-class.ss
defines the public type next to (class …), and a jinst now reports java.util.Date
(was :jolt/inst). So the core emits the taxonomy and the java layer remaps it in one
place. unit.edn's type suite updated to the class names. make test green.
Co-authored-by: Yogthos <yogthos@gmail.com>
Class/forName claimed every java.*/clojure.* name found (and any "x.y.Class"
matched the registered Class via a short-name fallback), so a library's
(class-found? "optional.Dep") feature-probe always said yes — tools.logging then
tried to build the java.util.logging / log4j backends jolt lacks and crashed.
Resolve forName by exact registry lookup + an honest prefix that excludes the
unbacked optional packages (java.util.logging, javax.management), so the probe
sees them absent and skips the backend.
class of a persistent collection / namespace now reports its JVM class name
(clojure.lang.PersistentHashSet, …Namespace, …) instead of jolt's internal :set/
:object tag, and isa? consults JVM class assignability — Object as every class's
root plus a modeled clojure.lang/java.util hierarchy — so (isa? (class x) C) and a
class-keyed multimethod dispatch like the JVM (e.g. (isa? Keyword Object) was
false). Adds the bare class tokens (Fn/Namespace/Set/…) these dispatch on.
(type x) is unchanged — it keeps jolt's documented internal-keyword form. Six
JVM-certified corpus rows. make test green, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
A library that throws an ex-info envelope carrying a JVM :class (jolt.host/
tagged-table with a "class" entry — e.g. http-client's UnknownHostException on a
DNS failure) was caught only by a broad (catch Throwable …): (class e) read the
class but (instance? C e) — which catch dispatch lowers to — always returned
false, so clj-http-lite's (catch UnknownHostException e …) for :ignore-unknown-host?
never matched and the condition propagated.
Add an instance? arm matching such an envelope against the carried class or any
ancestor (full name or last segment), and register the common exception hierarchy
(Throwable/Exception/RuntimeException/IOException + the java.net socket/host
exceptions) so (catch IOException e) / (instance? Throwable e) also match. A
non-throwable class (RuntimeException over an IOException, String) stays false.
Fixes http-client's :ignore-unknown-host? test (116/0/0, was 1 error). make test
green, 0 new divergences. Runtime .ss, no re-mint.
Co-authored-by: Yogthos <yogthos@gmail.com>
A value class libraries return from getters (java.time, and the java.net.http
client shim that aws-api's java backend builds on). Statics of/ofNullable/empty,
methods isPresent/isEmpty/get/orElse/orElseGet/ifPresent/toString, value-equal so
(= (Optional/of x) (Optional/of x)). Five JVM-certified corpus rows. Runtime .ss,
no re-mint.
Co-authored-by: Yogthos <yogthos@gmail.com>
parse-ms ignored sub-second fractions: a formatter parse of
"2020-07-06T10:59:13.417Z" dropped the .417 (and the ISO_OFFSET_DATE_TIME pattern
"…ssXXX" then mis-aligned, reading ".417Z" as the offset). java.time's ISO
formatters accept an optional fractional second, so jolt should too.
After parsing the seconds field, consume an optional .fff from the input (to
millis) when the pattern carries no fraction field — which is how the ISO_*
constants are modeled here (ss, no S). Also handle the S pattern letter for
explicit .SSS patterns. Carry the fraction into the result.
Fixes aws-api shape iso8601 parse (1 FAIL -> 0; cognitect.aws.util/parse-date now
returns the right Date). Two JVM-certified corpus rows. make test green, 0 new
divergences. Runtime .ss — no re-mint.
Co-authored-by: Yogthos <yogthos@gmail.com>
tick's fields-test walks every ChronoField a temporal supports and reads it,
which crashed on fields jolt didn't implement. Fill the gaps:
- LocalTime: CLOCK_HOUR_OF_DAY (1..24), HOUR_OF_AMPM, CLOCK_HOUR_OF_AMPM,
MICRO_OF_DAY — both isSupported and getLong.
- LocalDate: the aligned-* group (ALIGNED_DAY_OF_WEEK_IN_MONTH/_YEAR,
ALIGNED_WEEK_OF_MONTH/_YEAR).
- LocalDateTime field routing now asks which part supports the field instead of a
hardcoded date list, so a date field never misroutes to the time part (the actual
cause of "LocalTime has no field ALIGNED_DAY_OF_WEEK_IN_MONTH" — a ZonedDateTime's
date field fell through to its time).
- Year / YearMonth gain isSupported / get / getLong.
tick api_test 344/0/1 -> 599/0/0. Seven JVM-certified corpus rows. make test green,
0 new divergences. Runtime .ss — no re-mint.
Co-authored-by: Yogthos <yogthos@gmail.com>
A devirtualized protocol call resolved its impl with devirt-resolve on EVERY call
— but the tag/proto/method are compile-time constants, so the resolved fn is a
runtime constant (closed world). That per-call find-protocol-method (three
hashtable lookups) was the cost: on mono-dispatch, dispatch was ~75% of the time
(ablation: same arithmetic direct-call 166ms vs dispatch 673ms).
Resolve once. When emitting a direct-link def, each devirt site gets a fresh cache
cell, bound to #f in a let wrapping the def (so it persists across calls and is
shared by every invocation); the site resolves into it on first use ((or cell
(let ((_f (devirt-resolve ..))) (set! cell _f) _f))) and reuses it after — the
inline cache the JVM gets for free. First call still passes the real receiver, so
the Object/host-tag fallback (devirt-resolve) is unchanged.
mono-dispatch 673ms -> 214ms (~3.15x), 47.5x -> ~15x JVM, near the 166ms
direct-call floor. run-devirt.ss gains the cached-path checks (cell present, 1st
call caches + 2nd reuses, both == dispatch). make test / shakesmoke green, selfhost
holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
A record-or-nil (a protocol method whose impls return a record in one branch and
nil in another, or an `if` over a ctor and nil) now types as a NILABLE record
instead of widening to :any. A nilable record still bare-indexes its field reads
(jrec-field-at falls back to jolt-get on nil), but some?/nil? do NOT fold on it, so
a runtime guard is preserved — and inside (if (some? x) ..) / (if x ..) the then-
branch narrows x to the non-nil record, so its reads bare-index AND unbox there.
This is what lets the bounced ray type without a hint: scatter returns
ScatterResult-or-nil (Metal absorbs some rays), and the consumer reads
(:ray scattered) only under (if (some? scattered) ..). The narrowing proves
scattered non-nil there.
lattice: :nil type; :nil ∨ struct -> nilable struct, ∨ anything else -> :any;
nilability is contagious through a struct join, which also now preserves :type when
both sides agree (needed so a record ∨ its nilable self stays that record).
truthy-type?/field-type/pred-on treat a nilable struct as maybe-nil. types: nil
literal -> :nil; an `if` whose test is (some? x)/(nil? x)/x narrows the nilable
local x in the proven branch.
Ray tracer with NO hints: 38.4s -> 23.9s (~1.6x) — hit-sphere now types fully
(0 jolt-get, 57 jrec-field-at, 38 fl-ops), identical to the hand-hinted build.
run-narrow.ss gate, incl. the load-bearing check that the nil case still takes the
else branch (the guard is not folded away). make test / shakesmoke green, selfhost
holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
A protocol method whose impls all return the same record type has a monomorphic
return. collect-pm-rets! scans the unit's (register-(inline-)method ..) forms,
infers each impl fn's return type, and joins them per method; call-ret-type then
types a (method recv ..) call as that record, so a field read off the result
bare-indexes — e.g. (:ray (scatter m ..)) reads off a Ray. A disagreeing impl
joins to :any and keeps the generic path.
run-protoret.ss: a method with all-record impls bare-indexes + unboxes the field
read; a mixed-return method (one impl returns a number) stays generic. make test /
shakesmoke green, selfhost holds, 0 new divergences.
Foundation for auto-typing record values that flow through protocol dispatch. Does
not yet move the ray tracer: its scatter returns ScatterResult-or-nil (Metal
absorbs some rays), and the nil widens the join to :any — typing a nullable return
soundly needs flow-sensitive narrowing (a guarded (some? x) proves non-nil), filed
separately.
Co-authored-by: Yogthos <yogthos@gmail.com>
The whole-program fixpoint collects a self-recursive call's arg types into the
fn's own params. When a recursive call threads a param straight through unchanged
(same arg, same position — e.g. ray-cast passing `hittables` to itself), that arg's
type is the param's own current type: :any until external callers determine it. And
:any is absorbing, so collecting it pinned the param at :any forever — the type a
caller supplied (a vector of records) was lost, and the fn's field reads stayed
generic.
Skip a same-position pass-through arg in the self-recursion collection (contribute
the join identity). It can't add information — param i ⊇ param i is trivial — so
dropping it is sound; the param is still constrained by every external caller and
by any non-pass-through recursive arg. Applies to both self-recursion paths: a
`defn` recursing through its var, and a named fn literal recursing via its
self-local.
This is why ray-cast's `ray` typed (its recursion passes a fresh ray) but
`hittables` didn't (passed through). With the fix, hittables keeps its
vec<Sphere> element type, so hit-all's reduce element — and hit-sphere's reads —
type without any hint: ray tracer 38.4s -> 31.3s (~1.23x) with no annotations.
run-wp.ss: a recursive fn threading a vec param through keeps its element type.
make test / shakesmoke green, selfhost holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
jolt.host/record-ctor-key and record-type? were stubs returning nil since the
rehost, so phint-of always produced nil — a ^Record param hint (^Sphere s) was
silently dead. The inference only ever typed a record param when its callers
passed a concrete ctor (whole-program), so a fn called with an untyped value (a
vector element, a value threaded through recursion) read its fields generically.
Resolve the tag against the record registry (records.ss): chez-find-ctor-key maps
a ^Type tag to the ctor-key "ns/->Name" the inference seeds with, preferring the
compile ns and falling back to any registered record of that simple name (cross-ns
/ imported). record-type? / record-ctor-key call it. Runtime .ss — no re-mint (the
analyzer reaches record-ctor-key through the host var).
With this, a ^Sphere/^Ray-hinted hit-sphere in the ray tracer types its params, so
its 48 generic jolt-get field reads + boxed arithmetic become bare-index reads +
fl-ops: ray tracer 38.4s -> 23.9s (~1.6x), make test / shakesmoke green, selfhost
holds, 0 new divergences. A wrong hint stays safe — jrec-field-at falls back to
jolt-get for a non-record.
run-fieldnum.ss: a ^V-hinted param (no inferable caller) bare-indexes + unboxes,
with no generic jolt-get left.
Co-authored-by: Yogthos <yogthos@gmail.com>
A record field tagged ^double now reads back as a flonum and feeds the numeric
pass, so hintless arithmetic over those fields lowers to fl-ops — the leaf-numeric
analog of the ^Vec3 nested-field hints. Combined with the whole-program :double
param inference, a vec3-dot over a ^double-fielded record unboxes end to end with
no per-fn hints.
records.ss: a ^double field tag passes through resolution, and the ctor (and a
mutable-field set!) coerce a ^double field to a flonum — JVM primitive-field
parity (jolt returned an exact 1, not 1.0, before), and what makes reading the
field back as :double sound for an fl-op.
types.clj: field-type-from-tag maps "double" -> :double, and a keyword/get lookup
whose result is :double annotates the node :num-read :double. numeric.clj reads
that annotation and classifies the field read as a :double operand, so the
enclosing arithmetic specializes — the read itself keeps its jrec-field-at/jolt-get
emit.
run-fieldnum.ss gate: ctor coercion (int field -> flonum), field-field arithmetic
emitting fl*/fl+, and an untagged field staying generic. make test / shakesmoke
green, selfhost holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
The closed-world fixpoint (#226) flowed record types across fn boundaries; this
adds a numeric refinement so a hintless fn whose every call site passes a flonum
has its param unboxed to fl-ops, no ^double hint needed.
Lattice gains :double, a flonum refinement of :num: two doubles join to :double,
a double joined with anything else widens to :num — so a param is :double only
when every contributing value is a flonum, which is what makes the fl-op sound.
infer types a flonum literal and flonum arithmetic (+ - * / min max inc dec over
double/int-literal operands) as :double, and the fixpoint joins those across call
sites and return types like any other lattice value.
The bridge to the existing hint-directed pass is a synthetic [param :double]
nhint: wp-infer! stashes the :double params separately from the structural seeds,
and run-passes injects them as nhints before numeric/annotate, so the fl-op
emission and the exact->inexact entry coercion (a no-op on a proven flonum) apply
unchanged.
Sound subset only: :double, never :long — an untyped integer can be a bignum and
fx-ops would overflow/diverge from jolt's arbitrary precision. So an integer
caller leaves a param generic; an escaped fn (unknown callers) keeps :any.
run-numwp.ss gate: cross-fn :double propagation incl. through a flonum-returning
helper, the integer-caller and escape negatives, and the full run-passes path
emitting fl* + entry coercion. make test / shakesmoke green, selfhost holds, 0
new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
The devirtualized protocol call emitted find-protocol-method on the inferred
record tag, but a record can satisfy a protocol via an Object/host-tag default
rather than a direct impl — find-protocol-method on its own tag misses that,
while protocol-resolve walks to the default. So a record relying on
(extend-protocol P Object ...) resolved under ordinary dispatch but applied #f
under devirt and crashed. Closed-world opt builds only; the gate previously
covered just direct inline/extend-type impls so it shipped green.
Emit devirt-resolve, which tries the static tag and falls back to
protocol-resolve on a miss — same fast path, correct regardless of how the
record satisfies the protocol. Mirrors jrec-field-at falling back to jolt-get.
The receiver binds to one temp so it feeds the resolve and the application
without double-evaluating a side-effecting arg 0.
Also widen the whole-program fixpoint to :any on hitting the iteration cap: a
non-converged pre-fixpoint is more specific than the least fixpoint, so seeding
it would be unsound. Not reached in practice (~2 passes); a defensive floor.
run-devirt.ss gains an Object-default case. make test / shakesmoke green,
selfhost holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
When the inference types a keyword-lookup receiver as a record — it carries the
field-order :shape and :hint :struct from the whole-program fixpoint — the back
end reads the field by its declared slot via jrec-field-at instead of jolt-get.
That skips the jolt-get case-lambda, the dispatch fn, and the field-key
hashtable lookup, leaving a jrec? check + a static-index vector-ref.
jrec-field-at falls back to jolt-get when the receiver isn't the expected record
(a map downgraded by dissoc, or a value the inference mistyped), so it stays
correct if the static type is wrong. Only the no-default form takes the bare
path (a declared field is always present).
Sound only for non-nil records: a self-recursive param that can be nil (e.g.
binary-trees check-tree, whose untagged child is nil at leaves) types :any and
keeps jolt-get — the whole-program fixpoint demotes it. The target is non-nil
record params, like a Vec3 dot product (~5% there; boxed-flonum arithmetic
dominates the rest, a separate numeric lever).
run-fieldread.ss gate: emitted form uses jrec-field-at at the right slot and
matches jolt-get for each declared field; a non-field key and a default-arg form
keep the generic path. make test / shakesmoke green, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
When the inference proves a protocol call's receiver is one record type, the
back end resolves the impl by that static tag (find-protocol-method) instead of
routing through the protocol var -> jolt-invoke -> protocol-resolve, which
re-derives the tag and walks the type table. Same table lookup, minus the
var-deref, the rest-cons, and the receiver-type computation.
Fires only on a monomorphic site: a megamorphic receiver joins to :any and
carries no :devirt-type, so it keeps ordinary dispatch (the dispatch bench is
unaffected). The annotation comes from the whole-program fixpoint typing a
reduce/HOF element or a ctor return as a specific record.
Modest on the dispatch benchmarks (~6% on mono-dispatch) — float boxing in the
reduce accumulator dominates there, a separate numeric lever — but it removes
the dispatch overhead wherever a typed receiver is known.
run-devirt.ss gate: emitted form uses find-protocol-method, and evaluating it
matches ordinary dispatch for an inline impl, an extend-type impl, and the
non-devirt path. make test / shakesmoke green, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
Re-derive each app fn's param types from its call sites under --opt, so a
record type flows across fn boundaries: a ctor's return reaches a callee
param, and a typed vector's element reaches a HOF closure's param. The back
end can then bare-index field reads and devirtualize protocol calls at those
sites (it reads the resulting :hint/:devirt annotations; consuming them is
separate work).
This rebuilds the inter-procedural driver the Janet host had — the API
(infer-body/reinfer-def) survived the rehost but nothing drove it, and the
record-shapes/protocol-methods registries were empty stubs.
- records.ss: populate record-shapes (ctor key -> fields/tags/type, resolving
nested record field tags) and protocol-methods (method var -> [proto method])
registries at deftype/defprotocol load time; jolt.host accessors materialize
them.
- passes/types.clj: wp-infer! runs a closed-world fixpoint joining call-site
arg types into callee params; reinfer-def re-seeds each def at emit. Self-
recursive calls and fn-level recur are collected so a recursive fn's params
are constrained by its recursion, not just external callers — else a param
the recursion widens (e.g. binary-trees check-tree, whose untagged child can
be nil) would be unsoundly typed non-nil. A fn used in value position keeps
:any params (callers unknown). Megamorphic sites join to :any.
- build.ss: analyze all app forms and run the fixpoint before per-form emit.
- run-wp.ss: gate (cross-fn propagation, escape soundness, self-recursion).
make test / shakesmoke green, 0 new divergences, selfhost holds.
Co-authored-by: Yogthos <yogthos@gmail.com>
The persistent vector was a flat Scheme vector with copy-on-write: every conj
copied the whole backing array, so building an n-element vector was O(n^2). On the
collections bench that's vec-sum building 7500 elements with 227MB of copies, 90%
of the bench's allocation.
Replace it with Clojure's PersistentVector — a 32-way trie plus a trailing tail
chunk. conj appends to the tail and, when it fills, path-copies it into the trie,
so conj is O(1) amortized and a linear build is O(n). nth/assoc/pop are
O(log32 n). make-pvec (build a trie from a flat vector) and pvec-v (materialize
back) stay as compatibility shims, so the ~14 callers that read the backing array
— all one-shot conversions in =, hash, seq, meta-copy, transients, the reader —
are untouched; only this file's internals change.
vec-sum 70ms/227MB -> 1ms/3MB; collections 10.4x -> 4.0x vs JVM, under the 5x
target. 5 new corpus rows plus boundary stress (level transitions at 32 and 1024,
pop-collapse, assoc at every index) cover the trie.
Co-authored-by: Yogthos <yogthos@gmail.com>
A loop test like (or (>= i cap) (> ... 4.0)) desugars to
(let* [g (>= i cap)] (if (truthy? g) g (> ... 4.0))) and the whole thing was
wrapped in jolt-truthy? because returns-scheme-bool? only looked at :const and
:invoke nodes, not the let*/if an or/and expands to. The wrapper defeats Chez's
branch inlining on the hot loop edge.
Make returns-scheme-bool? recursive over :if (both branches bool), :let (body
bool, tracking which bound locals hold a Scheme boolean), and :local (in that
set). or/and over bool-returning ops then read as Scheme booleans and the outer
wrapper drops. Still sound: eliding only when the value is provably #t/#f — a
jolt-nil is a truthy record in Chez, so a false positive would be a real bug, and
the recursion only proves bool-ness through ops already known to return one.
No bench regression; the win lands on hinted float loops where the branch, not
boxed arithmetic, is the cost.
Co-authored-by: Yogthos <yogthos@gmail.com>
defprotocol emitted one variadic (fn [this & rest] (protocol-dispatch P m this
(list->cseq rest))) per method, so every protocol call — even a no-extra-arg one
like (area s) — consed a rest list, wrapped it in a cseq, var-deref'd
protocol-dispatch, and jolt-invoke'd it (consing again). On mono-dispatch that was
2.07GB of allocation, ~65% of the benchmark.
Emit one fixed-arity clause per declared arglist instead. The 1/2/3-param arities
call positional protocol-dispatch{1,2,3}, which resolve the impl (by record tag,
reify method, or host-tag extension — factored into protocol-resolve) and apply it
directly; no rest-list, no seq round-trip. The dispatchN entry points are in the
native-op table so the shim calls bind straight to the records.ss procedures
rather than var-deref. 4+ params fall back to the variadic protocol-dispatch.
mono-dispatch 1.5s/2.07GB -> 0.69s/280MB; dispatch 26x -> 12.2x, mono-dispatch
111x -> 51x vs JVM. 5 new corpus rows pin multi-arity methods, host-type args,
and protocol-method-as-value against JVM Clojure.
Co-authored-by: Yogthos <yogthos@gmail.com>
Records were a jrec holding an alist of (kw . val) conses: ~113B/node, built
fresh per construction, field reads a list scan. Replace that with a shared
per-type descriptor (tag + field keywords + an eq?-keyed keyword->index table)
plus a flat per-instance value vector and an extension map for any non-field
keys assoc'd on (jolt-nil when there are none). Construction now allocates one
vector instead of a cons chain and a field read is an index lookup. binary-trees
construction allocation drops 2.085GB -> 1.19GB.
That alone barely moved binary-trees wall-time: profiling showed the read loop,
not allocation, dominates, and the read loop's own allocation came from (nil? l)
lowering to (jolt-invoke (var-deref "clojure.core" "nil?") l), which conses its
args every call. Add nil?/some? to the backend native-op table so they inline to
jolt-nil?/jolt-some? (and drop the truthy wrapper, like the other predicates).
check-tree's read loop goes from 1.476GB allocated to zero; binary-trees 18.9x
-> 9.7x vs JVM. The remaining gap is the field-read dispatch chain (jolt-c3mw).
Two JVM divergences fixed along the way, both certified:
- dissoc of a declared field downgrades a record to a plain map (was kept as a
record); an extension key still drops cleanly.
- map->R keeps extension keys (was dropping anything outside the declared basis).
16 new corpus rows pin assoc/dissoc/count/keys/seq/=/hash/extension-field
behavior against JVM Clojure.
Co-authored-by: Yogthos <yogthos@gmail.com>
