Auditing the remaining cts baseline for R7 exposed real contract gaps hiding
among the model residue — all fixed to reference behavior:
- stale no-ratio-era stubs: numerator/denominator now work over jolt's exact
rationals (non-ratio is the Ratio cast failure); rational? includes decimals
- casts and pending: peek/pop demand an IPersistentStack (pop nil is nil),
realized? demands an IPending (a plain list/range throws), transient demands
an editable COLLECTION (non-colls throw; the RFC 0003 sorted/list/seq
superset keeps the copy-on-write fallback), empty on a plain record throws
- nil and empties: (nth nil i) is nil, (nth nil i d) is d, a nil index is NPE,
keys/vals of anything empty are nil, (conj nil) is nil
- lookups: contains? on a string is index-only (other keys IAE), get on an
array is lenient (nth still throws), a VECTOR invocation has nth semantics
(([1 2] 5) throws — call position and jolt-invoke both)
- into only transients editable collections; a PersistentQueue/sorted target
folds through conj (RT's IEditableCollection split)
- numbers: number?/num accept BigDecimal, quot/rem throw on an Infinite/NaN
quotient, even?/odd? demand integers
- ordering: keywords compare namespace-first with nil first (Symbol.compareTo)
- misc: run! honors reduced, eval self-evaluates non-form values, intern
demands an existing namespace, counted? excludes strings, seqable? includes
arrays, shuffle rejects maps, sort-by rejects a collection comparator,
when-let demands one binding pair, case*/deftype*/letfn*/reify*/& are
special symbols
Two mis-certified corpus rows fixed (they threw on the JVM too and hid in the
tolerated bucket): a raw \d string escape and duplicate literal map keys.
SPEC.md gains the baseline-traceability section: every one of the 146
remaining suite failures maps to a documented divergence (integer-box,
no-single-float, RFC 0003 transients, seq/chunking model, stm-refs,
parse-uuid strictness, vec-array adoption). cts baseline 5955 -> 6042 pass,
5 errors, 30 namespaces. 9 JVM-certified corpus rows.
The clojure.string case fns and searches now take any Object s through its
toString like the reference's ^CharSequence signatures ((upper-case :kw) is
":KW", (capitalize 1) is "1"); nil throws, and a nil substr in
starts-with?/ends-with? throws. some-fn re-ported with the reference
arities: (some-fn) is an arity error and a no-match result is the last
predicate's own falsy value (false, not nil). ifn? covers multimethods,
promises (which are now invocable — calling one delivers, via a cold-path
invoke-arm registry that costs the hot dispatch nothing), and deftypes
implementing IFn's invoke.
One structural find on the way: defmulti/defmethod deferred inside a fn
body (the deftest pattern) interned/resolved in whatever namespace was
current when they RAN, not the one they were written in — the macros now
bake their expansion ns and the setups honor it.
Also: Boolean/Integer/Double wrapper ctors, primitive TYPE statics
(Integer/TYPE etc.), .reduce on collections (IReduce), and Long/TYPE.
cts baseline 5857 -> 5904 pass, 58 -> 28 errors, 57 baselined namespaces —
the string cluster, some-fn, ifn-qmark, boolean-qmark, and reduce
namespaces are all fully clean. 7 JVM-certified corpus rows; spec entry.
byte/short/int/long/char silently wrapped or passed out-of-range values
through; the JVM range-checks (RT.byteCast family). One checked-cast
helper now carries the ranges: a double range-checks ITSELF before
truncating ((byte 1.1) is 1, (byte 127.000001) throws), NaN casts to 0,
ratios and bigdecs truncate, a non-number is CCE, and the throw carries
the JVM message. float range-checks against Float/MAX_VALUE. The
unchecked-* casts now genuinely wrap and sign-fold ((unchecked-byte 200)
is -56 — the old bit-and lost the sign) with doubles saturating like
Java's conversions; unchecked-long/int are host natives. double/float of
a bigdec convert instead of crashing. The no-single-float residue stays
accepted (SPEC.md).
Also fixes#290: a binary built by the SELF-CONTAINED joltc died with
'variable var-deref is not bound' when a namespace loaded at runtime.
The in-process build compiled flat.ss against a clean copy-environment,
which orphans every top-level define in locations the binary's runtime
eval can't see. It now compiles against the default interaction
environment (defines land in the real symbol cells, same as the legacy
fresh-Chez path) and a generated prologue pre-binds each kernel name the
runtime redefines to its kernel value, so the earliest boot reads match
the legacy path's primitive references. requiring-resolve is implemented
(the issue's dynamic-require pattern), and the release workflow smokes a
runtime require in a built binary.
Cast namespaces byte/short/int/long/char now fully clean; cts baseline
5805 -> 5857 pass, 67 baselined namespaces. 7 JVM-certified corpus rows.
Running the whole rewrite-clj test suite (159 tests) surfaced seven more bugs;
with these it passes 3377/0/0. Each is a general jolt/JVM divergence:
- *out* was pinned to the startup stdout port, so (.write *out* …) escaped a
with-out-str capture (z/print writes via *out*). It now resolves the live
current-output-port, like print/__write, so a redirect is seen.
- nth / assoc past the end of a vector or seq threw a bare Chez error (class
:object). Throw IndexOutOfBoundsException, matching the JVM.
- A number's .toString(radix) ignored the base. Render in the base, lowercase
(rewrite-clj rebuilds 0xff / 0377 / 2r1001 through it).
- A required namespace's own :as aliases leaked into its requirer: the loaded ns
form compiles while (chez-current-ns) is still the requirer, so ce-scan-requires!
registered the loaded ns's aliases under the wrong ns and clobbered a same-named
alias there. Register an (ns NAME …) form's aliases under NAME.
- A quoted collection dropped its metadata; now it keeps USER metadata (drops the
reader's :line/:column/:file), like a Clojure quoted constant.
- enumeration-seq only did (seq e); it now drives a java.util.Enumeration through
hasMoreElements/nextElement, and StringTokenizer implements them.
Regressions: corpus rows (with-out-str/*out*, nth/assoc bounds, toString radix,
quote metadata, enumeration-seq) certified against JVM; a smoke fixture for the
alias leak (a required ns's alias must not leak). tools.reader + rewrite-clj added
to docs/libraries.md. make test green.
data.priority-map's whole suite passes (4/4). It leans on deftype/collection
interop jolt got wrong; four general fixes:
- rseq dispatches to a deftype's clojure.lang.Reversible.rseq method instead of
always demanding a vector/sorted-coll (natives-seq.ss).
- a deftype method declared at two arities from two interfaces now dispatches by
arity: the priority-map has seq[this] (Seqable) and seq[this ascending]
(Sorted), so (.seq pm false) must reach the 2-arg one. find-method-any-protocol
now matches the call's arg count via procedure-arity-mask, and a deftype's own
declared method wins over the generic collection interop in dot-forms.
- (empty x) on a deftype/record with its own empty method uses it rather than
returning {} (jolt.host/jrec-method? gate in clojure.core/empty).
- clojure.lang.Sorted (comparator / entryKey / seqFrom) works on jolt's
sorted-map/set, so subseq/rsubseq run — including the priority-map delegating
.comparator to its backing sorted-map (dot-forms.ss + host-static.ss).
Listed in docs/libraries.md + the site. One re-mint (clojure.core/empty);
everything else runtime. make test green (0 new divergences), shakesmoke
byte-identical.
clojure.core's unchecked-* (and +/-/*/inc/dec under *unchecked-math*) are long
ops that WRAP on overflow; jolt's checked arithmetic is arbitrary-precision and
its unchecked-* were plain non-wrapping (+ x y), diverging from the JVM. Now they
truncate to the low 64 bits as a signed long, matching Clojure:
(unchecked-add 9223372036854775807 1) => -9223372036854775808
(unchecked-multiply 9223372036854775807 …) => 1
- host/chez/seq.ss: jolt-wrap64 + binary jolt-unc{add,sub,mul,inc,dec,neg}2 and
the variadic clojure.core/unchecked-* fns (def-var!'d in natives-seq.ss, where
def-var! is bound). The overlay's plain unchecked-* defns are removed.
- backend lng-ops: unchecked-+/-/* emit the wrapping jolt-unc* helpers (the
raising fx ops can't wrap on Chez's 61-bit fixnums); unchecked-inc/dec too.
- *unchecked-math* is honored: the analyzer reads it (jolt.host/unchecked-math?)
and rewrites +/-/*/inc/dec to their unchecked-* for the rest of a file that
(set!)s it, like the JVM.
- jolt->fx: a ^long value that overflows the 61-bit fixnum range passes through
as an exact integer instead of erroring (a full-width long from wrapping math).
Also adds Long/bitCount / numberOfLeadingZeros / reverse and Math/getExponent /
scalb (test.check's splittable PRNG uses them).
This lets clojure.test.check load and run quick-check on jolt. re-mint (analyzer/
backend/overlay are seed sources). make test green (+6 corpus rows, 0 new
divergences, numeric gate updated), shakesmoke byte-identical.
Running clojure/core.logic's own suite surfaced a batch of general jolt gaps.
None are core.logic-specific; each is a language/host behavior that was wrong or
missing. With these, the core relational engine (unify, run/fresh/conde,
conso/membero/appendo, reification to _0/_1, lcons) runs; the remaining failures
are in core.logic's constraint-logic-programming and finite-domain layers
(tracked separately).
- analyzer: accept the list-member dot form (. target (method args)), sugar for
(. target method args). Re-mint.
- identical? is reference identity (eq?), not value equality. It was aliased to =,
which infinite-loops when a deftype's .equals short-circuits on (identical? this o)
(core.logic's Substitutions) and is wrong for distinct equal collections.
- jrecs use a deftype's declared hashCode/equals/equiv for map/set keying instead
of structural field comparison, so metadata-wrapped keys still match (core.logic
keys substitutions on lvar id, ignoring metadata).
- meta/with-meta dispatch to a deftype's clojure.lang.IObj meta/withMeta methods
when present, so metadata threaded through the type's own assoc/withMeta survives
(previously kept in an identity side-table the reconstructed instances didn't share).
- coll?/seqable? on a deftype require IPersistentCollection (cons) or ISeq (first);
ILookup(valAt)/Indexed(nth)/Counted(count)/Seqable(seq) alone no longer qualify,
matching the JVM.
- syntax-quote resolves a bare symbol to the compile ns's own def before
clojure.core, so a name the ns excluded and redefined (core.logic's == after
:refer-clojure :exclude) qualifies correctly in macro output.
- reader: record literals #ns.Type{...} / #ns.Type[...] expand to the map->/->
factory call.
- structmap API: defstruct/create-struct/struct-map/struct/accessor (map-backed,
insertion-ordered). Re-mint.
- .hashCode on strings/symbols (Java String.hashCode, Symbol Util.hashCombine);
Class.isInstance; java.util.Collection.contains over vector/list/set;
clojure.lang.RT/nextID and clojure.lang.Util hash/hasheq/equiv/identical statics.
corpus.edn: 8 JVM-certified rows. unit.edn: a Counted+Seqable deftype is coll?=false
(was a stale expectation encoding the old behavior).
jolt maps were HAMTs with hash iteration order; Clojure keeps small maps as
PersistentArrayMap (insertion order), converting to PersistentHashMap past a
threshold. Map literals, array-map, assoc, into/transient, merge, zipmap,
select-keys, update-keys/vals, frequencies and group-by now iterate in insertion
order for <=8 entries, matching the JVM. hash-map and >8-entry maps stay hash
order; sets stay hash order.
The pmap record gains an order field (the insertion-order key list, or #f once
hashed); the HAMT still backs the values so equality/hash/lookup are unchanged.
pmap-fold visits an array-mode map last-to-first so the runtime's cons-accumulate
idiom reconstructs insertion order without touching its many call sites, and
hash-mode output stays byte-identical; pmap-fold-fwd visits in order for the few
sites that build a value directly. Transient maps track insertion order and
promote to hash past max(8, source-count), matching TransientArrayMap.
The hash-map native-op retargets to a hash-order builder so (hash-map ...) stays
hash-ordered while {...} literals are ordered; syntax-quote builds maps via the
hash builder (Clojure expands `{...} to apply hash-map). The core overlay map
builders seed from {} instead of (hash-map) to keep order.
Threshold is 8 for any key (the keyword exception in newer Clojure isn't in
1.12.5). honeysql now passes 832/0/0; 19 JVM-certified corpus rows added.
A type-aware audit (~190 collection expressions vs reference Clojure) found four
divergences the corpus missed — value-equality (= [0 1] '(0 1)) hides type and
laziness differences. Fixed, with type-predicate + over-infinite corpus rows that
pin them.
- partition-all [n coll] built vector chunks; JVM chunks are seqs. (The [n step
coll] arity was already correct, as is the partition-all transducer, whose
chunks are vectors in JVM too.) Now builds seq chunks.
- replace always returned a vector (mapv) and was eager; JVM is type-preserving —
a vector maps to a vector, any other seqable to a lazy seq.
- sequence eagerly realized its source (into-xform), so (first (sequence (map inc)
(range))) hung. Rewrote as a transformer iterator: pull one input at a time,
buffer the step outputs, emit lazily, run the completion to flush a stateful
xform. eduction builds on it (lazy, no longer an eager vector).
- mapcat and (apply concat coll-of-colls) hung over an infinite source because
jolt-apply seq->lists the trailing arg and mapcat seq->lists the map result.
Added lazy-concat-seq (lazily flatten a seq of colls); mapcat uses it directly,
and apply special-cases concat (its result is lazy) to route through it.
Docs: a cross-cutting return-type + laziness contract in docs/spec/09-core-library;
SPEC.md notes that = masks type/laziness so they need predicate / over-infinite
rows. EBNF is reader syntax only — unaffected.
Seed change (partition-all/replace/eduction are clojure.core overlay) -> re-mint;
selfhost holds. make test + shakesmoke + buildsmoke green, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
conj/assoc/dissoc/disj/pop/into and empty now thread the receiver's
metadata onto the result, matching Clojure (each op constructs a new
collection with meta() carried forward; coll.empty() is
EMPTY.withMeta(meta())). The metadata side-table is now weak so meta on
intermediate collections is reclaimed with them, and empty-list-t carries
an (unused) field so a metadata-bearing () is a distinct identity from the
shared singleton instead of leaking meta onto every ().
Unblocks metadata-driven walks (aero/integrant): (into (empty form) ...)
now preserves a vector/map/set's metadata, so a postwalk whose outer fn
reads (meta x) sees it.
The 1123-line collection tier is the largest source file. Cut it at two existing
section banners into 20-coll (predicates, printing, hierarchies, pure-over-core
leaves), 21-coll (rand/sort seams, the test runner, fn combinators), and 22-coll
(canonical Clojure ports, transduce/into, JVM-shape stubs). No macros in this tier,
so order is the only constraint; the emit-image manifest lists the three in
sequence. Re-minted seed is identical apart from gensym label renumbering.
