range, map, and filter were fully element-by-element lazy, so
(map f (range 1 50)) realized one element per first/nth where JVM
Clojure realizes a whole 32-element chunk. range is a chunked
LongRange on the JVM and map/filter are chunk-preserving, so the
observable side-effect timing differed.
Following clojure.lang.LongRange, ChunkedCons, ChunkBuffer and
core.clj, this adds a crest field to the cseq record and a
cseq-chunked constructor modeling ChunkedCons (a standalone chunk
pvec, an offset, and the after-chunk seq). The chunk accessors move
to seq.ss next to the representation they read. map/filter/remove
take a chunked branch when the source is chunked, realizing the whole
chunk and chunk-cons'ing it onto a lazy rest, so their output is
itself chunked and chained transforms each batch by 32. Bounded range
is now an eager chunked seq, and the reduce fast path flows through a
ChunkedCons rest. The chunk-buffer/chunk/chunk-cons builder API in
natives-array.ss now produces a real ChunkedCons.
Single-arg (range), multi-coll map, and plain lazy seqs stay
element-by-element, like the JVM.
Adds a lazy / chunking suite to the corpus that observes realization
timing via an atom counter: first over a chunked map realizes 32,
crossing a chunk boundary realizes 49, chained maps batch [32 32],
filter applies the predicate to the whole first block, and a plain
lazy seq still realizes one element at a time. Two cases that
documented the old over-laziness now assert the JVM value of 32 and
were dropped from the allowlist. certify against JVM Clojure 1.12.3
reports 0 new and 0 stale divergences.
Running clojure/core.logic's own suite surfaced a batch of general jolt gaps.
None are core.logic-specific; each is a language/host behavior that was wrong or
missing. With these, the core relational engine (unify, run/fresh/conde,
conso/membero/appendo, reification to _0/_1, lcons) runs; the remaining failures
are in core.logic's constraint-logic-programming and finite-domain layers
(tracked separately).
- analyzer: accept the list-member dot form (. target (method args)), sugar for
(. target method args). Re-mint.
- identical? is reference identity (eq?), not value equality. It was aliased to =,
which infinite-loops when a deftype's .equals short-circuits on (identical? this o)
(core.logic's Substitutions) and is wrong for distinct equal collections.
- jrecs use a deftype's declared hashCode/equals/equiv for map/set keying instead
of structural field comparison, so metadata-wrapped keys still match (core.logic
keys substitutions on lvar id, ignoring metadata).
- meta/with-meta dispatch to a deftype's clojure.lang.IObj meta/withMeta methods
when present, so metadata threaded through the type's own assoc/withMeta survives
(previously kept in an identity side-table the reconstructed instances didn't share).
- coll?/seqable? on a deftype require IPersistentCollection (cons) or ISeq (first);
ILookup(valAt)/Indexed(nth)/Counted(count)/Seqable(seq) alone no longer qualify,
matching the JVM.
- syntax-quote resolves a bare symbol to the compile ns's own def before
clojure.core, so a name the ns excluded and redefined (core.logic's == after
:refer-clojure :exclude) qualifies correctly in macro output.
- reader: record literals #ns.Type{...} / #ns.Type[...] expand to the map->/->
factory call.
- structmap API: defstruct/create-struct/struct-map/struct/accessor (map-backed,
insertion-ordered). Re-mint.
- .hashCode on strings/symbols (Java String.hashCode, Symbol Util.hashCombine);
Class.isInstance; java.util.Collection.contains over vector/list/set;
clojure.lang.RT/nextID and clojure.lang.Util hash/hasheq/equiv/identical statics.
corpus.edn: 8 JVM-certified rows. unit.edn: a Counted+Seqable deftype is coll?=false
(was a stale expectation encoding the old behavior).
jolt maps were HAMTs with hash iteration order; Clojure keeps small maps as
PersistentArrayMap (insertion order), converting to PersistentHashMap past a
threshold. Map literals, array-map, assoc, into/transient, merge, zipmap,
select-keys, update-keys/vals, frequencies and group-by now iterate in insertion
order for <=8 entries, matching the JVM. hash-map and >8-entry maps stay hash
order; sets stay hash order.
The pmap record gains an order field (the insertion-order key list, or #f once
hashed); the HAMT still backs the values so equality/hash/lookup are unchanged.
pmap-fold visits an array-mode map last-to-first so the runtime's cons-accumulate
idiom reconstructs insertion order without touching its many call sites, and
hash-mode output stays byte-identical; pmap-fold-fwd visits in order for the few
sites that build a value directly. Transient maps track insertion order and
promote to hash past max(8, source-count), matching TransientArrayMap.
The hash-map native-op retargets to a hash-order builder so (hash-map ...) stays
hash-ordered while {...} literals are ordered; syntax-quote builds maps via the
hash builder (Clojure expands `{...} to apply hash-map). The core overlay map
builders seed from {} instead of (hash-map) to keep order.
Threshold is 8 for any key (the keyword exception in newer Clojure isn't in
1.12.5). honeysql now passes 832/0/0; 19 JVM-certified corpus rows added.
meta-copy keyed metadata on the SAME cseq/lazyseq object (the else branch), so
(with-meta xs m) mutated the original list in place — Clojure's PersistentList
is immutable and withMeta returns a new list. (with-meta xs {:k xs}) thus built
a self-referential cycle (the list's metadata pointed at the list), which looped
*print-meta* printing forever — the root of aero's meta-preservation hang. Now
copies the cseq/lazyseq node like the other collections. aero suite completes:
58/0/1 (was hanging).
(type x) was jolt's internal taxonomy keyword (:string/:set/:jolt/inst), which
breaks any library dispatching a multimethod on [(type a) (type b)] against
java/clojure.lang classes (e.g. clojure.tools.logging.test's matchers). Make the
PUBLIC clojure.core/type Clojure's (or (:type meta) (class x)).
The taxonomy keyword stays the core model: natives-meta.ss keeps jolt-type and
exposes it as __type-tag, which print-method/print-dup dispatch on (so #uuid/#regex/
records still print). The JVM mapping lives in the java host layer — host-class.ss
defines the public type next to (class …), and a jinst now reports java.util.Date
(was :jolt/inst). So the core emits the taxonomy and the java layer remaps it in one
place. unit.edn's type suite updated to the class names. make test green.
Co-authored-by: Yogthos <yogthos@gmail.com>
Records were a jrec holding an alist of (kw . val) conses: ~113B/node, built
fresh per construction, field reads a list scan. Replace that with a shared
per-type descriptor (tag + field keywords + an eq?-keyed keyword->index table)
plus a flat per-instance value vector and an extension map for any non-field
keys assoc'd on (jolt-nil when there are none). Construction now allocates one
vector instead of a cons chain and a field read is an index lookup. binary-trees
construction allocation drops 2.085GB -> 1.19GB.
That alone barely moved binary-trees wall-time: profiling showed the read loop,
not allocation, dominates, and the read loop's own allocation came from (nil? l)
lowering to (jolt-invoke (var-deref "clojure.core" "nil?") l), which conses its
args every call. Add nil?/some? to the backend native-op table so they inline to
jolt-nil?/jolt-some? (and drop the truthy wrapper, like the other predicates).
check-tree's read loop goes from 1.476GB allocated to zero; binary-trees 18.9x
-> 9.7x vs JVM. The remaining gap is the field-read dispatch chain (jolt-c3mw).
Two JVM divergences fixed along the way, both certified:
- dissoc of a declared field downgrades a record to a plain map (was kept as a
record); an extension key still drops cleanly.
- map->R keeps extension keys (was dropping anything outside the declared basis).
16 new corpus rows pin assoc/dissoc/count/keys/seq/=/hash/extension-field
behavior against JVM Clojure.
Co-authored-by: Yogthos <yogthos@gmail.com>
Four general gaps, shaken out by loading clojure.spec.alpha:
- Special forms were shadowable by a same-named macro. analyze-list
macroexpanded before checking special forms, so a ns that redefs def/and/or
(spec excludes them via :refer-clojure :exclude) made a bare def resolve to
the macro instead of the special form, breaking every defn after. Now a head
in the special-form set is never macroexpanded, matching the reference
macroexpand1 isSpecial check.
- reify dropped all but the last arity of a multi-arity protocol method (spec
reifies (specize* [s]) and (specize* [s _])). The macro keyed methods by name
and overwrote; now it groups arities into one multi-arity fn.
- reify instances did not implement IObj: with-meta threw and (instance?
clojure.lang.IObj r) was false. Every Clojure reify carries metadata. with-meta
now copies the reify to a fresh identity (shared method table) and keys its
meta; instance? IObj/IMeta is true for any reify. This was the registry bug —
spec's with-name returned nil for specs, so get-spec missed.
- (set! (. Class field) val) was rejected. spec toggles
clojure.lang.RT/checkSpecAsserts this way; the analyzer now lowers it to a
jolt.host/set-static-field! call over a mutable-statics table, and a plain
Class/field read consults that table.
Also: .name/.getName on a Namespace and .ns/.sym on a Var (spec's ns-qualify /
->sym). analyzer + reify are seed sources (re-minted). spec.alpha now does
valid?/conform/cat/keys/explain-str/check-asserts. tick.alpha.interval-test still
needs time-literals data readers (separate).
conj/assoc/dissoc/disj/pop/into and empty now thread the receiver's
metadata onto the result, matching Clojure (each op constructs a new
collection with meta() carried forward; coll.empty() is
EMPTY.withMeta(meta())). The metadata side-table is now weak so meta on
intermediate collections is reclaimed with them, and empty-list-t carries
an (unused) field so a metadata-bearing () is a distinct identity from the
shared singleton instead of leaking meta onto every ().
Unblocks metadata-driven walks (aero/integrant): (into (empty form) ...)
now preserves a vector/map/set's metadata, so a postwalk whose outer fn
reads (meta x) sees it.
with-meta/meta only recognized the eager collections, so (with-meta (map-indexed
…) m) — a lazy seq — threw "value does not support metadata". A lazy seq is IObj
on the JVM; add it to the metadatable set (keyed in the side-table by identity,
like cseq/procedure).
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.
Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.
Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.
Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
(type r) returned a symbol user.TyR, so (= (symbol (str (type r))) (type r))
was true; the JVM's type is a Class (not a Symbol) so it's false. jolt models
classes as strings, so a record's type is now its ns-qualified class-name
string — equal to (class r), as on the JVM where type and class coincide for a
record. The symbol-keyed print-method defmethods already fall through to the
default record printing, so they're unaffected. Closes type-of-record.
Rephrase comments that pointed at deleted Janet files (emit.janet, the seed
sources, 'the Janet back end punts ...') to present-tense descriptions of the
Chez behavior. Comment/docstring-only; the self-host fixpoint is unchanged
(comments don't affect the compiled seed).
Delete five files that were Janet-host shims with no Chez path: clojure.java.io
(provided natively by host/chez/io.ss), and jolt.{nrepl,png,interop,shell}
(the janet.* bridge, os/shell, janet.net — none exist on Chez).
jolt-cf1q.6
Implement (type x) on the Chez RT (host/chez/natives-meta.ss). Mirrors the
seed's core-type: the :type metadata wins when present, a record yields its
ns-qualified class-name SYMBOL (user.TyR — no-ns sentinel #f so it = the
overlay's (symbol (str t))), everything else a host-taxonomy keyword
(:number/:string/:vector/:map/:set/:seq/:fn/…). Total by construction — a
non-record value falling through to a crash would read as a divergence, so the
cond covers every value type incl. the host wrappers (atom/volatile/regex/var/
transient/uuid -> :jolt/*, a :jolt/type-tagged map like ex-info -> its tag,
sorted-set -> :jolt/sorted-set, sorted-map -> :map) and a final :object.
Also pin sequential?/seq? on lazy seqs (test/chez/_seqpred.janet): the inc M
seq? re-def-var! fix already covers sequential? transitively (sequential? is
overlay and delegates to seq?), so no code change — the earlier "still broken"
note was wrong, it assumed sequential? was native like seq?.
Prelude corpus parity 2000 -> 2002 (the two type cases), floor raised, 0 new
divergences. Gate: _type 37/37 + _seqpred 22/22 (both vs build/jolt oracle),
emit-test 321/321, full jpm test, conformance 355x3.
Capture a def's reader metadata on the Chez var. The :def emit now lowers a def
with non-empty metadata to def-var-with-meta!, which stores the user meta
(^:private / ^Type tag / docstring -> {:doc}) in an eq side-table keyed by the
var-cell. jolt-meta of a var-cell merges that onto {:ns :name} derived from the
cell, so every var reports {:ns :name} like Clojure with the def-time meta
layered on. (^{:map} metadata on a def name stays uncompilable for the compiler
generally — analyzer rejects it, the Janet back end punts to its interpreter,
which Chez lacks — so it's out of subset, not a meta-capture gap.)
Added natives-meta.ss to the prelude-cache fingerprint. Prelude parity
1969 -> 1972, 0 new divergences; the three var-metadata allowlist entries
(^:private / ^Type tag / docstring) dropped. New focused gate
test/chez/_var_meta.janet.
meta/with-meta resolved to jolt-nil. Chez values don't carry metadata, so
collections use an identity-keyed side-table: with-meta returns a fresh copy of
the value (new identity) and records its meta there, leaving the original
unchanged (immutable-with-meta) and dropping meta on a later copying op. Symbols
carry meta in their existing field; meta on a non-metadatable value is nil.
vary-meta works over these. with-meta on a fn stays fn? (jolt is lenient).
emit.janet carries a quoted symbol's reader metadata (^:foo bar) onto the
emitted jolt-symbol so (meta 'x) sees it; symbol = still ignores meta.
Prelude parity 1701 -> 1723, 0 new divergences. jolt-rkbc.
