The reader now rejects what the JVM reader rejects: a token that starts
like a number but doesn't parse is NumberFormatException (1a, 08, 0x2g,
2r2 — never a symbol); ratio parts are digit runs (1/-1 invalid) with a
zero denominator throwing ArithmeticException; empty ns/name parts are
invalid tokens (:, ::, foo/, /foo) while /, ns//, and :/ stay valid;
duplicate map keys and set elements throw at read; unsupported string
escapes and octal escapes past \377 throw; a stray close delimiter is
'Unmatched delimiter'; \r ends line comments. #inst validates its
calendar fields progressively (leap years included) and #uuid demands
canonical hex. 1-arg symbol splits its ns at the FIRST slash
(Symbol.intern): (symbol "foo/bar/baz") is foo/"bar/baz".
clojure.edn gets its own strict seam (__read-form-edn): auto-resolved
keywords are invalid there, every #_ discarded form validates through the
same :readers/:default pipeline (an unreadable tagged element throws even
when discarded), built-in tags win over :default, M literals construct
BigDecimals, lists satisfy list?, and EOF honors :eof — an opts map
without :eof makes end-of-input an error.
clojure.edn-test.read-string goes 246 pass / 46 fail / 5 errors -> 297/0/0
(fully clean). cts baseline 5904 -> 5955 pass, 23 errors, 56 baselined
namespaces. 9 JVM-certified corpus rows; reader spec section.
The clojure.string case fns and searches now take any Object s through its
toString like the reference's ^CharSequence signatures ((upper-case :kw) is
":KW", (capitalize 1) is "1"); nil throws, and a nil substr in
starts-with?/ends-with? throws. some-fn re-ported with the reference
arities: (some-fn) is an arity error and a no-match result is the last
predicate's own falsy value (false, not nil). ifn? covers multimethods,
promises (which are now invocable — calling one delivers, via a cold-path
invoke-arm registry that costs the hot dispatch nothing), and deftypes
implementing IFn's invoke.
One structural find on the way: defmulti/defmethod deferred inside a fn
body (the deftest pattern) interned/resolved in whatever namespace was
current when they RAN, not the one they were written in — the macros now
bake their expansion ns and the setups honor it.
Also: Boolean/Integer/Double wrapper ctors, primitive TYPE statics
(Integer/TYPE etc.), .reduce on collections (IReduce), and Long/TYPE.
cts baseline 5857 -> 5904 pass, 58 -> 28 errors, 57 baselined namespaces —
the string cluster, some-fn, ifn-qmark, boolean-qmark, and reduce
namespaces are all fully clean. 7 JVM-certified corpus rows; spec entry.
add-watch/remove-watch/set-validator!/get-validator were atom-only; the
atom ctor ignored :meta and :validator; watching a var crashed. Now the
ARef contract is one seam: atoms keep their record slots (hot path
unchanged), every other reference type registers a predicate and stores
watches/validators in identity-keyed side tables, and notifies at its
mutation points. Vars notify on root changes (def on a watched var,
var-set outside a thread binding, alter-var-root — thread-binding sets
don't notify, like the JVM); agents notify per action. The def-var! wrap
costs two weak-table probes per def and does IRef work only on a watched
var.
Ctor options follow ARef: the validator gates the initial value
(IllegalStateException 'Invalid reference state' — also the class for
rejected swap!/reset!), :meta must be a map (else ClassCastException),
nil allowed. meta reads any reference through the identity side-table
(the type-gated fall-through is gone); alter-meta!/reset-meta! work on
non-var references.
Runtime-only (no re-mint). 9 JVM-certified corpus rows; spec entry; cts
baseline 5781 -> 5805 pass, 73 baselined namespaces (the residual error
in the watch namespaces is their STM ref section — refs stay out of
scope).
derive/underive/ancestors/descendants/parents/isa? re-ported from
clojure.core with the argument assertions and throw contracts intact:
derive asserts tag/parent shapes (AssertionError) and throws on redundant
or cyclic derivation; underive/derive on a non-hierarchy value throw at the
parents lookup (the map is called as a function, like the reference);
(descendants h SomeClass) throws UnsupportedOperationException. isa? gains
the reference's supers arm (a relationship derived on a class's super
applies to the class).
The class arms now answer fully through the one class graph: parents of a
class are its direct supers (bases), ancestors are the transitive set
rooted at java.lang.Object for concrete classes (interfaces are marked and
don't root at Object, matching getSuperclass semantics). deftype/defrecord
classes register into the graph at definition — protocol interfaces they
implement appear as supers (JVM-munged ns spelling), records carry the
record interfaces (IRecord/IPersistentMap/... whose closure supplies
Associative/Seqable), bare deftypes carry IType. The type NAME var still
holds the ctor (a jolt-ism); class-key maps it back to the class so
(ancestors TypeName)/(isa? x TypeName) work. canonical-host-tag learned to
NOT canonicalize deftype names through the graph arm (extend-type on a
deftype was registering under the bare segment its values never report).
Five old corpus rows used non-namespaced derive tags that throw on the JVM
too; now namespaced. 8 new JVM-certified corpus rows; spec entries for the
hierarchy family; cts baseline 5730 -> 5781 pass (ancestors/derive/
descendants/parents/underive namespaces fully clean), 74 baselined
namespaces.
are let-bound its template vars, so a var inside quote never substituted:
(are [x] (special-symbol? 'x) if def) tested the literal symbol x twice.
Rebuild are on clojure.template/do-template (postwalk substitution), the
same architecture as upstream, with the same arg-count check.
This un-aborts every suite namespace whose are rows need substitution:
cts baseline moves 5302->5614 pass, 236->192 errors, 88->84 baselined
namespaces. The newly-reachable assertions also surface real divergences
now baselined and filed (edn reader strictness, Boolean ctor).
next.jdbc's own source needs clojure.datafy + clojure.java.data and its tests
need JVM JDBC drivers, so it doesn't run on jolt. Keep clojure.jdbc (via
jolt-lang/db's jdbc.core over FFI SQLite) as the supported JDBC surface; point
migratus at jolt-lang/db.
Document the register-class-supers! seam next to the other host-class hooks:
when a library class belongs to a hierarchy (a custom exception caught as
IOException, a value matching instance? across its supertypes and dispatching a
protocol extended to any of them), declare its supers once and instance?/isa?/
supers/extend-protocol all derive.
Running the whole rewrite-clj test suite (159 tests) surfaced seven more bugs;
with these it passes 3377/0/0. Each is a general jolt/JVM divergence:
- *out* was pinned to the startup stdout port, so (.write *out* …) escaped a
with-out-str capture (z/print writes via *out*). It now resolves the live
current-output-port, like print/__write, so a redirect is seen.
- nth / assoc past the end of a vector or seq threw a bare Chez error (class
:object). Throw IndexOutOfBoundsException, matching the JVM.
- A number's .toString(radix) ignored the base. Render in the base, lowercase
(rewrite-clj rebuilds 0xff / 0377 / 2r1001 through it).
- A required namespace's own :as aliases leaked into its requirer: the loaded ns
form compiles while (chez-current-ns) is still the requirer, so ce-scan-requires!
registered the loaded ns's aliases under the wrong ns and clobbered a same-named
alias there. Register an (ns NAME …) form's aliases under NAME.
- A quoted collection dropped its metadata; now it keeps USER metadata (drops the
reader's :line/:column/:file), like a Clojure quoted constant.
- enumeration-seq only did (seq e); it now drives a java.util.Enumeration through
hasMoreElements/nextElement, and StringTokenizer implements them.
Regressions: corpus rows (with-out-str/*out*, nth/assoc bounds, toString radix,
quote metadata, enumeration-seq) certified against JVM; a smoke fixture for the
alias leak (a required ns's alias must not leak). tools.reader + rewrite-clj added
to docs/libraries.md. make test green.
A :jolt/native spec can now carry a :static archive; `jolt build` links it
into the executable, so the app calls the C code with no shared object on the
target. --dynamic (or :jolt/build {:dynamic-natives true}) keeps the old
runtime load-shared-object behavior; a spec with no :static is unchanged.
The cc link force-loads the archive (-force_load on macOS, --whole-archive on
Linux) and exports the executable's symbols (-rdynamic on Linux) so the baked-in
symbols resolve via (load-shared-object #f) + foreign-procedure at startup. Build
step 1 evaluates the app's foreign-procedure forms in-process, so a static
archive is preloaded there as a throwaway shared object to resolve them.
The distributed self-contained joltc has no external cc/Chez but must build these
apps, so it now bundles the Chez kernel (libkernel.a + scheme.h) and the launcher
source and re-links a custom stub with the archives baked in — needing only a
system cc, no Chez. run/repl skip static-only specs (nothing to load); keep a
:darwin/:linux candidate to use such a lib interpreted.
Adds static-native-smoke (cc path) to ci and a static phase to the joltc
self-build smoke (distributed path).
Match babashka's spelling: the nREPL server now starts with
`bin/joltc --nrepl-server [port]` instead of `bin/joltc nrepl`. Port
parsing and JOLT_NREPL_PORT are unchanged.
Also wire up --help/-h to print usage (previously only the no-arg
invocation did), and fix the usage listing to show the real flag.
Smoke now asserts --help mentions --nrepl-server. Docs updated to match.
algo.monads' writer monad extends a protocol to clojure.lang.IPersistentList,
but jolt's lists only reported ASeq/ISeq in value-host-tags, so writer-m-add
didn't dispatch ("No method writer-m-add"). jolt models every seq as a list (no
distinct LazySeq — (class (map inc xs)) is PersistentList), so a seq now also
reports PersistentList / IPersistentList / IPersistentStack, in value-host-tags
and host-type-set. extend-protocol clojure.lang.IPersistentList then dispatches
on a list.
algo.monads passes its whole suite (11/11) over tools.macro. Listed in docs +
site. Runtime only, no re-mint. make test green (+1 corpus row, 0 new
divergences), shakesmoke byte-identical.
jolt modelled letfn as a special form directly, so (macroexpand-1 '(letfn …))
returned the form unchanged. Clojure's letfn is a macro that expands to letfn*,
and macroexpansion tooling (tools.macro, tools.analyzer) depends on that — its
special-form handlers key on letfn*, not letfn.
Split it the Clojure way:
- letfn* is now the special form (analyzer), taking flat name/fn-form pairs
[name1 fn1 name2 fn2 …] — the letrec :let lowering is unchanged.
- letfn is a macro (00-syntax) turning each (name [params] body*) spec into a
name + (fn name [params] body*) binding, so it expands to letfn*.
So (macroexpand-1 '(letfn [(f [x] x)] (f 1))) now yields
(letfn* [f (fn f [x] x)] (f 1)), and clojure.tools.macro passes its whole suite
(macrolet / symbol-macrolet / mexpand-all). Listed in docs + site.
make test green (+1 corpus row, 0 new divergences), shakesmoke byte-identical.
One re-mint (analyzer + the letfn macro); selfhost holds.
data.priority-map's whole suite passes (4/4). It leans on deftype/collection
interop jolt got wrong; four general fixes:
- rseq dispatches to a deftype's clojure.lang.Reversible.rseq method instead of
always demanding a vector/sorted-coll (natives-seq.ss).
- a deftype method declared at two arities from two interfaces now dispatches by
arity: the priority-map has seq[this] (Seqable) and seq[this ascending]
(Sorted), so (.seq pm false) must reach the 2-arg one. find-method-any-protocol
now matches the call's arg count via procedure-arity-mask, and a deftype's own
declared method wins over the generic collection interop in dot-forms.
- (empty x) on a deftype/record with its own empty method uses it rather than
returning {} (jolt.host/jrec-method? gate in clojure.core/empty).
- clojure.lang.Sorted (comparator / entryKey / seqFrom) works on jolt's
sorted-map/set, so subseq/rsubseq run — including the priority-map delegating
.comparator to its backing sorted-map (dot-forms.ss + host-static.ss).
Listed in docs/libraries.md + the site. One re-mint (clojure.core/empty);
everything else runtime. make test green (0 new divergences), shakesmoke
byte-identical.
Completes the JVM long-compatibility gap so clojure.test.check (and the
property-based suites built on it, e.g. data.codec) run on jolt.
A ^long is 64-bit but a Chez fixnum is only 61-bit, so the backend's fast fx
comparison / quot / min / max / inc / dec ops raised on a full-width long (one
from the PRNG or wrapping arithmetic). They now go through the jolt-l* macros
(host/chez/seq.ss): the fx fast path when the operands ARE fixnums, the generic
op otherwise — so e.g. ((fn [^long a ^long b] (< a b)) Long/MAX 1) is false, not
an error. Arithmetic +/-/* keep the raw fx ops (under *unchecked-math* they're
already the wrapping unchecked-*).
Also fixes unsigned-bit-shift-right: it was an arithmetic (sign-propagating)
shift, now a logical shift over the 64-bit two's-complement window, so
(unsigned-bit-shift-right -1 1) is 2^63-1 like the JVM.
Result: test.check 1.1.3 loads and runs (generators, quick-check, shrinking);
data.codec's base64 property suite passes (12/12 defspecs; the 2 deftests check
clojure.lang.IFn$OLLOL, a JVM primitive-fn interface, N/A). Both added to
docs/libraries.md + the site.
re-mint (backend/seed). make test green (+3 corpus rows, 0 new divergences,
numeric gate updated to the jolt-l* ops), shakesmoke byte-identical.
clojure.data.csv runs its whole suite on jolt (4/4 reading/writing/eof/line-
endings). Three general gaps fixed, all runtime, no re-mint, JVM-certified:
- The prefix-list form of :require/:use — (:require (clojure [string :as str]))
means clojure.string :as str — now expands (loader.ss). It silently failed
before, trying to load a "clojure" namespace.
- extend-protocol to java.io.Reader / Writer / StringReader / PushbackReader now
dispatches: those reader/writer host tags carry the right class names in
value-host-tags AND are in host-type-set, so extend-protocol registers under
the canonical tag instead of a local ns tag (records.ss). data.csv's
Read-CSV-From protocol extends to String / Reader / PushbackReader.
- (str StringWriter) returns its accumulated content (register-str-render for the
"writer" jhost), not the opaque host object — data.csv writes CSV to one and
reads it back.
Listed in docs/libraries.md + the site.
make test green (+2 corpus rows, 0 new divergences), shakesmoke byte-identical.
clojure.zip was missing xml-zip — a zipper over xml {:tag :content} elements,
which clojure.data.zip and any xml-zipper code needs. Added (runtime, loaded on
require). clojure.data.zip's whole xml suite (9/9) then passes, once XML parsing
is provided: clojure.xml/parse now ships in jolt-lang/xml over its
javax.xml.stream pull parser (committed there).
Listed in docs/libraries.md + the site.
clojure.core.contracts (over core.unify) now runs its whole suite on jolt —
14/14 across contracts/constraints/with-constraints/provide tests. Two general
gaps fixed:
- Symbol and Keyword now report IFn (and Fn/Runnable/Callable) in the modeled
class hierarchy, so a (class x)-dispatched multimethod with an IFn method
matches a symbol or keyword, like the JVM (both implement IFn — they're
callable). core.contracts' funcify* dispatches on (class constraint) and a
bare predicate symbol must hit the IFn arm. Runtime, no re-mint.
- A live Var value spliced into a form by a macro (defcurry-from resolves a var
and emits (~v l r)) now compiles: analyze treats a var-cell form as a
:the-var reference by ns+name, the same node as (var ns/name), mirroring the
existing spliced-namespace (~*ns*) case. analyzer.clj + host-contract.ss,
re-mint (prelude stays byte-identical; only the analyzer image changes).
Listed in docs/libraries.md + the site.
make test green (+2 corpus rows, 0 new divergences), shakesmoke byte-identical.
Adds clojure.core.async's higher-level dataflow API as a Clojure overlay
(stdlib/clojure/core/async.clj) over jolt's native channel primitives, plus
clojure.core.async.lab. The native layer (host/chez/java/async.ss) gains
offer!/poll!, put specs and :priority/:default in alts!, a transducer
ex-handler arg to chan, unblocking-buffer?, promise-buffer, and on-caller?
handling for put!/take!. The overlay covers alts!/pipe/pipeline/split/
reduce/transduce/into/take/mult/mix/pub-sub/map/merge/onto-chan/to-chan and
the deprecated map</map>/filter>/... family (rewritten as go-loops since the
JVM versions reify the impl handler protocol jolt doesn't expose).
Loading: the native primitives pre-seed clojure.core.async, so the loader now
drops it from the loaded set and a require pulls the overlay from the source
roots like clojure.test (AOT-bundled into built binaries).
Running clojure/core.async's own suite shook out two general bugs:
- :refer with a list form, (:require [ns :refer (a b c)]), dropped the names
(only the vector form was handled) — chez-register-spec! now accepts both.
- (range 0) / (range 5 5) returned nil instead of the empty seq () — empty
ranges now match Clojure, so (= () (range 0)) holds.
Suite: async_test 15/20, pipeline_test 7/7, timers_test 2/2, lab_test 2/2.
The five non-passing async_test cases all assert JVM go-machine limitations
jolt's thread-based model is a superset of (the 1024 pending-op cap, parking
ops that must throw outside a go block, expanding-transducer buffer
backpressure) or dispatch-thread identity, not data semantics.
make test green (0 new divergences, +4 range corpus rows), shakesmoke
byte-identical.
Both printers (jolt-pr-str, jolt-pr-readable) now thread a print depth and
read the two limit vars. *print-length* truncates each collection to N
elements + "...", walking seqs lazily so an infinite seq prints under the
limit without realizing it. *print-level* renders a collection at depth >=
the level as "#". The reader consults *default-data-reader-fn* for an
unregistered #tag before falling back (tagged form on the data seam, throw
on the edn seam). All three interned with nil defaults.
Spec coverage dashboard had 6 missing-portable and 24 dynamic-var entries. The
portable ones are now implemented (missing-portable -> 0, dynamic-var -> 14):
- Stateful matcher: re-matcher now returns a real mutable Matcher; re-find over
it steps through matches and re-groups returns the last match's groups (was an
inert tagged map). Closes re-groups.
- letfn is interned as a clojure.core var so (resolve 'letfn) matches the JVM. It
stays a special form (the value is never invoked, not marked a macro).
- *1 *2 *3 *e interned (nil outside a REPL).
- Portable dynamic vars whose default already matches jolt's behaviour:
*read-eval* *print-dup* *print-namespace-maps* *flush-on-newline*
*compile-files* *math-context* *command-line-args* *file*.
The remaining 14 dynamic-var entries are host-internal (compile-path,
compiler-options, fn-loader, reader-resolver, repl, source-path, ...) or deferred
pending printer/reader support (*print-length* *print-level*
*default-data-reader-fn*). Corpus rows added for each closed gap; coverage.md
regenerated.
irregex rejected two patterns the JVM accepts, which blocked library loads:
- [\w-_] errored with bad char-set because a - after a shorthand class was
read as a range start. Java reads it as a literal hyphen. Preprocess the
pattern to escape such a dash.
- (X+)* errored with duplicate repetition because sre-repeater? recurses
through submatch, treating a quantified group like a dangling a**. Override
it to a bare leading * / + check, matching the JVM (which only rejects the
dangling case).
Both in regex.ss (runtime). Unblocks cuerdas (was load-fail, now 292 passing)
and aws-api config-test. Also documents the host/chez/java source-layering rule
in host-interop.md.
jolt-l8so
A type-aware audit (~190 collection expressions vs reference Clojure) found four
divergences the corpus missed — value-equality (= [0 1] '(0 1)) hides type and
laziness differences. Fixed, with type-predicate + over-infinite corpus rows that
pin them.
- partition-all [n coll] built vector chunks; JVM chunks are seqs. (The [n step
coll] arity was already correct, as is the partition-all transducer, whose
chunks are vectors in JVM too.) Now builds seq chunks.
- replace always returned a vector (mapv) and was eager; JVM is type-preserving —
a vector maps to a vector, any other seqable to a lazy seq.
- sequence eagerly realized its source (into-xform), so (first (sequence (map inc)
(range))) hung. Rewrote as a transformer iterator: pull one input at a time,
buffer the step outputs, emit lazily, run the completion to flush a stateful
xform. eduction builds on it (lazy, no longer an eager vector).
- mapcat and (apply concat coll-of-colls) hung over an infinite source because
jolt-apply seq->lists the trailing arg and mapcat seq->lists the map result.
Added lazy-concat-seq (lazily flatten a seq of colls); mapcat uses it directly,
and apply special-cases concat (its result is lazy) to route through it.
Docs: a cross-cutting return-type + laziness contract in docs/spec/09-core-library;
SPEC.md notes that = masks type/laziness so they need predicate / over-infinite
rows. EBNF is reader syntax only — unaffected.
Seed change (partition-all/replace/eduction are clojure.core overlay) -> re-mint;
selfhost holds. make test + shakesmoke + buildsmoke green, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
Shaking out clojure.core.memoize (207 assertions, 0 fail) cleared several
general gaps:
- deref/@ on a deftype or reify implementing clojure.lang.IDeref dispatches to
its deref method (RetryingDelay / make-derefable).
- deftype mutable fields (^:unsynchronized-mutable / ^:volatile-mutable) are
read live: a set! within a method is observed by a later read in the same
invocation, not the entry-time capture. Needed for double-checked locking.
Immutable fields stay let-bound. Field reads rewrite to (.-field inst) with
lexical-shadow tracking.
- def metadata values are evaluated, like Clojure: ^{:k (f)} stores (f)'s
result and ^{:af some-fn} the fn. :tag stays a literal hint.
- try dispatches catch clauses by class in order via the exception supertype
hierarchy; a non-matching value re-throws, an untyped host condition is caught
by a RuntimeException/Exception/Throwable clause. Previously the last clause
won and the class was ignored.
- locking takes a real per-object monitor (recursive Chez mutex) now that
futures/agents/threads share one heap; it was a no-op.
- supers/ancestors reflect a small modeled JVM interface hierarchy, so
(ancestors (class f)) yields Runnable/Callable (core.memoize's arg check).
- AssertionError / Error constructors.
JOLT_FEATURES is gone from the docs: it isn't read anywhere on Chez, and the
reader already includes :clj in its default feature set. RFC 0002's
{:jolt :default} design was reverted in the reader; docs now match the code.
Raises the SCI floor 205 -> 210.
Finishes core.match — its full test suite (115/115) now passes, including the
two patterns the earlier work left out:
- Regex-literal patterns. A #"…" now reads as a regex VALUE (Clojure parity: the
reader constructs the Pattern, so a macro receives a regex, not jolt's tagged
form), and the analyzer compiles a regex value to the same :regex IR leaf via
its source. emit-quoted handles a quoted regex; a regex value carries the
java.util.regex.Pattern host tag so extend-protocol/instance? dispatch on it.
- Primitive-array patterns. A ^Type hint's :tag is now the SYMBOL (e.g. `ints`),
matching the JVM, so core.match's array-tag lookup engages the array
specialization (alength/aget). jolt's :tag consumers already tolerate a symbol
(hc-cell-num-ret normalizes; tag->nkind/def-meta handle both).
Also: a library-conformance directive in CLAUDE.md, and the supported-libraries
list (docs + site) simplified to one-line entries — a listed library is assumed
to work fully, so no tallies or feature enumerations. core.match + transit-jolt
added to the list.
Seed change (reader/backend/30-macros) -> re-minted; the rest runtime. JVM-
certified corpus rows; the stale `symbol hint -> :tag` divergence is dropped from
the allowlist (jolt now matches the JVM). make test + shakesmoke green.
Expand java.io so libraries that touch the filesystem work unchanged.
File: the full method surface — length, lastModified, can{Read,Write,Execute},
isHidden, list, mkdir(s), delete, createNewFile, renameTo, getParentFile,
get{Absolute,Canonical}File, compareTo/equals/hashCode — plus the statics
separator / pathSeparator / createTempFile / listRoots. A File now keeps the
path as given (new File("rel").getPath() is "rel", .isAbsolute false); a
relative path resolves against JOLT_PWD only when the filesystem is touched,
matching the JVM. slurp/spit and the dir helpers go through the same
resolution, fixing a spit-vs-slurp inconsistency.
Streams (host/chez/io-streams.ss) — each a jhost wrapping a Chez port, so
buffering, EOF and binary<->char transcoding come from Chez:
- FileInputStream / FileOutputStream / ByteArrayInputStream /
ByteArrayOutputStream / BufferedInputStream / BufferedOutputStream
- FileReader / FileWriter / InputStreamReader / OutputStreamWriter /
BufferedReader / BufferedWriter
Buffered* return the wrapped stream (Chez ports are already buffered).
clojure.java.io: input-stream/output-stream now yield real byte streams (were
aliased to the char reader/writer); added copy (byte-exact for byte sources),
make-parents, delete-file. with-open also closes file-writer/port-writer/
print-writer (a pre-existing gap).
All runtime shims, no re-mint. 15 JVM-certified corpus rows; make test +
shakesmoke green.
Shaking out tick's api and alpha.interval suites (api 353->359, interval
0->103 passing) cleared a set of general gaps:
- Named-zone DST. Zones resolved to a fixed representative offset, so
America/New_York in August read -05:00 not -04:00. Add US/EU DST rules
(compact transition-date math) and make instant<->zoned, the zone rules'
getOffset, and the zoned equality arm DST-aware.
- Nanosecond zoned/offset times. Instant is nanos but atZone/atOffset/
toInstant/withZoneSameInstant and Instant/parse went through epoch-ms,
truncating sub-ms. Route them through nanos.
- Locale month/day names. A formatter dropped its Locale; carry it and add
French names so MMM under Locale/FRENCH renders "mai".
- Callable records. A defrecord implementing clojure.lang.IFn (tick's
GeneralRelation) is now invokable: jolt-invoke dispatches to its inline
invoke method. Also give collections the Iterable host tag so a protocol
extended to Iterable matches vectors/seqs.
- Imported class short names. (:import [java.time ZonedDateTime]) then
(. ZonedDateTime parse s) resolved to nil; an otherwise-unresolved bare
Capitalized name that's a registered host class now resolves as a class.
- Data readers. A project's data_readers.{clj,cljc} is loaded into
*data-readers* (reader namespaces required eagerly); registered #tag
literals in source rewrite to (reader-fn 'form). clojure.core/read-string
now applies #inst/#uuid/#"regex" and *data-readers* like Clojure.
- Duration/between accepts zoned/offset date-times.
All runtime shims, no re-mint. docs/libraries.md: tick full pass + aero.
The EBNF and reader S7 already specified ::kw auto-resolution — the
implementation was out of spec, now aligned. S7 noted unknown ::alias/k MUST be a
read error; jolt is lenient (reads :alias/k), so record that as a deviation.
Corpus gains JVM-certified rows for ::kw resolution, clojure.edn :default
receiving a symbol tag, and with-meta on a lazy seq.
The source loader sets the current ns and registers :as aliases per file. The
build flattened every app namespace into one image with no such markers, so all
app forms ran under the last-set ns ("user"). Two breakages followed, both only
in a built binary:
- defmulti/defmethod resolve their target var through chez-current-ns, so they
registered the multifn under "user" while compiled var-derefs used the baked
ns — the multifn the app saw was uninitialized ("not a fn nil" on dispatch).
- a quoted alias-qualified symbol (a (defmethod ig/foo …) on an aliased multifn)
resolves its ns through chez-resolve-alias, but the stripped (ns …) form left
the alias table empty, so it landed in ns "ig".
bld-ns-prelude now emits (set-chez-ns! ns) plus chez-register-alias! for each
ns's :as aliases before that ns's forms, in both the normal and tree-shake emit
paths. The build-app fixture gains a :default multimethod and an aliased cross-ns
defmethod so buildsmoke covers both across all build modes.
Two small clarity moves from the review:
- dynamic-vars.ss -> dynamic-var-defaults.ss. It holds the default VALUES of a few
core dynamic vars (*clojure-version*, *assert*, …); the near-identical dyn-binding.ss
holds the binding-stack machinery. The names were easy to confuse.
- Pull the ~60-line %-format engine out of the natives-misc.ss grab-bag into
natives-format.ss (its ->long/pad-left/fmt-float helpers were local to it).
rt.ss loads + MODULES.md updated. Runtime .ss, no re-mint; make test green, format +
the dynamic vars verified.
The trio split on a fine axis (registry core / statics / object classes) but the
names didn't say so — 'static-statics'/'static-objects' and headers that read
'Continues X'. Rename:
host-static-statics.ss -> host-static-methods.ss (Class/member statics + fields)
host-static-objects.ss -> host-static-classes.ss (instantiable object classes)
host-static.ss stays the registry core. Headers rewritten to state each file's role
and what it covers instead of chaining. rt.ss loads + the one comment reference +
MODULES.md updated. No code moved; runtime .ss, make test green.
Navigability groundwork from the architecture review — zero behaviour change.
- docs/MODULES.md: the repo map. Area -> directory -> key files -> re-mint?, plus
per-feature touch points (tree-shaking, direct-linking, numeric fl/fx, inlining,
multimethods, deps) and where a given clojure.core fn lives. Answers "where does
X live / what's related to Y" in one read.
- docs/rfc/README.md: index the 7 RFCs; flags RFC 0007's stale "no code yet" status
(direct-linking + tree-shaking shipped) and the undocumented inlining/numeric work.
- CLAUDE.md: document the var-deref calling convention (public defns reached from the
.ss runtime by string lookup aren't dead), the def-var! native pattern, and the
overlay shadowing rule; point at MODULES.md.
- REFACTOR_PLAN.md: the prioritized, risk-tiered plan (working doc for this branch).
README + tools-deps.md cover --tree-shake and --direct-link: what tree-shaking does
(whole-program reachability over app + libraries + clojure.core, drop unreachable,
drop the compiler for no-eval apps), and why it bails to keep-all when reachable code
resolves vars by name at runtime (eval/resolve/ns-resolve/...), with the diagnostic
output and how to make an app shakeable. Notes the Stalin soundness model.
Release builds can legitimately want runtime dynamism (redefinition, eval,
load-string), so closed-world direct-linking shouldn't be forced on them. Gate it
behind an explicit --direct-link flag (or deps.edn :jolt/build {:direct-link
true}); off by default in every mode, including release and --opt.
build-binary takes an explicit direct-link? arg instead of deriving it from the
mode. build-smoke now covers the --direct-link path and asserts the cross-ns call
actually lowers to a jv$ binding; default release stays dynamically linked.
Build output landed in the CLI's cwd (the jolt repo, since bin/joltc cd's
there), not the project — so a bare -o path or the default binary appeared
in the wrong place. Resolve output against JOLT_PWD, and default it cargo-
style under the project's target/: target/release for release/--opt,
target/debug for --dev, named after the project dir. The <name>.build scratch
dir sits beside the binary, so it lands under the same target dir. -o is
honored — absolute as-is, relative against the project.
A built binary dropped its deps.edn :jolt/native declarations and its
resource roots, so an FFI+resources app (ring-app) failed at runtime:
sockets/sqlite gave 'no entry for socket' and io/resource returned nil.
The buildsmoke fixture is pure compute, so neither path was exercised.
The launcher now loads required + :process native libs before the app's
top-level forms (a library's defcfn resolves its foreign-procedure symbols
at top-level eval during startup, so the libs must be loaded first);
optional libs load in the scheme-start launcher, where a missing lib is
caught rather than aborting the heap build.
deps.edn :jolt/build {:embed [dirs]} bakes those dirs' files into the heap
(register-embedded-resource! at heap build), so io/resource serves them with
no files on disk. Non-embedded resources resolve at runtime against JOLT_PWD,
and io/file reads (e.g. config.edn) stay external.
build-binary now takes the encoded natives, embed dirs, and project paths
from cmd-build; deps/resolve-project surfaces them. Buildsmoke fixture grows
an embedded resource + a :process native to cover both paths.
Restores the standalone-binary capability the Janet host had. `bin/joltc build
-m NS -o OUT` AOT-compiles an app into a single self-contained executable — the
whole runtime, clojure.core, stdlib and compiler embedded, no Chez install or
jolt source needed at runtime.
Pipeline (host/chez/build.ss, host primitive jolt.host/build-binary driven by
jolt.main's build command): resolve deps, load the entry namespace recording the
app namespaces in dependency order, re-emit each to Scheme, textually inline the
cli.ss runtime load sequence into one flat source + the app + a launcher, then
compile-file -> make-boot-file -> embed the boot as C bytes -> cc-link against
libkernel.a.
Two non-obvious bits: the compile pass runs in a fresh Chez, not the loaded
runtime (regex.ss shadows top-level `error`, which otherwise bakes a broken
reference into the boot); and the launcher installs scheme-start rather than
running -main at top level, since boot top-level forms execute during heap build
before argv is set, so args only reach -main through scheme-start.
Loader: a require of an in-memory namespace with no source file now no-ops, so
AOT'd app namespaces satisfy require in a built binary.
Mode flags (--opt/--dev, default release) are plumbed; the optimization passes
they gate come in a later stage. RFC 0007 has the design. Gated by `make
buildsmoke`.
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.
Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.
Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.
Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
The line REPL was broken (read-line called nil — the __stdin-read-line host seam
the clojure.core *in* reader drives was never implemented on Chez) and didn't
load the project, so (require '[some.lib]) failed. Now:
- __stdin-read-line reads a line from stdin (get-line); read-line / read / the
REPL work.
- repl resolves the project first (deps on the roots, native libs loaded), so
libraries are available — same context a run gets.
- jolt.nrepl: a jolt-native nREPL server (bencode over a loopback jolt.ffi
socket) speaking the real protocol — clone / describe / eval / load-file /
close, with stdout capture, :ns-scoped eval (in-ns; binding *ns* doesn't drive
load-string resolution here), and real error text. 'joltc nrepl [port]' applies
the project then serves; writes .nrepl-port. Editors (CIDER/Calva/Cursive)
connect and develop live; project libraries load in the session.
- ex-message returns nil for raw Chez conditions, so jolt.host/condition-message
exposes the condition text; the REPL and nREPL surface it instead of an opaque
#<compound condition>.
Why native, not real nREPL: nrepl.server is welded to java.util.concurrent
executors, two compiled Java helper classes, a DynamicClassLoader, Compiler
internals and a JVMTI agent — not faithfully shimmable. The wire protocol, which
is what clients depend on, is small and implemented directly.
Runtime .ss + jolt-core, no re-mint. Full gate green.
Bring the docs in line with the actual implementation now that Chez is the sole
substrate.
Deleted the migration/spike/handoff artifacts that only documented the Janet
era or the port effort: the port plan, phase-0 and foundational-runtime spike
writeups (+ the stray root-level copy), the self-hosting design notes, the
architecture-refactor plan, and spike/chez/RESULTS.md.
Rewrote the current reference docs against the Chez facts: building-and-deps and
tools-deps (no jpm/build step — bin/joltc off the checked-in seed, deps via
jolt.deps into ~/.jolt/gitlibs), libraries (SQLite is built-in jdbc.core over
libsqlite3, not a Janet driver), the conformance/spec test-flow docs (the Chez
corpus runner + certify, no .janet harnesses), and the transient / type-hint /
seed-overlay design notes (Chez representations: mutable transients, flat
copy-on-write vectors, HAMT maps, the seed/overlay twin). Fixed the README
collections line (vectors aren't 32-way tries) and added the ffi/transient gate
targets. rfc 0001's numerics open-question is resolved (the Scheme tower).
Renamed the built-in HTTP adapter to jolt.http.server only (dropped the
ring-janet.adapter alias — a Janet-era name).
Bring the formal definition in line with this session's language work:
- grammar.ebnf: numbers are a real tower (exact integer / Ratio / double); the M
suffix reads a real BigDecimal, N an exact integer (drop the stale Janet note).
- 02-reader S5: M is a real java.math.BigDecimal with scale-insensitive equality.
- 03-special-forms: document the read -> macroexpand -> analyze order (macros
expand before special-form dispatch); special-form heads are not shadowable but
macros are and value-position locals may be named like a special; set! on a var
sets the innermost binding (else root); letfn is a primitive with letrec*
semantics.
Revise the epic's direction from a minimal Janet shim to ripping Janet
out entirely — Chez becomes the sole substrate. The missing spine: the
compiler pipeline itself only runs on Janet today (the analyzer executes
on the Janet host; the IR->Scheme emitter is host/chez/emit.janet). Phase
3 is re-scoped to self-host the compiler on Chez (emitter -> Clojure
jolt.backend-scheme, reader -> jolt-core, compile-from-source bootstrap
fixpoint). Phase 5 becomes a hard delete of both src/jolt/*.janet and
host/chez/*.janet. Sequencing: core parity first, then self-host, then
delete.
