A ^double/^long param hint (or a float literal) now drives Chez flonum/fixnum
ops instead of generic arithmetic — JVM-style primitive hints, available in every
build and at -e (not gated on direct-linking or whole-program inference).
New pass jolt.passes.numeric: a local forward type-flow seeded from ^double/^long
fn-param hints (analyzer attaches :nhints per arity) and float literals,
propagated through let inits / arithmetic / if / do. It tags an arithmetic invoke
:num-kind :double|:long when every operand is that kind (an integer literal is a
wildcard, coerced to a flonum in a double op). The back end lowers a tagged node
to fl+/fl-/fl*/fl//fl<?/... or fx+/fx*/fx1+/fxquotient/... (unchecked-add etc.
join the fixnum path; == too). Runs last in run-passes, both branches.
Soundness: :long is seeded ONLY from an explicit ^long hint, never a bare integer
literal, so un-hinted integer code keeps jolt's arbitrary-precision numbers — no
fixnum-overflow surprise, no corpus divergence. :double comes from ^double hints
and float literals (flonum arithmetic is always flonum, matching the generic
result). A ^long hint is a promise the value is a fixnum: fx+ raises on overflow,
like a JVM fixed-width long.
Numeric-hinted params coerce at fn entry (exact->inexact / jolt->fx), the way the
JVM coerces a primitive parameter — so the body's fl*/fx* ops can rely on the
type even when a caller passes an exact int (e.g. Chez's (* 0 1.0) => exact 0).
Round 1 specializes hinted straight-line / fn-body arithmetic. fl-ops are ~4x
generic in a tight Chez loop, but realizing that on loop-carried accumulators
needs loop-var typing — round 2. Sound foundation, gated by test/chez/numeric-test.ss.
A release/optimized `jolt build` is a closed world: every app def is final, so
an app->app call can bind to the def's Scheme binding directly instead of going
through (jolt-invoke (var-deref ns name)).
The emitter gains a direct-link mode (off for the seed mint, runtime -e/repl, and
dev builds). With it on, a top-level app def also emits a binding jv$<ns>$<name>
that def-var! aliases; an app->app call or value-ref to a name already emitted in
the unit lowers to that binding, skipping both the var-table lookup and the
generic IFn dispatch. ^:dynamic/^:redef defs and nested defs (a defonce's inner
def) opt out and stay indirect. Off direct-link mode, emit-top-form is exactly
emit, so the seed and runtime eval are byte-unchanged (selfhost holds).
build.ss turns it on for release + optimized; the defined-set accumulates across
the dependency-ordered namespaces so a dep's defs are linkable by the time the
entry that calls them is emitted. App->core calls stay indirect for now (core is
the baked seed); that's a later stage.
~1.74x on a hot cross-namespace call loop (26.5s -> 15.2s).
A built binary dropped its deps.edn :jolt/native declarations and its
resource roots, so an FFI+resources app (ring-app) failed at runtime:
sockets/sqlite gave 'no entry for socket' and io/resource returned nil.
The buildsmoke fixture is pure compute, so neither path was exercised.
The launcher now loads required + :process native libs before the app's
top-level forms (a library's defcfn resolves its foreign-procedure symbols
at top-level eval during startup, so the libs must be loaded first);
optional libs load in the scheme-start launcher, where a missing lib is
caught rather than aborting the heap build.
deps.edn :jolt/build {:embed [dirs]} bakes those dirs' files into the heap
(register-embedded-resource! at heap build), so io/resource serves them with
no files on disk. Non-embedded resources resolve at runtime against JOLT_PWD,
and io/file reads (e.g. config.edn) stay external.
build-binary now takes the encoded natives, embed dirs, and project paths
from cmd-build; deps/resolve-project surfaces them. Buildsmoke fixture grows
an embedded resource + a :process native to cover both paths.
jolt could call C (foreign-fn -> foreign-procedure) but C could not call back
into jolt, which GTK signals (and any callback-taking C API) require. Add the
inverse: jolt.ffi/foreign-callable wraps a jolt fn as a C-callable function
pointer, mirroring the foreign-fn pipeline.
A new jolt.ffi/__ccallable special form carries the fn as a child expression
(analyzed + walked by the passes; ir.clj gains an :ffi-callable arm in both
child walks) plus literal arg/ret type keywords. The back end lowers it to a
locked Chez foreign-callable and returns its entry-point address as a jolt
pointer; host/chez/ffi.ss registers the code object so the collector keeps it,
and free-callable unlocks it. :collect-safe emits the convention that
reactivates the thread on entry, for callbacks fired while it is parked in a
:blocking call (a GTK main loop).
Test: ffi-binding-test.ss sorts an int array through libc qsort with a jolt
comparator (C -> jolt -> C). Re-minted seed.
Thread/yield was a no-op and Thread/interrupted always returned false. Now:
- yield calls libc sched_yield (resolved once via the process symbols), so a
spin loop relinquishes the CPU. Falls back to a zero-length park if the symbol
can't be resolved.
- each OS thread carries an interrupt flag (a box, thread-local). currentThread
returns a handle wrapping the calling thread's flag, so .interrupt from another
thread sets the target's flag. .isInterrupted reads without clearing; the static
Thread/interrupted reads and clears — JVM semantics.
Consolidates the Thread surface: currentThread + the instance methods live in
io.ss (where the handle and its classloader are built), the flag box + yield +
the interrupted static in host-static.ss. Unit cases cover yield, the read/clear
split, and a cross-thread interrupt over a future.
Restores the standalone-binary capability the Janet host had. `bin/joltc build
-m NS -o OUT` AOT-compiles an app into a single self-contained executable — the
whole runtime, clojure.core, stdlib and compiler embedded, no Chez install or
jolt source needed at runtime.
Pipeline (host/chez/build.ss, host primitive jolt.host/build-binary driven by
jolt.main's build command): resolve deps, load the entry namespace recording the
app namespaces in dependency order, re-emit each to Scheme, textually inline the
cli.ss runtime load sequence into one flat source + the app + a launcher, then
compile-file -> make-boot-file -> embed the boot as C bytes -> cc-link against
libkernel.a.
Two non-obvious bits: the compile pass runs in a fresh Chez, not the loaded
runtime (regex.ss shadows top-level `error`, which otherwise bakes a broken
reference into the boot); and the launcher installs scheme-start rather than
running -main at top level, since boot top-level forms execute during heap build
before argv is set, so args only reach -main through scheme-start.
Loader: a require of an in-memory namespace with no source file now no-ops, so
AOT'd app namespaces satisfy require in a built binary.
Mode flags (--opt/--dev, default release) are plumbed; the optimization passes
they gate come in a later stage. RFC 0007 has the design. Gated by `make
buildsmoke`.
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.
Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.
Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.
Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
Two nREPL divergences the library shakeout surfaced — both have a real host
mechanism on Chez.
Interrupt (jolt-amzy): Chez's engine timer (set-timer + thread-local
timer-interrupt-handler) is polled at procedure-call / loop back-edges, so a
running computation — even a tight loop — can be aborted from another thread.
concurrency.ss adds jolt.host/{make-interrupt, interrupt!, run-interruptible}: an
interrupt token is a shared box; run-interruptible arms a periodic timer whose
handler escapes (call/cc) when the token is set, throwing {:jolt/interrupted true}.
The eval thread is reused, not abandoned. (A thread blocked in a __collect_safe
foreign call only sees it on return — like the JVM not killing native code.)
Thread-local *ns* (jolt-6rld): chez-current-ns is now a Chez thread-parameter, so
each session worker / future has its own current ns (vars stay global, only the
pointer is per-thread). *ns* reads derive from it (dyn-binding.ss), and a bound
*ns* drives chez-current-ns — so (binding [*ns* the-ns] (load-string code))
resolves against the-ns, and concurrent in-ns across threads don't clobber each
other. Single-threaded behaviour is unchanged. All runtime .ss — no re-mint.
A reify that doesn't implement a given protocol method now dispatches to that
protocol's extended impls over the reify's host tags (e.g. an Object/default
extension) instead of erroring 'No reified method'. This is malli's pattern: it
reifies some protocols and relies on RegexSchema's default for the rest. A method
with neither a reify impl nor a default still errors.
Surfaced running the DB libraries (migratus) on the jolt db library:
- java.sql.SQLException .getNextException / .getStackTrace / .printStackTrace on
jolt throwables (conditions + ex-info) return nil/empty, so a library walking
the exception chain doesn't crash.
- java.net.URL ctor + .getProtocol (file/http), alongside the existing url shim.
- A generic java.lang.ClassLoader: getSystemClassLoader / a thread's
contextClassLoader resolve a named resource against the source roots (the same
model as clojure.java.io/resource) — file: URL or nil. Thread/currentThread.
These are generic host capabilities, not DB-specific.
The jolt-lang/db next.jdbc surface now runs migratus far enough to connect, build
the migrations table, and discover migrations; migratus's remaining dependency is
java.nio.file (FileSystems/Path/PathMatcher glob), a JVM filesystem API kept out
of core.
Part of the java.* host-class gap (jolt-1nnn). String/format delegates to the
core format engine; NumberFormat getInstance/getNumberInstance/getIntegerInstance
group the integer part and honor min/max fraction digits.
- java.text.SimpleDateFormat.parse parses the RFC1123/1036/asctime patterns
(day/month names, 2-digit-year sliding window, tz token), and .format renders
z/Z/X timezone tokens (GMT/+0000/Z) instead of emitting them literally. Date
gains toLocalDate/toLocalDateTime/before/after/equals. Fixes ring.util.time.
- read / read+string work over a host java.io reader (StringReader wrapped in a
PushbackReader): drain, parse one form, push the tail back. Fixes cuerdas
istr / << string interpolation (and selmer's <<), which read embedded forms
from the template via (read pushback-reader).
Shake-out from the conformance-library sweep. Host-side fixes (runtime .ss,
no re-mint) plus one analyzer change (re-minted):
- Exception fidelity: ex-info and host-constructed throwables (RuntimeException.
etc.) now carry their JVM class, so (class e), instance? across the exception
hierarchy, .getMessage, and clojure.test thrown?/thrown-with-msg? all work.
- .getBytes returns a seqable/countable byte-array and honors UTF-16/UTF-32;
String. decodes them. ->bytevector accepts byte-arrays (Base64).
- Universal .getClass / .toString / .indexOf / .lastIndexOf on any value/seq.
- record? uses the host jrec? predicate (the old (get x :jolt/deftype) crashed
on a sorted-map by invoking its comparator).
- extend-protocol to abstract host types (clojure.lang.Fn/IFn/APersistentVector,
java.net.URI) dispatches.
- New host classes: clojure.lang.PersistentQueue, java.util.ArrayList,
java.net.URI, java.io.File / java.util.UUID ctors, Double/Float ctors+statics,
regex instance? Pattern, System/setProperty.
- *assert* / *print-readably* are real settable/bindable vars.
- (symbol "ns/name") splits the namespace at the last slash.
- letfn fn params desugar destructuring (analyzer; re-minted).
unit.edn gains exinfo/hostobj/queue/hostctor/destructure regression rows.
The host carries bytes two ways: Chez bytevectors (what String/.getBytes
produce) and jolt byte-arrays (what byte-array / the Java-array shims use). They
didn't interconvert, so code mixing the two — like clj-http-lite, which buffers
into (byte-array n) but encodes via .getBytes and decodes via (String. ^[B body
charset) — broke.
- byte-array now also accepts a bytevector or a string (UTF-8 bytes), so the two
representations convert freely at interop seams.
- (String. bytes [charset]) decodes a bytevector OR a jolt byte-array with the
named charset (UTF-8 default; ISO-8859-1/latin1/ascii = one byte/char). It
previously only took a bytevector and ignored the charset.
Runtime .ss shims, no re-mint. Unit covers both directions + charset.
read-bytes/write-bytes go through UTF-8 (with a latin1 fallback), which mangles
arbitrary binary — gzip payloads, TLS records, any non-text body. An HTTP client
moving bytes between jolt byte-arrays and foreign socket/zlib/OpenSSL buffers
needs byte-exact transfer. read-array returns a fresh byte-array of n bytes from
foreign memory; write-array copies a byte-array's bytes into a pointer. Test
covers a round-trip preserving high bytes (200, 255).
clj-http-lite drives java.net URL/HttpURLConnection and java.io byte streams
through .method interop. The Chez host had __register-class-ctor!/-statics! (what
router/reitit needs) but no way to register instance methods on a shim object or
to extend instance?. Add both, plus jolt.host/table?:
- tagged-table .method dispatch: an htable-arm on record-method-dispatch routes
(.m obj a*) through a per-tag method registry keyed off the table's jolt/type;
unregistered methods fall through (sorted colls are htables too).
- __register-instance-check! installs (fn [class-name val] -> true|false|nil),
nil = fall through; chained ahead of the base instance-check.
Runtime .ss shims, no re-mint. Unit covers dispatch, args, instance? both ways.
The HTTP server moves out of the host into the jolt-lang/ring-janet-adapter
library, which binds sockets itself via jolt.ffi and shuts down cleanly. Drop
host/chez/http-server.ss and the obsolete ffi-server-test (the FFI collect-safe
path is covered by ffi-binding-test; the server by the adapter's own CI).
A library binding a blocking native call (accept/recv/connect/...) needs it
emitted __collect_safe so the thread deactivates for the call and doesn't pin
the stop-the-world collector. foreign-fn / defcfn take an optional trailing
:blocking; the backend emits (foreign-procedure __collect_safe ...). Needed for
the ring-janet-adapter socket-server port. ffi-binding-test asserts a thread
parked in a :blocking call doesn't block (collect).
The sqlite/jdbc functionality moves out of the host into the jolt-lang/db
library, which binds libsqlite3 (and libpq) itself via jolt.ffi. A baked
built-in jdbc.core would shadow the library's, so it's removed here. ring-app
gets jdbc.core from the db git dep instead.
A jolt library can now bind its own native dependencies and expose a Clojure API
over them — no jolt built-in required. This is the foundation for moving the
http-client / db / adapter functionality out of the host and into real libraries.
- jolt.ffi/foreign-fn (sugar: defcfn) is a compiler special form: a compile-time
-typed C signature lowers to a real Chez foreign-procedure (analyzer :ffi-fn ->
backend foreign-procedure), so calls are typed and marshaled, not eval'd.
- host/chez/ffi.ss provides the rest under jolt.ffi: load-library, alloc/free,
read/write/sizeof, ptr<->string, null/null?. Loaded after the loader snapshot
so a library's (require '[jolt.ffi]) still loads the macro side.
- Types: int/uint/long/ulong/int64/uint64/size_t/ssize_t/iptr/uptr/double/float/
pointer/string/void/uint8/char.
Validated end to end: a pure-Clojure file binds libc (getpid/strlen/abs) and
libsqlite3 (open/prepare/step/column/finalize over out-param pointers) and runs a
query. Gate test test/chez/ffi-binding-test.ss (make ffi); selfhost holds.
Bring the docs in line with the actual implementation now that Chez is the sole
substrate.
Deleted the migration/spike/handoff artifacts that only documented the Janet
era or the port effort: the port plan, phase-0 and foundational-runtime spike
writeups (+ the stray root-level copy), the self-hosting design notes, the
architecture-refactor plan, and spike/chez/RESULTS.md.
Rewrote the current reference docs against the Chez facts: building-and-deps and
tools-deps (no jpm/build step — bin/joltc off the checked-in seed, deps via
jolt.deps into ~/.jolt/gitlibs), libraries (SQLite is built-in jdbc.core over
libsqlite3, not a Janet driver), the conformance/spec test-flow docs (the Chez
corpus runner + certify, no .janet harnesses), and the transient / type-hint /
seed-overlay design notes (Chez representations: mutable transients, flat
copy-on-write vectors, HAMT maps, the seed/overlay twin). Fixed the README
collections line (vectors aren't 32-way tries) and added the ffi/transient gate
targets. rfc 0001's numerics open-question is resolved (the Scheme tower).
Renamed the built-in HTTP adapter to jolt.http.server only (dropped the
ring-janet.adapter alias — a Janet-era name).
The Chez port had landed transients as copy-on-write — each conj!/assoc!/etc.
rebuilt the whole persistent collection. Semantics were right but a transient
vector was O(n^2) to build (the persistent vector is a flat array, so every
conj! copied it); maps/sets were ~O(n log n) since the HAMT only path-copies.
This restores the Janet host's approach: true mutable backing, snapshot once on
persistent!.
vec : a growable Scheme vector (capacity + fill count); conj!/pop! amortized
O(1), persistent! hands off the buffer (exact fit) or trims once.
map : a Chez hashtable keyed by key-hash/jolt= (value equality, nil-safe);
persistent! folds it into a pmap.
set : a Chez hashtable; persistent! folds into a pset.
cow : fallback for anything else (e.g. a sorted coll) keeps the old
copy-on-write path, preserving jolt's superset.
get/count/contains?/nth see through each representation. Building a 400k vector
went from minutes (quadratic) to ~50ms (linear). assoc! keeps the variadic
dangling-key nil-pad on both vectors and maps. test/chez/transient-test.ss pins
the invariants and the linear-time property; wired in as `make transient`.
Two thread-safety bugs in the native FFI layer.
The HTTP server's accept/recv/send were plain foreign-procedures. A thread
inside a foreign call stays active for the stop-the-world collector, so the
accept loop sitting idle in accept() froze GC for the whole process whenever
another thread (a future, an async block) allocated. Mark the three blocking
calls __collect_safe so the thread deactivates for the call's duration —
collection proceeds while the accept thread waits. The args are an fd and
foreign-alloc'd buffers (outside the Scheme heap), so a collection mid-call has
nothing to move.
jolt.http-client built its -D header-file path from an unguarded (set! counter
(+ counter 1)) and counter mod 90000, with no per-process component. Concurrent
requests could compute the same path and clobber each other's headers. Use a
mutex-guarded monotonic counter plus the pid.
test/chez/ffi-server-test.ss exercises both (a (collect) while the server is
idle in accept(), temp-path uniqueness across threads, and a live request) and
is wired into the gate as `make ffi`.
deftype fields tagged ^:unsynchronized-mutable / ^:volatile-mutable can now be
reassigned in place from a method, as on the JVM. A jrec stores fields as cons
cells, so a new jolt-set-field! mutates the pair with set-cdr!. The deftype macro
rewrites (set! mutable-field v) in a method body to (set! (.-field inst) v), and
the analyzer compiles a (set! (.-field obj) v) target to jolt-set-field! — so
both the rewritten symbol form and an explicit interop (set! (.-root this) v) go
through one path. Field reads remain a snapshot at method entry, which is correct
for the universal read-then-set pattern (a repeated set! of the same field in one
call would read the entry value).
Closes the set!-of-local SCI failures: SCI load 202 -> 205/218.
Found in a read/eval review: a local named like a special form wrongly took
over operator position. (let [if (fn ...)] (if true 1 2)) returned the fn, but
per spec section 3 (and the reference) special-form heads are not shadowable;
only macros are. Two fixes: drop the (not shadowed) guard on the special-form
branch of analyze-list (so an (if ...) head is always the special), and prefix
a local whose name is a Scheme keyword when emitting (so a value local legally
named if does not shadow the (if ...) the back end emits). Value-position
locals named if/or/case still work.
(type r) returned a symbol user.TyR, so (= (symbol (str (type r))) (type r))
was true; the JVM's type is a Class (not a Symbol) so it's false. jolt models
classes as strings, so a record's type is now its ns-qualified class-name
string — equal to (class r), as on the JVM where type and class coincide for a
record. The symbol-keyed print-method defmethods already fall through to the
default record printing, so they're unaffected. Closes type-of-record.
(def ^String tv ...) left (:tag (meta (var tv))) as the unresolved "String";
the JVM compiler resolves the hint to java.lang.String at def time. Add a
resolve-class-hint host seam (built from the existing class-token table) and
resolve a def's :tag through it in the analyzer. The reader path
(read-string "^String x") stays unresolved, matching the JVM (only the
compiler resolves). Closes ^Type-tag-on-var.
- (class x) returns per-type JVM class names (Long/Double/Ratio/Character/Atom),
not a blanket java.lang.Number.
- register fully-qualified class tokens (java.lang.Long, clojure.lang.Keyword,
clojure.lang.Atom, ...) that self-evaluate to their name, so (= (class 1)
java.lang.Long) and (instance? clojure.lang.Atom x) resolve.
- instance? recognizes Long/Double/Ratio/Character/Symbol/Atom/IFn built-ins.
Closes class number/string/keyword/name, instance? Atom, atom?. Corpus 2699->2705.
- == 1-arg returns true for any value (Clojure short-circuits before the number
check), not 'requires numbers'.
- current-time-ms wired to now-millis so the time macro works.
- subvec truncates float/ratio indices via long (Scheme quotient rejects flonums).
- defonce checks bound? not var-get — in a top-level do the name is already an
unbound interned cell, which var-get throws on.
- drop the line-seq corpus row (used janet/spit, N/A); allowlist char-array
(needs Class/forName "[C").
Corpus 2678->2683, floor raised. Re-minted. Full gate green; CI green.
jolt-cf1q.7
Remove the 17 rows that exercised the Janet FFI bridge (interop/janet bridge,
interop/jolt.interop) and the Janet build-time env scrub (host-interop/bake env
scrub, janet.os/setenv) — none exist on any non-Janet host, so they only added
crash noise. Portable interop is covered elsewhere. corpus 2920->2903 rows;
gate 2678/2742 0 new div, certify 0 new/0 stale.
Rewrite the README, CLAUDE.md build/architecture sections, test/chez/README,
and conformance SPEC for the Janet-free world: bin/joltc + make test, the
self-hosting bootstrap, the frozen JVM-sourced corpus. CI installs Chez + JDK/
Clojure and runs 'make test' (was Janet/jpm).
jolt-cf1q.6
Remove the Janet seed (src/jolt/*.janet: reader, value layer, vars/ns, the
tree-walking interpreter, the Janet backend, the optimizing compiler), the
Janet->Scheme cross-compiler (host/chez/{driver,emit,jolt-chez}.janet),
bin/jolt-chez, the jpm build (project.janet) and the Janet test runner
(run-tests.janet), plus the entire Janet test suite. jolt now builds and runs
on Chez alone: bin/joltc off the checked-in seed, bootstrap.ss to rebuild it.
The portable Clojure stays: jolt-core/**, host/chez/**.ss, and the stdlib +
tooling under src/jolt/clojure + src/jolt/jolt (read by the seed build, no
Janet). The gate is 'make test' (self-host, corpus, unit, cli smoke, certify).
Drop the sci and clojure-test-suite submodules (used only by deleted Janet
integration tests); irregex stays.
Filesystem corpus/unit cases that probed project.janet now probe README.md.
jolt-cf1q.6
Add a Janet-free gate so correctness can be judged with only Chez + Clojure:
- host/chez/run-corpus.ss: corpus.edn vs JVM expecteds, lifting the per-case ns
isolation from the old Janet driver; reads corpus.edn via the jolt reader.
- host/chez/run-unit.ss + test/chez/unit.edn: the host-specific unit cases,
evaluated in-process and compared to baked expecteds.
- host/chez/selfcheck.sh: self-host fixpoint (bootstrap.ss rebuild == checked-in seed).
- host/chez/smoke.sh: real bin/joltc CLI smoke.
- host/chez/remint.sh: re-mint the seed to a byte-fixpoint after a source change.
- Makefile: 'make test' runs the lot; 'make remint' rebuilds the seed.
Numbers match the Janet gate: corpus 2679/2757 0 new div, unit 450/450, certify
0 new/0 stale.
jolt-cf1q.6
The unit tests in test/chez/_*.janet now drive bin/joltc (the zero-Janet
spine) and judge against baked expected values instead of a live build/jolt
run. Ten of them captured the oracle from build/jolt per case; their values
are now literals (one env-dependent javastatic case became a predicate so it
stays portable). The rest already had literal expecteds with a redundant
build/jolt sanity check, now dropped.
Retire emit-test/emit-parity/reader-parity: they compared the Chez/Clojure
path against a live Janet evaluation, emitter, or reader. That migration check
is done, and run-corpus-zero-janet (Chez analyzer vs the JVM corpus) plus
certify.clj cover correctness now.
Rewrite the README for the current zero-Janet gate.
jolt-5oci
run-corpus.janet drove a target binary (default build/jolt, the Janet host)
through the corpus and run-corpus-chez.janet was the all-flonum subset probe.
Both compare against corpus.edn, which is now JVM-sourced, so the all-flonum
hosts diverge on every numeric/host case. The zero-Janet spine gate
(run-corpus-zero-janet.janet) plus certify.clj are the oracles; drop these and
the stale test/chez/known-divergences.edn allowlist they used.
corpus.edn :expected is now the value reference JVM Clojure produces, set by the
new test/conformance/regen-corpus.clj (one JVM process, per-row thread watchdog).
167 rows moved to the JVM value: ratios (/ 1 2)=>1/2, doubles (double 3)=>3.0,
shared-heap concurrency (the future/pmap/agent cases), clojure.math doubles. The
JVM is the spec; jolt is measured against it.
known-divergences.edn shrinks to the rows whose JVM value is an opaque host object
that can't round-trip to source (Java arrays, transients, atoms, beans, proxies,
chunks all print as #object[..@addr]) plus (fn* foo) and a few racy concurrency
cases (:flaky). The zero-janet gate's allowlist becomes the set of host gaps vs the
JVM spec (no Class/array/BigDecimal, :jolt reader, jolt's own printing).
Math/clojure.math sqrt/pow/floor/trig now return doubles (Chez returns exact for
exact args, e.g. (sqrt 9)=>3); JVM always returns a double.
extract-corpus.janet no longer writes corpus.edn unless asked (the test runner
imported it and was silently overwriting the JVM corpus with the spec sources'
placeholder answers). The prelude parity gate is deleted — the zero-janet spine +
certify.clj are the oracles.
zero-janet 2678 (0 new divergences), certify 0 new / 0 stale, emit-test 330/330.
jolt was all-flonum (one :number type, inherited from Janet whose only number
type is a double). The Chez runtime has a full numeric tower, so the zero-Janet
path now carries it = JVM Clojure semantics:
(/ 1 2) => 1/2 (exact Ratio, was 0.5)
(integer? 3) => true (integer? 3.0) => false (float? 3.0) => true
(ratio? (/ 1 2)) => true (= 3 3.0) => false (== 3 3.0) => true
(+ 1 2) => 3 (exact) (/ 1.0 2) => 0.5 (double)
jolt= was already exactness-aware (values.ss) and == is value-equality, so
=/== match the JVM split. The reader preserves exactness (integer literals exact,
a/b ratios exact rationals, decimals/exponents flonums); backend_scheme emit-const
renders exact ints/ratios and flonums faithfully; the value-position arithmetic,
count, int, compare, bit ops, parseLong, string .length/.indexOf, range,
timestamps, and array bytes return exact integers (= JVM int/long) instead of
coercing to flonum. double/parseDouble/clojure.math floor|ceil|signum stay double.
Only the zero-Janet path carries the tower (the Janet reader loses exactness into
a double before emit). The prelude/all-flonum path is unaffected for compiled code;
the runtime reader is shared, so a couple of all-flonum reader assertions become
value (==) assertions. ~16 numeric corpus cases now give the JVM tower value vs the
Janet-era :expected and are allowlisted as tower divergences (Chez == reference
JVM) pending the corpus flip to JVM (jolt-ecz0). No BigDecimal type (1M).
Re-minted. zero-janet 2682 (floor 2698->2682, the reclassified tower cases), 0 new
divergences; fixpoint 10/10, bootstrap 6/6, spine 35/35, cli 49/49; Janet gate 155
files 0 failed.
Two Chez reader bugs, both JVM-parity gaps:
inc'/+'/foo' (trailing apostrophe) were mis-read as a symbol followed by a
quote macro, because the reader treated ' as a terminator. In Clojure ' is a
NON-terminating macro char (constituent after the first char). Since the seed
is minted on Chez, (def inc' inc) became (def inc 'inc), clobbering inc's var
cell with its own symbol -- so (var-get (var inc)) returned the symbol, not the
fn. Drop ' from the token terminator set; a leading ' still quotes.
^bytes [b] / ^String [x y] return-type hints: the Chez reader lowered ^meta on
a collection to a (with-meta vec meta) form, but emitted a QUALIFIED
clojure.core/with-meta while the Janet reader emits a bare with-meta -- so the
fn/defn macros' unwrap logic (matching the bare head) slipped past it and choked
on a non-vector arglist. Emit bare with-meta to match Janet, and unwrap a
(with-meta <vec> _) arglist in analyze-fn as a backstop.
Re-minted the seed. zero-janet 2699, prelude 2652, Janet gate 155/0, fixpoint
10/10, bootstrap 6/6, all 0 new divergences.
realized? threw 'not supported on' for a jolt-lazyseq record (the overlay
reads :jolt/type); add a jolt-lazyseq? arm to the post-prelude wrapper
reading the record's own realized? flag.
conj! 1-arity (conj! coll) is the transducer-completion arity and returns
coll as-is on the JVM, no transient check — we threw 'not a transient'.
Both gates: zero-janet 2696->2698, prelude 2649->2652, 0 new divergences.
Audit of the janet.*/jolt.interop/STM corpus cases vs Chez equivalents: the Janet
FFI-bridge cases (janet/string, janet/type, janet.math/sqrt, janet.string/ascii-
upper) test functionality already covered by PORTABLE corpus cases (str 29, type 9,
sqrt 3, upper-case 9), so they're safe to delete in Phase 5. Two needed a Chez
equivalent so they pass instead of being lost:
- clojure.lang.LockingTransaction/isRunning -> false (no STM on jolt).
- line-seq: the corpus case used janet/spit setup but was the SOLE line-seq
coverage (0 other cases). Ported janet/spit->spit, and added a native Chez
line-seq (drain a jhost io/reader + split on newline; no trailing empty line)
that delegates a Janet map-reader to the overlay version.
zero-Janet 2694->2696, prelude floor 2648; self-host + Janet gate + JVM cert green.
jolt-cf1q.7 jolt-0obq
hc-list? required cseq-list? (a reader-built list), so a form built at runtime via
concat/map/cons — a lazy cseq with list?=#f — was rejected as "unsupported form".
In Clojure any seq is a valid call form, so accept any cseq. Lights up macros that
build their expansion with concat/list and (eval seq-form).
zero-Janet 2692->2694, 0 new divergences; self-host + Janet gate + JVM cert green.
jolt-cf1q.7
Date/time (inst-time.ss): java.util.Date / java.sql.Timestamp ctors accept ms or
another date value (ms-of) -> a jinst; java.text.SimpleDateFormat (pattern + .format
via the existing format-ms UTC engine; .setTimeZone accepted); java.util.TimeZone/
getTimeZone. instance? answers Date true / Timestamp false for a jinst (a Date is
not a Timestamp on the JVM).
clojure.edn/read over a reader (io.ss + post-prelude): the overlay edn.clj's
drain-reader is janet/type-coupled, so Chez drains the jhost StringReader/
PushbackReader to a string and reads the first EDN form. Unblocks jolt-uicd.
Native Chez throughout (no vendoring): Chez date arithmetic + string ports. zero-Janet
2688->2692, 0 new divergences; self-host + Janet gate + JVM cert green.
jolt-cf1q.7 jolt-dcmm jolt-7t3l jolt-uicd
deftype binds the type name as a VAR (the make-deftype-ctor closure), but (P. 5)
lowers to (host-new "P"), which only checked class-ctors-tbl -> "No constructor".
Fall back to resolving the class name as a var in the current ns / clojure.core
and invoking it — so (P. args) constructs the same jrec as the ->P factory, and
protocol method dispatch (.m / .-field) over it works.
zero-Janet 2685->2688 (no-constructor 5->2); prelude floor bumped to 2641 (the
delay batch's run). Self-host + Janet gates + JVM cert green.
jolt-cf1q.7
Add a thread-safe delay type (concurrency.ss): make-delay wraps a thunk; deref/
force run it once under a lock and cache (JVM delays are thread-safe + memoized).
delay? and realized?-on-a-delay are native; the overlay's `delay` macro
(-> make-delay) and `force` (-> deref) now work. realized? wrapper (post-prelude)
and the deref chain gain a delay arm. Removed the np-delay? stub from
natives-parity.ss (the real type lives in concurrency.ss).
Seed unchanged (no re-mint). zero-Janet 2673->2685, 0 new divergences; Janet gate
+ JVM cert green.
jolt-cf1q.7
The zero-Janet runner wrapped each case as (= EXPECTED ACTUAL) and checked the
result was true. That nests ACTUAL's top-level (do ...), so a case like
(do (defmacro m ...) (m 1)) can't use the macro it just defined — the analyzer
punted defmacro -> "uncompilable" (35 cases).
Match certify.clj's eval-isolated instead: carry EXPECTED and ACTUAL as separate
sources and evaluate ACTUAL as its own top-level program (jolt-compile-eval unrolls
the top-level do, so a macro defined earlier is usable later), then compare to
EXPECTED with =. Evaluating ACTUAL from SOURCE (not (eval (quote A))) preserves the
reader's map-literal source order, so the eval-order cases still pass.
eval-corpus-zero-janet / program-corpus-zero-janet now use a 3-field TSV
(label/expected/actual); run-corpus-zero-janet's per-case debug path evals both
sides too. Only run-corpus-zero-janet uses these (the prelude gate is untouched).
zero-Janet 2642->2673 (analyzer "uncompilable" 35->4); 1 new allowlisted divergence
(`{:a ~x :b ~y} syntax-quote map construction doesn't preserve source eval order —
pmap is unordered). Janet gate + JVM cert green.
jolt-cf1q.7
host/chez/natives-array.ss: a jolt-array over a mutable Chez vector + object/typed
constructors (object-array/int-array/.../byte-array/make-array/into-array/to-array/
aclone), typed aset-*, byte/short coercions, bytes?/bytes/ints/..., and eager
chunk-buffer/chunk-* (Jolt doesn't chunk). count/nth/seq/get and jolt.host/ref-put!
are extended to see a jolt-array, so the overlay's aget/aset/alength work over it.
Numbers it produces are flonums (jolt's rep) so exactness-aware = holds. char-array
stays in io.ss (a char-SEQ that io/reader/str/slurp consume).
natives-parity.ss adds: __reader-features / -set!, reader-conditional, re-matcher,
delay? (stub — no delay type yet), macroexpand / macroexpand-1 (via the host-contract
macro seams).
A Chez array is a DISTINCT object (= the JVM), not a seq, so comparing a BARE array
to a list diverges from the Janet array-as-seq stub — those cases are allowlisted on
both gates (element ops aget/aset/alength/seq/vec pass). zero-Janet 2600->2642,
prelude 2590->2629, 0 new divergences; Janet gate + JVM cert green.
jolt-cf1q.7
Native Chez shims for clojure.core fns that live in the Janet seed but had none
on the zero-Janet spine (they resolved to nil -> "not a fn"):
- host/chez/natives-parity.ss: hash / hash-combine / hash-ordered-coll /
hash-unordered-coll (24-bit masked like core_extra), transient?, rseq (vectors +
sorted colls), cat (transducer).
- jolt-invoke now dispatches a TRANSIENT vec/map/set as a fn (callable on the JVM):
((transient [10 20 30]) 1) -> 20.
- ns.ss: ns-resolve, ns-imports, remove-ns, intern, alias, ns-unalias, refer,
ns-refers, refer-clojure, alter-meta!, reset-meta!, and a real ns-aliases (was a
stub returning {}).
- runtime (require ...)/(use ...) now register :as/:refer into the Chez ns tables
(was a no-op). The Chez analyzer already pre-registers at analyze time, but when
the JANET analyzer compiled the form (prelude path) the Chez tables stayed empty,
so ns-aliases/ns-resolve over an alias diverged — this fixes both paths.
Seed unchanged (overlay doesn't reference these at mint time). zero-Janet corpus
2567->2600, prelude 2557->2590, 0 new divergences on either; Janet gate + JVM cert
green. Filed jolt-vgrp for the pre-existing var-get-of-scalar-native-op quirk.
jolt-cf1q.7
Mirrors test/bench/core-bench.janet's 8 compute programs + methodology (load the
runtime once, time compile+run of each, min of N) so the zero-Janet Chez path is
comparable to the Janet compile path.
Result: Chez is 3-33x faster than Janet across the set (~320ms vs ~5000ms total,
~15x), biggest on seq/map/reduce, smallest on fib (3.2x). So deleting Janet is a
perf win, which satisfies the "perf confirmed" precondition Phase 5 gates on. Chez
is ~2-28x slower than JVM Clojure, expected and not the bar (and the Janet-only
optimizing modes haven't been tried on Chez).
jolt-cf1q.5
Replace the Janet synchronous agent shim (agent = atom, send applies inline) with
JVM-style async agents: send/send-off enqueue an action and a single worker thread
per agent applies them in order; deref reads the current (maybe not-yet-updated)
state without blocking; await blocks until the queue drains. A validator rejection
or a thrown action puts the agent in an error state (agent-error) and halts the
queue; restart-agent clears it. send and send-off share one serialized worker (a
superset of the JVM's fixed/cached pool split). Native versions re-asserted in
post-prelude over the overlay; await/restart-agent are new.
Corpus: the two "send/send-off applies" cases do (send a f) (deref a) with no
await, so they now read state before the action runs — diverging like the JVM
(the suite was literally "synchronous shim"). Allowlisted on both gates; floors
-2 (zero-Janet 2569->2567, prelude 2559->2557). cli-test covers async agents via
await (ordered 100-send dispatch, error capture) — 49/49. Janet gate + JVM cert
green; 0 new divergences on either corpus.
jolt-byjr
