The host/chez directory mixed jolt's own runtime (value model, seq, reader,
vars, ns, multimethods) with the shims that emulate the JVM: java.* / javax.*
classes, clojure.lang interfaces, and the host-class registry they hang off.
Move that JVM-emulation layer into host/chez/java/ so it reads as a distinct
unit instead of being interleaved with the platform runtime.
Moved (content unchanged): host-static, host-static-methods,
host-static-classes, host-class, dot-forms, records-interop, byte-buffer,
io, io-streams, inst-time, java-time, bigdec, natives-queue, natives-str,
natives-array, math, concurrency, async, ffi.
The load paths in rt.ss/cli.ss and the build.ss runtime manifest are updated
to point at java/; the build inliner follows the (load ...) strings, so the
AOT path needs no other change. All runtime shims, no seed source touched
(the three .clj edits are doc comments), so no re-mint.
Gate green: make test (selfhost fixpoint, certify 0-new, sci 211, infer),
shakesmoke (4 apps byte-identical).
Running cognitect aws-api's pure test namespaces (signing/shapes/protocols/
util/retry/endpoints) surfaced general gaps:
- extend-protocol/extend-type accept a computed class type, e.g.
(Class/forName "[B") for the byte-array class — the byte-array idiom data.json
and aws-api use. The macro grouping handled only symbol/nil heads (it crashed on
a list type); type->name resolves a Class value via .getName; a byte-array
dispatches on the "[B" host tag.
- java.nio.ByteBuffer over a jolt byte-array (wrap/allocate/get/put/array/
remaining/position/limit/duplicate/flip), plus extend-protocol to it.
- java.util.Arrays (equals/copyOf/copyOfRange/fill) and java.util.Random
(nextBytes/nextInt/…).
- java.net.URI/create and clojure.lang.RT/baseLoader statics.
- clojure.core.async/promise-chan (deliver-once, peek-don't-pop).
- a failed java.time parse throws DateTimeParseException (typed), so
(catch DateTimeParseException …) matches it instead of leaking an untyped
condition.
The XML side lives in the jolt-lang/xml library (libxml2 over jolt.ffi); ByteBuffer
stays in core as a generic java.nio primitive.
Gate: make test green (corpus +6 JVM-certified rows, 0 NEW divergence; unit
553/553; SCI 211).
Shaken out getting ring-defaults (and its ring-core/anti-forgery/session stack)
to load and serve static resources on jolt. All general fixes, all runtime:
- Class/forName throws a catchable ClassNotFoundException for a class jolt can't
back (it returned a broken truthy value for any name, and crashed on use). Lets
the common (try (Class/forName "optional.Dep") (catch ...)) probe libraries use
to detect an absent dependency work — e.g. ring's joda-time check.
- deps: reconcile native libs (and source roots) in one step, deduped by library
identity, instead of the ad-hoc distinct at each call site. An app pulling two
libs that declare the same shared object (libcrypto via both jolt-crypto and
http-client) now includes and loads it once.
- io: a File answers getProtocol ("file") / getFile so resource-serving
middleware that expects io/resource to hand back a file: URL works; the
classloader gains getResources (every source root holding the resource).
- clojure.string/replace accepts a char match/replacement, like the JVM.
JVM-certified corpus rows for the Class/forName and string/replace behavior.
Expand java.io so libraries that touch the filesystem work unchanged.
File: the full method surface — length, lastModified, can{Read,Write,Execute},
isHidden, list, mkdir(s), delete, createNewFile, renameTo, getParentFile,
get{Absolute,Canonical}File, compareTo/equals/hashCode — plus the statics
separator / pathSeparator / createTempFile / listRoots. A File now keeps the
path as given (new File("rel").getPath() is "rel", .isAbsolute false); a
relative path resolves against JOLT_PWD only when the filesystem is touched,
matching the JVM. slurp/spit and the dir helpers go through the same
resolution, fixing a spit-vs-slurp inconsistency.
Streams (host/chez/io-streams.ss) — each a jhost wrapping a Chez port, so
buffering, EOF and binary<->char transcoding come from Chez:
- FileInputStream / FileOutputStream / ByteArrayInputStream /
ByteArrayOutputStream / BufferedInputStream / BufferedOutputStream
- FileReader / FileWriter / InputStreamReader / OutputStreamWriter /
BufferedReader / BufferedWriter
Buffered* return the wrapped stream (Chez ports are already buffered).
clojure.java.io: input-stream/output-stream now yield real byte streams (were
aliased to the char reader/writer); added copy (byte-exact for byte sources),
make-parents, delete-file. with-open also closes file-writer/port-writer/
print-writer (a pre-existing gap).
All runtime shims, no re-mint. 15 JVM-certified corpus rows; make test +
shakesmoke green.
jfile lacked .isAbsolute (path starts with /). clojure.core/default-data-readers
and *data-readers* were nil, so a library merging them (aero's reader opts)
couldn't resolve #inst. Define them keyed by symbol, like Clojure.
io/file already resolved a relative path against JOLT_PWD (the user's cwd before
the launcher cd's to the repo root), but slurp and io/reader on a relative string
path opened it against the process cwd — so a program run from its own directory
couldn't read its own relative files (e.g. aero's (read-config "config.edn")).
Route the string branches through project-relative, matching io/file. Internal
callers pass already-resolved paths through read-file-string and are unaffected.
The printer's two entry points (jolt-pr-str in rt.ss, jolt-pr-readable in printing.ss)
get register-pr-str-arm! / register-pr-readable-arm!, plus register-pr-arm! for the
types whose str and readable forms match (bigdec/inst/uuid/tagged/record/ns/var). The
normalize arms (sorted, lazy-seq, queue) and the uri readable arm register per-printer.
Also folds in the hash (dyn-binding var-cell), class (io uri/uuid/file), and get
(transients) arms missed earlier.
natives-array's get stays a case-lambda wrapper on purpose: its 2-arg path errors on
an out-of-bounds index while the 3-arg path returns the default, an arity distinction
the (coll k d) registry collapses — left as-is to preserve behaviour.
Completes jolt-lmot: all six dispatchers (hash/class/get/=/pr-str/pr-readable) off the
set!-rebind chains. make test green, 0 new corpus divergences; pr-str/str of inst,
uuid, bigdec, sorted-map, record-with-lazyseq, queue all verified.
A built binary dropped its deps.edn :jolt/native declarations and its
resource roots, so an FFI+resources app (ring-app) failed at runtime:
sockets/sqlite gave 'no entry for socket' and io/resource returned nil.
The buildsmoke fixture is pure compute, so neither path was exercised.
The launcher now loads required + :process native libs before the app's
top-level forms (a library's defcfn resolves its foreign-procedure symbols
at top-level eval during startup, so the libs must be loaded first);
optional libs load in the scheme-start launcher, where a missing lib is
caught rather than aborting the heap build.
deps.edn :jolt/build {:embed [dirs]} bakes those dirs' files into the heap
(register-embedded-resource! at heap build), so io/resource serves them with
no files on disk. Non-embedded resources resolve at runtime against JOLT_PWD,
and io/file reads (e.g. config.edn) stay external.
build-binary now takes the encoded natives, embed dirs, and project paths
from cmd-build; deps/resolve-project surfaces them. Buildsmoke fixture grows
an embedded resource + a :process native to cover both paths.
jolt-str-render-one and instance-check were each extended by a chain of
set!-wrapping closures spread across ~10 and ~5 host files, so the real
behavior of either was scattered and load-order-dependent. Give each a
registry the base file owns: converters.ss/records-interop.ss define the
registry plus a register-* helper, and each extending file registers one arm
instead of capturing %prev and set!-ing the global.
str-render arms are type-disjoint; instance-check arms run newest-first (the
old outermost-wins order) and may return 'pass to defer. The string-token ->
symbol normalization the natives-array arm did for every inner arm moves to
the dispatcher head; array tokens stay strings for that arm to decide.
jolt-ogib.14. Runtime-only shims, no re-mint.
Round 1 (correctness + dead code):
- Fix duplicate java.util.HashMap registration in host-static.ss: the alist
impl shadowed the hashtable ctor while leaving the hashtable methods bound,
so .keySet/.values/.remove/.clear crashed. Drop the alist version.
- Delete jolt-core/jolt/reader.clj: a 463-line dead duplicate reader, never
required or compiled (the live reader is host/chez/reader.ss) and drifted.
- Remove dead defs: ir/rt + :rt op + unused ir/op; the Janet branch in
clojure.edn/drain-reader; a shadowed first clojure.string/trim-newline;
io.ss jolt-char-array + the reader def-var (both shadowed by natives-array);
concurrency.ss jolt-future-done?*; compile-eval.ss jolt-analyze-emit.
Round 2 (perf + determinism):
- emit-quoted-map-value / quoted sets now emit sorted by emitted text instead
of host-hash order, which isn't stable across Chez versions (jolt-8479).
- jolt-into folds through a transient, so into/vec/mapv/filterv onto a vector
are O(n) instead of O(n^2).
- deps resolve-deps walks its queue with an index cursor (was subvec-per-pop).
- async channel and agent action queues use amortized-O(1) FIFOs; ArrayList is
backed by a growable vector (O(1) add/get) instead of a list.
Thread/yield was a no-op and Thread/interrupted always returned false. Now:
- yield calls libc sched_yield (resolved once via the process symbols), so a
spin loop relinquishes the CPU. Falls back to a zero-length park if the symbol
can't be resolved.
- each OS thread carries an interrupt flag (a box, thread-local). currentThread
returns a handle wrapping the calling thread's flag, so .interrupt from another
thread sets the target's flag. .isInterrupted reads without clearing; the static
Thread/interrupted reads and clears — JVM semantics.
Consolidates the Thread surface: currentThread + the instance methods live in
io.ss (where the handle and its classloader are built), the flag box + yield +
the interrupted static in host-static.ss. Unit cases cover yield, the read/clear
split, and a cross-thread interrupt over a future.
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.
Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.
Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.
Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
Surfaced running the DB libraries (migratus) on the jolt db library:
- java.sql.SQLException .getNextException / .getStackTrace / .printStackTrace on
jolt throwables (conditions + ex-info) return nil/empty, so a library walking
the exception chain doesn't crash.
- java.net.URL ctor + .getProtocol (file/http), alongside the existing url shim.
- A generic java.lang.ClassLoader: getSystemClassLoader / a thread's
contextClassLoader resolve a named resource against the source roots (the same
model as clojure.java.io/resource) — file: URL or nil. Thread/currentThread.
These are generic host capabilities, not DB-specific.
The jolt-lang/db next.jdbc surface now runs migratus far enough to connect, build
the migrations table, and discover migrations; migratus's remaining dependency is
java.nio.file (FileSystems/Path/PathMatcher glob), a JVM filesystem API kept out
of core.
- java.text.SimpleDateFormat.parse parses the RFC1123/1036/asctime patterns
(day/month names, 2-digit-year sliding window, tz token), and .format renders
z/Z/X timezone tokens (GMT/+0000/Z) instead of emitting them literally. Date
gains toLocalDate/toLocalDateTime/before/after/equals. Fixes ring.util.time.
- read / read+string work over a host java.io reader (StringReader wrapped in a
PushbackReader): drain, parse one form, push the tail back. Fixes cuerdas
istr / << string interpolation (and selmer's <<), which read embedded forms
from the template via (read pushback-reader).
Shake-out from the conformance-library sweep. Host-side fixes (runtime .ss,
no re-mint) plus one analyzer change (re-minted):
- Exception fidelity: ex-info and host-constructed throwables (RuntimeException.
etc.) now carry their JVM class, so (class e), instance? across the exception
hierarchy, .getMessage, and clojure.test thrown?/thrown-with-msg? all work.
- .getBytes returns a seqable/countable byte-array and honors UTF-16/UTF-32;
String. decodes them. ->bytevector accepts byte-arrays (Base64).
- Universal .getClass / .toString / .indexOf / .lastIndexOf on any value/seq.
- record? uses the host jrec? predicate (the old (get x :jolt/deftype) crashed
on a sorted-map by invoking its comparator).
- extend-protocol to abstract host types (clojure.lang.Fn/IFn/APersistentVector,
java.net.URI) dispatches.
- New host classes: clojure.lang.PersistentQueue, java.util.ArrayList,
java.net.URI, java.io.File / java.util.UUID ctors, Double/Float ctors+statics,
regex instance? Pattern, System/setProperty.
- *assert* / *print-readably* are real settable/bindable vars.
- (symbol "ns/name") splits the namespace at the last slash.
- letfn fn params desugar destructuring (analyzer; re-minted).
unit.edn gains exinfo/hostobj/queue/hostctor/destructure regression rows.
The line REPL was broken (read-line called nil — the __stdin-read-line host seam
the clojure.core *in* reader drives was never implemented on Chez) and didn't
load the project, so (require '[some.lib]) failed. Now:
- __stdin-read-line reads a line from stdin (get-line); read-line / read / the
REPL work.
- repl resolves the project first (deps on the roots, native libs loaded), so
libraries are available — same context a run gets.
- jolt.nrepl: a jolt-native nREPL server (bencode over a loopback jolt.ffi
socket) speaking the real protocol — clone / describe / eval / load-file /
close, with stdout capture, :ns-scoped eval (in-ns; binding *ns* doesn't drive
load-string resolution here), and real error text. 'joltc nrepl [port]' applies
the project then serves; writes .nrepl-port. Editors (CIDER/Calva/Cursive)
connect and develop live; project libraries load in the session.
- ex-message returns nil for raw Chez conditions, so jolt.host/condition-message
exposes the condition text; the REPL and nREPL surface it instead of an opaque
#<compound condition>.
Why native, not real nREPL: nrepl.server is welded to java.util.concurrent
executors, two compiled Java helper classes, a DynamicClassLoader, Compiler
internals and a JVMTI agent — not faithfully shimmable. The wire protocol, which
is what clients depend on, is small and implemented directly.
Runtime .ss + jolt-core, no re-mint. Full gate green.
Gaps the clj-http-lite suite hit running over jolt-lang/http-client:
- with-open now closes a tagged-table stream shim via its .close method (was a
:close field lookup that only matched the Janet-era tables).
- slurp accepts a bytevector / jolt byte-array / a byte input-stream shim and
honors :encoding — clj-http-lite slurps response bodies and :as :stream bodies.
- (.getBytes s charset) and Charset/forName respect the charset (ISO-8859-1 etc.,
one byte per char), not always UTF-8; Charset/forName returns the name string.
- (class e) reads the JVM :class off a throwable tagged-table, so clojure.test's
(thrown? Class …) / (= Class (class e)) match a library's typed exceptions.
- Registered the HTTP/IO exception class names (IOException, UnknownHostException,
SocketTimeoutException, SSLException, …) so the FQ literals self-evaluate.
All runtime .ss shims, no re-mint. Full gate green.
clj-http-lite drives bytes through clojure.java.io/copy (response body into a
ByteArrayOutputStream, request body out) and resolves URLs via io/as-url. Neither
worked for a library shim:
- io/copy was absent. Add it: raw bytes / string / a jhost reader write in one
shot; any other source drains via .read into a buffer and .write to the dest,
both through method dispatch — so a library's tagged-table streams copy without
the host knowing their layout.
- io/as-url ignored a library-registered URL class, so it and (URL. spec)
disagreed (the file-only jhost has no getProtocol/getHost/...). It now honors a
registered URL ctor, falling back to the jhost.
Runtime .ss shims, no re-mint.
Shaking the ring-app example's real library stack out against jolt surfaced a
batch of divergences from JVM Clojure, the biggest being evaluation order.
backend_scheme: call and recur arguments were emitted as bare Scheme operands,
so Chez's unspecified (right-to-left) order won out. Clojure evaluates left to
right, which selmer's reader loop relies on: (recur (add-node ... rdr) (read-char
rdr)) consumed a char early and dropped the first chars of every {{tag}}. Bind
operands to fresh temps in a let* (only when two or more can have side effects,
so hot calls over locals/consts stay un-wrapped). emit-ordered already did this
for collection literals; generalize it.
host-contract: syntax-quote now resolves the alias part of a qualified symbol
(impl/foo -> clojure.tools.logging.impl/foo) instead of leaving it bare, which
limped along via short-name matching until two loaded namespaces (reitit.impl,
clojure.tools.logging.impl) shared the short name and it broke.
collections: key-hash masks with bitwise-and, not fxand — jolt-hash is set!-
decorated per type (records return their own hash) and Chez's equal-hash can be a
bignum, so a key's hash isn't always a fixnum.
seq: even?/odd? handle bignums (JVM accepts any integer; the fxand crashed).
records: Keyword/Symbol .sym/.getName/.toString (honeysql's :clj branch reads
(.sym k)); Throwable .getMessage/.toString over a Chez condition.
host-static: __register-class-ctor!/__register-class-statics! so a host shim
(reitit.trie-jolt) can mirror a Java class.
natives-str: String.intern returns the string.
sqlite: jdbc.core fetch/fetch-one kebab-case column keys (the jolt-lang/db
convention; created_at -> :created-at).
io: a relative io/file path resolves against JOLT_PWD (the user's cwd), not the
repo root the launcher cd'd to — matches JVM cwd semantics, so config.edn loads.
cli: render an uncaught jolt throw (ex-info message + ex-data, or a condition)
instead of Chez's opaque "non-condition value" dump.
Gaps surfaced loading ring-core / hiccup / config / tools.logging on Chez:
- clojure.java.io/resource — resolve a named resource against the loader's
source roots (no classpath), returning a slurp-able File.
- (Object.) constructor — a fresh distinct value (lock / unique sentinel).
- *out* / *err* as dynamic vars over a port-writer, so (binding [*out* *err*] …)
and #'*out* compile and run (tools.logging, selmer).
- :refer :all now registers a refer-all relation, so an unqualified var from a
(require '[ns :refer :all]) resolves at compile time — including #'var.
- java.lang.Character interop ((.toString \+) etc.).
- StringReader accepts a char[] ((StringReader. (char-array s))), not just a
String.
- the \p{L} translation stops just below the UTF-16 surrogate gap (\x{D7FF})
instead of \x{10FFFF} — a range across the surrogates made irregex's char-set
construction call integer->char on a surrogate and crash.
Reader / loader:
- #?@ splicing reader conditionals now actually splice the matched collection's
items into the enclosing sequence; the splice flag was read but ignored, so a
binding vector like [a #?@(:clj [b (.foo b)])] lost its alignment.
- the file loader reads by position and skips a top-level form that reads as
nothing (a :cljs-only #?, a #_ discard, a trailing comment) instead of
treating it as EOF — which silently dropped the rest of a large .cljc file.
- jolt's reader feature set now includes :clj (was {:jolt :default}). jolt is a
Clojure/JVM-compatible host that emulates clojure.lang.* and java.* interop,
so it reads the :clj branch of a .cljc library, not :cljs. This also lets four
more reader-conditional corpus cases pass (floor 2726 -> 2730).
Backend:
- munge-name escapes ' (prime) -> _PRIME_; a Clojure symbol like f' otherwise
emitted a bare ' into Scheme, which is the quote reader macro and unbalanced
the output.
Host shims:
- clojure.java.io/writer (pass through a StringWriter, file-back a path) and a
readLine on the string reader, so line-seq over (io/reader …) works (markdown).
A better "unsupported destructuring pattern: <pat>" error message.
Rephrase comments that pointed at deleted Janet files (emit.janet, the seed
sources, 'the Janet back end punts ...') to present-tense descriptions of the
Chez behavior. Comment/docstring-only; the self-host fixpoint is unchanged
(comments don't affect the compiled seed).
Delete five files that were Janet-host shims with no Chez path: clojure.java.io
(provided natively by host/chez/io.ss), and jolt.{nrepl,png,interop,shell}
(the janet.* bridge, os/shell, janet.net — none exist on Chez).
jolt-cf1q.6
Audit of the janet.*/jolt.interop/STM corpus cases vs Chez equivalents: the Janet
FFI-bridge cases (janet/string, janet/type, janet.math/sqrt, janet.string/ascii-
upper) test functionality already covered by PORTABLE corpus cases (str 29, type 9,
sqrt 3, upper-case 9), so they're safe to delete in Phase 5. Two needed a Chez
equivalent so they pass instead of being lost:
- clojure.lang.LockingTransaction/isRunning -> false (no STM on jolt).
- line-seq: the corpus case used janet/spit setup but was the SOLE line-seq
coverage (0 other cases). Ported janet/spit->spit, and added a native Chez
line-seq (drain a jhost io/reader + split on newline; no trailing empty line)
that delegates a Janet map-reader to the overlay version.
zero-Janet 2694->2696, prelude floor 2648; self-host + Janet gate + JVM cert green.
jolt-cf1q.7 jolt-0obq
Date/time (inst-time.ss): java.util.Date / java.sql.Timestamp ctors accept ms or
another date value (ms-of) -> a jinst; java.text.SimpleDateFormat (pattern + .format
via the existing format-ms UTC engine; .setTimeZone accepted); java.util.TimeZone/
getTimeZone. instance? answers Date true / Timestamp false for a jinst (a Date is
not a Timestamp on the JVM).
clojure.edn/read over a reader (io.ss + post-prelude): the overlay edn.clj's
drain-reader is janet/type-coupled, so Chez drains the jhost StringReader/
PushbackReader to a string and reads the first EDN form. Unblocks jolt-uicd.
Native Chez throughout (no vendoring): Chez date arithmetic + string ports. zero-Janet
2688->2692, 0 new divergences; self-host + Janet gate + JVM cert green.
jolt-cf1q.7 jolt-dcmm jolt-7t3l jolt-uicd
clojure.java.io/reader as an in-memory StringReader over slurp/string/char[]/
File; File .toURL/.toURI returning a url jhost (.toString/.getPath); slurp drains
a StringReader; char-array; with-open's __close seam over jhost readers and plain
:close maps. All in host/chez/io.ss (Chez-native, no analyzer change). Corpus
2191->2202, 0 new divergences. clojure.edn/read over a PushbackReader stays
jolt-r8ku (runtime read).
A File is a path-backed jfile record: (instance? java.io.File f) is true,
str/slurp coerce it to its path, and the File method surface (getName/
getPath/exists/isDirectory/isFile/listFiles/getParent) dispatches through
record-method-dispatch. slurp/spit/flush run over Chez's filesystem
primitives; file-seq's dir primitives (__file?/__dir?/__list-dir) and the
overlay's File branch (.isDirectory/.listFiles, which emit to jolt-host-call)
are jfile-aware. clojure.java.io/file + as-file are def-var!'d natively.
New host/chez/io.ss, a Chez-native implementation -- the seed's
clojure.java.io (io.clj) is a Janet-backed shim over janet.*/janet.file, so
it can't be reused. The analyzer resolves io/file because the seed ctx has
clojure.java.io loaded; only a runtime def-var! is needed. type/instance-
check/str-render/jolt-host-call are extended via the set!-wrap pattern (type
also re-def-var!'d since the var cell captured the old value).
Reader/StringReader-coupled io (io/reader, line-seq over a file, .toURL,
slurp over a reader) deferred to jolt-at0a.
Parity 2176 -> 2191, 0 new divergences. New test/chez/_io.janet 20/20.
