types.clj drove inference through ~14 module-level atoms; the infer walk was
non-reentrant and depended on hidden set-*! install order. Thread one immutable
env (mk-env) through infer instead: it snapshots the installed config
(rtenv/vtypes/record-shapes/protocol-methods/map-shapes?) and carries the
per-run flags and accumulator/guard cells (diags/calls/checking-set/diag-memo).
A fresh env per run makes the pass re-entrant — isolated-diag-count's probe now
runs under a sub-env with its own diags cell instead of save/restoring a shared
atom.
Only state whose lifecycle spans separate API calls stays module-level: a
config-box the set-*! API writes, the escapes/user-sig sweep registries, and a
bridge holding the last checking run's diags for take-diags!. record-type-from-
entry/field-type-from-tag now take the shapes map directly rather than reading a
global.
jolt-ogib.10. Behavior pinned by the new infer gate (23 cases) plus selfhost +
buildsmoke. Re-minted seed.
The corpus/unit gates compile through run-passes' const-fold-only branch, so
the type-inference walk runs only under jolt build --opt — buildsmoke hit one
trivial app and checked stdout. run-infer.ss drives the pass directly: analyze
a source string, then call check-form / infer-body / the set-*! registries and
assert diagnostic counts and collected calls/escapes. Wired into make ci.
Gives the inference pass real behavior coverage so refactoring its internal
state is gate-validatable. jolt-ogib.10 groundwork.
jolt-str-render-one and instance-check were each extended by a chain of
set!-wrapping closures spread across ~10 and ~5 host files, so the real
behavior of either was scattered and load-order-dependent. Give each a
registry the base file owns: converters.ss/records-interop.ss define the
registry plus a register-* helper, and each extending file registers one arm
instead of capturing %prev and set!-ing the global.
str-render arms are type-disjoint; instance-check arms run newest-first (the
old outermost-wins order) and may return 'pass to defer. The string-token ->
symbol normalization the natives-array arm did for every inner arm moves to
the dispatcher head; array tokens stay strings for that arm to decide.
jolt-ogib.14. Runtime-only shims, no re-mint.
The lattice-split commit staged its seed before make remint ran, so image.ss
lagged the source. Commit the correct re-minted image (gensym renumbering only).
types.clj was 852 lines mixing the pure structural-type algebra with the
inference engine, checker, and driver. Move the lattice — scalar/struct/vec/set/
union types, join-t, depth-cap, shape, and the numeric/vector return-fn sets —
into jolt.passes.types.lattice (no inference state, no requires). types.clj
requires it; the engine is now ~720 lines. Compiled into the image before
jolt.passes.types. Re-minted seed differs only by gensym label renumbering.
records.ss mixed the record/protocol/reify model with JVM exception emulation.
Move the contiguous ex-info accessors, the exception supertype hierarchy,
instance-check, case-string, and the instance-check def-var into records-interop.ss,
loaded right after records.ss. Those are only called at runtime, so the relocated
forms resolve fine; records.ss is now the value model and protocol dispatch.
Runtime shim — no seed change.
The 929-line interop registry split three ways: host-static.ss keeps the
registries, the jhost record, the emit entry points and coercion helpers;
host-static-statics.ss holds the java.lang/util static-method registrations;
host-static-objects.ss holds the host object classes (ArrayList, HashMap, the
reader/writer/tokenizer shims, ctors, URL codecs) and the instance? hook. rt.ss
loads the three in order. Runtime shim — no seed change.
The 1123-line collection tier is the largest source file. Cut it at two existing
section banners into 20-coll (predicates, printing, hierarchies, pure-over-core
leaves), 21-coll (rand/sort seams, the test runner, fn combinators), and 22-coll
(canonical Clojure ports, transduce/into, JVM-shape stubs). No macros in this tier,
so order is the only constraint; the emit-image manifest lists the three in
sequence. Re-minted seed is identical apart from gensym label renumbering.
check-user-call rebuilt the all-:any env once per parameter (O(params^2)) and
re-inferred a callee body at every call site. Build the env once and memoize each
probe by [key i argtype] (and the baseline by [:base key]), cleared per form in
check-form. The global type-env is stable within a form's check and the probe's
calls/escapes side effects aren't read there, so a skipped repeat is observably
identical. (The inline-side re-walk the audit flagged is moot: hc-inline-ir is a
no-op on Chez, so try-inline never reaches body-size/body-closed?.)
ei-emit-ns (emit-image) and bld-emit-ns (build) were near-verbatim copies that had
drifted: the minter guard-wraps and skips failing forms, the build is strict, and
since the passes were wired the build also runs run-passes. Fold both into
ei-emit-ns* with optimize?/guard? flags; ei-emit-ns and bld-emit-ns become one-line
callers. Output is byte-identical (selfhost fixpoint and build smoke stay green).
- str-join delegates to str-join-strs (it only adds the per-element render).
- loader: extract resolve-on-roots; find-ns-file and load both use it.
- NumberFormat registers its short + FQ names from one shared member list.
- inst-time's private floor-div/floor-mod renamed inst-floor-* so they don't read
as the math.ss reals version.
Left the fold/inline/types pure-fn sets and the keyword/symbol ns builders alone:
those are file-local and semantically distinct (e.g. the intern key uses NUL, not
"/"), so merging them would be wrong.
- take-last / drop-last return seqs, not vectors: take-last wraps in seq; drop-last
is the JVM (map (fn [x _] x) coll (drop n coll)) form (lazy, () when empty).
- cycle is lazy ((lazy-seq (concat coll (cycle coll)))) so it no longer counts its
argument and terminates on a lazy/infinite input.
- fold's foldable-call catch uses :default, matching the rest of jolt-core and
also catching a raw host condition from a folding primitive.
- alts! rejects non-channel ports with a clear error (put specs / :default are
unsupported) instead of crashing inside ac-poll!.
- Misc: drop the unreachable second getCause clause; jolt-nth on a string raises
'nth "index out of bounds" like the vector branch; name the inline fixpoint cap;
bld-sh-capture rejoins lines with newlines; clarify a couple of comments.
map-ir-children single-sourced the child layout for rewrite passes; the read-only
analyses each re-enumerated ops by hand. Add a fold companion, reduce-ir-children,
and rebuild body-size, pure?, and body-closed? on it (each reduces to a leaf value
+ the special ops it actually needs). local-escapes? stays an explicit walk — its
default is conservatively true and it inspects node shape beyond child purity, so
folding an unhandled op over its children would be unsound for scalar replacement.
analyze-special inlined def (~35 lines) and set! while try/letfn/fn* were already
helpers. Pull both out and move field-head? above analyze-special so its set! arm
and analyze-list reach it without a forward reference — the file's "only analyze
is forward-declared" invariant holds again. Pure code motion.
The fold/inline/types passes and the jolt.passes façade were baked into neither
seed half and never invoked: compile-eval and build went analyze -> emit directly,
and `jolt build --opt` flipped an optimize flag that nothing consumed.
- Compile the passes into the image (emit-image manifest): fold, inline, types,
then the jolt.passes façade, after jolt.ir.
- compile-eval and build.ss now run jolt.passes/run-passes between analyze and
emit. Off the direct-link path it is a pure const-fold; `jolt build --opt`
turns on inline + flatten + scalar-replace + type inference (it sets
hc-optimize?, which inline-enabled? reads).
- The seed minter (emit-image) stays analyze -> emit, so the seed is built
un-optimized and the self-host fixpoint is unaffected.
build-smoke already exercised --opt; it now actually optimizes and still matches
the release binary's output. Corpus floor and the fixpoint are green.
Round 1 (correctness + dead code):
- Fix duplicate java.util.HashMap registration in host-static.ss: the alist
impl shadowed the hashtable ctor while leaving the hashtable methods bound,
so .keySet/.values/.remove/.clear crashed. Drop the alist version.
- Delete jolt-core/jolt/reader.clj: a 463-line dead duplicate reader, never
required or compiled (the live reader is host/chez/reader.ss) and drifted.
- Remove dead defs: ir/rt + :rt op + unused ir/op; the Janet branch in
clojure.edn/drain-reader; a shadowed first clojure.string/trim-newline;
io.ss jolt-char-array + the reader def-var (both shadowed by natives-array);
concurrency.ss jolt-future-done?*; compile-eval.ss jolt-analyze-emit.
Round 2 (perf + determinism):
- emit-quoted-map-value / quoted sets now emit sorted by emitted text instead
of host-hash order, which isn't stable across Chez versions (jolt-8479).
- jolt-into folds through a transient, so into/vec/mapv/filterv onto a vector
are O(n) instead of O(n^2).
- deps resolve-deps walks its queue with an index cursor (was subvec-per-pop).
- async channel and agent action queues use amortized-O(1) FIFOs; ArrayList is
backed by a growable vector (O(1) add/get) instead of a list.
Thread/yield was a no-op and Thread/interrupted always returned false. Now:
- yield calls libc sched_yield (resolved once via the process symbols), so a
spin loop relinquishes the CPU. Falls back to a zero-length park if the symbol
can't be resolved.
- each OS thread carries an interrupt flag (a box, thread-local). currentThread
returns a handle wrapping the calling thread's flag, so .interrupt from another
thread sets the target's flag. .isInterrupted reads without clearing; the static
Thread/interrupted reads and clears — JVM semantics.
Consolidates the Thread surface: currentThread + the instance methods live in
io.ss (where the handle and its classloader are built), the flag box + yield +
the interrupted static in host-static.ss. Unit cases cover yield, the read/clear
split, and a cross-thread interrupt over a future.
The apt chezscheme package ships petite+scheme only — no kernel dev files — so
the standalone-binary gate skipped on CI, leaving the whole jolt build pipeline
and the --opt inference passes uncovered on Linux. Build Chez v10.4.1 from
source (cached) to get libkernel.a + scheme.h, install the libs the kernel links
against, and set the Linux link flags. buildsmoke now runs for real in CI.
The standalone-binary build needs Chez's kernel dev files (libkernel.a,
scheme.h) and a C compiler. A distro chezscheme package ships neither, so the
gate failed on CI (apt installs petite+scheme only). Preflight for the csv dir
and cc and skip cleanly when they're missing — same pattern as certify skipping
without Clojure. Where the toolchain exists (dev machines), it runs as before;
the discovered csv dir is pinned via JOLT_CHEZ_CSV so the build uses exactly it.
Wire the optimization gate to build mode. inline-enabled? (which gates the
inference + flatten-lets + scalar-replace passes in jolt.passes/run-passes) was
hardwired off, so those passes had never run on Chez at all. host-contract now
exposes a settable hc-optimize? flag; `jolt build --opt` flips it on during app
emission.
Kept off for the default release build for now: the passes are sound by design
(RFC 0005/0006) but unexercised on Chez, so release stays on the proven
var-deref codegen until they're validated against the corpus. --opt is the
opt-in fast path. buildsmoke checks both modes produce the same result.
This does not yet deliver direct call binding — the backend has no direct-link
emission path (every :var call still routes through jolt-invoke/var-deref) and
the inline-ir host stash is still a stub. Those are the remaining stage-4 levers.
Restores the standalone-binary capability the Janet host had. `bin/joltc build
-m NS -o OUT` AOT-compiles an app into a single self-contained executable — the
whole runtime, clojure.core, stdlib and compiler embedded, no Chez install or
jolt source needed at runtime.
Pipeline (host/chez/build.ss, host primitive jolt.host/build-binary driven by
jolt.main's build command): resolve deps, load the entry namespace recording the
app namespaces in dependency order, re-emit each to Scheme, textually inline the
cli.ss runtime load sequence into one flat source + the app + a launcher, then
compile-file -> make-boot-file -> embed the boot as C bytes -> cc-link against
libkernel.a.
Two non-obvious bits: the compile pass runs in a fresh Chez, not the loaded
runtime (regex.ss shadows top-level `error`, which otherwise bakes a broken
reference into the boot); and the launcher installs scheme-start rather than
running -main at top level, since boot top-level forms execute during heap build
before argv is set, so args only reach -main through scheme-start.
Loader: a require of an in-memory namespace with no source file now no-ops, so
AOT'd app namespaces satisfy require in a built binary.
Mode flags (--opt/--dev, default release) are plumbed; the optimization passes
they gate come in a later stage. RFC 0007 has the design. Gated by `make
buildsmoke`.
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.
Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.
Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.
Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
Two nREPL divergences the library shakeout surfaced — both have a real host
mechanism on Chez.
Interrupt (jolt-amzy): Chez's engine timer (set-timer + thread-local
timer-interrupt-handler) is polled at procedure-call / loop back-edges, so a
running computation — even a tight loop — can be aborted from another thread.
concurrency.ss adds jolt.host/{make-interrupt, interrupt!, run-interruptible}: an
interrupt token is a shared box; run-interruptible arms a periodic timer whose
handler escapes (call/cc) when the token is set, throwing {:jolt/interrupted true}.
The eval thread is reused, not abandoned. (A thread blocked in a __collect_safe
foreign call only sees it on return — like the JVM not killing native code.)
Thread-local *ns* (jolt-6rld): chez-current-ns is now a Chez thread-parameter, so
each session worker / future has its own current ns (vars stay global, only the
pointer is per-thread). *ns* reads derive from it (dyn-binding.ss), and a bound
*ns* drives chez-current-ns — so (binding [*ns* the-ns] (load-string code))
resolves against the-ns, and concurrent in-ns across threads don't clobber each
other. Single-threaded behaviour is unchanged. All runtime .ss — no re-mint.
A reify that doesn't implement a given protocol method now dispatches to that
protocol's extended impls over the reify's host tags (e.g. an Object/default
extension) instead of erroring 'No reified method'. This is malli's pattern: it
reifies some protocols and relies on RegexSchema's default for the rest. A method
with neither a reify impl nor a default still errors.
Surfaced running the DB libraries (migratus) on the jolt db library:
- java.sql.SQLException .getNextException / .getStackTrace / .printStackTrace on
jolt throwables (conditions + ex-info) return nil/empty, so a library walking
the exception chain doesn't crash.
- java.net.URL ctor + .getProtocol (file/http), alongside the existing url shim.
- A generic java.lang.ClassLoader: getSystemClassLoader / a thread's
contextClassLoader resolve a named resource against the source roots (the same
model as clojure.java.io/resource) — file: URL or nil. Thread/currentThread.
These are generic host capabilities, not DB-specific.
The jolt-lang/db next.jdbc surface now runs migratus far enough to connect, build
the migrations table, and discover migrations; migratus's remaining dependency is
java.nio.file (FileSystems/Path/PathMatcher glob), a JVM filesystem API kept out
of core.
Part of the java.* host-class gap (jolt-1nnn). String/format delegates to the
core format engine; NumberFormat getInstance/getNumberInstance/getIntegerInstance
group the integer part and honor min/max fraction digits.
- java.text.SimpleDateFormat.parse parses the RFC1123/1036/asctime patterns
(day/month names, 2-digit-year sliding window, tz token), and .format renders
z/Z/X timezone tokens (GMT/+0000/Z) instead of emitting them literally. Date
gains toLocalDate/toLocalDateTime/before/after/equals. Fixes ring.util.time.
- read / read+string work over a host java.io reader (StringReader wrapped in a
PushbackReader): drain, parse one form, push the tail back. Fixes cuerdas
istr / << string interpolation (and selmer's <<), which read embedded forms
from the template via (read pushback-reader).
Shake-out from the conformance-library sweep. Host-side fixes (runtime .ss,
no re-mint) plus one analyzer change (re-minted):
- Exception fidelity: ex-info and host-constructed throwables (RuntimeException.
etc.) now carry their JVM class, so (class e), instance? across the exception
hierarchy, .getMessage, and clojure.test thrown?/thrown-with-msg? all work.
- .getBytes returns a seqable/countable byte-array and honors UTF-16/UTF-32;
String. decodes them. ->bytevector accepts byte-arrays (Base64).
- Universal .getClass / .toString / .indexOf / .lastIndexOf on any value/seq.
- record? uses the host jrec? predicate (the old (get x :jolt/deftype) crashed
on a sorted-map by invoking its comparator).
- extend-protocol to abstract host types (clojure.lang.Fn/IFn/APersistentVector,
java.net.URI) dispatches.
- New host classes: clojure.lang.PersistentQueue, java.util.ArrayList,
java.net.URI, java.io.File / java.util.UUID ctors, Double/Float ctors+statics,
regex instance? Pattern, System/setProperty.
- *assert* / *print-readably* are real settable/bindable vars.
- (symbol "ns/name") splits the namespace at the last slash.
- letfn fn params desugar destructuring (analyzer; re-minted).
unit.edn gains exinfo/hostobj/queue/hostctor/destructure regression rows.
The line REPL was broken (read-line called nil — the __stdin-read-line host seam
the clojure.core *in* reader drives was never implemented on Chez) and didn't
load the project, so (require '[some.lib]) failed. Now:
- __stdin-read-line reads a line from stdin (get-line); read-line / read / the
REPL work.
- repl resolves the project first (deps on the roots, native libs loaded), so
libraries are available — same context a run gets.
- jolt.nrepl: a jolt-native nREPL server (bencode over a loopback jolt.ffi
socket) speaking the real protocol — clone / describe / eval / load-file /
close, with stdout capture, :ns-scoped eval (in-ns; binding *ns* doesn't drive
load-string resolution here), and real error text. 'joltc nrepl [port]' applies
the project then serves; writes .nrepl-port. Editors (CIDER/Calva/Cursive)
connect and develop live; project libraries load in the session.
- ex-message returns nil for raw Chez conditions, so jolt.host/condition-message
exposes the condition text; the REPL and nREPL surface it instead of an opaque
#<compound condition>.
Why native, not real nREPL: nrepl.server is welded to java.util.concurrent
executors, two compiled Java helper classes, a DynamicClassLoader, Compiler
internals and a JVMTI agent — not faithfully shimmable. The wire protocol, which
is what clients depend on, is small and implemented directly.
Runtime .ss + jolt-core, no re-mint. Full gate green.
jolt-lang/http-client (clj-http-lite over jolt.ffi) replaces it, the same way
ring-janet-adapter replaced the built-in HTTP server. An HTTP client is a library
concern — jolt core no longer ships one or shells out to curl. Apps depend on the
library; (require '[jolt.http-client]) now resolves to its source.
Full gate green.
Gaps the clj-http-lite suite hit running over jolt-lang/http-client:
- with-open now closes a tagged-table stream shim via its .close method (was a
:close field lookup that only matched the Janet-era tables).
- slurp accepts a bytevector / jolt byte-array / a byte input-stream shim and
honors :encoding — clj-http-lite slurps response bodies and :as :stream bodies.
- (.getBytes s charset) and Charset/forName respect the charset (ISO-8859-1 etc.,
one byte per char), not always UTF-8; Charset/forName returns the name string.
- (class e) reads the JVM :class off a throwable tagged-table, so clojure.test's
(thrown? Class …) / (= Class (class e)) match a library's typed exceptions.
- Registered the HTTP/IO exception class names (IOException, UnknownHostException,
SocketTimeoutException, SSLException, …) so the FQ literals self-evaluate.
All runtime .ss shims, no re-mint. Full gate green.
clj-http-lite drives bytes through clojure.java.io/copy (response body into a
ByteArrayOutputStream, request body out) and resolves URLs via io/as-url. Neither
worked for a library shim:
- io/copy was absent. Add it: raw bytes / string / a jhost reader write in one
shot; any other source drains via .read into a buffer and .write to the dest,
both through method dispatch — so a library's tagged-table streams copy without
the host knowing their layout.
- io/as-url ignored a library-registered URL class, so it and (URL. spec)
disagreed (the file-only jhost has no getProtocol/getHost/...). It now honors a
registered URL ctor, falling back to the jhost.
Runtime .ss shims, no re-mint.
The host carries bytes two ways: Chez bytevectors (what String/.getBytes
produce) and jolt byte-arrays (what byte-array / the Java-array shims use). They
didn't interconvert, so code mixing the two — like clj-http-lite, which buffers
into (byte-array n) but encodes via .getBytes and decodes via (String. ^[B body
charset) — broke.
- byte-array now also accepts a bytevector or a string (UTF-8 bytes), so the two
representations convert freely at interop seams.
- (String. bytes [charset]) decodes a bytevector OR a jolt byte-array with the
named charset (UTF-8 default; ISO-8859-1/latin1/ascii = one byte/char). It
previously only took a bytevector and ignored the charset.
Runtime .ss shims, no re-mint. Unit covers both directions + charset.
read-bytes/write-bytes go through UTF-8 (with a latin1 fallback), which mangles
arbitrary binary — gzip payloads, TLS records, any non-text body. An HTTP client
moving bytes between jolt byte-arrays and foreign socket/zlib/OpenSSL buffers
needs byte-exact transfer. read-array returns a fresh byte-array of n bytes from
foreign memory; write-array copies a byte-array's bytes into a pointer. Test
covers a round-trip preserving high bytes (200, 255).
clj-http-lite drives java.net URL/HttpURLConnection and java.io byte streams
through .method interop. The Chez host had __register-class-ctor!/-statics! (what
router/reitit needs) but no way to register instance methods on a shim object or
to extend instance?. Add both, plus jolt.host/table?:
- tagged-table .method dispatch: an htable-arm on record-method-dispatch routes
(.m obj a*) through a per-tag method registry keyed off the table's jolt/type;
unregistered methods fall through (sorted colls are htables too).
- __register-instance-check! installs (fn [class-name val] -> true|false|nil),
nil = fall through; chained ahead of the base instance-check.
Runtime .ss shims, no re-mint. Unit covers dispatch, args, instance? both ways.
The HTTP server moves out of the host into the jolt-lang/ring-janet-adapter
library, which binds sockets itself via jolt.ffi and shuts down cleanly. Drop
host/chez/http-server.ss and the obsolete ffi-server-test (the FFI collect-safe
path is covered by ffi-binding-test; the server by the adapter's own CI).
Derive os.name from Chez's machine-type (*osx -> Mac OS X, else Linux/Windows).
OS-branching code (socket sockaddr layout, etc.) needs the truth; a library
binding sockets via jolt.ffi reads os.name to pick the platform struct layout.
A library binding a blocking native call (accept/recv/connect/...) needs it
emitted __collect_safe so the thread deactivates for the call and doesn't pin
the stop-the-world collector. foreign-fn / defcfn take an optional trailing
:blocking; the backend emits (foreign-procedure __collect_safe ...). Needed for
the ring-janet-adapter socket-server port. ffi-binding-test asserts a thread
parked in a :blocking call doesn't block (collect).
The sqlite/jdbc functionality moves out of the host into the jolt-lang/db
library, which binds libsqlite3 (and libpq) itself via jolt.ffi. A baked
built-in jdbc.core would shadow the library's, so it's removed here. ring-app
gets jdbc.core from the db git dep instead.
A jolt library can now bind its own native dependencies and expose a Clojure API
over them — no jolt built-in required. This is the foundation for moving the
http-client / db / adapter functionality out of the host and into real libraries.
- jolt.ffi/foreign-fn (sugar: defcfn) is a compiler special form: a compile-time
-typed C signature lowers to a real Chez foreign-procedure (analyzer :ffi-fn ->
backend foreign-procedure), so calls are typed and marshaled, not eval'd.
- host/chez/ffi.ss provides the rest under jolt.ffi: load-library, alloc/free,
read/write/sizeof, ptr<->string, null/null?. Loaded after the loader snapshot
so a library's (require '[jolt.ffi]) still loads the macro side.
- Types: int/uint/long/ulong/int64/uint64/size_t/ssize_t/iptr/uptr/double/float/
pointer/string/void/uint8/char.
Validated end to end: a pure-Clojure file binds libc (getpid/strlen/abs) and
libsqlite3 (open/prepare/step/column/finalize over out-param pointers) and runs a
query. Gate test test/chez/ffi-binding-test.ss (make ffi); selfhost holds.
Three related namespace-resolution fixes surfaced porting the clojure.tools.logging
library, all general:
- chez-register-spec! treats a :use :only vector like :require :refer, so a
(:use [ns :only [names]]) clause actually brings those names in. Before, a bare
reference to an :only'd name resolved to nil.
- syntax-quote (hc-sq-symbol) qualifies a referred name to its SOURCE namespace,
not the compile namespace — so a macro that syntax-quotes a referred var (e.g.
clojure.tools.logging/spy reaching clojure.pprint's pprint) expands correctly.
- resolve consults :as aliases and :refers like ns-resolve does; (resolve
'alias/name) was returning nil because the alias wasn't expanded.
Bring the docs in line with the actual implementation now that Chez is the sole
substrate.
Deleted the migration/spike/handoff artifacts that only documented the Janet
era or the port effort: the port plan, phase-0 and foundational-runtime spike
writeups (+ the stray root-level copy), the self-hosting design notes, the
architecture-refactor plan, and spike/chez/RESULTS.md.
Rewrote the current reference docs against the Chez facts: building-and-deps and
tools-deps (no jpm/build step — bin/joltc off the checked-in seed, deps via
jolt.deps into ~/.jolt/gitlibs), libraries (SQLite is built-in jdbc.core over
libsqlite3, not a Janet driver), the conformance/spec test-flow docs (the Chez
corpus runner + certify, no .janet harnesses), and the transient / type-hint /
seed-overlay design notes (Chez representations: mutable transients, flat
copy-on-write vectors, HAMT maps, the seed/overlay twin). Fixed the README
collections line (vectors aren't 32-way tries) and added the ffi/transient gate
targets. rfc 0001's numerics open-question is resolved (the Scheme tower).
Renamed the built-in HTTP adapter to jolt.http.server only (dropped the
ring-janet.adapter alias — a Janet-era name).
The Chez port had landed transients as copy-on-write — each conj!/assoc!/etc.
rebuilt the whole persistent collection. Semantics were right but a transient
vector was O(n^2) to build (the persistent vector is a flat array, so every
conj! copied it); maps/sets were ~O(n log n) since the HAMT only path-copies.
This restores the Janet host's approach: true mutable backing, snapshot once on
persistent!.
vec : a growable Scheme vector (capacity + fill count); conj!/pop! amortized
O(1), persistent! hands off the buffer (exact fit) or trims once.
map : a Chez hashtable keyed by key-hash/jolt= (value equality, nil-safe);
persistent! folds it into a pmap.
set : a Chez hashtable; persistent! folds into a pset.
cow : fallback for anything else (e.g. a sorted coll) keeps the old
copy-on-write path, preserving jolt's superset.
get/count/contains?/nth see through each representation. Building a 400k vector
went from minutes (quadratic) to ~50ms (linear). assoc! keeps the variadic
dangling-key nil-pad on both vectors and maps. test/chez/transient-test.ss pins
the invariants and the linear-time property; wired in as `make transient`.
Two thread-safety bugs in the native FFI layer.
The HTTP server's accept/recv/send were plain foreign-procedures. A thread
inside a foreign call stays active for the stop-the-world collector, so the
accept loop sitting idle in accept() froze GC for the whole process whenever
another thread (a future, an async block) allocated. Mark the three blocking
calls __collect_safe so the thread deactivates for the call's duration —
collection proceeds while the accept thread waits. The args are an fd and
foreign-alloc'd buffers (outside the Scheme heap), so a collection mid-call has
nothing to move.
jolt.http-client built its -D header-file path from an unguarded (set! counter
(+ counter 1)) and counter mod 90000, with no per-process component. Concurrent
requests could compute the same path and clobber each other's headers. Use a
mutex-guarded monotonic counter plus the pid.
test/chez/ffi-server-test.ss exercises both (a (collect) while the server is
idle in accept(), temp-path uniqueness across threads, and a live request) and
is wired into the gate as `make ffi`.
malli loads and validates now. Three divergences surfaced building its registry
and :map schema:
dot-forms: (.iterator coll) on a map fell into the map-as-object branch and was
mis-read as a missing :iterator key (nil), so malli's -vmap got nil and crashed
on .hasNext. Route iterator through the collection-interop path — a jiterator
over the seq (the entry iterator for a map).
host-static: register clojure.lang.LazilyPersistentVector/createOwning (-vmap
fills an object-array then hands it over) and PersistentArrayMap/createWithCheck
(malli's eager entry parser relies on its duplicate-key throw; a missing class
was caught and mis-reported as ::duplicate-keys on every map schema).
Shaking the ring-app example's real library stack out against jolt surfaced a
batch of divergences from JVM Clojure, the biggest being evaluation order.
backend_scheme: call and recur arguments were emitted as bare Scheme operands,
so Chez's unspecified (right-to-left) order won out. Clojure evaluates left to
right, which selmer's reader loop relies on: (recur (add-node ... rdr) (read-char
rdr)) consumed a char early and dropped the first chars of every {{tag}}. Bind
operands to fresh temps in a let* (only when two or more can have side effects,
so hot calls over locals/consts stay un-wrapped). emit-ordered already did this
for collection literals; generalize it.
host-contract: syntax-quote now resolves the alias part of a qualified symbol
(impl/foo -> clojure.tools.logging.impl/foo) instead of leaving it bare, which
limped along via short-name matching until two loaded namespaces (reitit.impl,
clojure.tools.logging.impl) shared the short name and it broke.
collections: key-hash masks with bitwise-and, not fxand — jolt-hash is set!-
decorated per type (records return their own hash) and Chez's equal-hash can be a
bignum, so a key's hash isn't always a fixnum.
seq: even?/odd? handle bignums (JVM accepts any integer; the fxand crashed).
records: Keyword/Symbol .sym/.getName/.toString (honeysql's :clj branch reads
(.sym k)); Throwable .getMessage/.toString over a Chez condition.
host-static: __register-class-ctor!/__register-class-statics! so a host shim
(reitit.trie-jolt) can mirror a Java class.
natives-str: String.intern returns the string.
sqlite: jdbc.core fetch/fetch-one kebab-case column keys (the jolt-lang/db
convention; created_at -> :created-at).
io: a relative io/file path resolves against JOLT_PWD (the user's cwd), not the
repo root the launcher cd'd to — matches JVM cwd semantics, so config.edn loads.
cli: render an uncaught jolt throw (ex-info message + ex-data, or a condition)
instead of Chez's opaque "non-condition value" dump.
Gaps surfaced loading ring-core / hiccup / config / tools.logging on Chez:
- clojure.java.io/resource — resolve a named resource against the loader's
source roots (no classpath), returning a slurp-able File.
- (Object.) constructor — a fresh distinct value (lock / unique sentinel).
- *out* / *err* as dynamic vars over a port-writer, so (binding [*out* *err*] …)
and #'*out* compile and run (tools.logging, selmer).
- :refer :all now registers a refer-all relation, so an unqualified var from a
(require '[ns :refer :all]) resolves at compile time — including #'var.
- java.lang.Character interop ((.toString \+) etc.).
- StringReader accepts a char[] ((StringReader. (char-array s))), not just a
String.
- the \p{L} translation stops just below the UTF-16 surrogate gap (\x{D7FF})
instead of \x{10FFFF} — a range across the surrogates made irregex's char-set
construction call integer->char on a surrogate and crash.
sqlite.ss: jolt.sqlite + jdbc.core (jolt-lang/db's API) over the system
libsqlite3 — open/close, exec, a prepared query returning row maps, text/int/
double parameter binding, last_insert_rowid. The sqlite3 C API is non-variadic
so it binds directly.
http-server.ss: a minimal HTTP/1.1 server over BSD sockets (socket/bind/listen/
accept/recv/send via FFI), one connection at a time on a background accept
thread, synchronous Ring handlers. Parses the request line + headers + a
Content-Length body into a Ring request map (:body a StringReader), formats a
Ring response map back. Exposed as jolt.http.server and, for the example, as
ring-janet.adapter/run-server. macOS and Linux socket-option constants handled.
A synchronous HTTP client def-var!'d into jolt.http-client (get/post/put/delete/
head/request -> {:status :headers :body}), with :headers, :body, :query-params,
:content-type and :insecure?. It shells out to the system `curl` rather than a
direct libcurl FFI: on Apple Silicon curl_easy_setopt is variadic and Chez's
fixed-signature foreign-procedure can't place the value arg on the stack where
the ABI expects it, so a direct bind silently drops the option. curl gives the
same native TLS/redirect/gzip with no per-platform C shim.
format now honours width and the -/0 flags (%-30s, %5d, %05d), not just %.Nf
precision — it was emitting the directive literally.
