Pivot from a jolt reimplementation to running the upstream library verbatim.
Vendors the real clojure/tools/logging.clj; jolt provides the backend and the
host primitives it needs. Language features (broadly useful for real Clojure
libs), all covered in 3-mode conformance + spec suites:
- defmacro: multi-arity dispatch (jolt-q8l) and a docstring + attr-map + params
head (jolt-qnr) — the 4-arity log macro and every level macro need these.
- syntax-quote resolves an alias-qualified symbol to its target ns (jolt-9av),
so a macro template (impl/get-logger) resolves at the use site.
- the ns macro unwraps ^{:map} metadata on the ns name (jolt-8w2 workaround,
matching def/defn/defmacro).
- a namespace object self-evaluates, so ~*ns* can be spliced into a template.
Host shims (ported from / modeled on clojure where applicable):
- clojure.string/trim-newline (ported, CharSequence interop -> count/subs)
- agent/send-off/send (minimal synchronous stubs; jolt has no thread pool/STM)
- clojure.lang.LockingTransaction/isRunning -> false
- a minimal clojure.pprint (pprint/with-pprint-dispatch/code-dispatch, for spy)
- clojure.tools.logging.impl: a jolt stderr LoggerFactory backend (the library's
designed pluggable extension point)
docs/libraries.md lists tools.logging; grammar.ebnf metadata note clarified.
Conformance 355/355 x3 modes; full jpm test gate green.
Everything reitit-core needs to load unmodified from git under :clj features:
- The baked binary now re-reads JOLT_FEATURES at startup (like JOLT_PATH).
reader-features-set! runs at module load = BUILD time for a binary, so a
process opting into :clj (to read a lib's :clj branches) was ignored, and
unmatched #?(...) forms silently spliced to nothing — defn of a fn with an
empty arglist, hence the cryptic index errors.
- (get s i) indexes a string and returns the char, as in Clojure (nth did;
get returned nil). reitit's path parser is (get path i)-based — without
this every route read as static.
- Class-shim registration exposed to Clojure: __register-class-statics! /
__register-class-methods! / __register-class-ctor!, so a library can mirror
a Java class jolt doesn't ship (the reitit.Trie mirror lives in jolt-lang/
router on top of these).
- Java surface reitit's :clj branches call: .getMessage (on exceptions and
strings) and a small universal object-method set, .intern, java.util.HashMap
(a mutable map wrapper). Plus defprotocol already took keyword options.
Gate green; clojure-test-suite 4715 -> 4718 (the get-on-strings fix).
Clojure's defprotocol takes an optional docstring and leading keyword
options (:extend-via-metadata true) before the signatures; jolt's macro
fed the option keyword to (first sig). honeysql declares its InlineValue
protocol exactly that way — with the fix, all four honeysql namespaces
load unmodified from git and the formatter produces correct sqlvecs for
selects/inserts/updates/deletes/joins/:inline. Listed in libraries.md.
The vendored spork/http is gone — jpm owns janet packages. In its place:
- The janet.* bridge autoloads jpm-installed modules on first reference:
janet.spork.http/server requires spork/http from the module path and
caches its bindings (failures are negatively cached). Works for any
module, in every mode, including inside net/server connection fibers.
- deps.edn grows a :jpm/module coordinate: jolt-deps verifies the module is
importable at resolve time, optionally running `jpm install` on the
:jpm/install package once when it isn't, and otherwise fails with the
install hint. Contributes no source roots. ring-app declares spork/http
this way.
Docs: README's interop section, docs/tools-deps.md (:jpm/module reference),
and the ring-app README (including the jpm-version caveat for spork HEAD's
.janet native sources, which older jpm rejects).
Three canonical-conformance fixes from the post-shrink batch:
- bit-and/bit-or/bit-xor/bit-and-not get Clojure's variadic arities as
20-coll shells folding the binary host ops (now __bit-* seams). 2-arg call
sites still compile to the native janet op via the backend's native-ops
table. The passes.clj constant-fold table now names the seams — the public
fns are overlay and don't exist when the compiler loads (this briefly broke
every compile-mode init).
- core-set? recognizes the :jolt/sorted-set representation (jolt-dpn):
(set? (sorted-set 1)) was false, and ifn? on sorted sets inherited the bug.
- (if) / (if test) / (if test then else extra) throw in both the analyzer
and the interpreter — spec 03-special-forms X1, now marked verified.
Suite 4704 -> 4706; bench and the greeter example benchmark are flat.
Fixed arities now throw Clojure's ArityException shape — 'Wrong number of
args (N) passed to: name' — on any count mismatch; variadic arities on fewer
than their fixed params. The compiled path already enforced fixed arities via
janet's native fn check and multi-arity dispatch; this adds the check to the
interpreter's single-arity closures (the oracle was silently dropping extra
args and giving a raw tuple-index error for missing ones) and guards the
compiled single-variadic wrapper's minimum. Messages carry the fn name when
there is one. 16 spec rows; the update.cljc suite row flipped green (4703 ->
4704).
Enforcement exposed that seq-to-map-for-destructuring had drifted: the spec
row called the 1-arity fn with two args, and the body silently dropped a
trailing unpaired element. Replaced with the canonical Clojure 1.11 version
(even pairs build the map, a single trailing element passes through — so
(f {:b 2}) kwargs calls work — and an unpaired key throws).
Also: transients RFC notes tuple support from the seed-shrink rounds.
Git clones now default to a shared, sha-immutable cache —
$JOLT_GITLIBS, else <config-dir>/gitlibs — instead of a per-project
./jpm_tree, the tools.gitlibs ~/.gitlibs model. Passing tree
explicitly still works (tests do). The resolved-roots cache moves
out of the clone tree to the project-local .cpcache/jolt-deps.jdn,
since roots depend on the project while clones don't.
deps.edn grows :tasks, the honest subset of babashka's: a string
task is a shell command, a map task is {:main-opts [...] :doc}.
jolt-deps tasks lists them (merged user+project), jolt-deps task
NAME runs one. Bare-expression tasks are out of scope: the reader
hands back parsed data and round-tripping to source is fragile.
Also fixes load-config skipping the symbol-key normalization when
only one config file existed — :tasks/:deps keys stayed raw reader
symbols (which embed positions and never compare equal), so lookups
missed. Regression rows in deps-tasks-test; docs updated for the
whole tools.deps surface (aliases, -A/-M, user config, conflicts,
gitlibs cache, tasks).
require-:as wrote the string-keyed :imports table (which resolution reads)
while ns-aliases read the symbol-keyed :aliases table (which nothing wrote)
— so (ns-aliases) was always empty and the alias fn had to write both as a
bridge. :aliases (alias-name string -> ns-name string) is THE store now:
require :as and the alias fn write it, both resolution paths read it first
(falling back to :imports for class imports, which is all that table holds
now), ns-unalias removes one entry, and ns-aliases presents Clojure's
{alias-symbol -> namespace object} shape built from it. ns-resolve's
qualified path goes through the same lookup.
Also: the coverage dashboard's last 'resolvable-not-interned' entry was '.'
— which (resolve '.) returns nil for on the JVM too; the tool now classifies
it as the special form it is, and that category reads ZERO.
7 new unified-alias spec rows (require/alias/ns-unalias round-trips through
both the resolution and introspection views); the white-box namespace test
tracks the accessor rename. Gate exit 0.
test/spec/untested-vars-spec.janet adds 143 rows asserting jolt's documented
behavior for the whole implemented-untested category — primed arithmetic,
the array/aset/coercion stubs, unchecked-*, the chunk family, JVM-shape
stubs (class/bean/proxy/memfn as resolve-only or :throws), ns/REPL
machinery, and the misc seqs. tools/spec_coverage.py now checks each var as
a whole TOKEN in the test sources (call-position-only matching missed *1,
+', ., .., /, and bare transducer refs like cat).
Writing rows from probed truth surfaced five real bugs, all fixed:
- comp with a jolt-IFn stage silently returned nil ((comp seq :content)) —
raw Janet keyword application is not jolt invoke. comp is the canonical
overlay defn now (fixed-arity composed fn, so the hot 1-arg path is two
direct calls); the seed keeps a private td-comp only for the transducer
machinery. hof bench +9% vs native, the price of correct IFn dispatch.
- extend (the fn) was a nil-expanding stub MACRO shadowing any definition;
it's a real fn over register-method now, and extends? (a constant-false
stub) is real over extenders
- (.. x f g) hit the 'ClassName.' constructor branch (a name ending in a
dot) and died; .. is the canonical threading macro now
- aclone errored on pvecs; ns-interns/ns-imports returned live host tables
that count/seq reject (now structs)
Thread/sleep + Thread/yield land as Thread statics beside Math/: sleep parks
the WORKER's own event loop (each future thread has one), which makes timed
deref provably fire — futures-spec gains the timeout-fires, sleep-in-body,
and timed-out-future-still-completes rows. The futures impl itself already
ran on real OS threads (ev/spawn-thread + marshalled results); jolt-ejx was
stale.
Dashboard: implemented+tested 433 -> 564 of 694; implemented-untested and
missing-portable are both EMPTY. Gate: jpm exit 0, all tests passed.
The dashboard's missing-portable category is now EMPTY (was 35 when the issue
was filed; this session's io/leaf work had already landed most of them).
The final seven:
- extenders — ctx-capturing clojure.core fn over the protocol type-registry:
the type-tags implementing a protocol, as symbols; nil when none
- find-keyword — keyword: jolt keywords have no intern table, so it always
finds (babashka makes the same call)
- inst-ms* — the raw Inst method; one inst representation, so = inst-ms
- read+string — over the 50-io readers, which now expose :buf and :fill-fn;
returns [form exact-text-consumed], EOF throws or yields [eof-value ""]
with the 3-arity, works for string AND stdin readers
- with-local-vars — fresh free-standing var cells (__local-var seam) bound as
locals; var-get/var-set work on any cell
- with-open — canonical recursive expansion closing through the __close seam:
a map-like value's :close fn or a host file (no .close interop here);
nested closes run inner-first, finally runs on throw
- with-precision — body evaluates with precision/:rounding accepted and
ignored (doubles, no BigDecimal context) — documented divergence
30 new spec rows (test/spec/missing-vars-spec.janet); coverage.md
regenerated: implemented+tested 426 -> 433, missing-portable 7 -> 0.
Gate: jpm exit 0, all tests passed.
Pins down what a transient is in Jolt (tagged table over a native Janet
array/table, canonical-keyed for maps/sets), where behavior deviates from the
JVM (O(n) transient/persistent! edges with O(1) native ops between, no
owner-thread check — same as Clojure 1.7+, transient-of-list leniency), and
the three reasons the machinery is seed-resident rather than a migration
candidate: it IS the mutation kernel, it sits under the seed's own dispatch,
and the value layer is declared irreducible. Exists so the kernel-shrink
ladder (jolt-tzo) doesn't revisit transients every round.
*ns* is interned in clojure.core holding the current NAMESPACE OBJECT, kept
in sync by ctx-set-current-ns through a var table cached on the env (one
table put on the hot path; core-bench A/B neutral). A thread binding
(binding [*ns* ...]) shadows the root through var-get as usual. in-ns now
returns the namespace object (Clojure); str renders a namespace as its name
and pr-str as #namespace[name]. The ns-designator helper accepts namespace
objects (they are tagged STRUCTS — the old table?-based check never matched
them), so (ns-aliases *ns*) / (ns-unalias *ns* 'a) work — SCI and the jank
syntax-quote corpus use exactly that shape.
Known divergence (documented): inside an interpreted fn, *ns* reflects the
fn's defining ns (jolt's resolution model rebinds current-ns per call);
top-level and load-time reads match Clojure.
Gate green (conformance 326x3, suite 4572 >= 4540, all batteries,
specs+unit +8 *ns* rows); bench neutral.
The names turn 2a's leak removal exposed as honestly missing, now proper:
- slurp/spit/flush (host-classified): path-based IO over Janet files; spit
takes :append; flush flushes *out*. printf prints formatted (no newline)
over the existing format. file-seq walks paths via two host dir primitives
through the overlay's tree-seq.
- ns-map / ns-unmap / ns-refers (ctx fns). ns-refers required fixing the
refer MODEL: refer/use/:refer now map the SOURCE VAR into the target ns
(the Clojure model) instead of copying its value into a new var — so
source-ns redefinitions propagate, the :macro flag travels for free, and
refers are identifiable by the var's home :ns.
- Thread-binding family: with-bindings*/with-bindings, bound-fn*/bound-fn,
bound?, thread-bound?, get-thread-bindings. The captured binding map is a
Janet struct keyed by the var tables — the exact frame representation
var-get reads — so it re-pushes correctly (a phm frame is invisible to
var lookup).
- load-string and eval interned as VALUES at the api layer (they need the
loader's compile-or-interpret routing); the eval special form still
handles direct calls.
Suite 4532 -> 4572 (baseline floor 4540 across timeout variance, clean 86),
conformance 326x3, stdlib battery, all specs+unit (+21 turn-2b rows).
Coverage: missing-portable 27 -> 10 (left: the *in*-model readers, the
with-local-vars/with-precision/extenders tail).
resolve-sym's last resort silently resolved any unknown Clojure symbol
against Janet's root environment — leaking Janet builtins with JANET
semantics into Clojure code: (type 1) was Janet's :number, (gensym) returned
Janet symbols (the long-documented (symbol (str (gensym))) macro landmine
existed BECAUSE of this), compare/slurp/int?/any? likewise. The explicit
janet/ prefix is the deliberate interop channel; the implicit fallback is
gone — an unresolved symbol is an error.
What the leak was masking, now proper interned vars:
- gensym: jolt's own (already existed, never interned) — returns real jolt
symbols; the macro landmine is dead
- compare: full Clojure total order (nil-first, numbers, strings, keywords,
symbols by ns/name, booleans, chars, uuid/inst, vectors by length then
elementwise; cross-type throws)
- type: :type metadata override, deftype/record tag as symbol, else a
taxonomy keyword (host-classified)
- int?: core-integer? — which had a latent bug the leak hid: (integer?
##Inf) was true (floor of inf is inf); NaN/infinities now excluded
- any?: constantly true (Clojure 1.9; SCI's namespaces.cljc needs it)
- jolt.interop/janet-type now uses the explicit (janet/type x) channel
- evaluator-test uses init (a bare make-ctx resolved EVERYTHING via the leak)
Suite 4470 -> 4532+ pass / 86-87 clean (proper compare unlocks the sort
files); baselines raised. Conformance 326x3 (+5 rows), +22 predicate spec
rows, stdlib battery green, all specs+unit. Coverage dashboard now counts
previously-leak-resolvable names honestly (missing-portable 19 -> 27).
#inst (jolt-rnh): an instant is an immutable tagged struct
{:jolt/type :jolt/inst :ms <epoch-millis>} — equality and map-key hashing by
INSTANT, so different offsets denoting the same moment are =. The reader
parses RFC3339 with Clojure's partial-timestamp defaults (#inst "2020" is
2020-01-01T00:00:00.000Z) and errors on malformed input; inst?/inst-ms in
the overlay; pr-str prints the canonical
#inst "yyyy-MM-ddThh:mm:ss.fff-00:00" and round-trips; str gives the bare
RFC3339 string. Self-evaluating in the evaluator (like uuid/chars).
Syntax-quote (jolt-l2a): a syntax-quoted self-evaluating literal (string,
number, boolean, nil, keyword, char) collapses to the literal at READ time,
matching Clojure's reader — so nested/adjacent backticks over literals are
inert: (= "meow" ```"meow") is true (jank pass-adjacent). Symbols still
qualify and collections still template. General nested syntax-quote over
non-literals remains UNVERIFIED in spec S25.
Tests: inst-spec (18 cases incl. partial defaults, offsets, round-trip),
+10 literal-collapse reader rows, +5 conformance rows (321x3). Spec
02-reader S20/S25 updated to normative. Suite stable 4470/86.
One documentation root: the prose docs (building-and-deps, self-hosting
architecture/compiler, tools-deps, grammar.ebnf) join the spec and RFCs under
docs/. References in README and deps-conformance-test updated.
jolt no longer satisfies :clj in reader conditionals. The shortcut was a
measured net liability: :clj branches carry JVM interop and JVM-specific
test expectations jolt fails, and they shadowed :default branches jolt
passes. A/B over the suite: clj,default = 4967 assertions / 4324 pass / 119
errors; jolt,default = 5069 / 4470 / 81 (+146 pass, -38 errors, +8 clean
files). Baselines raised to 4470/86.
Matching is now by CLAUSE order like Clojure — the first clause whose key is
in the feature set wins (#?(:default 5 :clj 6) is 5 everywhere); the old code
scanned for :clj first, then :default, regardless of position.
Foreign clj-targeted libraries are a property of the LOADING CONTEXT, not the
platform: reader-features-set! opts a load into a compatibility set, and the
SCI bootstrap/runtime tests load SCI under ["jolt" "clj" "default"] (its
.cljc selects implementations via :clj with no :jolt branches).
JOLT_FEATURES remains the process-wide override.
RFC 0002 records the decision with the measured data; spec 02-reader S18 is
now normative (clause order, documented feature set, per-context override).
Reader tests updated to the portable set + an opt-in round-trip.
The lexical-syntax chapter, granularity modeled on jank's 62-file
per-construct reader corpus: token grammar (whitespace/comments, collections
with read-time duplicate checks, numbers incl. the N/M tower question,
symbols/keywords incl. ::auto-resolution, strings/chars), the quote-family
sugars, the full #-dispatch catalog with normative entries (anonymous fn
%-derivation, discard composition, reader conditionals, symbolic floats,
tagged literals), and the syntax-quote contract (core/alias/current-ns
qualification, template-stable gensyms, ~' idiom, distribution through
collections).
Adapting the corpus surfaced and filed three findings, recorded as labeled
divergences/UNVERIFIED in the chapter: nested syntax-quote doesn't collapse
(S25, (= "meow" ```"meow") is false), #inst reads as a bare string (identity
data reader, no instant type), and jolt satisfies :clj in reader
conditionals (feature-key policy under review).
reader-forms-spec gains 11 chapter-cited rows (discard stacking, ##Inf/
##-Inf/##NaN, :default conditionals, qualified var-quote identity, gensym
stability within vs across templates) — all passing.
Fifteen vars from the spec coverage gap (docs/spec/coverage.md):
parse-long/parse-double/parse-boolean (strict validation; scan-number alone
accepts 0x10), newline, current-time-ms (host clock for time), update-keys/
update-vals (PHM base, collisions last-wins), partitionv/partitionv-all/
splitv-at (lazy seqs of vectors; splitv-at's tail stays a seq, matching the
reference), with-redefs/with-redefs-fn (roots restored on throw), time,
macroexpand (expand-1 to fixpoint), alias/ns-unalias (write BOTH alias stores
— require :as uses string-keyed :imports while ns-aliases reads :aliases;
split filed), ns-publics (symbol-keyed map; publics == interns, no privacy).
Three pre-existing bugs fixed along the way:
- (partition n step pad coll) misparsed pad as the coll and returned ()
- (require 'bare.symbol) rejected — only vector specs were accepted
- analyze-form leaked the interpreted analyzer's ns on a punt: a throw out of
the analyzer left current-ns=jolt.analyzer and :compile-ns set, so the
fallback interpretation resolved user vars against the wrong namespace
(bit (var user-sym) under compile mode)
And one overlay-authoring landmine documented in-code: a 20-coll fn must not
use 30-macros macros (with-redefs-fn's dotimes compiled as a forward ref that
resolved to the macro fn at runtime) — loop/recur instead.
Gate: conformance 316x3 (+14 rows), suite 4324 pass / 78 clean (was 4081/72;
parse_*/update_* files now contribute), baselines raised, all specs+unit,
fixpoint, self-host, sci, staged. Coverage: missing-portable 35 -> 20.
RFC 0001 proposes a normative, implementation-independent Clojure language
spec (the reader, evaluation model, special forms, data types, seq/laziness
contracts, namespaces/vars, and the portable clojure.core surface) to the
standard of R7RS/Racket — Clojure has none, and every alternative
implementation re-derives semantics from the reference and folklore. The
spec is executable-first: every numbered normative statement cites its
conformance test or is marked UNVERIFIED.
docs/spec/ carries the front matter (conformance terms, entry format, host
classification), the special-form catalog with worked normative entries for
if and let*, the core-library entry format with worked entries for first,
reduce, and parse-uuid, and a generated coverage dashboard over the 694-var
ClojureDocs inventory (tools/spec_coverage.py cross-references the surface
against jolt's interned+resolvable vars and the test suites).
Measured baseline: 380 implemented+tested, 154 implemented-untested, 35
portable-but-missing (filed), 22 resolvable-but-not-interned (filed — seed
fns invisible to resolve/ns-publics), rest classified host/JVM/concurrency.
