The jolt-vs-hand-Janet-vs-JVM mandelbrot comparison splits the 15.4x floor
into two layers: a Janet-VM floor (~10.8x JVM, optimal while-loop Janet over
unboxed doubles — only native codegen moves it) plus a ~1.43x jolt loop-
lowering overhead on top. The overhead is entirely the loop/recur -> recursive-
closure-called-per-iteration lowering; hand-Janet written the same way matches
jolt, while a while+var/set version is 1.43x faster. So a cheap backend win
(jolt-v28u) sits above the structural native-codegen lever.
Adds the spike artifacts under bench/ and the results writeup; marks the spike
done in the handoff. No source changes.
Co-authored-by: Yogthos <yogthos@gmail.com>
The targeted-specialization work (jolt-ffn) concluded that the constant-factor
gap vs JVM is structural, not per-form: three targeted passes (field-read,
inline cache, ctor descriptor-bake) all came back flat. mandelbrot (pure
compute) is ~15x off JVM and that's the floor — Janet bytecode VM + mark-sweep
GC + indirect calls.
This doc hands off the successor epic (jolt-5vsp): the foundational levers
(native codegen, GC-pressure reduction, deeper devirt+inline) and, importantly,
the spike to run first — localize the 15x floor by comparing jolt-compiled vs
hand-written-Janet vs JVM mandelbrot before committing to any big lever. Also
records what not to repeat.
Co-authored-by: Yogthos <yogthos@gmail.com>
* Bind *command-line-args* after the deps-image cache swap (jolt-4mui)
Under whole-program (deps-image cache active), `jolt -m NS ARG` dropped ARG:
run-main set *command-line-args* on the current ctx, but a cache HIT then
replaced ctx with the saved image (via `set ctx cached`), whose *command-line-
args* was whatever got baked when the image was saved. The stale binding won at
`(apply NS/-main *command-line-args*)`, so -main ran with the wrong (usually
default) args — silently, for any optimized -m program.
Move set-command-line-args to AFTER the cache swap so it binds on the final ctx.
Repro/regression in deps-cache-args-test.janet: first run builds the image
(arg "first"), second run (cache hit) must echo "second", not the baked "first".
* docs: RFC 0003 — phm is a HAMT, sorted colls a red-black tree
The transients RFC described phm as "bucket-based copy-on-write" and mused about
"if it ever becomes a HAMT" — it is one now (jolt-684u), and sorted maps/sets are
a red-black tree (jolt-0hbr). Update the deviation/future-work notes accordingly.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Dependency resolution now lives in the `jolt` CLI itself instead of a separate
jolt-deps executable. `jolt` resolves a deps.edn into JOLT_PATH/JOLT_APP_PATHS
in-process and dispatches the deps subcommands:
jolt -M:alias [args] run the alias :main-opts
jolt -A:alias CMD run CMD with the alias paths
jolt run FILE resolve, then run FILE
jolt path | tasks | task NAME
A deps.edn in the working dir is auto-resolved for the runnable commands
(repl/-m/-e/nrepl-server/FILE), so e.g. `jolt -M:nrepl` (or plain
`jolt nrepl-server`) starts an nREPL with the project and its deps loaded.
The runtime core stays deps-agnostic — it only reads JOLT_PATH. The resolver
(deps.janet) is reached only from the CLI entry and loads jpm lazily, so a run
with no deps.edn never touches it and an app baked from its own jolt/api entry
never links it. resolve-deps-argv only resolves on an explicit deps command or
when a deps.edn is present; help/version never do.
jolt-deps stays as a thin deprecation shim that forwards to `jolt`, so existing
scripts keep working. Docs (README, CLAUDE.md, building-and-deps, tools-deps)
and the help text updated.
Co-authored-by: Yogthos <yogthos@gmail.com>
* Reader: #() params survive syntax-quote (auto-gensym names)
#(...) named its synthesized params with bare gensyms, so a #() written inside a
syntax-quote had its params qualified to the current ns by sq-symbol — and a
qualified symbol isn't a valid fn param. hiccup's compiler emits
`(let [sb# ..] (iterate! #(.append sb# %) ..)), which broke with "Unable to
resolve symbol: ns/_NNNN".
Name the params with a trailing # (auto-gensym suffix, like Clojure's p1__N#) so
syntax-quote maps them consistently and leaves them unqualified. Harmless outside
a backtick (just a regular symbol name).
* interop: String/valueOf static + String is a CharSequence
Two interop gaps surfaced bringing up hiccup and malli:
- String/valueOf(Object): hiccup's compiler stringifies attribute values with
(String/valueOf (or arg "")). Added the static — "null" for nil, else core-str.
- (instance? CharSequence s) returned false for a string; String implements
CharSequence, and malli's :re validator gates on it before matching, so :re
schemas always failed. instance-check now answers true for strings.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
5d: document the seed↔overlay boundary and add a drift check. core fns split
across a Janet seed (core-X, registered in core-bindings) and a Clojure overlay;
five names (char?/sorted?/sorted-map?/sorted-set?/transduce) carry a defn in
both, with the overlay copy authoritative and the seed copy internal-only. The
into-vs-transduce home asymmetry was undocumented. Adds docs/seed-overlay-registry.md,
SEED-TWIN: comments at the five seed sites, and a build-time drift check
(test/unit/seed-overlay-registry-test.janet) that recomputes the twin set from
source and fails if it diverges or a twin leaks into core-bindings.
5e: rep↔API pointer comments in pv/plist/phm/phs/lazyseq (representation lives
here; Clojure-facing ops dispatch in core_coll/core_types) and back-pointers in
core_coll. No behavior change — comments, docs, one source-analysis test.
Full gate green (suite ≥4695 pass / ≥88 clean files), drift check passes.
* Protocol/interop fixes to run metosin/malli
Bringing up malli (schema validation) surfaced a batch of protocol and host-interop
gaps. m/validate now works across the schema vocabulary (predicates, :map incl.
nested/optional, :vector, :tuple, :enum, :maybe, :and, bounded int/string).
- extend-type and reify now accept MULTIPLE protocols in one form (each bare
symbol switches the current protocol). reify records every protocol it
implements, so instance?/satisfies? recognise all of them.
- Protocol method params support destructuring: reify/extend-type/deftype/
defrecord emit (fn ...) (which desugars patterns) instead of raw fn*.
- instance? of a PROTOCOL works like satisfies? for reify/record instances,
matching short names across qualified/bare protocol references.
- @x reads as the qualified clojure.core/deref, so it still derefs where a ns
excludes and rebinds deref (malli does). Updated reader-test + the reader
spec/grammar (S11, deref rule).
- Java collection interop on jolt collections: .nth/.count/.valAt/.get/.seq/
.containsKey route to the clojure.core equivalent (1-arg and 0-arg paths).
- java.util.HashMap capacity/load-factor constructors + .putAll.
- A class used as a value resolves to its instances' type, so Pattern -> the
regex type (malli keys class-schemas by it).
- Shims for malli's load path: LazilyPersistentVector/createOwning and
PersistentArrayMap/createWithCheck statics.
m/explain not yet working (jolt-fjb1). Full gate green.
* satisfies? recognizes reify, consistent with instance?
A reify's protocol methods are instance-local, so they aren't in the global type
registry that type-satisfies? consults — satisfies? returned false for a reify
even when it implemented the protocol. Check the protocols the reify records on
itself (the same :jolt/protocols list instance? uses), matching short names like
instance? does. Covers single- and multi-protocol reify.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
* Add architecture refactor plan
Synthesizes a six-part architectural review into phased, gate-validated cleanup
work. Targets LLM-maintainability: one home per feature, no god-files, explicit
checked contracts, no copy-paste dispatch. No code changes yet — the plan only.
* Refactor phase 0: dead code + isolated bugs
Pure cleanup ahead of the structural phases (docs/architecture-refactor-plan.md).
No behavior change except the two bug fixes, which are covered by a regression row.
Dead code (all verified zero-reference or overridden):
- core-resolve / core-satisfies? / core-type->str seed stubs + bindings —
resolve and satisfies? are interned by install-stateful-fns! (the seed copies
were shadowed); type->str was an inert SCI stub with no callers.
- find defined twice in 20-coll.clj; the dead copy returned a plain vector
(wrong — the live def at :787 returns a real map-entry) with a comment that
contradicted it.
- mark-hint (passes.clj), phs-to-struct (phm), shape-vals / ns-imports-fn
(types) — unreferenced.
- redundant local pad2 in javatime (module-level one already in scope).
Bugs:
- File.toURL stored :url but every :jolt/url method reads :spec, so a URL from
(.toURL file) returned nil from all its methods. Now stores :spec (+ spec row).
- pl-rest had a no-op (if (plist? r) r r); collapsed to r.
- :map-shapes? was missing from the deps-image cache key — two runs differing
only in map-shapes could reuse each other's image.
Also dropped read-quote's unused pos param. Full gate green.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Running a program is a closed world — every namespace is required, then it runs
to completion — so make it direct-link by default (inlining, record shapes, the
inference's specialization), and for a -m/-M entry auto-enable the whole-program
cross-namespace inference pass. A decomposed multi-namespace program was ~3.7x
slower than the same code in one namespace purely because per-namespace
inference can't see a caller in a not-yet-loaded namespace; this closes that for
the common case with no flags and no hints.
Interactive modes (repl, -e, nrepl-server) stay indirect/open — they have to let
you redefine vars, which direct-linking seals against. Opt-outs:
JOLT_NO_DIRECT_LINK forces the open path even for a program run (hot-reload,
runtime redefinition); JOLT_NO_WHOLE_PROGRAM keeps direct-linking but per-ns;
JOLT_DIRECT_LINK / JOLT_WHOLE_PROGRAM still force-on. Namespaces required inside
-main (after the batch pass) fall back to per-ns inference.
The success checker (RFC 0006) rides on the inference for free, but a casual
program run shouldn't spam type warnings just because it now direct-links, so its
default-on is suppressed when direct-linking was auto-enabled (:direct-link-auto?);
an explicit JOLT_DIRECT_LINK or JOLT_TYPE_CHECK still turns it on. whole-program-
test and devirt-test opt their per-ns baseline out of the new auto-default.
Docs: RFC 0005 gains 'Compilation modes and defaults' + 'Cross-namespace
inference'; RFC 0004 documents cross-ns/param hints; self-hosting-compiler and
--help updated. Full gate green.
Pivot from a jolt reimplementation to running the upstream library verbatim.
Vendors the real clojure/tools/logging.clj; jolt provides the backend and the
host primitives it needs. Language features (broadly useful for real Clojure
libs), all covered in 3-mode conformance + spec suites:
- defmacro: multi-arity dispatch (jolt-q8l) and a docstring + attr-map + params
head (jolt-qnr) — the 4-arity log macro and every level macro need these.
- syntax-quote resolves an alias-qualified symbol to its target ns (jolt-9av),
so a macro template (impl/get-logger) resolves at the use site.
- the ns macro unwraps ^{:map} metadata on the ns name (jolt-8w2 workaround,
matching def/defn/defmacro).
- a namespace object self-evaluates, so ~*ns* can be spliced into a template.
Host shims (ported from / modeled on clojure where applicable):
- clojure.string/trim-newline (ported, CharSequence interop -> count/subs)
- agent/send-off/send (minimal synchronous stubs; jolt has no thread pool/STM)
- clojure.lang.LockingTransaction/isRunning -> false
- a minimal clojure.pprint (pprint/with-pprint-dispatch/code-dispatch, for spy)
- clojure.tools.logging.impl: a jolt stderr LoggerFactory backend (the library's
designed pluggable extension point)
docs/libraries.md lists tools.logging; grammar.ebnf metadata note clarified.
Conformance 355/355 x3 modes; full jpm test gate green.
Update the status, strictness levels, and open questions to reflect what
landed: bounded unions (jolt-pz5), user-fn domains behind
JOLT_TYPE_CHECK_USER (jolt-zo1), precise file:line:col (jolt-fqy), and the
checker folded into one inference walk that piggybacks on direct-link
specialization (on by default there, opt-in in plain builds). Align the
error-reporting example with the actual output format.
0005 proposes replacing the ad-hoc inference lattice with one recursive
structural type (a struct carries its field types, a vector its element type,
recursively), so a lookup returns its field's type and nested access is typed
end to end. It unifies :struct tracking with field tracking, subsumes the
current inference phases, and is the soft-typing (HM + a dynamic top) design:
structural types + core-fn type schemes, solved by lattice join with :any as
top instead of unify-or-fail. Includes the depth cap for termination and an
explicit design-problems section.
0006 (follow-up, depends on 0005) reuses the inference as a loose type checker
in the success-typing discipline (Dialyzer): report only PROVABLY-wrong code
(a concrete type in an operation's throwing error-domain), accept everything
ambiguous, never a false positive. Curated error-domain table, strictness
levels (off/warn/error), clear located messages, and the soundness boundaries
(closed-world, macros, unions).
Builds on the ^:struct keyword-lookup hint:
- ^TypeName for records. A tag naming a defrecord/deftype now resolves to the
struct fast path: record instances are tables tagged :jolt/deftype (not
:jolt/type), so a raw keyword get is correct for them. A new host contract fn
record-type? detects a record by its ->Name constructor; a non-record tag
(^String, ^long, ...) is ignored, as before.
- (get m :k) and (get m :k default) now get the same inlined keyword lookup as
(:k m): the representation guard fast path when unhinted, and the bare get
when the subject is ^:struct/^Record. A variable/number/string key still
falls through to core-get. The two call shapes share one emitter
(emit-kw-lookup).
- JOLT_CHECK_HINTS=1 turns a violated hint into a clear runtime error (naming
the local and key) by keeping the guard and throwing on the tagged arm. It is
off by default with zero cost to normal builds (a hinted lookup still emits a
bare get), and is part of the image-cache fingerprint. This is the answer to
"a lying hint is silent": opt into checking during development.
- Docs: RFC 0004 records the design, soundness contract, and measurements; the
reader spec gains S12b (hints are semantically transparent; jolt recognizes
^:struct and ^Record as lookup-optimization assertions).
There is no Clojure keyword equivalent for "plain map / fast keyword access"
(Clojure hints are class names), so ^:struct stays a jolt-specific flag,
analogous to ^:dynamic.
Verified: conformance 335/335 in all three modes and the full jpm test pass; a
seeded ray-tracer render is byte-identical hinted vs unhinted; the struct-hint
test covers record hints, the get-form, inline propagation, and the checked-mode
error. Full render with hints holds at 13.3s -> 10.9s (1.22x).
Everything reitit-core needs to load unmodified from git under :clj features:
- The baked binary now re-reads JOLT_FEATURES at startup (like JOLT_PATH).
reader-features-set! runs at module load = BUILD time for a binary, so a
process opting into :clj (to read a lib's :clj branches) was ignored, and
unmatched #?(...) forms silently spliced to nothing — defn of a fn with an
empty arglist, hence the cryptic index errors.
- (get s i) indexes a string and returns the char, as in Clojure (nth did;
get returned nil). reitit's path parser is (get path i)-based — without
this every route read as static.
- Class-shim registration exposed to Clojure: __register-class-statics! /
__register-class-methods! / __register-class-ctor!, so a library can mirror
a Java class jolt doesn't ship (the reitit.Trie mirror lives in jolt-lang/
router on top of these).
- Java surface reitit's :clj branches call: .getMessage (on exceptions and
strings) and a small universal object-method set, .intern, java.util.HashMap
(a mutable map wrapper). Plus defprotocol already took keyword options.
Gate green; clojure-test-suite 4715 -> 4718 (the get-on-strings fix).
Clojure's defprotocol takes an optional docstring and leading keyword
options (:extend-via-metadata true) before the signatures; jolt's macro
fed the option keyword to (first sig). honeysql declares its InlineValue
protocol exactly that way — with the fix, all four honeysql namespaces
load unmodified from git and the formatter produces correct sqlvecs for
selects/inserts/updates/deletes/joins/:inline. Listed in libraries.md.
The vendored spork/http is gone — jpm owns janet packages. In its place:
- The janet.* bridge autoloads jpm-installed modules on first reference:
janet.spork.http/server requires spork/http from the module path and
caches its bindings (failures are negatively cached). Works for any
module, in every mode, including inside net/server connection fibers.
- deps.edn grows a :jpm/module coordinate: jolt-deps verifies the module is
importable at resolve time, optionally running `jpm install` on the
:jpm/install package once when it isn't, and otherwise fails with the
install hint. Contributes no source roots. ring-app declares spork/http
this way.
Docs: README's interop section, docs/tools-deps.md (:jpm/module reference),
and the ring-app README (including the jpm-version caveat for spork HEAD's
.janet native sources, which older jpm rejects).
Three canonical-conformance fixes from the post-shrink batch:
- bit-and/bit-or/bit-xor/bit-and-not get Clojure's variadic arities as
20-coll shells folding the binary host ops (now __bit-* seams). 2-arg call
sites still compile to the native janet op via the backend's native-ops
table. The passes.clj constant-fold table now names the seams — the public
fns are overlay and don't exist when the compiler loads (this briefly broke
every compile-mode init).
- core-set? recognizes the :jolt/sorted-set representation (jolt-dpn):
(set? (sorted-set 1)) was false, and ifn? on sorted sets inherited the bug.
- (if) / (if test) / (if test then else extra) throw in both the analyzer
and the interpreter — spec 03-special-forms X1, now marked verified.
Suite 4704 -> 4706; bench and the greeter example benchmark are flat.
Fixed arities now throw Clojure's ArityException shape — 'Wrong number of
args (N) passed to: name' — on any count mismatch; variadic arities on fewer
than their fixed params. The compiled path already enforced fixed arities via
janet's native fn check and multi-arity dispatch; this adds the check to the
interpreter's single-arity closures (the oracle was silently dropping extra
args and giving a raw tuple-index error for missing ones) and guards the
compiled single-variadic wrapper's minimum. Messages carry the fn name when
there is one. 16 spec rows; the update.cljc suite row flipped green (4703 ->
4704).
Enforcement exposed that seq-to-map-for-destructuring had drifted: the spec
row called the 1-arity fn with two args, and the body silently dropped a
trailing unpaired element. Replaced with the canonical Clojure 1.11 version
(even pairs build the map, a single trailing element passes through — so
(f {:b 2}) kwargs calls work — and an unpaired key throws).
Also: transients RFC notes tuple support from the seed-shrink rounds.
Git clones now default to a shared, sha-immutable cache —
$JOLT_GITLIBS, else <config-dir>/gitlibs — instead of a per-project
./jpm_tree, the tools.gitlibs ~/.gitlibs model. Passing tree
explicitly still works (tests do). The resolved-roots cache moves
out of the clone tree to the project-local .cpcache/jolt-deps.jdn,
since roots depend on the project while clones don't.
deps.edn grows :tasks, the honest subset of babashka's: a string
task is a shell command, a map task is {:main-opts [...] :doc}.
jolt-deps tasks lists them (merged user+project), jolt-deps task
NAME runs one. Bare-expression tasks are out of scope: the reader
hands back parsed data and round-tripping to source is fragile.
Also fixes load-config skipping the symbol-key normalization when
only one config file existed — :tasks/:deps keys stayed raw reader
symbols (which embed positions and never compare equal), so lookups
missed. Regression rows in deps-tasks-test; docs updated for the
whole tools.deps surface (aliases, -A/-M, user config, conflicts,
gitlibs cache, tasks).
require-:as wrote the string-keyed :imports table (which resolution reads)
while ns-aliases read the symbol-keyed :aliases table (which nothing wrote)
— so (ns-aliases) was always empty and the alias fn had to write both as a
bridge. :aliases (alias-name string -> ns-name string) is THE store now:
require :as and the alias fn write it, both resolution paths read it first
(falling back to :imports for class imports, which is all that table holds
now), ns-unalias removes one entry, and ns-aliases presents Clojure's
{alias-symbol -> namespace object} shape built from it. ns-resolve's
qualified path goes through the same lookup.
Also: the coverage dashboard's last 'resolvable-not-interned' entry was '.'
— which (resolve '.) returns nil for on the JVM too; the tool now classifies
it as the special form it is, and that category reads ZERO.
7 new unified-alias spec rows (require/alias/ns-unalias round-trips through
both the resolution and introspection views); the white-box namespace test
tracks the accessor rename. Gate exit 0.
test/spec/untested-vars-spec.janet adds 143 rows asserting jolt's documented
behavior for the whole implemented-untested category — primed arithmetic,
the array/aset/coercion stubs, unchecked-*, the chunk family, JVM-shape
stubs (class/bean/proxy/memfn as resolve-only or :throws), ns/REPL
machinery, and the misc seqs. tools/spec_coverage.py now checks each var as
a whole TOKEN in the test sources (call-position-only matching missed *1,
+', ., .., /, and bare transducer refs like cat).
Writing rows from probed truth surfaced five real bugs, all fixed:
- comp with a jolt-IFn stage silently returned nil ((comp seq :content)) —
raw Janet keyword application is not jolt invoke. comp is the canonical
overlay defn now (fixed-arity composed fn, so the hot 1-arg path is two
direct calls); the seed keeps a private td-comp only for the transducer
machinery. hof bench +9% vs native, the price of correct IFn dispatch.
- extend (the fn) was a nil-expanding stub MACRO shadowing any definition;
it's a real fn over register-method now, and extends? (a constant-false
stub) is real over extenders
- (.. x f g) hit the 'ClassName.' constructor branch (a name ending in a
dot) and died; .. is the canonical threading macro now
- aclone errored on pvecs; ns-interns/ns-imports returned live host tables
that count/seq reject (now structs)
Thread/sleep + Thread/yield land as Thread statics beside Math/: sleep parks
the WORKER's own event loop (each future thread has one), which makes timed
deref provably fire — futures-spec gains the timeout-fires, sleep-in-body,
and timed-out-future-still-completes rows. The futures impl itself already
ran on real OS threads (ev/spawn-thread + marshalled results); jolt-ejx was
stale.
Dashboard: implemented+tested 433 -> 564 of 694; implemented-untested and
missing-portable are both EMPTY. Gate: jpm exit 0, all tests passed.
The dashboard's missing-portable category is now EMPTY (was 35 when the issue
was filed; this session's io/leaf work had already landed most of them).
The final seven:
- extenders — ctx-capturing clojure.core fn over the protocol type-registry:
the type-tags implementing a protocol, as symbols; nil when none
- find-keyword — keyword: jolt keywords have no intern table, so it always
finds (babashka makes the same call)
- inst-ms* — the raw Inst method; one inst representation, so = inst-ms
- read+string — over the 50-io readers, which now expose :buf and :fill-fn;
returns [form exact-text-consumed], EOF throws or yields [eof-value ""]
with the 3-arity, works for string AND stdin readers
- with-local-vars — fresh free-standing var cells (__local-var seam) bound as
locals; var-get/var-set work on any cell
- with-open — canonical recursive expansion closing through the __close seam:
a map-like value's :close fn or a host file (no .close interop here);
nested closes run inner-first, finally runs on throw
- with-precision — body evaluates with precision/:rounding accepted and
ignored (doubles, no BigDecimal context) — documented divergence
30 new spec rows (test/spec/missing-vars-spec.janet); coverage.md
regenerated: implemented+tested 426 -> 433, missing-portable 7 -> 0.
Gate: jpm exit 0, all tests passed.
Pins down what a transient is in Jolt (tagged table over a native Janet
array/table, canonical-keyed for maps/sets), where behavior deviates from the
JVM (O(n) transient/persistent! edges with O(1) native ops between, no
owner-thread check — same as Clojure 1.7+, transient-of-list leniency), and
the three reasons the machinery is seed-resident rather than a migration
candidate: it IS the mutation kernel, it sits under the seed's own dispatch,
and the value layer is declared irreducible. Exists so the kernel-shrink
ladder (jolt-tzo) doesn't revisit transients every round.
*ns* is interned in clojure.core holding the current NAMESPACE OBJECT, kept
in sync by ctx-set-current-ns through a var table cached on the env (one
table put on the hot path; core-bench A/B neutral). A thread binding
(binding [*ns* ...]) shadows the root through var-get as usual. in-ns now
returns the namespace object (Clojure); str renders a namespace as its name
and pr-str as #namespace[name]. The ns-designator helper accepts namespace
objects (they are tagged STRUCTS — the old table?-based check never matched
them), so (ns-aliases *ns*) / (ns-unalias *ns* 'a) work — SCI and the jank
syntax-quote corpus use exactly that shape.
Known divergence (documented): inside an interpreted fn, *ns* reflects the
fn's defining ns (jolt's resolution model rebinds current-ns per call);
top-level and load-time reads match Clojure.
Gate green (conformance 326x3, suite 4572 >= 4540, all batteries,
specs+unit +8 *ns* rows); bench neutral.
The names turn 2a's leak removal exposed as honestly missing, now proper:
- slurp/spit/flush (host-classified): path-based IO over Janet files; spit
takes :append; flush flushes *out*. printf prints formatted (no newline)
over the existing format. file-seq walks paths via two host dir primitives
through the overlay's tree-seq.
- ns-map / ns-unmap / ns-refers (ctx fns). ns-refers required fixing the
refer MODEL: refer/use/:refer now map the SOURCE VAR into the target ns
(the Clojure model) instead of copying its value into a new var — so
source-ns redefinitions propagate, the :macro flag travels for free, and
refers are identifiable by the var's home :ns.
- Thread-binding family: with-bindings*/with-bindings, bound-fn*/bound-fn,
bound?, thread-bound?, get-thread-bindings. The captured binding map is a
Janet struct keyed by the var tables — the exact frame representation
var-get reads — so it re-pushes correctly (a phm frame is invisible to
var lookup).
- load-string and eval interned as VALUES at the api layer (they need the
loader's compile-or-interpret routing); the eval special form still
handles direct calls.
Suite 4532 -> 4572 (baseline floor 4540 across timeout variance, clean 86),
conformance 326x3, stdlib battery, all specs+unit (+21 turn-2b rows).
Coverage: missing-portable 27 -> 10 (left: the *in*-model readers, the
with-local-vars/with-precision/extenders tail).
resolve-sym's last resort silently resolved any unknown Clojure symbol
against Janet's root environment — leaking Janet builtins with JANET
semantics into Clojure code: (type 1) was Janet's :number, (gensym) returned
Janet symbols (the long-documented (symbol (str (gensym))) macro landmine
existed BECAUSE of this), compare/slurp/int?/any? likewise. The explicit
janet/ prefix is the deliberate interop channel; the implicit fallback is
gone — an unresolved symbol is an error.
What the leak was masking, now proper interned vars:
- gensym: jolt's own (already existed, never interned) — returns real jolt
symbols; the macro landmine is dead
- compare: full Clojure total order (nil-first, numbers, strings, keywords,
symbols by ns/name, booleans, chars, uuid/inst, vectors by length then
elementwise; cross-type throws)
- type: :type metadata override, deftype/record tag as symbol, else a
taxonomy keyword (host-classified)
- int?: core-integer? — which had a latent bug the leak hid: (integer?
##Inf) was true (floor of inf is inf); NaN/infinities now excluded
- any?: constantly true (Clojure 1.9; SCI's namespaces.cljc needs it)
- jolt.interop/janet-type now uses the explicit (janet/type x) channel
- evaluator-test uses init (a bare make-ctx resolved EVERYTHING via the leak)
Suite 4470 -> 4532+ pass / 86-87 clean (proper compare unlocks the sort
files); baselines raised. Conformance 326x3 (+5 rows), +22 predicate spec
rows, stdlib battery green, all specs+unit. Coverage dashboard now counts
previously-leak-resolvable names honestly (missing-portable 19 -> 27).
#inst (jolt-rnh): an instant is an immutable tagged struct
{:jolt/type :jolt/inst :ms <epoch-millis>} — equality and map-key hashing by
INSTANT, so different offsets denoting the same moment are =. The reader
parses RFC3339 with Clojure's partial-timestamp defaults (#inst "2020" is
2020-01-01T00:00:00.000Z) and errors on malformed input; inst?/inst-ms in
the overlay; pr-str prints the canonical
#inst "yyyy-MM-ddThh:mm:ss.fff-00:00" and round-trips; str gives the bare
RFC3339 string. Self-evaluating in the evaluator (like uuid/chars).
Syntax-quote (jolt-l2a): a syntax-quoted self-evaluating literal (string,
number, boolean, nil, keyword, char) collapses to the literal at READ time,
matching Clojure's reader — so nested/adjacent backticks over literals are
inert: (= "meow" ```"meow") is true (jank pass-adjacent). Symbols still
qualify and collections still template. General nested syntax-quote over
non-literals remains UNVERIFIED in spec S25.
Tests: inst-spec (18 cases incl. partial defaults, offsets, round-trip),
+10 literal-collapse reader rows, +5 conformance rows (321x3). Spec
02-reader S20/S25 updated to normative. Suite stable 4470/86.
One documentation root: the prose docs (building-and-deps, self-hosting
architecture/compiler, tools-deps, grammar.ebnf) join the spec and RFCs under
docs/. References in README and deps-conformance-test updated.
jolt no longer satisfies :clj in reader conditionals. The shortcut was a
measured net liability: :clj branches carry JVM interop and JVM-specific
test expectations jolt fails, and they shadowed :default branches jolt
passes. A/B over the suite: clj,default = 4967 assertions / 4324 pass / 119
errors; jolt,default = 5069 / 4470 / 81 (+146 pass, -38 errors, +8 clean
files). Baselines raised to 4470/86.
Matching is now by CLAUSE order like Clojure — the first clause whose key is
in the feature set wins (#?(:default 5 :clj 6) is 5 everywhere); the old code
scanned for :clj first, then :default, regardless of position.
Foreign clj-targeted libraries are a property of the LOADING CONTEXT, not the
platform: reader-features-set! opts a load into a compatibility set, and the
SCI bootstrap/runtime tests load SCI under ["jolt" "clj" "default"] (its
.cljc selects implementations via :clj with no :jolt branches).
JOLT_FEATURES remains the process-wide override.
RFC 0002 records the decision with the measured data; spec 02-reader S18 is
now normative (clause order, documented feature set, per-context override).
Reader tests updated to the portable set + an opt-in round-trip.
The lexical-syntax chapter, granularity modeled on jank's 62-file
per-construct reader corpus: token grammar (whitespace/comments, collections
with read-time duplicate checks, numbers incl. the N/M tower question,
symbols/keywords incl. ::auto-resolution, strings/chars), the quote-family
sugars, the full #-dispatch catalog with normative entries (anonymous fn
%-derivation, discard composition, reader conditionals, symbolic floats,
tagged literals), and the syntax-quote contract (core/alias/current-ns
qualification, template-stable gensyms, ~' idiom, distribution through
collections).
Adapting the corpus surfaced and filed three findings, recorded as labeled
divergences/UNVERIFIED in the chapter: nested syntax-quote doesn't collapse
(S25, (= "meow" ```"meow") is false), #inst reads as a bare string (identity
data reader, no instant type), and jolt satisfies :clj in reader
conditionals (feature-key policy under review).
reader-forms-spec gains 11 chapter-cited rows (discard stacking, ##Inf/
##-Inf/##NaN, :default conditionals, qualified var-quote identity, gensym
stability within vs across templates) — all passing.
Fifteen vars from the spec coverage gap (docs/spec/coverage.md):
parse-long/parse-double/parse-boolean (strict validation; scan-number alone
accepts 0x10), newline, current-time-ms (host clock for time), update-keys/
update-vals (PHM base, collisions last-wins), partitionv/partitionv-all/
splitv-at (lazy seqs of vectors; splitv-at's tail stays a seq, matching the
reference), with-redefs/with-redefs-fn (roots restored on throw), time,
macroexpand (expand-1 to fixpoint), alias/ns-unalias (write BOTH alias stores
— require :as uses string-keyed :imports while ns-aliases reads :aliases;
split filed), ns-publics (symbol-keyed map; publics == interns, no privacy).
Three pre-existing bugs fixed along the way:
- (partition n step pad coll) misparsed pad as the coll and returned ()
- (require 'bare.symbol) rejected — only vector specs were accepted
- analyze-form leaked the interpreted analyzer's ns on a punt: a throw out of
the analyzer left current-ns=jolt.analyzer and :compile-ns set, so the
fallback interpretation resolved user vars against the wrong namespace
(bit (var user-sym) under compile mode)
And one overlay-authoring landmine documented in-code: a 20-coll fn must not
use 30-macros macros (with-redefs-fn's dotimes compiled as a forward ref that
resolved to the macro fn at runtime) — loop/recur instead.
Gate: conformance 316x3 (+14 rows), suite 4324 pass / 78 clean (was 4081/72;
parse_*/update_* files now contribute), baselines raised, all specs+unit,
fixpoint, self-host, sci, staged. Coverage: missing-portable 35 -> 20.
RFC 0001 proposes a normative, implementation-independent Clojure language
spec (the reader, evaluation model, special forms, data types, seq/laziness
contracts, namespaces/vars, and the portable clojure.core surface) to the
standard of R7RS/Racket — Clojure has none, and every alternative
implementation re-derives semantics from the reference and folklore. The
spec is executable-first: every numbered normative statement cites its
conformance test or is marked UNVERIFIED.
docs/spec/ carries the front matter (conformance terms, entry format, host
classification), the special-form catalog with worked normative entries for
if and let*, the core-library entry format with worked entries for first,
reduce, and parse-uuid, and a generated coverage dashboard over the 694-var
ClojureDocs inventory (tools/spec_coverage.py cross-references the surface
against jolt's interned+resolvable vars and the test suites).
Measured baseline: 380 implemented+tested, 154 implemented-untested, 35
portable-but-missing (filed), 22 resolvable-but-not-interned (filed — seed
fns invisible to resolve/ns-publics), rest classified host/JVM/concurrency.
