RFC 0006 error reporting wanted file:line:col but IR nodes carried no
position, so diagnostics read only "type error in <ns>: <msg>". Now:
type error /tmp/scene.clj:5:5: `inc` requires a number, but argument 1 is a string
The reader records each LIST form's absolute start offset in a table keyed
by form identity (lists are fresh arrays, never interned), gated behind a
flag the loaders enable only when JOLT_TYPE_CHECK is on — zero cost off.
Keying by identity makes positions survive macroexpansion exactly when the
user's own sub-form is spliced through, and absent for macro-synthesized
structure: a `(inc :k)` written inside `(when c ...)` reports at its own
line, never at the expansion's generated if/do.
The analyzer stamps the offset onto :invoke nodes (form-position host
contract fn); the checker carries it into each diagnostic as :pos; the
loaders stash the file's source + path on the env (save/restored across
nested requires); backend/type-check! converts offset -> line:col via the
reader's line-col and renders the RFC format. Falls back to the ns when no
position is available (synthetic forms), so it is never worse than before.
Gate green, conformance 335/335, suite 4718, runtime bench even (positions
are compile-time only; off by default).
The success checker fired only against core-fn error domains (stable, not
redefinable). This adds reporting of a call that passes a provably-wrong
type to a USER fn whose body requires otherwise — e.g. a fn that only does
arithmetic on a param, called with a string.
As check-walk sees defs it registers each non-redefinable single-fixed-arity
user fn's {:params :body} in module state (user-sig-box, accumulating across
forms like rtenv-box — a def must precede its call). At a call site (strict
mode only) the body is re-checked with ONE parameter bound to its concrete
argument type, others :any; if that produces a diagnostic the all-:any body
did not, the argument alone is provably wrong and the call is reported.
Monotonic — binding a concrete type can only add error-domain hits — so still
no false positives. A cycle guard (checking-box) terminates mutual recursion.
Gated behind JOLT_TYPE_CHECK_USER (orthogonal to the warn/error level)
because it rests on the closed-world assumption, weaker than the core-fn
case. check-form gains a strict? arity; the default path is unchanged and
user-fn code runs only when the checker is enabled. ^:redef/^:dynamic and
multi/variadic fns are not registered (their body is no stable requirement).
Gate green, suite 4718, conformance 335/335.
The success checker (RFC 0006) used to lose differing if-branches to :any
and accept the use. (inc (if c "a" :k)) typed the if as :any — sound but
imprecise, since the value is provably {:str | :kw}, every member of which
is in inc's error domain.
Adds {:union #{T...}} to the lattice: join-t forms a scalar union of
differing branches instead of collapsing to :any, capped at 4 distinct
scalars (the member space is the five scalar tags, so the lattice stays
finite and the inter-procedural fixpoint still terminates). The checker's
not-number?/not-seqable? report a union only when EVERY member is in the
error domain — any valid member accepts the call, so still no false
positives. type-name renders "a string or a keyword".
Unions are scalar-only and carry no :struct/:vec/:set key, so every
structural predicate already treats them as opaque — specialization sees
them exactly as :any and codegen is unchanged. Gate green, suite 4718,
conformance 335/335, bench even.
Reuse the structural inference from RFC 0005 as a loose type checker. It reports
a core-fn call only when an argument's inferred type is concrete and lies in
that op's throwing error domain, and accepts everything ambiguous (:any, a
union that joined to :any, :truthy). By construction it never produces a false
positive: a correct program has nothing to report even in error mode.
The curated error-domain table starts with the clearest throwing cases:
arithmetic on a provable non-number, and count/first/rest/next/seq/nth on a
provable non-seqable scalar. Lenient operations like (get 5 :k) and (:k 5),
which return nil rather than throw, are deliberately not listed.
Checking is decoupled from specialization: it runs whenever JOLT_TYPE_CHECK is
warn or error, regardless of :inline?, reading the knob at compile time so no
rebuild is needed. warn prints to stderr, error fails the form's compilation,
off (the default) skips it entirely. Core init stays clean under the flag.
jolt-y3b
Replace the ad-hoc inference lattice (a flat :struct-map tag plus {:vec ELEM})
with one recursive structural type: {:struct {field -> T}}, {:vec T}, {:set T},
scalar tags, and :any. A keyword lookup now returns its field's type, so nested
access like (:r (:direction ray)) is typed end to end and drops its guard. join
is field-wise and element-wise with a depth cap of 4 so the inter-procedural
fixpoint still terminates.
The back end honors a struct hint on any subject node, not just locals, so an
inferred field type on a nested lookup specializes. The orchestrator's fixpoint
joins through the portable join-types so compound types no longer collapse to
:any.
Ray tracer goes 12.8s to 11.0s with no hints, matching the explicit ^:struct
version (10.9s). Render checksum unchanged (1915337), full gate green,
conformance x3 modes pass.
jolt-5uj
The inference now tags a :local it proved to be a vector with :hint :vector, and
the back end specializes (count v) -> pv-count (skipping core-count's dispatch
chain) and the 3-arg (nth v i default) -> pv-nth. The 2-arg nth is deliberately
NOT specialized: pv-nth returns nil out-of-bounds where Clojure nth throws.
Sound, conformance 335/335 x3 and full jpm test pass; type-infer-phase2-test
pins the specialization and the 2-arg exclusion.
Extends the inference lattice with a parametric vector type {:vec ELEM} and
threads element types through the program:
- vector literals, conj/into, and range produce element-typed vectors;
- reduce/map/mapv/filter/filterv seed their closure's element (and reduce's
accumulator) param, so a lookup inside the closure over a vector-of-structs
specializes (the HOF-element-awareness piece);
- a var reference carries a VALUE type — a fn var is :truthy (non-nil, sealed
root), a def var carries its inferred init type (e.g. a color table is
{:vec :struct-map}); element-returning fns (rand-nth/first/nth/...) yield the
collection's element type. These let the dynamically-built scene's sphere
maps type as structs.
The inter-procedural fixpoint now also infers non-fn def value types, and the
recompile re-emits the WHOLE unit callee-first (reverse-topological) so a
caller re-embeds its recompiled, now-specialized callees and a call site
compiled after the pass links the whole chain.
Result on the ray tracer (no hints): the chain closes — hittables infers to
{:vec :struct-map}, hit-sphere's hittable param to :struct-map — and the render
goes 13.1s -> 12.8s. That is only ~3%, far short of the explicit hint's 1.22x.
The remaining gap is nested field access: a lookup RESULT like (:direction ray)
is :any, so (:r (:direction ray)) stays guarded, and the vec3 fns (called with
such values) can't be typed struct. The hint asserts the vec3 params directly
and propagates through inlining; matching it needs field-shape types
(ray.direction : vec3, vec3.r : number) — a structural extension (Phase 4).
Sound: a seeded full render produces an identical checksum (1915337);
conformance 335/335 x3 and the full jpm test pass; type-infer-phase3-test pins
the element-typing + HOF mechanism. Phase 2 (vector nth/count specialization)
was deprioritized — it is orthogonal to this benchmark.
Closed-world (optimization mode): after a unit loads, infer-unit! runs a
whole-unit fixpoint over the call graph and recompiles. A fn's param types are
the lub of its in-unit call-site arg types; its return type is the lub of its
tail positions; iterated to a least fixpoint. Param types are RECOMPUTED FRESH
each iteration (not accumulated) because :any is the lattice top — joining an
early-iteration :any would poison the result permanently. Closures inherit the
enclosing tenv so captured locals keep their types (their own params shadow to
:any). A fn whose var escapes as a VALUE keeps :any params (its callers aren't
all visible). Each fn is then re-inferred with its param types seeded and
re-emitted; recompiled bodies are semantically identical, so correctness holds
regardless of order. Sound under source distribution + whole-program compile
(the consumer compiles all call sites together).
Plumbing: the portable pass (jolt.passes) gained inter-procedural primitives —
set-rtenv!, infer-body (types a body, collects its call sites), reinfer-def
(seeds param types), and escape tracking. The back end stashes each
single-fixed-arity defn's :def IR (:infer-ir); the evaluator triggers
infer-unit! after a unit loads (via an env hook, opt mode only).
Result and honest finding: the fixpoint correctly types scalar-flowing params
(ray-cast/hit-all/hit-sphere all get the ray param as :struct-map, no hint),
but the ray tracer does NOT speed up — its dominant lookups are on `hittable`,
the element of the `hittables` vector threaded through `reduce`, which stays
:any. Typing it needs collection-element types (vector<struct>) plus HOF-element
awareness (knowing reduce applies the closure to elements), which is beyond
inter-procedural param inference. The explicit ^:struct hint reaches it (it
types the reduce closure param directly), which is why the hinted run is 1.22x.
Verified: conformance 335/335 x3, full jpm test; new type-infer-phase1-test
pins the fixpoint, the escape gate, the seeded re-inference, and correctness.
A forward, soft-typing-style pass (simplified HM: monovariant, never-fails,
lattice top = :any) in jolt.passes, run after the inline/scalar-replace
fixpoint when the optimization mode is on. It types expressions from literals
and arithmetic, flows the type through let bindings, and joins at if-branches.
Where a keyword-lookup subject is PROVEN to be a plain struct map it sets
:hint :struct (the same channel a manual hint uses, so the back end drops the
:jolt/type guard); where the type is :any it leaves the dynamic guard in place.
Sound by construction: a concrete type is assigned only when proven (scalar
keys with non-nil/non-false values for a struct-map), so a wrong bare get can't
happen. This is the foundation; on its own it mostly overlaps Route 1
scalar-replacement (which already eliminates non-escaping let-bound maps), so
its standalone win is small. Phase 1 (inter-procedural) is where escaping
params get typed.
Verified: conformance 335/335 x3, full jpm test; new type-infer-test pins the
flow rules and the sound :any fallback (cases force the map to escape so the
test isolates inference from scalar-replacement).
Builds on the ^:struct keyword-lookup hint:
- ^TypeName for records. A tag naming a defrecord/deftype now resolves to the
struct fast path: record instances are tables tagged :jolt/deftype (not
:jolt/type), so a raw keyword get is correct for them. A new host contract fn
record-type? detects a record by its ->Name constructor; a non-record tag
(^String, ^long, ...) is ignored, as before.
- (get m :k) and (get m :k default) now get the same inlined keyword lookup as
(:k m): the representation guard fast path when unhinted, and the bare get
when the subject is ^:struct/^Record. A variable/number/string key still
falls through to core-get. The two call shapes share one emitter
(emit-kw-lookup).
- JOLT_CHECK_HINTS=1 turns a violated hint into a clear runtime error (naming
the local and key) by keeping the guard and throwing on the tagged arm. It is
off by default with zero cost to normal builds (a hinted lookup still emits a
bare get), and is part of the image-cache fingerprint. This is the answer to
"a lying hint is silent": opt into checking during development.
- Docs: RFC 0004 records the design, soundness contract, and measurements; the
reader spec gains S12b (hints are semantically transparent; jolt recognizes
^:struct and ^Record as lookup-optimization assertions).
There is no Clojure keyword equivalent for "plain map / fast keyword access"
(Clojure hints are class names), so ^:struct stays a jolt-specific flag,
analogous to ^:dynamic.
Verified: conformance 335/335 in all three modes and the full jpm test pass; a
seeded ray-tracer render is byte-identical hinted vs unhinted; the struct-hint
test covers record hints, the get-form, inline propagation, and the checked-mode
error. Full render with hints holds at 13.3s -> 10.9s (1.22x).
A constant-keyword lookup (:k m) currently emits a guarded form,
(if (get m :jolt/type) (core-get m k) (get m k)), to tell a plain struct
(raw get is correct) from a phm/sorted/transient (needs core-get). On a
struct that guard is a second get, so the lookup costs ~36ns where a bare
get is ~20ns. Profiling the ray tracer (jolt-dad) showed keyword lookups are
~50% of a render and the guard is the only avoidable part, but dropping it
needs to know statically that the subject is a plain struct.
Type hints are exactly that information, and jolt already parses them and
otherwise ignores them. This wires one through: a local hinted ^:struct
asserts a plain struct/record map, so a (:k local) lookup on it skips the
guard and emits a bare get. The hint rides on the binding symbol into the
analyzer, which records it per-local and attaches it to :local IR nodes; the
back end reads it on the lookup subject. It also propagates through inlining:
when the inliner let-binds a non-trivial arg to a fresh local, it carries the
called fn's param hint onto that local, so lookups inside the spliced body
keep the bare path. This is a programmer assertion, like a Clojure type hint
(an inaccurate hint just makes the raw get return the wrong value, the same
contract as a wrong ^String), so it stays opt-in and off by default.
On the ray tracer (with inlining on) this is 13.3s to 10.9s, 1.22x, taking it
to 7.8x JVM from 9.4x after the inline pass. The unhinted path emits identical
code (the fast arm is just factored out), so nothing changes without hints.
Verified: a seeded full render produces an identical checksum hinted vs
unhinted; conformance 335/335 in all three modes and the full jpm test pass;
new test/integration/struct-hint-test.janet pins the guard removal, the
inline propagation, and that an accurate hint is correctness-preserving.
Adds two IR passes to jolt.passes that run when a unit opts into
direct-linking (JOLT_DIRECT_LINK=1, off by default). The inline pass splices
small direct-linked fns at their call sites, copy-propagating trivial args so
that scalar replacement can then see map literals across the call boundary.
Scalar replacement is AOT escape analysis: a map allocation whose only use is
constant-keyword lookup is dropped and each (:k m) is replaced with the value
at :k, both for a literal lookup subject and for a non-escaping let-bound map.
Inlining and scalar replacement iterate to a capped fixpoint, since inlining
exposes literals that scalar replacement then collapses.
The back end stashes the body IR of each single-fixed-arity defn on its var
cell (inline-stash!), and the portable pass reads it through two new jolt.host
contract fns (inline-enabled?, inline-ir). Inlining is gated on :inline?, which
is off for all of init so core and the self-hosted compiler compile exactly as
before (const-fold only); api/init and main re-read JOLT_DIRECT_LINK so the
flag works both for a freshly built context and for the build-time-baked one in
the shipped binary.
Only inline-safe targets are spliced: a single fixed arity, no recur/loop/fn/
try crossing the boundary, within a size budget, a closed body (no free locals
beyond the params, so a self-recursive fn's name reference can't dangle), and
not ^:redef / ^:dynamic. Bodies are fully alpha-renamed so no spliced name can
collide with a caller local.
On the ray tracer this is 15.3s -> 13.0s (1.18x). The ceiling is honest: that
workload's cost is dominated by lookups on maps that genuinely escape (rays,
hits, materials) and by dynamic dispatch (the reduce closure, the :scatter fn),
which escape analysis cannot remove. On allocation-bound code where the
temporaries are local it is far larger: a vec3 reflect+dot loop goes 9.3s ->
0.38s (25x), with the loop body reduced to pure arithmetic.
Verified: full jpm test passes (inline off, no regression); conformance 335/335
in all three modes and the clojure-test-suite both pass with inline on; new
inline-sra-test pins the transform and its semantics.
A typo'd symbol used to auto-intern an unbound var and die later as
'Cannot call nil as a function' with no hint which symbol. Now:
$ jolt -e '(undefined-fn 1)'
Error: Unable to resolve symbol: undefined-fn in this context
The analyzer's :unresolved fallthrough now punts to the interpreter
(whose resolver raises the message above when the form runs) instead of
emitting a var-ref that interned the var. A punt rather than a hard
throw because runtime-interning forms (defmulti's setup) legitimately
reference the var they're about to create from a nested do.
Pulling that thread surfaced three real bugs the leniency was masking:
- h-resolve-global resolved unqualified symbols against ctx-current-ns,
which during analysis is jolt.analyzer — so user-ns vars NEVER
resolved through it; the lenient arm happened to emit the right ns.
Now resolves against the compile ns like the qualified branch.
- Top-level (do ...) wasn't split: Clojure compiles and EVALS each
child in sequence so earlier children's runtime effects (defmulti's
intern) are visible while later children compile. eval-toplevel now
splits.
- The stdlib itself had forward references the auto-intern hid:
10-seq's transducers used vreset!/vswap! from 20-coll (moved to
10-seq); in 20-coll qualified-ident?/realized?/list*/underive
referenced defs declared later in the file (reordered); sorted? and
partition-all are genuinely later-tier and got (declare ...).
Test rows updated where they encoded the old leniency: ir-passes'
dead-branch row (unresolved in a dead branch is an error, as in
Clojure), compile-mode's ctx-isolation row (other ctx now errors
instead of reading nil), cli rows assert the new message. Gate green,
conformance 335/335 x3, suite 4718 steady, bench within noise.
Before: (+ 1 "a") printed 'could not find method :+ for 1 or :r+ for "a"'
followed by three janet frames pointing at jolt internals. After:
Error: Cannot add 1 and "a" — + expects numbers
at app.deep/level3
Round 1 — compiled fns carry their Clojure identity:
- The analyzer's recur target (which doubles as the compiled janet fn's
name) is now ns/fn-name (_r$app.deep/level3--N), so janet stack traces
name the user's fns; defn passes the self-name through to fn.
- eval-toplevel re-raises with propagate instead of protect+error — the
failing fiber's stack was being discarded, which is why every trace began
at eval-toplevel.
- require/maybe-require-ns route loaded namespaces through the loader's
compile-or-interpret eval-toplevel via a ctx hook (the evaluator can't
import the loader). Previously REQUIRED namespaces always ran interpreted:
slower, and their fns were anonymous in traces.
Round 2 — report-error presents for users (rephrase-inspired):
- The full trace text is stashed at the innermost eval-toplevel boundary
(janet's debug/stacktrace walks the fiber propagation chain; debug/stack
cannot), then filtered: _r$ frames demangled to ns/fn-name, jolt-internal
and [eval] frames dropped. JOLT_DEBUG=1 restores the raw janet trace.
- Message rewrites: janet arithmetic dispatch -> 'Cannot add X and Y — +
expects numbers'; compiled arity -> Clojure's 'Wrong number of args (N)
passed to: ns/fn'; nil-call gets an undefined-symbol hint (round 3 will
fix resolution properly).
6 cli-test rows assert the exact user-visible output. Gate green, suite
4718 steady, bench within noise.
Clojure's defprotocol takes an optional docstring and leading keyword
options (:extend-via-metadata true) before the signatures; jolt's macro
fed the option keyword to (first sig). honeysql declares its InlineValue
protocol exactly that way — with the fix, all four honeysql namespaces
load unmodified from git and the formatter produces correct sqlvecs for
selects/inserts/updates/deletes/joins/:inline. Listed in libraries.md.
(^bytes [b])-style return hints reach fn as a (with-meta [b] {:tag ...})
form; unhint sheds the wrapper through the rebuild path so the clause
representation never changes. The host-interop hint rows exercise it.
The interop surface ring.util.codec needs (registered through the javatime
shim registries): URLEncoder/URLDecoder (www-form-urlencoded in pure janet),
Charset/forName, Base64 encoder/decoder, Integer/valueOf with radix +
parseInt, StringTokenizer, clojure.lang.MapEntry (a 2-tuple), a String ctor
from bytes, .getBytes on the String surface, and a java.lang.Number method
surface (byteValue and friends).
Protocol fixes: extend-protocol on java.util.Map/Set/List now dispatches
(maps — phm/struct/sorted/records — never produced host tags and fell to
Object), lazy seqs gained their ISeq tags, and a nil extension arm works
(group-by-head and extend-type both choked on the nil head). reduce
dispatches to a reified clojure.lang.IReduceInit's own reduce method, which
is how ring-codec tokenizes.
jolt-deps learns :deps/root (tools.deps monorepo subdirectory checkouts —
ring-core lives inside ring-clojure/ring). spork/http, when jpm-installed,
reaches the jolt layer as janet.spork.http/* through the janet.* bridge
(soft: nothing requires it unless used).
The protocol fixes alone let 29 more clojure-test-suite assertions execute:
5319 -> 5348 run, 4706 -> 4715 pass.
The first per-type migration print-method unlocked: uuid, regex, transient,
and channel rendering move from host pr-render branches to io-tier
defmethods (exact same output). The renderer's tagged fallthrough now
dispatches ANY remaining :jolt/* value through the print-method hook before
the raw pairs view — so every tagged type is user-overridable, atoms
included ((defmethod print-method :jolt/atom ...) fires nested), and future
per-type migrations are pure overlay additions.
Hot types (numbers, strings, symbols, collections) stay native, and inst/
namespace/var stay host for now — their formatters (rfc3339, display names)
live there anyway. A transient's :kind is read with jolt.host/ref-get: get
on a transient is the dispatched collection lookup (same trap as sorted
colls).
Before the hook is wired (init-time error messages) tagged values fall
through to the pairs view — bootstrap rendering only.
print-method/print-dup are now multimethods in the io tier with Clojure's
exact dispatch ((:type meta) keyword, else type — core.clj 3693). On jolt the
dispatch value for a record is its quoted full-name symbol, since class names
aren't values here.
Records used to pr-str as the raw janet table; the renderer's record branch
now prints Clojure's #ns.Type{:k v} syntax, and first consults a callback the
api wires up after the overlay loads — so a user defmethod on a record type
fires everywhere: top level, nested in collections, through pr/prn/pr-str.
Builtin overrides (a :number method) fire only on direct print-method calls;
pr keeps the native fast path (documented divergence).
java.io.Writer arrives as a shim beside the StringReader/StringBuilder ones:
a :jolt/writer tagged value with write/append/flush/toString, a StringWriter
ctor, and a sink variant the renderer callback uses.
Two latent host bugs fixed on the way: the interpreted syntax-quote splice
blew up on ~@nil (an interpreted macro's empty & rest binds nil — first tier
user of defmulti found it; d-realize now treats nil as the empty seq), and
(print-method x nil) now throws like the JVM instead of returning nil.
10 spec rows; bench dead even (sandwich run); greeter green on a fresh
binary.
Batch 2 of the post-shrink sweep, all pure compositions or documented stubs:
enumeration-seq/iterator-seq (seq), promise/deliver (an atom — deref of an
undelivered promise stays nil, single-threaded host), bean, uri?,
special-symbol? (an evaluated set of quoted symbols — a QUOTED set literal
stays an unevaluated reader form on jolt, which the first version tripped
over), print-method/print-dup (inert until jolt-g1r), and the whole proxy
surface (mappings/call-with-super/init/update pass through, the constructive
half throws). The seed loses 16 defns and bindings; nothing kept.
Three canonical-conformance fixes from the post-shrink batch:
- bit-and/bit-or/bit-xor/bit-and-not get Clojure's variadic arities as
20-coll shells folding the binary host ops (now __bit-* seams). 2-arg call
sites still compile to the native janet op via the backend's native-ops
table. The passes.clj constant-fold table now names the seams — the public
fns are overlay and don't exist when the compiler loads (this briefly broke
every compile-mode init).
- core-set? recognizes the :jolt/sorted-set representation (jolt-dpn):
(set? (sorted-set 1)) was false, and ifn? on sorted sets inherited the bug.
- (if) / (if test) / (if test then else extra) throw in both the analyzer
and the interpreter — spec 03-special-forms X1, now marked verified.
Suite 4704 -> 4706; bench and the greeter example benchmark are flat.
Fixed arities now throw Clojure's ArityException shape — 'Wrong number of
args (N) passed to: name' — on any count mismatch; variadic arities on fewer
than their fixed params. The compiled path already enforced fixed arities via
janet's native fn check and multi-arity dispatch; this adds the check to the
interpreter's single-arity closures (the oracle was silently dropping extra
args and giving a raw tuple-index error for missing ones) and guards the
compiled single-variadic wrapper's minimum. Messages carry the fn name when
there is one. 16 spec rows; the update.cljc suite row flipped green (4703 ->
4704).
Enforcement exposed that seq-to-map-for-destructuring had drifted: the spec
row called the 1-arity fn with two args, and the body silently dropped a
trailing unpaired element. Replaced with the canonical Clojure 1.11 version
(even pairs build the map, a single trailing element passes through — so
(f {:b 2}) kwargs calls work — and an unpaired key throws).
Also: transients RFC notes tuple support from the seed-shrink rounds.
Round 6 of the seed shrink (the printer round, scoped by the perf wall). The
five wrappers move to 20-coll over two new host seams: __write (push a string
to *out*) and __pr-str1 (render one value readably). The renderer itself
stays in the seed — it's representation-coupled (pvec/phm/phs/sorted
internals) and shared with the hot str, and rendering through overlay calls
would pay the per-element call cost everywhere big values get printed.
print-method as a real multimethod is follow-up work.
The new spec rows caught a renderer bug: string bodies were never escaped, so
(pr-str "a\"b") didn't round-trip through the reader. pr-render now
escapes quote/backslash/control chars per Clojure.
Round 5 of the seed shrink. transduce is the canonical 5-liner over reduce
(which already honors reduced and steps lazy seqs); eduction composes with
comp and stays eager into a vector (documented divergence, as before);
td-comp — eduction's last caller — is deleted from the seed. transient
accepts tuples now (reader vectors / map entries), so (into [] (first {:a 1}))
keeps working everywhere a vector does.
into was moved, benched, and moved back: the overlay call layers cost the
into-vec suite ~11% back-to-back (536 vs 480ms), the same per-call wall that
sent even?/odd? home in round 4. A transient conj! fast path didn't pay for
itself either (jolt call overhead dominates, not the per-element conj). The
seed keeps core-into + its private transduce machinery; the binding count
still drops by three.
Round 4 of the seed shrink. zero?, pos?, every? move to the syntax tier
(empty? and the analyzer use them — raw def+fn* per the file constraint);
char? joins the tagged-value predicates in 20-coll. coll? stays seed: host
set? doesn't cover sorted sets (filed jolt-dpn) and the tag check from the
overlay would hit the sorted-coll get trap. pos? guards number? explicitly —
the staged recompile emits bare > as the native janet op, which orders
strings (zero? gets the same guard; spec rows lock both plus neg?).
The canonical every? seq-walks its coll, which exposed that rest/next over
sets, phms, struct maps and sorted colls fell into core-rest's indexed
fall-through and walked the wrapper table's INTERNAL fields — (next #{1 2})
was (nil nil), (clojure.set/subset?) broke. core-rest now seqs those
representations (branches placed AFTER the hot vector/lazy paths; the first
ordering cost seq-pipe 4x). Suite rises 4700 -> 4703; baseline 4660 -> 4695.
even?/odd? are back in the seed after the bench A/B: (filter even? ...) pays
an extra call layer per element through the overlay (seq-pipe 262 -> 1100ms).
They join the perf-wall list with the lazy hot fns.
Round 3 of the seed shrink. To the overlay: identity, constantly, neg?,
even?, odd? (20-coll, ahead of their first in-tier uses), not= and unreduced
(00-syntax — the kernel and seq tiers use them), ==, ensure-reduced,
halt-when, parse-boolean, parse-uuid, newline, seque, array-seq, to-array-2d,
and the masking unchecked-byte/short/char/float/double coercions. parse-uuid
validates via re-matches over a new __make-uuid host binding (overlay source
can't write :jolt/type map literals). memfn moves to 30-macros as a working
macro over the .method call sugar instead of a fn that throws.
Behavior fixes toward Clojure, each with spec rows: == now throws on
non-numbers instead of comparing them, and halt-when is the canonical
::halt-map version (the halt value replaces the whole reduction result, no
double completion). list? and map-entry? stay in the seed — both are
representation-coupled (plist/tuple checks).
clojure-test-suite goes 4701 -> 4700: update.cljc expects
(update {:k 1} :k identity 1 2 3 4) to throw an arity error, and jolt fns
don't enforce fixed arity anywhere (pre-existing, language-wide — the seed's
Janet identity threw natively). Filed as jolt-6xn; fixing it should flip
several suite rows at once.
Jolt numbers don't overflow, so +'/-'/*'/inc'/dec' and the whole unchecked-*
family are just the checked ops — now one-line defs in core/20-coll.clj
instead of ~25 seed bindings. int? and num move the same way.
unchecked-divide-int now goes through quot, so dividing by zero throws like
the JVM instead of silently truncating infinity. unchecked-int/long gain
char handling via int, matching Clojure ((unchecked-int \a) => 97). The
masking byte/short/char coercions are not aliases and stay in the seed for
a later round.
Also drops a second duplicate set of unchecked defns that was shadowing the
first at module load.
Loading these libs via require worked (load-ns-source interprets, macros
expand lazily) but the same code inlined by uberscript routes through
eval-toplevel and compiled, surfacing four gaps:
- a ^{:map} metadata def name reads as (def (with-meta name m) v); the
analyzer died extracting the name (config.core's defonce env). It now
throws uncompilable so the interpreter, which handles it, takes over.
- declare was a no-op, so a compiled forward reference to a declared
name that collides with a janet root binding bound to the host fn
(selmer.parser's (declare parse) compiled to janet's 1-arg parse).
declare now expands to no-init defs, the interpreter interns them,
and the analyzer routes no-init def to the interpreter.
- class? was missing (selmer.util's exception macro calls it at
expansion time). Always false, like ratio? — no Class objects here.
- require of an unlocatable namespace silently left an empty ns behind,
deferring the failure to an unresolved symbol far from the cause. It
now throws like Clojure's FileNotFoundException. Namespaces entered
in-session count as loaded (Clojure puts them in *loaded-libs*), and
the SCI bootstrap opts out via :lenient-require? since its
clj-targeted requires can't all exist on this host.
Selmer now loads and renders templates on jolt: variables, filters
(upper, date with JVM patterns), if/for tags, nested lookups, HTML
escaping, and render-file with its last-modified template cache.
New src/jolt/javatime.janet provides the java.time surface Selmer's
date filters use (DateTimeFormatter/Instant/ZoneId/LocalDateTime/
FormatStyle/Locale, epoch-ms backed, host-local timezone) plus the
java.io/java.lang/java.net shims its template reader needs
(StringReader, StringBuilder, URL, File/separator, Class/forName).
Everything registers through three new evaluator registries
(class-statics, tagged-methods, class-ctors), so the module is data
plus an install call.
Fixes shaken out along the way, each load-bearing for Selmer and
correct on their own:
- :refer :all silently referred nothing (it iterated the :all keyword)
- ns :import ignored vector specs and didn't share deftype ctor vars
- dot calls on deftype/reify instances never consulted the protocol
registry, so (.render-node node ctx) failed where (render-node ...)
worked
- instance? rejected expression type args like (Class/forName "[C")
- char-array didn't accept a string
- io/resource now searches the loader's source roots (the classpath
analog); io/reader handles char arrays, URLs, readers, and returns
an in-memory reader with :read-line-fn for file paths
- String .split (regex, JVM trailing-empty semantics), file-path
methods (.toURI/.toURL/.getPath/.lastModified/.exists)
- System/getProperty (os.name & co), the janet/* bridge now works
inside env-less fibers, and qualified class names that syntax-quote
mangles (selmer.util/StringBuilder) fall back to the ctor registry
Spec rows cover the shim surface; test/integration/selmer-test.janet
runs the real Selmer from ~/src/selmer (skips cleanly when absent).
ifn? (jolt-1vx) is the canonical IFn set in the overlay: fns, keywords,
symbols, maps (sorted included), sets, vectors, and vars — NOT lists. The
seed version said true for lists and false for struct maps and vars.
Mutable-mode caveat documented (vectors and lists share the array repr
there). 13 predicate rows.
Multimethod dispatch (jolt-heo) now collects EVERY isa-matching method key
and picks the dominant one — x dominates y when prefer-method'd over it or
(isa? x y) — and two matches with no dominant is an ambiguity ERROR, as in
Clojure. It used to take whichever key the table yielded first, silently
ignoring prefer-method. The prefers store upgrades to Clojure's
{x -> set-of-dominated} shape, shared between the dispatch closure and
prefer-method-setup via the var; prefers becomes a macro over a setup fn
(the store lives on the VAR — the multifn value can't carry it, so the old
fn read {} forever). 6 multimethod rows + the conformance row updated to
the canonical shape (335x3).
The reader (jolt-ou8) kept the pending KEY when a comment or #_ sits in a
map's VALUE slot: the old code dropped both, desyncing kv pairing — the
real value became the next key and the closing brace landed in value
position ('Unmatched closing brace'). Selmer's deps.edn (a '; for
development (REPL, etc)' comment between key and value) now parses; 6
reader rows incl. nested commented maps.
Gate: jpm exit 0, conformance 335x3, all tests passed.
jolt.passes is the new portable pipeline stage between the analyzer and the
back end: pure IR -> IR rewrites, total over node :ops (unknown ops pass
through with folded children), loaded with the compiler namespaces and
resolved lazily by analyze-form (JOLT_NO_IR_PASSES=1 disables — the same
escape-hatch pattern as the macro oracle). The shape is flatiron's opt.clj
applied to the jolt IR, which is what jolt-2om asked for.
The first pass is constant folding: a call of a foldable numeric SEED fn
(the later tiers don't exist when the compiler loads) whose args are all
constant numbers becomes a constant, and an if with a constant test becomes
the taken branch (dead-branch elimination — the untaken side never even
resolves). Folding computes with the ACTUAL jolt fns, so results match
runtime semantics by construction; a fold that would throw (mod 5 0) is
left for runtime.
Two walk lessons paid for in debugging: let/loop bindings are
[name init-ir] PAIRS, not maps (assoc'ing :init into a pair corrupts it);
and a throw inside the interpreted pass unwinds past the interpreter's ns
restores, so analyze-form restores the compile ns after the (protected)
pass call — without that, one pass error left current-ns in jolt.passes and
the rest of the tier compile resolved against the wrong namespace (sort-by
landed on the 2-arg JANET builtin).
ir-passes-test pins folds, conservatism (free vars, throwing folds), and
end-to-end eval. Gate exit 0.
Every walk over a lazy seq created FRESH wrapper tables around the shared
rest-thunks (ls-rest, ls-seq/ls-count, realize-for-iteration, the printers,
reduce — each had its own make-lazy-seq loop), so independent walks re-ran
the thunks: side effects duplicated, and a doall'd seq of futures was
re-spawned serially by the deref walk. Every walker now goes through
ls-rest-cached, which memoizes the rest wrapper on its node — thunks run
exactly once, as in Clojure. Costs ~10% on walk-heavy benches (the per-node
cache get/put — Clojure's LazySeq pays the same); net still -9% vs the
pre-linear-walks baseline. Three regression rows pin once-only effects and
value stability across walks.
On top of that: pmap/pcalls/pvalues (jolt-oeu) over the real-thread futures
— spawn-all-then-deref (the once-only fix is what makes the doall actually
mean that), snapshot semantics documented, multi-coll arity via the
canonical vector-zip. System/currentTimeMillis + nanoTime land as System
statics (the realtime clock — os/time is whole seconds, which quantized
every elapsed measurement to 1000ms). Seven pmap rows incl. a generous-
margin parallelism check (4 x 200ms sleeps under 700ms after warmup).
test/spec/untested-vars-spec.janet adds 143 rows asserting jolt's documented
behavior for the whole implemented-untested category — primed arithmetic,
the array/aset/coercion stubs, unchecked-*, the chunk family, JVM-shape
stubs (class/bean/proxy/memfn as resolve-only or :throws), ns/REPL
machinery, and the misc seqs. tools/spec_coverage.py now checks each var as
a whole TOKEN in the test sources (call-position-only matching missed *1,
+', ., .., /, and bare transducer refs like cat).
Writing rows from probed truth surfaced five real bugs, all fixed:
- comp with a jolt-IFn stage silently returned nil ((comp seq :content)) —
raw Janet keyword application is not jolt invoke. comp is the canonical
overlay defn now (fixed-arity composed fn, so the hot 1-arg path is two
direct calls); the seed keeps a private td-comp only for the transducer
machinery. hof bench +9% vs native, the price of correct IFn dispatch.
- extend (the fn) was a nil-expanding stub MACRO shadowing any definition;
it's a real fn over register-method now, and extends? (a constant-false
stub) is real over extenders
- (.. x f g) hit the 'ClassName.' constructor branch (a name ending in a
dot) and died; .. is the canonical threading macro now
- aclone errored on pvecs; ns-interns/ns-imports returned live host tables
that count/seq reject (now structs)
Thread/sleep + Thread/yield land as Thread statics beside Math/: sleep parks
the WORKER's own event loop (each future thread has one), which makes timed
deref provably fire — futures-spec gains the timeout-fires, sleep-in-body,
and timed-out-future-still-completes rows. The futures impl itself already
ran on real OS threads (ev/spawn-thread + marshalled results); jolt-ejx was
stale.
Dashboard: implemented+tested 433 -> 564 of 694; implemented-untested and
missing-portable are both EMPTY. Gate: jpm exit 0, all tests passed.
The dashboard's missing-portable category is now EMPTY (was 35 when the issue
was filed; this session's io/leaf work had already landed most of them).
The final seven:
- extenders — ctx-capturing clojure.core fn over the protocol type-registry:
the type-tags implementing a protocol, as symbols; nil when none
- find-keyword — keyword: jolt keywords have no intern table, so it always
finds (babashka makes the same call)
- inst-ms* — the raw Inst method; one inst representation, so = inst-ms
- read+string — over the 50-io readers, which now expose :buf and :fill-fn;
returns [form exact-text-consumed], EOF throws or yields [eof-value ""]
with the 3-arity, works for string AND stdin readers
- with-local-vars — fresh free-standing var cells (__local-var seam) bound as
locals; var-get/var-set work on any cell
- with-open — canonical recursive expansion closing through the __close seam:
a map-like value's :close fn or a host file (no .close interop here);
nested closes run inner-first, finally runs on throw
- with-precision — body evaluates with precision/:rounding accepted and
ignored (doubles, no BigDecimal context) — documented divergence
30 new spec rows (test/spec/missing-vars-spec.janet); coverage.md
regenerated: implemented+tested 426 -> 433, missing-portable 7 -> 0.
Gate: jpm exit 0, all tests passed.
sort-by, rand-int, shuffle, random-uuid, char-escape-string, and
char-name-string move to 20-coll over the two host seams that stay (rand and
sort — they ARE the randomness/ordering primitives). Canonical upgrades ride
along: sort-by defaults its comparator to compare, so nil sorts FIRST (the
kernel fn used host ordering and put nil last); rand-int truncates toward
zero via int (the kernel fn floored, wrong for negative n); shuffle is a
pure-functional Fisher-Yates over vector assoc and rejects non-collections
(a string is seqable but not shuffleable, as on the JVM — the honest gate
caught that one); random-uuid builds over rand-int and validates through
parse-uuid; the char tables are char-keyed Clojure maps (Clojure's shape —
the seed keeps its private code-keyed copies for pr-render).
22 new spec rows. Gate: jpm test exit 0 verified, suite 4698 >= 4660, bench
parity with main back-to-back (4733 vs 4817).
recompile-defns! is the defn analog of recompile-macros!: pre/at-kernel
overlay defns (00-syntax's destructure and friends; the kernel tier too in
interpret mode) load as interpreted closures, the evaluator stashes their fn
source on the var (:defn-src, scoped by a flag only api/load-core-overlay!
sets), and the end-of-init pass compiles them and swaps the var root. With
that in place, keys/vals/empty? — the fns the 00-syntax expanders call at
expansion time — move to the top of 00-syntax as raw fn* defs (canonical:
keys/vals project (seq m), so sorted maps come back in comparator order and
(keys {}) is nil; empty? keeps O(1) count dispatch with seq's cell check only
for the lazy/list fallback). The sorted tier drops its now-dead :keys/:vals
ops.
Correctness fixes that surfaced once the gate was run with a REAL exit code
(the previous 'jpm test | grep' gates reported grep's exit and masked spec
failures across #48-#50):
- map conj is strict again: a non-nil/non-map arg must be a 2-element vector
('Vector arg to map conj must be a pair'), and merge inherits it — the
batch-2 canonical merge had silently dropped the validation
- conj onto a lazy seq prepends (it fell into the MAP fallback); upstream
clojure.data/diff relies on (conj seq x) via set/union over keys, so diff
now matches Clojure exactly
- (seq {}) / (seq #{}) / empty phm are nil, not ()
- key/val are strict (a plain vector is not an entry); find mints a REAL
entry as the first entry of a one-entry map, nil values intact
- the sci avoid-method-too-large stub passes its registry map through
instead of returning a raw host table (strict conj rejected it; sci's
clojure-core registry is also no longer discarded)
Test updates: lazy-infinite pins take-nth realization at 5 (was 7 — the
canonical lazy impl realizes fewer); self-host asserts the analyzer IS loaded
in interpret mode (compiled expanders, PR #50) and is NOT in the
:compile-macros? false oracle. 18 new maps-spec rows.
Gate: jpm test exit 0 (verified directly, not through a pipe), conformance
326x3, suite 4698 >= 4660.
empty, assoc-in, and update-in move to 20-coll.clj as the canonical recursive
ports; interpose and take-nth move to the lazy tier WITH their canonical
transducer arities (volatile-based), so the seed's td-interpose/td-take-nth
helpers go too. (empty lazy-seq) is () now — the kernel fn returned a bare
host table for it.
keys/vals/empty? stay put for now: they're expander-coupled — 00-syntax's
when/and/or/cond/destructure expanders call them at expansion time, which
happens during the kernel-tier compile, before any later tier exists. They
move when early defns get the staged-recompile treatment macros already have.
26 new spec rows (incl. transducer arities through sequence/into and laziness
checks against (range)). Gate green: conformance 326x3, suite >= baseline,
full jpm test.
key/val/select-keys/zipmap/merge/merge-with/get-in/memoize/partial/
trampoline/some?/true?/false?/max/min/reverse move to 20-coll.clj as the
canonical Clojure definitions, plus find — which was previously missing from
jolt entirely (select-keys/merge-with/memoize build on it). Two behavior
fixes ride along: memoize now caches nil results (the kernel fn re-computed
them — canonical find-based impl), and conj of nil onto a map is a no-op as
in Clojure (it errored; the canonical merge relies on it). max/min keep the
JVM NaN behavior by construction (pairwise >/<). not= stays: the kernel tier
(subvec) uses it.
One new tier-ordering rule, learned the hard way: a tier may only use macros
from tiers that load BEFORE it — memoize's if-let (30-macros) broke compiled
init while interpret mode passed, because compile expands macros at tier
load and the interpreter expands lazily. Now documented in the migration
workflow note.
MIGRATION.md is gone — task tracking lives in beads (jolt-ded; the per-batch
workflow, tier-order rules, perf wall, and remaining candidates are in bd
memory core-migration-workflow). The doc's candidate lists had gone stale
against the actual seed anyway.
43 new spec rows. Gate green: conformance 326x3, suite >= baseline, full
jpm test, bench at parity with main back-to-back (4851 vs 4831 TOTAL).
Nine more seed leaves move to 20-coll.clj (verified leaf-by-leaf: defn +
core-bindings entry only, no internal callers). fnil is upgraded to Clojure's
canonical 2/3/4-arity — it patches only the first 1-3 arguments; the old
kernel fn patched every position it had a default for, which Clojure does
not. The rest carry their kernel semantics over unchanged (bigdec is a
double, numerator/denominator throw, supers is #{}, munge rewrites dashes).
16 new spec rows incl. the fnil arity-contract cases. Gate green:
conformance 326x3, suite 4577, full jpm test (2:18 — first full run with the
ctx image cache on main).
*in*, read-line, read, with-in-str, and line-seq land as a new overlay IO
tier (core/50-io.clj). *in* is a dynamic var holding a reader — a plain map
of two closures, :read-line-fn (next line, nil at EOF) and :read-fn (next
form, advancing past exactly that form). The default *in* reads real stdin
with a shared leftover buffer, so read and read-line interleave correctly;
with-in-str rebinds *in* to a string reader over one atom-held buffer —
(read) consumes its form, a following (read-line) returns the rest of that
line, as in Clojure. read has the 0/1/3 arities (EOF throws, or returns
eof-value when eof-error? is false).
The Janet seed grows two seams next to read-string: __stdin-read-line (one
line off stdin, newline stripped) and __parse-next (one form off a string ->
[form rest], nil at end of input) — and loses the line-seq stub.
Two traps hit and documented for future tiers: a map LITERAL with :jolt/type
as a key is read as a tagged form (don't tag overlay value maps), and a
leftover seed stub holding the same name breaks direct-linked self-recursion
— the overlay line-seq's recursive call bound to the stub's root, truncating
after one line. The stub's string-splitting behavior is kept as a documented
extension.
20 new io-spec rows (read-line EOF/interleave, read arities + eval round-trip,
line-seq incl. real-stdin paths). Gate green: conformance 326x3, suite 4577,
full jpm test.
sorted-map/sorted-map-by/sorted-set/sorted-set-by/sorted?/sorted-map?/
sorted-set?/subseq/rsubseq now live in their own overlay tier (25-sorted.clj).
A sorted coll is a tagged host table with a comparator-ordered :entries
vector, a 3-way :cmp, and the tier's op implementations ATTACHED to the value
(:ops map): the seed's conj/assoc/get/seq/count/... branches are each a
one-line call through (coll :ops), so the ops travel with the value — correct
across contexts, forks, and AOT images, no module-level hooks to re-wire.
The host surface grows by three minimal value primitives: jolt.host/
tagged-table, ref-put! (already there), and ref-get — a raw field read,
because plain get on a sorted coll IS the comparator lookup and reading
:entries with it recurses.
This fixes a pile of Clojure-correctness gaps the Janet kernel had:
- lookup/membership now go through the COMPARATOR: (contains? (sorted-set 1)
1.0) was a deep= scan, (conj (sorted-set 1) 1.0) and assoc of a
comparator-equal key now no-op/replace as in Clojure
- equality is representation-agnostic: (= (sorted-map :a 1) {:a 1}) and
(= (sorted-set 1 2) #{1 2}) were false
- iteration was broken: (map inc (sorted-set 3 1 2)) errored
(realize-for-iteration and coll->cells had no sorted branches)
- empty?/empty saw the host wrapper, not the collection: (empty? (sorted-map))
was false, (empty sc) returned a bare table; it now keeps the comparator
- sorted colls canonicalize as map keys; comparator fns may be boolean
predicates or 3-way (Clojure's fn->comparator)
- sorted-map throws on odd kv count; conj nil is a no-op
Also fixes jolt-h86 en passant: into-conj had no branch for sets (or sorted
colls) and silently returned the target unchanged — (into #{} [:a :b]) was
#{}. The fallback now folds conj. Regression rows in sets-spec.
sorted-spec grows to 77 rows (comparator-based membership, equality,
empty/rseq/printing, seq-fn interop, subseq/rsubseq on maps). Gate green:
conformance 326x3, suite 4577 (vs 4566 prior — the battery gained rows),
sorted+sets specs, full jpm test, bench at parity with main back-to-back
(4521ms vs 4619ms TOTAL under identical load).
make-hierarchy/derive/underive/isa?/parents/ancestors/descendants are now
Clojure in the overlay — Clojure's own pure-map implementation: a hierarchy
is {:parents {tag #{..}} :ancestors {..} :descendants {..}}, the 3-arity
forms are PURE (derive returns a new hierarchy), and the 1/2-arity forms swap
a private global-hierarchy atom.
This fixes three correctness gaps the Janet kernel had: multi-parent derive
(the kernel's :parents held a single parent per tag), TRANSITIVE descendants
(the kernel tracked direct children only), and vector-pair isa?
((isa? [child1 child2] [parent1 parent2])). Cyclic and duplicate derives now
behave like Clojure (throw / no-op).
Multimethod dispatch was the kernel's only internal caller: defmulti-setup's
dispatch closure now calls the overlay's isa? through a lazily-resolved var
(cached per multimethod); a :hierarchy option is an atom (deref per dispatch,
matching Clojure's var semantics) or a plain hierarchy map. The Janet kernel
(types.janet) and the core-* wrappers are deleted.
20 new hierarchy spec rows (pure 3-arity incl. cycle/duplicate edges, global
+ dispatch incl. custom :hierarchy atoms). Gate green: conformance 326x3,
suite 4566 >= 4540, all batteries, bench in the session band.
The representation predicates (sequential?/associative?/counted?/indexed?/
reversible?/seqable?, boolean?/double?/float?/infinite?, the qualified/simple
ident predicates), the realization boundaries doall/dorun, list*, find,
realized?/force, pop, the print-str family, and rand-nth/random-sample are
now Clojure in 20-coll, expressed over the overlay's own predicates and
primitives — no Janet representations referenced.
Several got MORE correct in the move: dorun honors its bounded-n arity (the
seed ignored n); find works on vectors by index (contains? gives it free);
indexed? no longer claims seq results; seqable? no longer claims arbitrary
tagged tables (atoms were "seqable"); realized? reads the tagged slots via
the established get pattern.
Seed: 3427 -> ~3200 lines, 371 -> ~345 core-* fns. Gate green (conformance
326x3, suite 4569 >= 4540, stdlib battery, all specs+unit); bench in noise.
The names turn 2a's leak removal exposed as honestly missing, now proper:
- slurp/spit/flush (host-classified): path-based IO over Janet files; spit
takes :append; flush flushes *out*. printf prints formatted (no newline)
over the existing format. file-seq walks paths via two host dir primitives
through the overlay's tree-seq.
- ns-map / ns-unmap / ns-refers (ctx fns). ns-refers required fixing the
refer MODEL: refer/use/:refer now map the SOURCE VAR into the target ns
(the Clojure model) instead of copying its value into a new var — so
source-ns redefinitions propagate, the :macro flag travels for free, and
refers are identifiable by the var's home :ns.
- Thread-binding family: with-bindings*/with-bindings, bound-fn*/bound-fn,
bound?, thread-bound?, get-thread-bindings. The captured binding map is a
Janet struct keyed by the var tables — the exact frame representation
var-get reads — so it re-pushes correctly (a phm frame is invisible to
var lookup).
- load-string and eval interned as VALUES at the api layer (they need the
loader's compile-or-interpret routing); the eval special form still
handles direct calls.
Suite 4532 -> 4572 (baseline floor 4540 across timeout variance, clean 86),
conformance 326x3, stdlib battery, all specs+unit (+21 turn-2b rows).
Coverage: missing-portable 27 -> 10 (left: the *in*-model readers, the
with-local-vars/with-precision/extenders tail).
resolve-sym's last resort silently resolved any unknown Clojure symbol
against Janet's root environment — leaking Janet builtins with JANET
semantics into Clojure code: (type 1) was Janet's :number, (gensym) returned
Janet symbols (the long-documented (symbol (str (gensym))) macro landmine
existed BECAUSE of this), compare/slurp/int?/any? likewise. The explicit
janet/ prefix is the deliberate interop channel; the implicit fallback is
gone — an unresolved symbol is an error.
What the leak was masking, now proper interned vars:
- gensym: jolt's own (already existed, never interned) — returns real jolt
symbols; the macro landmine is dead
- compare: full Clojure total order (nil-first, numbers, strings, keywords,
symbols by ns/name, booleans, chars, uuid/inst, vectors by length then
elementwise; cross-type throws)
- type: :type metadata override, deftype/record tag as symbol, else a
taxonomy keyword (host-classified)
- int?: core-integer? — which had a latent bug the leak hid: (integer?
##Inf) was true (floor of inf is inf); NaN/infinities now excluded
- any?: constantly true (Clojure 1.9; SCI's namespaces.cljc needs it)
- jolt.interop/janet-type now uses the explicit (janet/type x) channel
- evaluator-test uses init (a bare make-ctx resolved EVERYTHING via the leak)
Suite 4470 -> 4532+ pass / 86-87 clean (proper compare unlocks the sort
files); baselines raised. Conformance 326x3 (+5 rows), +22 predicate spec
rows, stdlib battery green, all specs+unit. Coverage dashboard now counts
previously-leak-resolvable names honestly (missing-portable 19 -> 27).
#inst (jolt-rnh): an instant is an immutable tagged struct
{:jolt/type :jolt/inst :ms <epoch-millis>} — equality and map-key hashing by
INSTANT, so different offsets denoting the same moment are =. The reader
parses RFC3339 with Clojure's partial-timestamp defaults (#inst "2020" is
2020-01-01T00:00:00.000Z) and errors on malformed input; inst?/inst-ms in
the overlay; pr-str prints the canonical
#inst "yyyy-MM-ddThh:mm:ss.fff-00:00" and round-trips; str gives the bare
RFC3339 string. Self-evaluating in the evaluator (like uuid/chars).
Syntax-quote (jolt-l2a): a syntax-quoted self-evaluating literal (string,
number, boolean, nil, keyword, char) collapses to the literal at READ time,
matching Clojure's reader — so nested/adjacent backticks over literals are
inert: (= "meow" ```"meow") is true (jank pass-adjacent). Symbols still
qualify and collections still template. General nested syntax-quote over
non-literals remains UNVERIFIED in spec S25.
Tests: inst-spec (18 cases incl. partial defaults, offsets, round-trip),
+10 literal-collapse reader rows, +5 conformance rows (321x3). Spec
02-reader S20/S25 updated to normative. Suite stable 4470/86.
Fifteen vars from the spec coverage gap (docs/spec/coverage.md):
parse-long/parse-double/parse-boolean (strict validation; scan-number alone
accepts 0x10), newline, current-time-ms (host clock for time), update-keys/
update-vals (PHM base, collisions last-wins), partitionv/partitionv-all/
splitv-at (lazy seqs of vectors; splitv-at's tail stays a seq, matching the
reference), with-redefs/with-redefs-fn (roots restored on throw), time,
macroexpand (expand-1 to fixpoint), alias/ns-unalias (write BOTH alias stores
— require :as uses string-keyed :imports while ns-aliases reads :aliases;
split filed), ns-publics (symbol-keyed map; publics == interns, no privacy).
Three pre-existing bugs fixed along the way:
- (partition n step pad coll) misparsed pad as the coll and returned ()
- (require 'bare.symbol) rejected — only vector specs were accepted
- analyze-form leaked the interpreted analyzer's ns on a punt: a throw out of
the analyzer left current-ns=jolt.analyzer and :compile-ns set, so the
fallback interpretation resolved user vars against the wrong namespace
(bit (var user-sym) under compile mode)
And one overlay-authoring landmine documented in-code: a 20-coll fn must not
use 30-macros macros (with-redefs-fn's dotimes compiled as a forward ref that
resolved to the macro fn at runtime) — loop/recur instead.
Gate: conformance 316x3 (+14 rows), suite 4324 pass / 78 clean (was 4081/72;
parse_*/update_* files now contribute), baselines raised, all specs+unit,
fixpoint, self-host, sci, staged. Coverage: missing-portable 35 -> 20.
uuid support was broken end to end (jolt-6s2): random-uuid built malformed
strings ((string/format "%x") with no zero-padding, so hex groups came out
short, and no variant bits), uuid? was hardcoded false, the #uuid data reader
was identity (bare string), and parse-uuid didn't exist. Nothing tested it.
A UUID is now an immutable tagged struct {:jolt/type :jolt/uuid :str <lower>}
(make-uuid, types.janet) — struct value equality gives case-insensitive = and
map-key/set hashing for free, and the evaluator treats it as self-evaluating
(like chars). random-uuid emits a correct RFC 4122 v4 (zero-padded 8-4-4-4-12,
version nibble 4, variant 8-b). parse-uuid validates the canonical shape,
returns nil on a bad string, throws on a non-string (Clojure 1.11). uuid? is
an overlay tag predicate. str renders the bare string; pr-str renders
#uuid "..." and round-trips.
Tests: uuid-spec (30 cases: format, parse edge cases from the suite, value
semantics, reader literal), 6 conformance cases x3 modes. The suite's
parse_uuid/random_uuid/uuid_qmark files now contribute: 4049 -> 4074 pass,
71 clean files; baselines raised.
prefer-method/remove-method/remove-all-methods/get-method/methods become
overlay macros over ctx-capturing *-setup fns (a multimethod's method table
lives on its VAR, so the name passes quoted — the defmulti/defmethod shape).
instance? likewise (class names don't evaluate to values); satisfies? is a
plain ctx-capturing fn (evaluated args). locking and defonce become overlay
macros — locking now also evaluates the monitor expr (the old arm skipped it
and any body form past the first); defonce keeps the existing-root check.
read-string and macroexpand-1 are ctx-capturing fns.
Removed all eleven from the evaluator special arms + special-symbol?,
host_iface special-names, and compiler uncompilable-heads. evaluator-test's
locking/instance? cases use init now (overlay macros need the full env).
Surfaced pre-existing (filed): multimethod dispatch records prefer-method
preferences on the var but never consults them in ambiguous isa dispatch.
Gate: conformance 296x3 (+11 tier-6c cases), fallback-zero 73/3, fixpoint,
self-host, sci, staged, suite 4049>=4034, all specs+unit.
* core: fix jolt-265 — syntax-quote fully-qualifies core syms to clojure.core/
Re-attempt of the deferred gap, now unblocked. The earlier uberscript break
wasn't qualification itself but an aliased-ref resolution bug it exposed:
resolve-var looked up ns-aliases (e.g. g/foo) against the analyzer's REBOUND
ctx-current-ns (jolt.analyzer, since the analyzer runs interpreted in its own
ns) instead of the namespace being compiled. A user's aliased refs failed
mid-compile (g/hello -> nil in the bundled standalone).
- resolve-var resolves aliases against :compile-ns when set (same ns
h-current-ns uses), falling back to ctx-current-ns — correct for both modes.
- sq-symbol qualifies a resolved clojure.core name to clojure.core/<name>
(Clojure hygiene); special forms stay bare; unresolved syms -> current ns.
Tests updated to the qualified behavior (features-test, reader-forms-spec).
* test: comprehensive spec — regex + sorted colls + random/predicate gaps
Filling the biggest untested clojure.core areas found in a coverage audit
(168 of 506 provided fns had no spec). New + expanded suites:
- regex-spec.janet (20): #"…" literals, regex?, re-find/re-matches/re-seq
(match/no-match/groups), re-pattern, and clojure.string split/replace with
regex (incl $1 backrefs). Whole area was previously unspecced.
- sorted-spec.janet (14): sorted-map/sorted-set construction + ordering, sorted?,
subseq/rsubseq. Pins the working subset — get/conj/assoc/keys/vals on sorted
colls and the by-comparator ctors are not yet first-class (jolt-ti9).
- predicates-spec +14: seqable?, integer?, reduced?/unreduced, not-empty.
- numbers-spec +5: rand/rand-int/rand-nth invariants.
Fix: sorted? was bound to core-sorted-map? so it returned false for sorted-sets;
now true for both sorted maps and sets (core-sorted?).
Filed: jolt-ti9 (sorted collections incomplete: get/conj/assoc/keys/vals don't
operate on the sorted wrapper; sorted-*-by ignore the comparator).
Gate green incl full jpm build + jpm test.
* core: close surfaced gaps — first-class sorted colls, with-out-str, rand arity, deref reduced
jolt-ti9: sorted-map/sorted-set are now first-class across the collection fns —
get/assoc/dissoc/conj/contains?/keys/vals/disj and call-as-fn all operate on the
wrapper and preserve sort order. The by-comparator constructors (sorted-map-by/
sorted-set-by) now thread the user comparator (numeric or boolean-predicate) through
all derived colls. Sorted predicates/ctors/ops moved above core-conj so the
collection fns can branch on them; jolt-invoke (interpreter) gets inline branches.
jolt-rfw: add with-out-str (binds output to a string buffer) + the macro.
jolt-ek3: (rand n) arity and deref-of-reduced (uuid? still deferred).
Specs: new io-spec.janet; sorted-spec expanded to pin the now-working map/set ops
and by-comparator ordering; predicate/number spec restorations.
* remove old doc
* core: fix stale comment — by-comparator sorted ctors are implemented
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
