- host-interop-spec: add an 'interop / janet bridge' suite — janet/<name> and
janet.<module>/<name> resolution, the value-representation boundary (a Jolt
vector crosses as a Janet table), explicit-only (unprefixed module not
exposed), and unknown-symbol errors.
- nrepl-test: assert a def's value renders as #'ns/name (pr-str loops on a
var's cyclic ns refs, so jolt.nrepl renders vars itself).
Add a general Janet interop bridge and an nREPL implemented on top of it.
Interop bridge (evaluator):
- A qualified symbol whose namespace is `janet` or `janet.<module>` resolves
against Janet's environment: `janet/<name>` -> root binding (janet/slurp),
`janet.<module>/<name>` -> module binding (janet.net/server, janet.os/clock).
The explicit `janet` segment marks every crossing into host code (where
Clojure semantics, e.g. collection representation, no longer hold). This makes
the whole Janet stdlib — networking included — reachable from Clojure.
jolt.nrepl (Clojure, src/jolt/jolt/nrepl.clj):
- bencode codec (encode + streaming decode), ported from nrepl.bencode.
- server: accept loop via janet.net/accept + janet.ev/call (Janet's built-in
handler arity-checks Jolt closures, so we drive accept ourselves); ops clone/
describe/eval/load-file/close/ls-sessions/interrupt/eldoc following
babashka.nrepl response shapes. eval captures *out* by rebinding Janet's :out,
reports ns, streams out, and isolates the eval namespace (current-ns is global
ctx state) restoring it afterward. Vars are rendered as #'ns/name (pr-str
loops on a var's cyclic ns refs).
- client: connect / request / client-eval / client-clone / client-close.
CLI: `jolt nrepl [port]` starts the server and writes .nrepl-port; the Clojure
source is embedded at build time so the binary is self-contained from any cwd.
Tests: test/spec/nrepl-spec.janet (bencode), test/integration/nrepl-test.janet
(server+client over a real TCP/bencode wire, server in a subprocess).
Implement clojure.core futures backed by Janet's ev/thread for genuine
parallelism (CPU-bound work can use a second core, unlike cooperative go
blocks):
- future / future-call, deref + (deref f timeout-ms timeout-val), future?,
future-done?, future-cancel, future-cancelled?; realized? on futures.
- A worker OS thread computes and marshals back a [:ok v]/[:error e] result
over a thread-chan; a parent-side collector fiber caches it and closes a
broadcast latch so any number of deref-ers unpark.
- Snapshot semantics: separate heaps mean the body + captured state are copied
to the worker and only the result is copied back (mutating a captured atom
does not propagate). Documented in README.
- future-cancel can't interrupt a Janet OS thread, so it marks the future
cancelled/done (deref throws, predicates flip) while the worker runs out.
clojure-test-suite baseline 3915 -> 3913: implementing future unskips
realized_qmark.cljc's (when-var-exists future ...) block, which depends on
JVM Thread/sleep + real thread interruption jolt can't provide; deref then
re-raises the unresolved-Thread/sleep error. Documented at the baseline.
Spec: test/spec/futures-spec.janet (18 cases).
(chan n xform) applies a transducer on the put side: a jolt transducer is a
directly-callable closure, composed over a reducing fn whose step gives each
output value into the channel (honoring its buffer kind). One put may yield zero
or more values; a reduced result (e.g. from take) closes the channel; close!
runs the transducer completion arity to flush stateful remainders. Works with
map/filter/mapcat/take/comp/etc.
Buffers: (buffer n) fixed, (dropping-buffer n) drops new values when full,
(sliding-buffer n) drops the oldest. Implemented via a non-blocking give —
(ev/select [ch v] closed-chan) detects a full buffer without parking.
harness: run-spec flushes per suite. spec: core.async/channel-transducers (5),
core.async/buffers (3). jpm test green.
Note: distinct/dedupe/partition-all/partition-by still lack a 0-coll transducer
arity in core (separate gap), so they can't yet be used as channel xforms.
Jolt's dynamic-var binding stack was a single global array, so concurrent go
blocks interleaved each other's bindings and a go block didn't see the bindings
in effect when it was spawned.
Move the binding stack into Janet's fiber-local dyn (:jolt/binding-stack): each
fiber (go block) lazily gets its own array, so bindings can't interleave. Janet
ev/go fibers inherit the parent's dyn, but go-spawn now snapshots the binding
stack at spawn time and installs a private copy in the new fiber — Clojure
binding conveyance. A go block's own (binding ...) shadows the conveyed frame.
types.janet: cur-binding-stack / snapshot-bindings / install-bindings; var-get/
var-set/push/pop use the fiber-local stack. async.janet: go-spawn conveys.
spec: core.async/binding-conveyance (4 cases — conveyance, isolation, no leak to
root, inner shadowing). jpm test green.
Janet fibers are stackful coroutines, so a go block is just its body run in a
fiber that parks on channel ops by yielding to the event loop — the interpreter
call stack rides along, no CPS/state-machine transform. So <!/>! work anywhere
(inside try, nested fns, loops), unlike Clojure's go macro.
src/jolt/async.janet implements chan/chan?/close!/<!/>!/<!!/>!!/go/go-loop/
thread/alts!/timeout/put!/take! over ev/ channels and fibers, installed as the
clojure.core.async namespace (pre-populated in init, so require finds it).
A channel is a pair of ev/chans (:ch values + :done close-signal); a take is
(ev/select :ch :done), which drains buffered values before the close signal —
giving Clojure's drain-then-nil semantics without the buffer loss of
ev/chan-close, and with no leaked fibers (close! just closes :done).
Single-threaded cooperative scheduling: <! (park) and <!! (block) coincide.
Dynamic-var conveyance (Phase 2) and channel transducers (Phase 3) are TODO.
spec: test/spec/core-async-spec (16 cases — go/channels, buffering+close,
go-loop pipelines, alts!/timeout, parking inside try/nested-fn). jpm test green.
- grammar.ebnf: rewrite the number rule to cover the literal syntaxes the reader
now accepts — 0x/0X hex, N (bigint) / M (bigdec) suffixes, ratios a/b, radixed
integers (NrXXX, base 2..36), exponents, and the ##Inf/##-Inf/##NaN symbolic
floats — noting Jolt reads them as plain Janet numbers.
- README: Numbers bullet notes the literal syntaxes read; conformance count.
- reader-syntax-spec: drop the stale 'ratio not supported' case; add coverage
for hex-uppercase/N/M/ratio/radix/exponent/##Inf/##NaN.
- PLAN.md: refresh the stale Current State snapshot for the 3-layer test
structure (spec/integration/unit), ~3,920 suite assertions, 218/218
conformance, current source size.
jpm test green.
- A map entry is a 2-element tuple (Jolt produces tuples only from map iteration;
vector literals are pvecs, lists are arrays). key/val/map-entry? now accept a
2-tuple and reject a plain vector, matching Clojure's MapEntry-vs-vector
distinction — no metadata needed, the representations already differ.
- min-key/max-key reproduce Clojure's NaN-aware folding (2-arg strict </>, then
<=/>=) and require numeric keys (NaN allowed, strings throw).
- subvec coerces float/NaN indices like (int ...) (truncate, NaN->0) then
bounds-checks, instead of throwing on non-integers.
min_key 35/14 -> 49/0 (clean); key/val recover the 2-vector cases; subvec floats
fixed. clojure-test-suite pass 3898->3921. Updated conformance-test (key/val now
needs a real entry). spec: map/map-entry-&-key-ordering (14). jpm test green.
- subs requires a string and validates 0<=start<=end<=count (no Janet
from-end/clamping); negative/out-of-range/nil indices throw
- assoc! on a transient vector bounds-checks the index (0..count)
subs 11-fail -> 24/5 (5 remaining are byte-vs-codepoint Unicode, platform);
assoc_bang 32/6 -> 35/3. clojure-test-suite pass 3889->3898.
spec: string/subs-strictness (7), transient/assoc!-bounds (4). jpm test green.
merge now throws when a non-first arg is a scalar, a set, a list, or a
wrong-length vector (a length-2 vector or a map still merge). Other map-like
tables (records/sorted-maps/host tables, e.g. SCI's namespaces) keep the lenient
conj path so the SCI bootstrap still loads.
merge 29/11/2 -> 35/3/4. clojure-test-suite pass 3874->3880.
spec: 5 merge strictness cases. jpm test green.
- first throws on scalars (numbers/keywords/booleans/char & symbol structs)
- rseq is vector/sorted-only (throws on strings/maps/numbers/seqs)
- assoc requires an even kv count and a map/vector/nil receiver
first clean; rseq 12/1; assoc 39/3. clojure-test-suite pass 3864->3874.
spec: seq/more-strictness (11). jpm test green.
- assoc on a vector bounds-checks the index (0..count); out-of-range/negative throw
- dissoc throws on non-maps (numbers/sequences/sets/scalars); nil ok; records/
sorted-maps/meta-maps still handled
- count throws on scalars (numbers/keywords/symbols/booleans/chars)
- subvec validates vector type and 0<=start<=end<=count
- numerator/denominator always throw (Jolt has no ratio type)
- min-key/max-key throw on no values
count 18/2; dissoc 19/3; assoc 36/6; subvec 26/3/5; numerator/denominator throw
cases pass. clojure-test-suite pass 3840->3864. spec: map/strictness (16). jpm test green.
- peek/pop are stack-only (vectors/lists): throw on sets/maps/strings/scalars,
and pop throws on an empty vector/list
- vec throws on non-seqable args (numbers/keywords/transients)
- key/val require a map entry (2-element vector); throw on nil/numbers/maps/sets
pop clean; peek 9/2; vec 17/2/1; key/val 12/2/1 (remaining are the
2-vector-vs-MapEntry / tuple-from-seq divergences). pass 3824->3840.
spec: seq/accessor-strictness (16). jpm test green.
conj!/assoc!/dissoc!/disj!/pop!/persistent! now throw on a non-transient (or
wrong transient kind) instead of falling back to the persistent op, matching
Clojure. conj! keeps its special arities: (conj!) -> (transient []), (conj! coll)
-> coll. conj! onto a transient map accepts a [k v] pair or a map (merge), and
throws on a list/set/seq.
pop_bang/dissoc_bang clean; conj_bang 13/1/22->47/3/1; persistent_bang 9/8->16/1.
clojure-test-suite pass 3781->3824. spec: transient/strictness (10). jpm test green.
- zero?/pos?/neg? throw on non-numbers; odd?/even? throw on non-integers
(nil, infinities, NaN, fractional) via need-num/need-int helpers
- comparisons < > <= >= throw on non-number args (1-arity stays true, no check)
- max/min throw on non-number args
- quot/rem/mod throw on zero divisor and non-finite operands
odd?/even?/lt/gt/lt_eq/gt_eq suite files now clean; pass 3691->3738.
Updated systematic-coverage-test (zero? nil now throws). spec:
numbers/strictness (15). jpm test green.
Numbers printed via Janet's (string v) rendered infinities/NaN as inf/-inf/nan.
Add fmt-number so str/pr-str (and collection rendering) emit Infinity/-Infinity/
NaN. str.cljc 3->32 pass. Remaining str fails are the integer-valued-double
divergence ((str 0.0) is "0" not "0.0" since 0.0 == 0 in Janet).
spec: numbers/printing-of-inf-&-nan (5). jpm test green.
- case: quote list literals (read as arrays) in constant position so a wrapped
list ((a b c)) matches by value instead of being evaluated as a call; symbol
constants already quoted. Vector/map/set constants already worked. case errors
in the suite drop to 0 (60 pass).
- associative?: true only for vectors (pvec) and maps (phm/struct/sorted-map),
not lists/tuples-from-seq-fns/lazy-seqs/sets.
- reversible?: true for vectors and sorted-map/sorted-set only.
- update: coerce f via as-fn so (update m k :kw)/(update m k a-set) work; extra
args already handled.
- nth: (nth nil i)/(nth nil i default) returns nil/default instead of throwing.
clojure-test-suite pass 3649->3678, errors 122->105, clean files 44->46.
associative?/reversible? files now fully clean. spec: predicates + control/case.
jpm test green.
Biggest fix: jolt keywords are Janet keywords and maps are Janet structs/phm, but
(:a struct) at the Janet level returns nil (not a Clojure accessor) and errors on
a phm — so core-map/filter/sort-by/group-by/etc. calling (f x) directly broke the
ubiquitous keyword/set/map-as-function idioms ((map :a coll), (sort-by :k coll),
(filter a-set coll), (group-by :type coll)). Added as-fn coercion (keyword/symbol
-> key lookup, map -> key lookup, set -> membership) applied at the entry of
map/filter/remove/keep/mapv/filterv/sort-by/group-by/partition-by/some/
not-any?/not-every?/take-while/drop-while/min-key/max-key.
Also:
- realize-for-iteration treats nil as an empty seq (Clojure semantics), fixing
nth/nthrest/nthnext/take-last/reduce/doseq over nil.
- take-nth and interpose gained their 1-arg transducer arities.
- sort-by gained the 3-arg (keyfn comparator coll) form.
- min-key/max-key: single item returns it without calling f; ties keep the last.
- underive gained the 2-arg global-hierarchy form.
clojure-test-suite pass 3535->3649, errors 177->122, clean files 39->44.
spec: seq/IFn-values-as-functions (11). jpm test green.
Reader gaps caused the clojure-test-suite worker to crash whole deftests on
literals it could not parse (0N, 1.5M, 2r1010, 1/2), losing every assertion in
the file. read-number now handles:
- N (bigint) / M (bigdec) suffixes -> plain number (Jolt has no bignum/bigdec)
- ratios a/b -> double quotient
- radix integers NrDDD (2r1010, 16rFF, 36rZ) parsed by base
- exponents (1e3, 1.5e-2) and 0X hex
Also fixed suite measurement: when-var-exists now skips silently (its SKIP
print to stdout was corrupting the worker's count line, dropping whole files),
and the worker emits counts on an @@COUNTS sentinel line (robust against test
bodies that print, e.g. with-out-str). Runner parses the sentinel; deftest
crashes now report the underlying message.
Impact: clojure-test-suite 210->231 files run, pass 1955->3535, clean files
24->39. Baseline raised to 3450/38.
spec: numbers/literal-syntax (13 cases). jpm test green.
The reader now reads the symbolic float values ##Inf, ##-Inf and ##NaN. Added
infinite? and NaN? predicates. Fixed intval? to exclude infinity (floor(inf)=inf
but inf isn't integer-valued), so float?/double? are true for ##Inf and
int?/pos-int?/nat-int?/neg-int? are false for it.
This unblocked many number-test files whose forms previously failed to
READ (##Inf/##NaN literals), so clojure-test-suite jumped from 2241 to 2539
assertions and pass 1719 -> 1955. Baseline raised to 1900. NaN_qmark now runs.
float?/double? on integer-valued doubles (1.0) remain false: Janet represents
an integer and an integer-valued double identically, so they're inherently
indistinguishable — documented in the README Numbers section.
spec: numbers/floats-&-symbolic-values (15 cases). jpm test green.
Closes jolt-fy8 (fixable parts; int-vs-float ambiguity is a documented divergence).
Real transient correctness gaps surfaced by the clojure-test-suite:
- Transients are now invokable for read-only lookup like their persistent forms:
((transient v) i), ((transient m) k [default]), (:k (transient m)),
((transient s) x). jolt-invoke/coll-lookup gained a transient branch
(transient-lookup) that indexes :arr / canon-keyed :tbl. Added phm/canon as a
public canonicalizer so collection keys compare by value here too.
- assoc! accepts an ODD arg count (a missing final value is nil), unlike assoc —
core-assoc! now uses nil-safe get for the value instead of erroring.
- Using a transient after persistent! (or a second persistent!, or pop! on an
empty transient vector) now throws, via a :jolt/persistent invalidation flag
checked by the mutating ops. Catches the classic transient footgun.
spec: transients-spec gains invokable-lookup (7), assoc!-odd-args (4),
invalidation (4). clojure-test-suite pass 1704->1719.
Remaining transient fails are accepted lenient divergences (jolt doesn't throw
on bad-shape conj!/assoc!/pop! of wrong-typed args; nil set elements). jpm test green.
transduce-reduce and core-reduce both called realize-for-iteration, fully
realizing the coll before the reduce loop — so an infinite lazy seq hung before
the reduced short-circuit could fire. (into [] (take 5) (range)) etc. never
terminated.
Add reduce-with-reduced, which steps a lazy seq one cell at a time
(realize-ls/ls cell protocol), checking reduced? after each element so a take/
take-while transducer (or any reducing fn returning reduced) terminates over an
infinite seq. Route transduce-reduce and both core-reduce arities through it;
2-arg reduce now seeds from the first element and reduces the rest lazily.
spec: transducers/short-circuit + reduce/honors-reduced (13 cases).
clojure-test-suite timeouts 7->6, pass 1683->1688. jpm test green.
Fixes jolt-kxb.
Replace the correctness-only transient aliases with real mutable scratch
collections via host interop:
- transient vector -> a Janet array; conj!/assoc!/pop! mutate in place
- transient map -> a Janet table keyed by canonical key (collection keys still
compare by value); assoc!/dissoc!/conj! mutate in place
- transient set -> a Janet table; conj!/disj! mutate in place
- persistent! freezes back to a pvec / phm / phs
- count/nth/get/contains? work on transients; transient? predicate added
Building a map/set this way avoids the persistent path's per-step bucket-array
copying (transient map build ~35% faster at 20k here); vectors are comparable
since pvec conj is already ~O(1). The mutating ops return the transient and the
source collection is untouched.
spec/transients-spec (34 cases). conformance 218/218, jpm test green.
Mine the remaining integration 'ported Clojure' batteries into the spec layer
and delete them (clojure-atom/control/for/logic/macros, core, logic). A broad
function-coverage diff confirmed they exercised no clojure.core fn the spec
lacked; their distinctive value was the truthiness/boolean contract, now
captured in a dedicated spec.
New/expanded spec coverage:
- spec/truthiness-spec: only nil/false are falsy (0, "", [], {}, #{} are truthy);
not / and / or return-value & short-circuit semantics; if-not/when-not/boolean
- assert (exceptions-spec), get-validator (state-spec)
Layout now: spec 23 files / 732 cases; integration trimmed to 10 genuine
cross-cutting batteries (conformance, SCI bootstrap/runtime, jank, compile-mode,
api, namespace, bootstrap, features, systematic-coverage). conformance 218/218,
jpm test green.
Close jolt-dd5.
- catch now binds the originally-thrown value (unwrapping the :jolt/exception
envelope), so (catch ... e (throw e)) rethrows the same exception instead of
nesting another envelope, and (catch ... e e) on (throw 42) yields 42.
- var-set updates the innermost thread-binding frame for the var (replacing the
stack slot) when the var is dynamically bound, matching Clojure; it falls back
to the root otherwise.
Restored spec cases: exceptions rethrow + catch-binds-thrown-value, namespaces
var-set-in-binding. conformance 218/218, jpm test green.
Mine the cljs port batteries into the spec layer and delete them (their behavior
is covered by spec; the unique functions they exercised are now specced).
Removed test/integration/ports/ entirely.
New/expanded spec coverage from the mining:
- spec/exceptions-spec: try/catch/finally, throw, ex-info/ex-message/ex-data/ex-cause
- doto, pr-str, keyword/symbol constructors, atom?, dynamic var binding
Two rare edges filed (jolt-...): rethrow of a caught ex-info re-wraps it; var-set
on a dynamic var inside binding no-ops. Core try/catch/ex-info and binding work.
Test layout is now spec (22 files, ~677 cases) / integration / unit / support.
conformance 218/218, jpm test green.
Mine the phase* port batteries into the spec layer and delete them (their
behavior is now covered by spec): phase5 (hierarchies), phase6 (tagged
literals/reader-conditionals), phase7 (lazy/sets), phase8+phase12 (protocols;
phase12 was a duplicate of phase8), phase10 (strings/set), phase13
(reify/walk). Unique cases mined into spec: reader #inst/#uuid/#?@ splice,
(Type. args) dot constructor, lazy-seq body-runs-once, keywordize-keys/
stringify-keys, custom :default key and explicit :hierarchy dispatch.
phase6-final tested the COMPILE-MODE path ({:compile? true}), distinct from the
interpreter-based spec — kept and renamed integration/compile-mode-test.
jpm test green, conformance 218/218.
Close jolt-do7. Maps/sets keyed by a collection (a map, vector, ...) now compare
by value instead of Janet identity:
- PHM/PHS hash and compare keys through an injected canonicalizer (collection
keys -> value-hashable struct/tuple); keys are still stored as-is
- map literals and core-assoc promote to a phm when a key is a collection
- frequencies/group-by use a phm base so collection elements/keys dedup by value
- set equality is value-based (from earlier)
Real bugs found and fixed along the way:
- set literals #{(inc 1)} did not evaluate their elements (stored raw forms!)
- the REPL printer rendered phm maps as {} (they hit the deftype branch); now a
phm branch prints entries
Added spec cases (maps/collection-keys, sets/literals & value elements).
conformance 218/218, jpm test green.
Final spec areas. Bugs caught and fixed:
- (def name docstring value) used the docstring as the value; now the 3-arg
docstring form binds the value and records :doc meta
- resolve was a nil stub; now a special form resolving a symbol to its var
(nil if unresolved). Added find-ns (non-creating lookup) and ns-name.
- in-ns didn't evaluate its arg, so (in-ns 'foo) failed; now evaluates it per
Clojure (the integration test's unquoted form updated to the quoted idiom)
- #(... %& ...) built %& as a positional param instead of a & rest param;
now emits (fn* [... & gen] ...) so %& captures the rest
Full public-API spec layer now in place. conformance 218/218, jpm test green.
Add spec suites covering lazy-seq/lazy-cat laziness, infinite + self-referential
seqs (ones/nats/fib), realized?; transducers (map/filter/take/cat/keep/mapcat as
xforms, comp, into/transduce/sequence/eduction/completing); and metadata
(with-meta/meta/vary-meta/^ reader). All passed against jolt as written — no
implementation changes needed. jpm test green.
Add spec suites for stateful refs (atoms/volatiles/delays/promises), multimethods
(dispatch + hierarchies), and protocols/types/records. Bugs caught and fixed:
- reify with multiple methods registered only the first (stepped i by 2 over
one-element specs); now collects every method spec
- get-method/methods/remove-method evaluated their arg to the dispatch fn, but a
multimethod's methods live on its var — now resolve the var (get-method and
methods are special forms; remove-method fixed). get-method/methods added.
- no global hierarchy: 2-arg derive modified a throwaway, isa? was hardcoded
false, parents missing, ancestors/descendants returned arrays (so contains?
failed), and dispatch ignored derive. Now a global hierarchy backs the 1/2-arg
derive/isa?/parents/ancestors/descendants (returning sets) and multimethod
dispatch falls back to it.
Corrected a phase5 port assertion that encoded the old isa? bug.
conformance 218/218, jpm test green.
Add spec suites for conditionals/logic/let/loop/iteration/threading, functions
(definition, application, combinators), and destructuring across let/fn/loop/
doseq/for. Bug caught and fixed: loop* assumed simple-symbol bindings, so
(loop [[a b] [1 2]] ...) errored; it now destructures each binding via
destructure-bind, supporting patterns in loop bindings (and recur). jpm test green.
Add spec suites for numbers/arithmetic, strings (str + clojure.string), and
type/value predicates. Bugs they caught and fixed:
- <, >, <=, >= were binary-only; now variadic ((< 1 2 3) chains)
- seq? was true for vectors; vectors are not ISeq in Clojure, so (seq? [1])
is now false (true only for lists/lazy-seqs)
- clojure.string/index-of and last-index-of were 1-based (stray inc); now
0-based per Clojure
Corrected a systematic-coverage assertion that encoded the old seq? bug.
jpm test green, conformance 218/218.
Add behavioral spec suites for the collection API (~180 cases). Writing them
surfaced and fixed real bugs:
- (count nil) errored; now 0 (Clojure semantics)
- (repeat x) and (repeatedly f) — the infinite 1-arg arities — were unsupported;
now return lazy infinite seqs
- set equality used deep= (representation-sensitive), so a set of map literals
could differ from an equal set built differently; now value-based (each
element value-equal to some element of the other)
Known limitation filed: phm-typed maps (from hash-map) used as set elements /
map keys hash by identity (map *literals* are structs and work).
jpm test green.
