README and the self-hosting design doc now describe the current pipeline:
compile-by-default with an always-correct interpreter fallback, var-cell late
binding, parity validation, and AOT images. The staged path marks
var-indirection, hybrid+coverage, and compile-by-default+AOT done; self-hosting
the compiler and moving core to Clojure remain.
The reader expanded ^X form to (with-meta form X), which evaluated the tag (so
^String errored 'Unable to resolve symbol: String') and, as a param, was no
longer a bare symbol (so the arg bound to nil). Now a keyword/symbol/string hint
on a symbol attaches to the symbol's :meta and the symbol stays bare, so type
hints are transparent in params, lets, and bodies. Map metadata still uses a
runtime with-meta form.
meta now returns a symbol's :meta, and def applies the name's metadata
(^:dynamic, ^:private, ^Type tag, ^{:doc}) to the var, so (meta (var x)) is
consistent. Specs in metadata-spec; grammar note in the ebnf.
README notes regex \p{...} as unsupported (separate from this).
Add --version/version, -e/--eval, -f/--file, -m/--main (require NS and apply
its -main to *command-line-args*), and nrepl-server/--nrepl-server taking an
optional [host:]port (nrepl stays an alias). repl/help/-h round it out.
Also rewrite doc/tools-deps.md as design notes for the now-implemented deps
support (it still read as pre-implementation research), and note the new flags
in the README. Test: cli-test runs the flags from source.
Move build details, JOLT_PATH/namespace resolution, and the jolt-deps/deps.edn
usage into doc/building-and-deps.md instead of growing the README; README Build
now links to it.
Add a general Janet interop bridge and an nREPL implemented on top of it.
Interop bridge (evaluator):
- A qualified symbol whose namespace is `janet` or `janet.<module>` resolves
against Janet's environment: `janet/<name>` -> root binding (janet/slurp),
`janet.<module>/<name>` -> module binding (janet.net/server, janet.os/clock).
The explicit `janet` segment marks every crossing into host code (where
Clojure semantics, e.g. collection representation, no longer hold). This makes
the whole Janet stdlib — networking included — reachable from Clojure.
jolt.nrepl (Clojure, src/jolt/jolt/nrepl.clj):
- bencode codec (encode + streaming decode), ported from nrepl.bencode.
- server: accept loop via janet.net/accept + janet.ev/call (Janet's built-in
handler arity-checks Jolt closures, so we drive accept ourselves); ops clone/
describe/eval/load-file/close/ls-sessions/interrupt/eldoc following
babashka.nrepl response shapes. eval captures *out* by rebinding Janet's :out,
reports ns, streams out, and isolates the eval namespace (current-ns is global
ctx state) restoring it afterward. Vars are rendered as #'ns/name (pr-str
loops on a var's cyclic ns refs).
- client: connect / request / client-eval / client-clone / client-close.
CLI: `jolt nrepl [port]` starts the server and writes .nrepl-port; the Clojure
source is embedded at build time so the binary is self-contained from any cwd.
Tests: test/spec/nrepl-spec.janet (bencode), test/integration/nrepl-test.janet
(server+client over a real TCP/bencode wire, server in a subprocess).
- Add tests workflow status badge.
- Replace the stale 'Janet >= 1.36' line: futures/core.async use threaded ev/
channels; developed and CI-tested against 1.41.
- Note that jpm build can serve a stale build/jolt (use jpm clean) while jpm
test runs from source.
Implement clojure.core futures backed by Janet's ev/thread for genuine
parallelism (CPU-bound work can use a second core, unlike cooperative go
blocks):
- future / future-call, deref + (deref f timeout-ms timeout-val), future?,
future-done?, future-cancel, future-cancelled?; realized? on futures.
- A worker OS thread computes and marshals back a [:ok v]/[:error e] result
over a thread-chan; a parent-side collector fiber caches it and closes a
broadcast latch so any number of deref-ers unpark.
- Snapshot semantics: separate heaps mean the body + captured state are copied
to the worker and only the result is copied back (mutating a captured atom
does not propagate). Documented in README.
- future-cancel can't interrupt a Janet OS thread, so it marks the future
cancelled/done (deref throws, predicates flip) while the worker runs out.
clojure-test-suite baseline 3915 -> 3913: implementing future unskips
realized_qmark.cljc's (when-var-exists future ...) block, which depends on
JVM Thread/sleep + real thread interruption jolt can't provide; deref then
re-raises the unresolved-Thread/sleep error. Documented at the baseline.
Spec: test/spec/futures-spec.janet (18 cases).
- .github/workflows/tests.yml: build Janet (cached) + jpm, init vendor/sci
submodule, run jpm test on every push and PR.
- README: explain the per-form eval router — interpreted (default) vs
compiled (:compile?), the always-interpret carve-out, and shared context.
Two changes unlock native Janet speed in compile mode:
- Hot numeric primitives (+ - * < > <= >=) emit as native Janet SYMBOLS rather
than the variadic core fns, so Janet's compiler uses its arithmetic/compare
opcodes. = / not= / quot / rem / mod / division stay as core fns (their
semantics differ from Janet's). Trade-off: the strict non-number checks are
relaxed under compilation (documented perf-mode divergence).
- emit-invoke emits a DIRECT call (f arg...) when the callee is a function
reference (core/local/symbol/fn), instead of wrapping every call in jolt-call.
jolt-call is kept only for keyword/collection literals in call position
((:k m), ({:a 1} :a)) so IFn dispatch still works.
compiled fib(30): 3.4s -> 0.076s (native ceiling), faster than jank's 0.8s.
Updated compiler-test string assertions (core-+ -> +); compile-mode-test gains
native-op + IFn-dispatch cases; README documents compile mode. jpm test green.
(chan n xform) applies a transducer on the put side: a jolt transducer is a
directly-callable closure, composed over a reducing fn whose step gives each
output value into the channel (honoring its buffer kind). One put may yield zero
or more values; a reduced result (e.g. from take) closes the channel; close!
runs the transducer completion arity to flush stateful remainders. Works with
map/filter/mapcat/take/comp/etc.
Buffers: (buffer n) fixed, (dropping-buffer n) drops new values when full,
(sliding-buffer n) drops the oldest. Implemented via a non-blocking give —
(ev/select [ch v] closed-chan) detects a full buffer without parking.
harness: run-spec flushes per suite. spec: core.async/channel-transducers (5),
core.async/buffers (3). jpm test green.
Note: distinct/dedupe/partition-all/partition-by still lack a 0-coll transducer
arity in core (separate gap), so they can't yet be used as channel xforms.
Janet fibers are stackful coroutines, so a go block is just its body run in a
fiber that parks on channel ops by yielding to the event loop — the interpreter
call stack rides along, no CPS/state-machine transform. So <!/>! work anywhere
(inside try, nested fns, loops), unlike Clojure's go macro.
src/jolt/async.janet implements chan/chan?/close!/<!/>!/<!!/>!!/go/go-loop/
thread/alts!/timeout/put!/take! over ev/ channels and fibers, installed as the
clojure.core.async namespace (pre-populated in init, so require finds it).
A channel is a pair of ev/chans (:ch values + :done close-signal); a take is
(ev/select :ch :done), which drains buffered values before the close signal —
giving Clojure's drain-then-nil semantics without the buffer loss of
ev/chan-close, and with no leaked fibers (close! just closes :done).
Single-threaded cooperative scheduling: <! (park) and <!! (block) coincide.
Dynamic-var conveyance (Phase 2) and channel transducers (Phase 3) are TODO.
spec: test/spec/core-async-spec (16 cases — go/channels, buffering+close,
go-loop pipelines, alts!/timeout, parking inside try/nested-fn). jpm test green.
- grammar.ebnf: rewrite the number rule to cover the literal syntaxes the reader
now accepts — 0x/0X hex, N (bigint) / M (bigdec) suffixes, ratios a/b, radixed
integers (NrXXX, base 2..36), exponents, and the ##Inf/##-Inf/##NaN symbolic
floats — noting Jolt reads them as plain Janet numbers.
- README: Numbers bullet notes the literal syntaxes read; conformance count.
- reader-syntax-spec: drop the stale 'ratio not supported' case; add coverage
for hex-uppercase/N/M/ratio/radix/exponent/##Inf/##NaN.
- PLAN.md: refresh the stale Current State snapshot for the 3-layer test
structure (spec/integration/unit), ~3,920 suite assertions, 218/218
conformance, current source size.
jpm test green.
Strictness work brought the suite to ~3900 passing; the leniency divergence is
largely gone. Remaining failures are bignum/ratio/bigdec, integer/float
identity, 64-bit/Unicode, eager-vector seqs, and the map-entry-as-2-vector case.
~3700 suite assertions pass; the remainder are accounted for by the documented
platform/design differences (no bignum/ratio/bigdec, integer/float identity,
64-bit/Unicode, leniency vs throwing, eager-vector seqs).
The reader now reads the symbolic float values ##Inf, ##-Inf and ##NaN. Added
infinite? and NaN? predicates. Fixed intval? to exclude infinity (floor(inf)=inf
but inf isn't integer-valued), so float?/double? are true for ##Inf and
int?/pos-int?/nat-int?/neg-int? are false for it.
This unblocked many number-test files whose forms previously failed to
READ (##Inf/##NaN literals), so clojure-test-suite jumped from 2241 to 2539
assertions and pass 1719 -> 1955. Baseline raised to 1900. NaN_qmark now runs.
float?/double? on integer-valued doubles (1.0) remain false: Janet represents
an integer and an integer-valued double identically, so they're inherently
indistinguishable — documented in the README Numbers section.
spec: numbers/floats-&-symbolic-values (15 cases). jpm test green.
Closes jolt-fy8 (fixable parts; int-vs-float ambiguity is a documented divergence).
Run the external cross-dialect clojure-test-suite (lread/clojure-test-suite,
240 per-fn .cljc files in clojure.test format) against Jolt:
- test/support/clojure_test.clj: minimal clojure.test + portability shims
(deftest/is/testing/are + when-var-exists/thrown?/big-int?/lazy-seq?) — just
enough surface to load the suite and tally pass/fail/error. Pre-loaded so the
suite's (require [clojure.test ...]) finds it already populated.
- test/integration/suite-worker.janet: one-shot worker that loads the shim,
the suite's number-range helper ns, and a single .cljc file, then prints
'pass fail error'.
- test/integration/clojure-test-suite-test.janet: spawns a worker per file
under an ev/with-deadline wall-clock budget (so infinite-seq tests that hang
Jolt's eager evaluator are auto-contained, not a manual skip-list) and asserts
pass/clean-file counts stay at/above a baseline. References ~/src/clojure-test-suite
if present; skips cleanly when absent, like the jank battery.
Current: 210 files run, 7 timed out, 2233 assertions -> 1683 pass / 350 fail /
200 error, 23 clean files. Remaining fails are genuine divergences (float/ratio/
bigint, lenient transients where Clojure throws), tracked separately.
Fixes two real evaluator bugs the suite surfaced:
- :refer now preserves a referred macro's :macro flag (was interned as a plain
value, degrading referred macros to functions).
- resolve-var now resolves ns aliases (like resolve-sym), so aliased macros
(e.g. p/thrown? via :as p) dispatch as macros instead of being called as fns.
Replace the correctness-only transient aliases with real mutable scratch
collections via host interop:
- transient vector -> a Janet array; conj!/assoc!/pop! mutate in place
- transient map -> a Janet table keyed by canonical key (collection keys still
compare by value); assoc!/dissoc!/conj! mutate in place
- transient set -> a Janet table; conj!/disj! mutate in place
- persistent! freezes back to a pvec / phm / phs
- count/nth/get/contains? work on transients; transient? predicate added
Building a map/set this way avoids the persistent path's per-step bucket-array
copying (transient map build ~35% faster at 20k here); vectors are comparable
since pvec conj is already ~O(1). The mutating ops return the transient and the
source collection is untouched.
spec/transients-spec (34 cases). conformance 218/218, jpm test green.
Mine the remaining integration 'ported Clojure' batteries into the spec layer
and delete them (clojure-atom/control/for/logic/macros, core, logic). A broad
function-coverage diff confirmed they exercised no clojure.core fn the spec
lacked; their distinctive value was the truthiness/boolean contract, now
captured in a dedicated spec.
New/expanded spec coverage:
- spec/truthiness-spec: only nil/false are falsy (0, "", [], {}, #{} are truthy);
not / and / or return-value & short-circuit semantics; if-not/when-not/boolean
- assert (exceptions-spec), get-validator (state-spec)
Layout now: spec 23 files / 732 cases; integration trimmed to 10 genuine
cross-cutting batteries (conformance, SCI bootstrap/runtime, jank, compile-mode,
api, namespace, bootstrap, features, systematic-coverage). conformance 218/218,
jpm test green.
Mine the cljs port batteries into the spec layer and delete them (their behavior
is covered by spec; the unique functions they exercised are now specced).
Removed test/integration/ports/ entirely.
New/expanded spec coverage from the mining:
- spec/exceptions-spec: try/catch/finally, throw, ex-info/ex-message/ex-data/ex-cause
- doto, pr-str, keyword/symbol constructors, atom?, dynamic var binding
Two rare edges filed (jolt-...): rethrow of a caught ex-info re-wraps it; var-set
on a dynamic var inside binding no-ops. Core try/catch/ex-info and binding work.
Test layout is now spec (22 files, ~677 cases) / integration / unit / support.
conformance 218/218, jpm test green.
Reorganize the flat 49-file test/ into three layers (jpm test recurses, so all
are still discovered):
- test/unit/ white-box component tests (reader, evaluator, types,
persistent-map, lazy-seq, macro, interop, compiler)
- test/integration/ cross-cutting + regression batteries (conformance, jank,
sci-bootstrap/runtime, features, systematic-coverage, api,
core, namespaces, ported clojure suites) and
.../ports/ ported clojure/cljs test batches pending consolidation
- test/spec/ the behavioral contract (built out in following commits)
- test/support/harness.janet shared defspec table runner (cases compared via
Jolt's own =, with a :throws sentinel) + expect= helpers
Files moved with git mv (history preserved) and import paths fixed for depth.
jpm test green. README Test section updated.
Next: build out test/spec/ to cover the public API area-by-area, mining the
integration batteries and filling gaps.
Update stale claims: bigint/biginteger (int/s64), +'/*'/inc' aliases, transients
(correctness-only), and the byte-array-via-buffer / array family are now
implemented. Note clojure.repl/template and JVM reflection/proxy remain out.
Round 3 of the persistent-collections work. Lists were immutable Janet arrays,
so conj/cons-prepend was an O(n) copy (O(n^2) to build a list) — a large perf
gap vs Clojure's PersistentList.
Add src/jolt/plist.janet: an immutable cons-cell list (first/rest/count), same
algorithm as Clojure/CLJS/jank PersistentList. conj/cons onto a list now creates
an O(1) node that shares the existing list as its tail (no copy), with a cached
O(1) count. Repeated conj is O(n) total instead of O(n^2).
Hooked plist through first/rest/next/seq/count/peek/pop/nth/empty/empty?, the
predicates (list?/seq?/coll?/sequential?), realize-for-iteration, =, coll->cells
(concat/lazy), both printers, destructuring, and instance? tags. (list ...) and
quoted lists stay arrays; only conj/cons introduce plist nodes, so the surface
and risk stay small.
Verified: reduce-conj of 200k elements runs in ~0.4s (was effectively O(n^2)).
conformance 206/206, features 78/78 (+7 list regressions), jank 120 (+1).
Round 2 of the persistent-collections work.
Add a real 32-way branching-trie persistent vector (src/jolt/pv.janet) with a
tail buffer: O(log32 n) conj/assoc/nth/pop, with unchanged subtrees shared by
identity. Vector literals and vec/vector/conj/assoc/subvec/etc. now produce and
maintain these in the default (immutable) build, replacing the old tuple-based
vectors. Every core seq op, the destructurer, IFn application, the printers, =,
and the evaluator's literal/splice paths were taught to handle the pvec type.
Define several Clojure seq fns that were silently leaking to Janet builtins
(some, keep, interleave, flatten, mapcat, interpose) and broke once vectors
became tables; normalize collections through realize-for-iteration everywhere.
Build-time JOLT_MUTABLE flag now selects fast Janet-native mutable collections:
in that mode vectors are arrays (conj appends in place, vector? true, print []),
sharing one representation with lists. Default build is immutable.
Tests: conformance 206/206, features 71/71, jank 119 (baseline). Test helpers
normalized so Janet-level = compares against tuple literals regardless of repr.
The 2 test-load-sci failures (bit-clear/get-method) pre-date this work.
