simon/jolt
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Docs + CI for the Chez-only substrate

Browse source 
Rewrite the README, CLAUDE.md build/architecture sections, test/chez/README,
and conformance SPEC for the Janet-free world: bin/joltc + make test, the
self-hosting bootstrap, the frozen JVM-sourced corpus. CI installs Chez + JDK/
Clojure and runs 'make test' (was Janet/jpm).

jolt-cf1q.6
This commit is contained in:
Yogthos 2026-06-21 11:34:48 -04:00
parent 58d03d67be
commit 750ce05716
5 changed files with 169 additions and 386 deletions
Download patch file Download diff file Expand all files Collapse all files

61
.github/workflows/tests.yml vendored
View file

@ -1,6 +1,7 @@
name: tests name: tests
# Run the full test suite (jpm test) on every push and pull request. # Run the gate (make test) on every push and pull request. Chez is the sole
# substrate; the JVM is used only as the conformance oracle (certify).
on: on:
push: push:
pull_request: pull_request:
@ -8,53 +9,27 @@ on:
jobs: jobs:
test: test:
runs-on: ubuntu-latest runs-on: ubuntu-latest
env:
JANET_VERSION: v1.41.2 # bump to match the version Jolt is developed against
# Per-file deadline for the clojure-test-suite battery. Finite files finish
# in well under 1s; the genuinely-infinite ones get killed at any deadline.
# A generous value gives slow CI runners headroom so a sub-second file
# spiking doesn't time out and drop total-pass below the baseline.
JOLT_SUITE_TIMEOUT: "20"
steps: steps:
- uses: actions/checkout@v5 - uses: actions/checkout@v5
with: with:
# Submodules: vendor/sci (SCI bootstrap/runtime tests) and submodules: recursive # vendor/irregex, used by the Chez regex shim
# vendor/clojure-test-suite (the cross-dialect conformance battery,
# asserted against a baseline by clojure-test-suite-test.janet).
submodules: recursive
- name: Cache Janet build - name: Install Chez Scheme
id: cache-janet
uses: actions/cache@v5
with:
path: /tmp/janet
key: janet-${{ env.JANET_VERSION }}-${{ runner.os }}
- name: Build Janet
if: steps.cache-janet.outputs.cache-hit != 'true'
run: | run: |
git clone --depth 1 --branch "$JANET_VERSION" https://github.com/janet-lang/janet.git /tmp/janet sudo apt-get update
make -C /tmp/janet sudo apt-get install -y chezscheme
# the gate invokes `chez`; Debian/Ubuntu installs it as `scheme`
if ! command -v chez >/dev/null 2>&1; then
sudo ln -s "$(command -v scheme)" /usr/local/bin/chez
fi
chez --version || true
- name: Install Janet - name: Install JDK + Clojure (certify oracle)
run: sudo make -C /tmp/janet install
- name: Install jpm
run: | run: |
git clone --depth 1 https://github.com/janet-lang/jpm.git /tmp/jpm sudo apt-get install -y default-jdk rlwrap
# bootstrap.janet resolves jpm/cli.janet relative to the cwd, so it curl -L -O https://github.com/clojure/brew-install/releases/latest/download/linux-install.sh
# must run from inside the jpm checkout. sudo bash linux-install.sh
cd /tmp/jpm clojure --version
sudo janet bootstrap.janet
- name: Janet version - name: Gate
run: janet -v run: make test
- name: Build executable
run: jpm build
- name: Run tests
# Parallel runner (same file set as `jpm test`, across CPU workers).
# The executable is built in the previous step, so binary-spawning tests
# don't skip.
run: janet run-tests.janet

93
CLAUDE.md
View file

@ -53,75 +53,64 @@ bd close <id> # Complete work
## Build & Test ## Build & Test
```bash No build step — `bin/joltc` runs off the checked-in seed (`host/chez/seed/`).
jpm build # build/jolt (one binary; ctx baked at build time) The gate is pure Chez (+ Clojure for the JVM oracle); no Janet.
jpm build; janet run-tests.janet # FULL gate, PARALLEL (across CPU workers) — build first so binary-spawning tests don't skip; JOLT_TEST_JOBS overrides worker count, -v shows all output
jpm test # FULL gate, serial (same file set; slower — recursive over test/)
janet test/spec/<f>.janet # one spec file
janet test/integration/conformance-test.janet # 3-mode conformance (interpret/compile/self-host)
JOLT_BENCH=1 janet test/bench/core-bench.janet # bench (opt-in; skipped in the gate) — back-to-back vs main, never absolute
```
**Run the gate with a REAL exit code.** `jpm test | grep ...` reports grep's
exit, not jpm's — this once shipped masked spec failures. Correct form:
```bash ```bash
jpm test > /tmp/gate.out 2>&1; echo "EXIT: $?" bin/joltc -e EXPR # run a Clojure expression on Chez
grep -E "non-zero exit|All tests" /tmp/gate.out make test # FULL gate (self-host + corpus + unit + smoke + certify)
make corpus # conformance corpus vs the JVM-sourced spec (floor 2678)
make unit # host-specific unit cases (test/chez/unit.edn)
make selfhost # bootstrap fixpoint (rebuild == checked-in seed)
make certify # JVM oracle (skips if clojure absent)
chez --script host/chez/run-corpus.ss # the corpus gate directly; JOLT_CORPUS_LIMIT=N for a fast stride
make remint # re-mint the seed after a seed-source change
``` ```
The literal `All tests passed.` line must be present. CI (.github/workflows/ **Re-mint after changing a seed source.** The reader (`host/chez/reader.ss`), the
tests.yml) runs the same gate on every push/PR. analyzer/IR/backend (`jolt-core/jolt/*.clj`), or the `clojure.core` overlay
(`jolt-core/clojure/core/*.clj`) are baked into the seed — change one and run
`make remint` (iterates `host/chez/bootstrap.ss` to a byte-fixpoint) or `make
selfhost` fails. Runtime-only `host/chez/*.ss` shims do NOT need a re-mint.
`jpm build` output goes STALE silently — `rm -rf build && jpm clean` before **Run the gate with a REAL exit code.** `make test > /tmp/gate.out 2>&1; echo
trusting the binary, or test from source (authoritative). "EXIT: $?"` — the final `OK: all gates passed` line must be present. CI
(`.github/workflows/tests.yml`) runs `make test` on every push/PR.
## Architecture Overview ## Architecture Overview
Clojure on Janet. A shrinking Janet seed (`src/jolt/*.janet`: reader, value Clojure on Chez Scheme — the sole substrate, no Janet. A small Chez runtime
layer, vars/ns, evaluator, the self-hosted pipeline's back end) hosts a (`host/chez/*.ss`: value model, persistent collections, seqs, vars/ns, host
Clojure overlay (`jolt-core/`): the analyzer/IR (`jolt-core/jolt/`) and interop) hosts a portable Clojure overlay (`jolt-core/`): the
`clojure.core` in dependency-ordered tiers (`jolt-core/clojure/core/NN-*.clj`, reader/analyzer/IR/backend (`jolt-core/jolt/`) and `clojure.core` in
loaded in order: 00-syntax, 00-kernel (bootstrap-compiled), 10-seq, 20-coll, dependency-ordered tiers (`jolt-core/clojure/core/NN-*.clj`, loaded in order:
25-sorted, 30-macros, 40-lazy, 50-io). Compile is the default path (analyzer 00-syntax, 00-kernel, 10-seq, 20-coll, 25-sorted, 30-macros, 40-lazy, 50-io).
-> IR -> Janet bytecode, hybrid with interpreter fallback); `JOLT_INTERPRET=1` The stdlib namespaces (`clojure.string`/`set`/`walk`/`edn`/`pprint`/…) are
forces the tree-walking interpreter, `JOLT_INTERPRET_MACROS=1` additionally portable Clojure under `src/jolt/clojure/`.
keeps macro expanders interpreted (the pure oracle). `api/init-cached` serves
a disk-cached ctx image (~5ms vs ~2.4s); the cache key fingerprints sources + `bin/joltc` (`host/chez/cli.ss`) loads the checked-in seed
env knobs — add any NEW ctx-shaping env var to `image-cache-path` in (`host/chez/seed/{prelude,image}.ss`) + the spine and compiles+evals on Chez
api.janet or tests will see stale language behavior. (read → analyze → IR → emit → eval). `host/chez/bootstrap.ss` rebuilds that seed
from source on pure Chez; the build is a self-hosting fixpoint (a rebuild
reproduces the checked-in seed byte-for-byte — `make selfhost`). The correctness
oracle is the JVM-sourced conformance corpus (`test/chez/corpus.edn`,
`test/conformance/`).
Issue tracking and design notes live in beads (`bd prime`, `bd memories`). Issue tracking and design notes live in beads (`bd prime`, `bd memories`).
## Conventions & Patterns ## Conventions & Patterns
Porting seed fns to the overlay (the jolt-tzo shrink ladder) — traps that have
each bitten at least once:
- **Verify leaf-ness first**: grep ALL `src/jolt/*.janet` for the `core-X`
name (defn + core-bindings entry only), and check that tiers loading
EARLIER than the target tier don't call it. Nothing the analyzer/ir use may
move below the kernel tier.
- **Delete the seed defn + binding in the same change.** A leftover stub
breaks direct-linked self-recursion: the overlay fn's recursive call binds
to the STUB's root at compile time (line-seq once truncated after one
element this way).
- **A tier may only use macros from tiers that load before it.** Compile mode - **A tier may only use macros from tiers that load before it.** Compile mode
expands macros at tier LOAD; the interpreter expands lazily — so an expands macros at tier LOAD, so an `if-let` (30-macros) inside a 20-coll fn
if-let (30-macros) inside a 20-coll fn passes every interpreted test and breaks compiled init even though it passes when expanded lazily. Same ordering
breaks compiled init. for expander-called fns (empty?/keys/vals live in 00-syntax).
- **Never read your own wrapper's fields with `get`** in attached-ops values - **Never read your own wrapper's fields with `get`** in attached-ops values
(sorted colls): `get` on the wrapper IS the dispatched lookup and recurses (sorted colls): `get` on the wrapper IS the dispatched lookup and recurses
forever. Use `jolt.host/ref-get`. forever. Use `jolt.host/ref-get`.
- **Map literals with `:jolt/type` as a key** parse as tagged reader forms — - **Map literals with `:jolt/type` as a key** parse as tagged reader forms —
don't tag overlay value maps in source. don't tag overlay value maps in source.
- **Expander-called fns live in 00-syntax** (empty?/keys/vals): expansion
first happens during the kernel-tier compile, before later tiers exist.
Early defns and expanders are interpreted during init and recompiled by the
staged passes (recompile-defns!/recompile-macros!) once the analyzer is
alive.
- **Fix latent bugs to match Clojure** rather than preserving them, with a - **Fix latent bugs to match Clojure** rather than preserving them, with a
regression spec row. Canonical Clojure definitions are preferred verbatim. regression case. Match the JVM (or provide a superset); the JVM-sourced corpus
- **Gate every batch**: conformance x3 modes, suite >= baseline is the contract.
(clojure-test-suite-test.janet — raise the baseline when it rises), full - **Gate every change**: `make test` with a real exit code (self-host fixpoint,
jpm test with a real exit code, bench back-to-back vs main. corpus floor, unit, cli smoke, certify). Re-mint if a seed source changed.

306
README.md
View file

@ -2,266 +2,106 @@
[![tests](https://github.com/jolt-lang/jolt/actions/workflows/tests.yml/badge.svg)](https://github.com/jolt-lang/jolt/actions/workflows/tests.yml) [![tests](https://github.com/jolt-lang/jolt/actions/workflows/tests.yml/badge.svg)](https://github.com/jolt-lang/jolt/actions/workflows/tests.yml)
A Clojure implementation on top of [Janet](https://janet-lang.org). Jolt reads Clojure source and, by default, compiles each form to native Janet bytecode — falling back to a tree-walking interpreter for forms the compiler doesn't handle, so results always match the interpreter. It ships a Clojure-compatible standard library. The goal is a Janet-hosted [SCI](https://github.com/borkdude/sci)-style runtime with a minimal bootstrap. A Clojure implementation on [Chez Scheme](https://cisco.github.io/ChezScheme/).
Jolt reads Clojure source, analyzes it to a host-neutral IR, emits Scheme, and
runs it on Chez. The compiler is self-hosted: it is written in Clojure
(`jolt-core/`) and compiles itself. It ships a Clojure-compatible standard library.
## Requirements
Only [Chez Scheme](https://cisco.github.io/ChezScheme/) (the gate invokes it as
`chez`). The conformance gate additionally uses Clojure on the JVM as an oracle,
but running jolt does not.
## Build ## Build
There is no build step. The bootstrap seed (`host/chez/seed/{prelude,image}.ss`)
is checked in, so a fresh clone runs immediately:
```bash ```bash
git clone https://github.com/jolt-lang/jolt.git git clone --recurse-submodules https://github.com/jolt-lang/jolt.git
cd jolt cd jolt
git submodule update --init # pulls vendor/sci and vendor/clojure-test-suite bin/joltc -e '(+ 1 2)' # => 3
jpm build # builds build/jolt (one binary)
``` ```
Requires `jpm` and a recent Janet (CI-tested against 1.41). See After changing a compiler source — the reader (`host/chez/reader.ss`), the
[docs/building-and-deps.md](docs/building-and-deps.md) for build details, the analyzer/IR/backend (`jolt-core/jolt/*.clj`), or the `clojure.core` overlay
`jpm clean` caveat, how namespaces are resolved (`JOLT_PATH`), and pulling (`jolt-core/clojure/core/*.clj`) — re-mint the seed:
Clojure libraries from a `deps.edn` (resolved by `jolt` itself — `jolt -M:…`,
`jolt run`, `jolt path`). ```bash
make remint # iterates host/chez/bootstrap.ss to a byte-fixpoint
```
## Run ## Run
```bash
bin/joltc -e EXPR # evaluate a Clojure expression and print the result
``` ```
build/jolt # start a REPL
build/jolt file.clj [args] # run a file (binds *command-line-args* and *file*)
build/jolt -e EXPR [args] # evaluate EXPR and print the result
build/jolt -m NS [args] # require NS and call its -main
build/jolt nrepl-server [addr] # start an nREPL server ([host:]port, default 7888)
build/jolt --version # print the version
build/jolt -h | --help # help
```
The REPL accumulates multi-line forms until they balance:
```
user=> (defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
#'user/fib
user=> (map fib (range 10))
(0 1 1 2 3 5 8 13 21 34)
```
Running a file evaluates its top-level forms:
```
$ echo '(println "hello" (* 6 7))' > hello.clj
$ build/jolt hello.clj
hello 42
```
## Use as a library
```janet
(use jolt/api)
(def ctx (init))
(eval-string ctx "(+ 1 2)") # → 3
(eval-string ctx "(map inc [1 2 3])") # → (2 3 4) ; a lazy seq, like Clojure
```
`(init)` returns a context with `clojure.core` loaded. Each context is isolated; use separate contexts for separate environments.
### Evaluation pipeline: interpreted and compiled
Every form passes through one router (`loader/eval-toplevel`) that decides *per
form* whether to tree-walk it or compile it to Janet bytecode. The shipped
runtime **compiles by default**; set `JOLT_INTERPRET=1` to force the interpreter.
**Hybrid, always correct.** The compiler is incomplete by design: a form it can't
compile correctly throws `jolt/uncompilable`, and the router falls back to the
tree-walking interpreter (`eval-form`) for that form. So the result *always*
matches the interpreter — compilation is a transparent speedup, never a semantic
change. Only the compile step is guarded; runtime errors in compiled code
propagate normally (no double-evaluation, no hidden errors).
What compiles: `def`/`defn`, multi-arity / named / variadic fns, `recur` (in
`loop` and directly in `fn`), `let`/`if`/`do`/`try`/`throw`/`quote`, map and
vector literals, and calls. What falls back to the interpreter: context-modifying
and definitional forms (`ns`, `defmacro`, `deftype`, `defprotocol`,
`defmulti`/`defmethod`, `reify`, `require`, `binding`, …), destructuring, regex
literals, and the handful of interpreter-only special forms.
**Live redefinition.** Compiled global references deref through Jolt **var cells**
(Janet early-binds plain symbols, which would freeze redefinition), so redefining
a `def`/`defn` at the REPL is visible to already-compiled callers — Clojure's var
model. Hot numeric primitives (`+ - * < > <= >=`) emit native Janet ops, and
calls compile to direct Janet calls.
```janet
(def ctx (init {:compile? true}))
(eval-string ctx "(defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))")
(eval-string ctx "(fib 30)") ; → 832040, native Janet bytecode
```
For compute-heavy code the compiled path is dramatically faster than tree-walking,
at native Janet speed.
**Validated at parity.** The conformance suite passes 258/258 under *all three*
execution paths — interpreter, compiler, and the self-hosted compiler
(`conformance-test.janet` runs all three in CI) — and the full clojure-test-suite
matches its baseline across ~4.6k assertions — evidence the hybrid path doesn't
diverge.
**AOT.** `aot.janet` marshals a compiled namespace to a Janet bytecode image
(`save-ns`) and loads it back into a fresh context (`load-ns-image`), skipping
parse/analyze/emit/compile on reload. Core fns are referenced by name against the
baked-in runtime; only user bytecode and var cells are serialized.
## Host interop
Jolt exposes CLJS-style host interop through `.` on any Janet table or struct — a field holding a function is called with the receiver as the first argument:
```clojure
(def obj {:greet (fn [self name] (str "Hello " name))})
(. obj greet "Alice") ; → "Hello Alice"
(.-greet obj) ; field access (reader sugar for (. obj :greet))
```
### The `janet` interop bridge
The whole Janet standard library is reachable from Clojure through an explicit
`janet` namespace segment, which marks every crossing into host code (where
Clojure semantics no longer hold):
```clojure
(janet.os/clock) ; → a Janet module fn: os/clock
(janet.string/join ["a" "b"] ",") ; → janet `string/join` (NB: takes a Janet
; tuple, not a Jolt vector — convert first)
(janet/slurp "deps.edn") ; → a Janet root builtin: slurp
(janet/type [1 2]) ; → :table
```
The rule is `janet/<name>` for a Janet root binding and `janet.<module>/<name>`
for a module binding. Because the boundary is explicit, you can tell at the call
site that a form drops into the host — and that values cross the boundary as
their Janet representations (a Jolt vector is a Janet table, etc.), so a Janet
function expecting a tuple needs an explicit conversion. The `jolt.interop`,
`jolt.shell`, and `jolt.http` namespaces are thin Clojure wrappers built on this.
This bridge is what makes networking (and everything else in Janet's stdlib)
available to ordinary Clojure — for example, `jolt.nrepl` (below) is plain
Clojure over `janet.net/*`. It also reaches any **jpm-installed module**:
the first reference to `janet.<module>/<name>` requires the module from the
janet module path and caches its bindings — so after `jpm install spork`,
`janet.spork.http/*` just works. The `jolt.http` client and the Ring adapter
in [examples/ring-app](https://github.com/jolt-lang/examples/tree/main/ring-app)
are built on spork/http this way, and a project can declare the requirement
in `deps.edn` with a `:jpm/module` coordinate (see docs/tools-deps.md).
```clojure
(require '[jolt.interop :as j])
(j/janet-type [1 2]) ; → :tuple
(j/janet-table-keys {:a 1 :b 2}) ; → [:b :a]
```
## nREPL
Jolt ships an [nREPL](https://nrepl.org) server and client (`jolt.nrepl`),
written in Clojure on top of the `janet.net/*` bridge. Start a server from the
CLI — it writes `.nrepl-port` so editors (CIDER, Calva, …) auto-connect:
```bash ```bash
jolt nrepl-server # listen on 127.0.0.1:7888, write .nrepl-port $ bin/joltc -e '(->> (range 10) (filter even?) (map (fn [x] (* x x))) (reduce +))'
jolt nrepl-server 12345 # choose a port 120
jolt nrepl-server 0.0.0.0:12345 # choose host and port (alias: nrepl) $ bin/joltc -e '(/ 1 2)'
1/2
``` ```
In a project with a `deps.edn`, `jolt nrepl-server` auto-resolves it, so the ## Architecture
server starts with the project and its dependencies on the path — connect your
editor, then `(require 'your.app)`. (Equivalently, add `:aliases {:nrepl
{:main-opts ["nrepl-server"]}}` and run `jolt -M:nrepl`.)
Supported ops: `clone`, `describe`, `eval`, `load-file`, `close`, `ls-sessions`, A small Chez runtime (`host/chez/*.ss`: value model, persistent collections, seqs,
`interrupt` (acknowledged; an in-flight eval can't actually be interrupted), and vars/namespaces, host interop) hosts a portable Clojure overlay (`jolt-core/`): the
`eldoc`. `eval` streams `out`, reports the current `ns`, evaluates each form in reader/analyzer/IR/backend (`jolt-core/jolt/`) and `clojure.core` in
the message, and returns an `eval-error` status (the session stays usable) on dependency-ordered tiers (`jolt-core/clojure/core/NN-*.clj`). The stdlib namespaces
failure. One Jolt runtime backs the server and sessions share it, so `def`s (`clojure.string`/`set`/`walk`/`edn`/`pprint`/…) are portable Clojure under
persist across a connection like a normal dev REPL. `src/jolt/clojure/`.
It's also usable as a library — embed a server, or drive another nREPL as a `bin/joltc` loads the checked-in seed and the spine, then compiles and evaluates on
client: Chez (read → analyze → IR → emit → eval). `host/chez/bootstrap.ss` rebuilds that
seed from source on pure Chez; the build is a self-hosting fixpoint (a rebuild
```clojure reproduces the checked-in seed byte-for-byte).
(require '[jolt.nrepl :as nrepl])
(def server (nrepl/start-server! {:port 7888}))
;; ... later ...
(nrepl/stop-server! server)
(def c (nrepl/connect {:port 7888}))
(def session (nrepl/client-clone c))
(nrepl/client-eval c "(+ 1 2)" session) ; → responses incl. {"value" "3"}
(nrepl/client-close c)
```
## Differences from Clojure ## Differences from Clojure
Jolt targets Clojure semantics but runs on Janet, not the JVM. The notable divergences: Jolt targets Clojure semantics but runs on Chez, not the JVM.
- **Host platform.** No JVM and no Java interop — `import`, `gen-class`, `proxy` of Java classes, and `java.*` are unavailable. `instance?` recognizes a small set of built-in types (`clojure.lang.Atom`, `Number`, `String`, …). - **Host platform.** No JVM and no Java interop — `import`, `gen-class`, `proxy` of
- **Numbers.** Janet integers and doubles. `(/ 1 3)` is `0.3333…` and large products lose precision. No ratios or `BigDecimal` (`ratio?` is always false, `bigdec` falls back to a double); `bigint`/`biginteger` use Janet's 64-bit `int/s64`, not arbitrary precision. The reader still accepts Clojure's numeric literal syntaxes — the BigInt/BigDecimal suffixes (`42N`, `1.5M`), ratios (`1/2`), radixed integers (`2r1010`, `16rFF`), and exponents (`1e3`) — but reads them as plain Janet numbers (a ratio becomes its double quotient). The auto-promoting `+'`/`-'`/`*'`/`inc'`/`dec'` are aliases for the plain ops, since Janet numbers don't overflow. `quot`/`rem`/`mod` follow Clojure's sign rules. The symbolic values `##Inf`/`##-Inf`/`##NaN` read, and `infinite?`/`NaN?` work. Janet represents an integer and an integer-valued double identically, so `1` and `1.0` are indistinguishable: `(float?/double? 1.0)` is `false` and `(int? 1.0)` is `true` — `float?`/`double?` are true only for values with a fractional part or `##Inf`/`##NaN`. Java classes, and `java.*` are unavailable. A class token resolves to a name; a
- **Collections.** By default Jolt uses immutable persistent data structures: vectors are 32-way branching tries (structural-sharing persistent vectors with O(log₃₂ n) `conj`/`assoc`/`nth`), lists are persistent singly-linked cons cells (O(1) `conj`/`cons` prepend with structural sharing), and maps/sets are persistent hash structures. Value equality and sequence operations are Clojure-compatible, but hash-map/hash-set iteration order is unspecified and differs from Clojure — use `sorted-map`/`sorted-set` when order matters. small set of host classes is recognized for `instance?`.
- **Mutable build mode.** Jolt can be compiled to use fast Janet-native *mutable* collections instead, via a build-time flag: `JOLT_MUTABLE=1 jpm build` (default `jpm build` is immutable). In mutable mode vectors and lists share one mutable array representation (so `conj` mutates in place and appends, and `vector?`/`list?` no longer distinguish them) — a performance/looseness trade-off. The default immutable build has full Clojure value semantics. - **Numbers.** The full Scheme numeric tower, matching the JVM: exact integers and
- **Concurrency / STM.** No refs, `dosync`, agents, or `send`; `locking` evaluates its body without real locking. Atoms, volatiles, promises, and delays are supported. bignums, exact ratios (`(/ 1 2)` ⇒ `1/2`), and flonum doubles. `=` is
- **Futures.** `future` runs its body on a *real* OS thread (Janet's `ev/thread`), so it can use a second core for CPU-bound work — unlike the cooperatively-scheduled `go` blocks. `deref`/`@` parks until the result is ready (with the optional `(deref f timeout-ms timeout-val)` arity); `future?`, `future-done?`, `realized?`, `future-cancel`, and `future-cancelled?` are supported. Two important divergences from the JVM: (1) **snapshot semantics** — Janet threads have separate heaps, so the body and the state it closes over are *copied* to the worker thread and only the return value is copied back; mutating a captured atom does not propagate to the parent (communicate via the return value). (2) **no thread interruption** — Janet OS threads can't be cancelled mid-run, so `future-cancel` marks the *future* cancelled (deref then throws and the predicates flip) but the underlying computation still runs to completion in the background. As on the JVM, a live future thread keeps the process alive until it finishes (the JVM's non-daemon future pool behaves the same). category-aware (`(= 3 3.0)` ⇒ `false`); `==` is value-equality (`(== 3 3.0)` ⇒
- **core.async.** `clojure.core.async` runs on Janet fibers and channels (`chan`, `go`, `go-loop`, `<!`/`>!`/`<!!`/`>!!`, `close!`, `alts!`, `timeout`, `put!`/`take!`, `buffer`/`dropping-buffer`/`sliding-buffer`, and channel transducers via `(chan n xform)`). Because Janet fibers are stackful coroutines, a `go` block is just its body run in a fiber — no CPS/state-machine rewrite — so `<!`/`>!` work *anywhere*, including inside `try`, nested `fn`s, and loops (positions Clojure's `go` macro forbids). Go blocks are cooperatively scheduled on one OS thread, so parking (`<!`) and blocking (`<!!`) coincide; `thread` runs cooperatively too. Dynamic-var bindings are conveyed into `go` blocks (each go block sees the bindings in effect when it was spawned). `true`). `integer?`/`int?` are exact integers, `float?`/`double?` are flonums,
- **Regex.** Compiled to Janet's PEG engine (Janet has no regex). Supported: capturing groups (`[whole g1 …]`), greedy and lazy quantifiers with backtracking, `(?:…)`, lookahead `(?=…)`/`(?!…)`, alternation, anchors `^ $ \b \B`, character classes, and the `(?i)` flag. Not supported: lookbehind, backreferences (`\1`), named groups (`(?<name>…)`), and Unicode property classes (`\p{Lu}`). `ratio?` is an exact non-integer. No `BigDecimal` (`decimal?` is always false).
- **Arrays.** Java-style arrays map onto Janet's native types: `byte-array` is a Janet buffer (contiguous, C-backed); `object-array`/`int-array`/`double-array`/etc. are Janet arrays. `aget`/`aset`/`alength`/`aclone` work over both. - **Concurrency.** `future`/`promise`/`agent`/`pmap` run on real OS threads over a
- **Transients.** `transient`/`conj!`/`assoc!`/`dissoc!`/`disj!`/`pop!`/`persistent!` are real mutable scratch collections backed by Janet's native arrays and tables (vectors → arrays, maps/sets → tables), so building a collection with them avoids the per-step copying of the persistent path (notably for maps/sets). `persistent!` freezes back to a persistent value. **shared heap**, matching JVM semantics (not isolated-heap snapshots). Atoms use a
- **Not implemented.** JVM reflection, `proxy`, and the `clojure.repl`/`clojure.template` namespaces. per-atom mutex with JVM-style CAS. `clojure.core.async` provides blocking channels
and `go`/`<!`/`>!`/`alts!`/`timeout`.
- **Regex.** Backed by [irregex](https://github.com/ashinn/irregex) (vendored),
PCRE/Java-style patterns.
- **Collections.** Immutable persistent vectors (32-way tries), cons lists, and HAMT
maps/sets. Hash-map/hash-set iteration order is unspecified — use
`sorted-map`/`sorted-set` when order matters. Transients are real mutable scratch
collections.
Supported and Clojure-compatible: chars as a distinct type, lazy/infinite sequences, transducers, destructuring, multimethods with hierarchies, protocols/records (`deftype`/`defrecord`/`reify`/`extend-protocol`), metadata, namespaces, and the reader (`#()`, `#_`, `#?`, tagged literals, `#"…"`). Supported and Clojure-compatible: lazy/infinite sequences, transducers,
destructuring, multimethods with hierarchies, protocols/records
(`deftype`/`defrecord`/`reify`/`extend-protocol`), metadata, namespaces, runtime
`eval`/`load-string`/`defmacro`, and the reader (`#()`, `#_`, `#?`, tagged literals,
`#"…"`).
## Test ## Test
``` ```bash
jpm test # full suite (recurses test/) make test # the full gate (no Janet)
janet test/spec/sequences-spec.janet # a single spec make corpus # conformance corpus vs the JVM-sourced spec
janet test/integration/conformance-test.janet make unit # host-specific unit cases
make selfhost # bootstrap fixpoint (rebuild == checked-in seed)
make smoke # bin/joltc CLI smoke
make certify # JVM oracle (skips if clojure is absent)
``` ```
Tests are organized in three layers: The conformance corpus (`test/chez/corpus.edn`) is a host-neutral language spec
whose expected values are sourced from reference JVM Clojure. See
- **`test/spec/`** — the contract. Black-box, behavior-defining tables (one file [test/conformance/SPEC.md](test/conformance/SPEC.md).
per public API area) that collectively pin down Jolt's defined behavior. This
is the authoritative description of what Jolt promises.
- **`test/integration/`** — cross-cutting and regression batteries: the Clojure
conformance suite (run in all three execution modes), SCI bootstrap/runtime
loading, jank conformance, the cross-dialect
[clojure-test-suite](https://github.com/jank-lang/clojure-test-suite) (a git
submodule at `vendor/clojure-test-suite`, run via a minimal `clojure.test` shim
and baseline-guarded), compile-mode tests, the library API, and a broad
systematic-coverage net.
- **`test/unit/`** — white-box tests for individual components (reader,
evaluator, types, persistent collections, regex, compiler).
`test/support/harness.janet` provides the shared `defspec` table runner (cases
are `["label" expected actual]`, compared with Jolt's own `=`) plus
`expect=`/`expect-throws` for unit tests.
The syntactic half of the contract — the surface syntax the reader accepts — is
specified as an EBNF grammar in [`docs/grammar.ebnf`](docs/grammar.ebnf), with
Jolt-vs-Clojure deviations noted inline. `test/spec/reader-syntax-spec.janet`
exercises it.
### clojure-test-suite conformance
The [clojure-test-suite](https://github.com/jank-lang/clojure-test-suite) battery
(vendored as a git submodule) runs ~3980 assertions green. Jolt validates its
arguments like Clojure — arithmetic on non-numbers, comparisons against `nil`,
out-of-range indices, malformed `conj!`/`assoc!`/`merge`, non-seqable
`first`/`seq`/`vec`, and lazy transformers (`map`/`filter`/…) realized over a
non-seqable all throw. The lazy seq fns return seqs (not vectors), so
`seq?`/`vector?`/`sequential?` of their results match Clojure. The assertions
that remain failing are accounted for by the platform/design differences above,
not by missing behavior:
- **No bignum/ratio/BigDecimal** — `bigint`/`numerator`/`denominator`/`bigdec`,
the `big-int?`/auto-promotion checks, and the `2N`/`1/2`/`1.0M` literals read
but don't carry those exact types.
- **Integer/float identity** — Janet represents `1` and `1.0` identically, so
`quot`/`rem`/`mod`'s `double?`/`int?` result-type assertions and many
`float?`/`double?` cases can't distinguish them (`(str 0.0)` is `"0"`).
- **64-bit integers / Unicode** — `bit-and` etc. on full-width 64-bit constants
lose precision (doubles), and `subs`/`count` work on bytes, not code points.
## License ## License

75
test/chez/README.md
View file

@ -1,60 +1,41 @@
# Chez test harness # Chez test harness
The correctness gate for jolt on Chez (epic jolt-cf1q). Correctness is judged The correctness gate for jolt. Pure Chez (+ Clojure for the JVM oracle), no Janet.
against the JVM-sourced conformance spec — no Janet decides pass/fail. The spec Correctness is judged against the JVM-sourced conformance spec; the spec itself
itself lives in `test/conformance/` (see its `SPEC.md`); this directory holds the lives in `test/conformance/` (see its `SPEC.md`). Run the whole gate with `make
Chez runners and unit tests. test` from the repo root.
## The spec corpus ## The spec corpus
- `corpus.edn` — the contract: ~2920 rows `{:suite :label :expected :actual}`, `corpus.edn` is the contract: ~2920 rows `{:suite :label :expected :actual}`, with
with `:expected` sourced from reference JVM Clojure. Valid as both EDN and Janet `:expected` sourced from reference JVM Clojure by `test/conformance/regen-corpus.clj`.
data. Generated by `test/conformance/regen-corpus.clj` (the answers) over the It is frozen (the canonical source) — add or change cases here, then re-source the
case list from `extract-corpus.janet` (the `:actual` strings, pulled from answers with `regen-corpus.clj` and re-certify with `test/conformance/certify.clj`.
`test/spec/*.janet` and `conformance-test.janet`).
- `extract-corpus.janet` — re-derives the case list when spec rows change. Writes
only when `JOLT_EXTRACT_CORPUS_OUT` is set, so a gate run never clobbers the
JVM-sourced `corpus.edn`. After re-deriving, re-source the answers with
`regen-corpus.clj`.
## The standing gate ## The gate runners (`host/chez/`)
`run-corpus-zero-janet.janet` runs every corpus case through the zero-Janet - `run-corpus.ss` — runs every corpus case through the spine (read → analyze → IR →
spine: read → analyze → IR → emit → eval, all on Chez (the analyzer is emit → eval, all on Chez), comparing each result by value-equality against the JVM
`jolt.analyzer` cross-compiled to Scheme over `host-contract.ss`). Each result is `:expected`. A `known-fail` allowlist covers cases jolt can't match because Chez has
compared by value-equality against the JVM `:expected`. A `known-fail` allowlist no JVM host (Java classes, arrays, `BigDecimal`, opaque host-object printers, …);
covers cases jolt can't match because Chez has no JVM host (Java classes, arrays, the gate fails only on a NEW divergence or if the pass count drops below the floor.
`BigDecimal`, opaque host-object printers, …); the gate fails only on a NEW
divergence or if the pass count drops below the floor.
JOLT_CHEZ_ZEROJANET_CORPUS=1 janet test/chez/run-corpus-zero-janet.janet chez --script host/chez/run-corpus.ss
JOLT_CORPUS_LIMIT=200 … # every-Nth stride, fast iteration JOLT_CORPUS_LIMIT=200 … # every-Nth stride, fast iteration
JOLT_CHEZ_ZJ_FLOOR=N … # override the floor (default 2678)
Floor: 2678 (`JOLT_CHEZ_ZJ_FLOOR` overrides). Raise it as host gaps close. - `run-unit.ss` — host-specific unit cases (`test/chez/unit.edn`) that aren't in the
JVM-portable corpus: dot-forms, java statics, io, reader, walk, vars/namespaces,
refs. Each `:expr` is evaluated in-process and its printed value compared to a baked
`:expected` (`:throws` asserts a raise).
## Unit tests - `selfcheck.sh` — self-host fixpoint: `bootstrap.ss` rebuild byte-equals the
checked-in seed (`host/chez/seed/`).
- `smoke.sh` — real `bin/joltc -e` CLI smoke.
`_*.janet` drive `bin/joltc` (the pure-Chez CLI; `JOLT_BIN` overrides) one ## Other Chez tests
subprocess per case and compare the last stdout line against a baked expected
value. `:throws` asserts a non-zero exit. Each file targets one area —
`_type`/`_class`/`_seqpred` (taxonomy + predicates), `_str`/`_strns`/`_reader`
(strings + reader), `_io`/`_ioreader`/`_insttime`/`_javastatic` (host interop),
`_dynbind`/`_var_meta`/`_ns` (vars + namespaces), `_atomwatch`/`_walk`/`_dotform`
/`_stdlib` (refs, walk, dot-forms, stdlib). Run one with
`janet test/chez/_type.janet`.
## Self-host gates - `values-test.ss` — the value model (nil/truthiness/collections). `make values`.
- `bench-chez.ss` — compute bench through the pipeline (opt-in; not in the gate).
- `spine-test.janet` — the zero-Janet compile/eval spine end to end. All runners assume `chez` on PATH.
- `cli-test.janet` — `bin/joltc -e` behavior.
- `bootstrap-test.janet` — `host/chez/bootstrap.ss` rebuilds the prelude + image
from source on Chez and matches the checked-in seed (`host/chez/seed/`).
- `fixpoint-test.janet` — the on-Chez compiler reproduces itself (stage2 == stage3,
prelude pstage3 == pstage4).
## Bench
`bench-pipeline.janet` (opt-in via `JOLT_CHEZ_BENCH=1`) times fib + mandelbrot
through the real pipeline against the spike ceiling. Skipped in the gate.
All Chez runners skip cleanly when `chez` is not on PATH.

20
test/conformance/SPEC.md
View file

@ -17,10 +17,9 @@ read one data file, run each case, compare, report.
| `test/conformance/certify.clj` | Certifies `:expected` against reference **JVM Clojure**; gates on new/stale divergences; emits the profile. | — | | `test/conformance/certify.clj` | Certifies `:expected` against reference **JVM Clojure**; gates on new/stale divergences; emits the profile. | — |
`corpus.edn` is **JVM-sourced**: `regen-corpus.clj` evaluates each case's `:actual` `corpus.edn` is **JVM-sourced**: `regen-corpus.clj` evaluates each case's `:actual`
on reference JVM Clojure and writes the JVM value as `:expected`. The case *list* on reference JVM Clojure and writes the JVM value as `:expected`. **`corpus.edn` is
(the `:actual` strings) comes from `test/spec/*-spec.janet` via `extract-corpus.janet`, the canonical, frozen contract**: it is what every runtime consumes, what
but the *answers* are the JVM's. **`corpus.edn` is the canonical contract**: it is `certify.clj` certifies, and where new cases are authored directly.
what every runtime consumes and what `certify.clj` certifies.
## Row schema ## Row schema
@ -101,17 +100,16 @@ implements. Current profile (≈2670 portable, ≈249 non-portable):
3. Run it. Your **conformance level** is the set of feature families with no 3. Run it. Your **conformance level** is the set of feature families with no
failures. Portable-only is the floor; each feature you implement raises it. failures. Portable-only is the floor; each feature you implement raises it.
The two reference harnesses already do exactly this on Chez: The reference harness does exactly this on Chez: `host/chez/run-corpus.ss` (the
`test/chez/run-corpus-prelude.janet` (Janet analyzer → Chez runtime) and analyzer runs on Chez → Chez runtime), with a regression floor. Run it via `make
`test/chez/run-corpus-zero-janet.janet` (Chez analyzer → Chez runtime), both with a corpus`.
regression floor.
## Maintaining the spec ## Maintaining the spec
- **Add/change cases**: edit `test/spec/*-spec.janet` or `conformance-test.janet`, - **Add/change cases**: edit `test/chez/corpus.edn` directly, then re-source the
then `janet test/chez/extract-corpus.janet` to regenerate `corpus.edn`. answers with `regen-corpus.clj`.
- **Re-certify**: `clojure -M test/conformance/certify.clj`. A new divergence is - **Re-certify**: `clojure -M test/conformance/certify.clj`. A new divergence is
either a real bug (file it, mark the allowlist entry `:bug` + `:bead`) or a either a real bug (file it, mark the allowlist entry `:bug` + `:bead`) or a
deliberate delta (classify it in `known-divergences.edn`). deliberate delta (classify it in `known-divergences.edn`).
- **Refresh the profile**: re-run with `--profile test/conformance/profile.edn`. - **Refresh the profile**: re-run with `--profile test/conformance/profile.edn`.
- **Re-floor the runtime gates** when parity rises (`run-corpus-*.janet`). - **Re-floor the runtime gate** when parity rises (`host/chez/run-corpus.ss`).