prefer-method/remove-method/remove-all-methods/get-method/methods become
overlay macros over ctx-capturing *-setup fns (a multimethod's method table
lives on its VAR, so the name passes quoted — the defmulti/defmethod shape).
instance? likewise (class names don't evaluate to values); satisfies? is a
plain ctx-capturing fn (evaluated args). locking and defonce become overlay
macros — locking now also evaluates the monitor expr (the old arm skipped it
and any body form past the first); defonce keeps the existing-root check.
read-string and macroexpand-1 are ctx-capturing fns.
Removed all eleven from the evaluator special arms + special-symbol?,
host_iface special-names, and compiler uncompilable-heads. evaluator-test's
locking/instance? cases use init now (overlay macros need the full env).
Surfaced pre-existing (filed): multimethod dispatch records prefer-method
preferences on the var but never consults them in ambiguous isa dispatch.
Gate: conformance 296x3 (+11 tier-6c cases), fallback-zero 73/3, fixpoint,
self-host, sci, staged, suite 4049>=4034, all specs+unit.
create-ns/remove-ns/find-ns/all-ns/the-ns/ns-interns/ns-aliases/ns-imports/
ns-resolve/resolve/refer become ctx-capturing clojure.core fns
(install-stateful-fns!) with evaluated-arg Clojure semantics, replacing
interpreter special arms that were loose: the-ns/ns-interns/ns-aliases/
ns-imports ignored their argument (always current ns), create-ns/remove-ns/
ns-resolve took theirs unevaluated. The optional-arg forms still default to
the current ns, preserving the prior 0-arg behavior. refer previously had a
special-names entry but NO interpreter arm — it errored everywhere; it now
refers the named ns's public vars (use-impl), :only/:exclude filters not yet
honored. ns-resolve does its lookup directly: types/ns-resolve keys ns-find
with the symbol struct instead of its name string and never finds anything.
Removed all of them from the evaluator arms, host_iface special-names, and
compiler uncompilable-heads (use/ns/require/in-ns stay punted in the bootstrap
compiler — it builds analyzer.clj, whose ns forms need the interpreter).
Gate: conformance 285x3 (+8 ns-fn cases), fallback-zero 62/3, fixpoint,
self-host, sci, staged, suite 4049>=4034 (+2 passes), all specs+unit.
var-get/var-set/var?/alter-var-root/alter-meta!/reset-meta! already had plain
core-bindings wrappers — the interpreter special arms shadowed them (and the
arm's alter-var-root silently dropped rest args, which the core wrapper
handles). find-var/intern need the ctx, so they join install-stateful-fns! as
ctx-capturing clojure.core fns; the old core-intern binding was a do-nothing
stub. Removed the eight from the evaluator special arms + special-symbol?,
host_iface special-names, and compiler uncompilable-heads.
Gate: conformance 277x3 (+6 var-fn cases incl. alter-var-root rest args),
fallback-zero 51/3, fixpoint, self-host, sci, staged, suite 4047>=4034 (one
new pass), all specs+unit.
deftype with inline protocol methods failed in compile mode with "Unable to
resolve symbol: jolt.analyzer/extend-type". deftype's expander template emits
(extend-type ...), but extend-type is defined AFTER deftype in 30-macros — so
when the expander compiles at defmacro time, syntax-quote lowering can't
resolve it and falls through to current-ns qualification. The analyzer runs
interpreted in jolt.analyzer, so ctx-current-ns mid-compile is jolt.analyzer,
baking the wrong qualification into the compiled template.
Same bug class jolt-265 fixed for resolve-var: read :compile-ns (the namespace
being compiled) when set, falling back to ctx-current-ns. Interpret and
self-host modes never hit it (interpreted expanders resolve at use time, when
extend-type exists) — which is also why no gate test caught it: nothing
EVALUATED a deftype-with-inline-methods in compile mode. Two 3-mode
conformance cases added.
Gate: conformance 271x3, fallback-zero, fixpoint, self-host, sci, staged,
suite 4046>=4034, all specs+unit.
The gate spent almost all its time rebuilding identical contexts: init is
~50 ms interpreted / ~900 ms compiled (tier loading, analyzer build, macro
recompilation), and both the conformance harness and the spec harness built a
fresh ctx PER CASE — 269 cases x 3 modes for conformance alone (~285 s), and
~1500 spec cases (~90 s). The suite battery additionally ran its 234 worker
subprocesses sequentially (~100 s incl. 5 x 6 s timeout files).
- api: snapshot/fork — marshal a fully-built ctx once (reverse-lookup dicts
from root-env, built at module load before any ctx exists), unmarshal cheap
fully-isolated deep copies (~2 ms). A fork shares nothing mutable with its
siblings, so per-case isolation is preserved exactly.
- conformance: one init per mode + fork per case (self-host pre-builds the
analyzer before snapshotting). 285 s -> 5 s, same 269x3 results.
- spec harness: jeval/run-spec/expect-throws fork from one lazy module-level
snapshot. Spec sweep ~90 s -> 9 s, all pass.
- clojure-test-suite: run the per-file worker subprocesses through a
token-channel worker pool (default 4, JOLT_SUITE_WORKERS to override,
capped at 8). 17.5 s with counts identical to the sequential run
(4046/520/116, 5 timeouts).
Full gate wall-clock: ~8 min -> 43 s, everything green (conformance 269x3,
fallback-zero, fixpoint, self-host, sci, staged, suite >= 4034/67, all
specs+unit).
* core: Stage 2 Task 2 tier 4b — compile ns (+ use/import/refer-clojure)
ns becomes a macro (00-syntax) expanding to (do (in-ns 'name) (require …)
(use …) (import …) (refer-clojure …)) — matching Clojure, where require is
a fn and ns expands to require calls. use/import/refer-clojure join the
namespace ops as ctx-capturing clojure.core fns (use-impl/import-impl/
refer-clojure-impl, interned by install-stateful-fns!). Removed the ns
interpreter special arm; dropped ns/use/import from host_iface special-names
and loader stateful-head?.
Placement matters: ns lives in the FIRST overlay tier (00-syntax), because
the self-hosted analyzer build is triggered while 10-seq loads and processes
jolt.analyzer's own (ns …) form — ns must exist by then. Its body resolves
fn/map/reduce/cond at expansion time, by which point all of 00-syntax has
loaded. The bootstrap compiler (compiler.janet) still PUNTS ns/in-ns/require/
use/import (it compiles analyzer.clj/ir.clj, whose ns forms must fall back to
the interpreter that expands the macro) — only the self-hosted analyzer
compiles them.
use-impl fixes a latent bug: it now refers the used ns's vars into the
CURRENT ns (the old special interned a ns into itself, a no-op).
ns/use/import now compile + interpret as plain invokes. fallback-zero: ns off
must-punt, onto must-compile.
Gate green: conformance 267x3, fallback-zero 38/4, bootstrap-fixpoint
stage1==2==3, self-host, staged-bootstrap, sci-bootstrap, clojure-test-suite
>=4034/67, namespace, deps-loader, cli, aot, embedded-stdlib, uberscript,
features 78/78, all unit + spec.
* test/core: cover ns form + :use; fix use-impl spec parsing
Checking test coverage for the namespace tiers surfaced a real bug nothing
exercised: use-impl checked (array? s) to find a spec's ns symbol, but a
vector spec like [src.x] is a pvec/tuple, not a Janet array — so (:use ...)
and standalone (use ...) errored. Fixed to coerce (pvec->array) then take the
head when the spec is indexed (matching require-impl).
Added coverage that was missing:
- conformance (x3 modes): the (ns …) form itself + :use refers.
- namespaces-spec: (ns …) form, :use, standalone use.
The :use/(use …) path had ZERO tests before, which is why the bug slipped
through earlier tiers.
EBNF needs no change — it's a reader/syntax grammar with no special-forms
enumeration; ns/require/protocol syntax is unchanged (the destructuring note
was already updated in jolt-f79).
Gate green: conformance 269x3, fallback-zero 38/4, self-host, sci-bootstrap,
bootstrap-fixpoint, staged-bootstrap, clojure-test-suite >=4034/67, namespace,
all unit + spec (namespaces 27/27).
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
* compiler: self-hosted analyzer compiles set literals (#{…})
Stage 1 Task 1. analyzer.clj punted set literals to the interpreter
((form-set? form) (uncompilable "set literal")); now it builds the set-node IR
(already defined in ir.clj) from (form-set-items form), and backend.janet emits
(make-phs e1 e2 …) — each element evaluated then the persistent set built,
mirroring compiler.janet's emit-set-expr and the interpreter's :jolt/set path.
Closes a self-hosted-analyzer vs bootstrap-compiler parity gap: #{…} no longer
forces interpreter fallback on the compile path.
Gate: conformance 262x3 (+4 set-literal cases incl computed elements / empty /
in-let), fixpoint, self-host, sci, suite 3981/66, specs+unit green; core-bench
neutral (A/B). set?/disj-as-fns remain deliberately interpreted (in-sync across
all three lists) — adjudicated in Task 2.
* test: fallback-zero harness — assert non-stateful forms compile (not interpret)
Stage 1 Task 3. self-host-test checks results but not which path ran. This runs
the portable analyzer (backend/analyze-form) on a corpus of non-stateful forms
and asserts NONE raise :jolt/uncompilable — i.e. the self-hosted analyzer
compiled them, not the interpreter fallback. Inverse sanity list confirms a few
intentional-interpret forms (ns/defmacro/require/set?/letfn) still punt, so the
harness can't pass by compiling everything.
29 must-compile (incl set literals from Task 1) + 5 must-punt, 0 failures. As
Stage 1 parity grows, forms move from the punt list into must-compile; when the
fallback set equals the frozen intentional stateful set, the bootstrap is
retireable.
* core: migrate 7 lazy seq fns from the Janet seed to the Clojure overlay (40-lazy)
Finishes a port the prior team started and reverted (bb4a3e0): the 40-lazy.clj
tier moved lazy seq fns Janet→Clojure but regressed the suite to 849 because
lazy-seq's expansion leaked as data in compile mode — that was jolt-r81, since
root-fixed (lazy-seq/lazy-cat moved to 00-syntax). With the wall gone, the port
works. This shrinks the Janet seed toward the north star (self-hosted
clojure-in-clojure on a minimal host bootstrap).
Moved to core/40-lazy.clj (wired as a loaded tier after 30-macros):
distinct keep keep-indexed map-indexed cycle repeat iterate
40-lazy.clj completed to full parity: distinct gains its transducer arity;
keep/keep-indexed/map-indexed already had both arities.
Removed from the Janet seed (core.janet): the 7 core-* fns + their core-bindings
entries, the now-dead td-keep/td-map-indexed transducer helpers (the CLJ versions
carry their own), and the already-dead core-partition-by/core-xml-seq (shadowed by
10-seq/20-coll). Net: core.janet −131 lines.
Deferred (kept in Janet, separate follow-ups): partition-all (a CLJ port via
take/drop realizes a non-minimal element count, tripping the §6.3 laziness
counters + a suite file) and repeatedly (canonical CLJ doesn't validate args, so
the repeatedly.cljc throw cases regress). Both need behavior-matching first.
Gate: conformance 262x3, lazy-infinite 44/44, clojure-test-suite 4004/66 (UP from
3981 — the CLJ versions add coverage, e.g. distinct value-equality), fixpoint,
self-host, sci 422/0, fallback-zero, specs+unit, core-bench all green.
* test: raise clojure-test-suite baseline 3981 -> 4004 (lazy-fn migration coverage)
* core: migrate partition-all to the Clojure overlay (minimal realization)
Resolves the deferred partition-all port (jolt-yo3). The earlier CLJ attempt via
lazy take/drop over-realized vs the Janet pstep, tripping the §6.3 laziness
counter. The collection arities now realize EXACTLY n per chunk with a first/rest
loop and continue from the advanced cursor (no re-drop), so (take 3 (partition-all
2 (map counting (range)))) realizes exactly 6 — matching minimal realization in
both interpret and compile modes. Keeps transducer + [n coll] + [n step coll]
arities. letfn-bound recursion sidesteps the compile-mode multi-arity closure bug
(jolt-zxw), like keep-indexed/map-indexed.
Removed from the Janet seed: core-partition-all + its binding + the now-dead
td-partition-all helper (the CLJ version carries its own transducer arity).
Gate: conformance 262x3, lazy-infinite 44/44 (incl the §6.3 partition-all
counter), clojure-test-suite 4004/66, fixpoint, self-host, specs+unit green.
* core: migrate repeatedly to Clojure + fix char-not-callable / take count validation
Resolves the deferred repeatedly port (jolt-8qx). The blockers were two jolt
leniencies vs Clojure, now fixed (and correct beyond repeatedly):
- A char (a :jolt/type-tagged struct) fell into the struct-as-map branch of both
jolt-call (compile path) and the interpreter's apply dispatch, so (\a) returned
nil instead of throwing. Now only an UNtagged struct (a map literal) — or a
record — is callable as a key lookup; tagged structs fall through to "Cannot
call … as a function". Symbols are still handled (keyword-style get).
- core-take didn't validate its count, letting Janet's >= silently compare an int
to a char/string. It now rejects a non-number n like Clojure.
With those, the canonical CLJ repeatedly matches: (first (repeatedly non-fn)) and
(repeatedly non-number f) throw. Moved repeatedly to core/40-lazy.clj; removed
core-repeatedly + its binding from the seed.
These correctness fixes help broadly: repeatedly.cljc goes clean (19/10 -> 29/0),
and the suite rises 4004 -> 4034 pass / 66 -> 67 clean. Baseline raised.
Gate: conformance 262x3, lazy-infinite 44/44, clojure-test-suite 4034/67,
fixpoint, self-host, sci, fallback-zero, specs+unit green.
* test: document the 4004 -> 4034 baseline raise (partition-all/repeatedly + char/take fixes)
* docs: partition-all letfn is for minimal realization, not jolt-zxw
jolt-zxw (multi-arity arity-param mis-capture in a nested lazy-seq under :compile?)
is no longer reproducible: an arity-direct partition-all now compiles correctly in
the overlay. The original failure was an artifact of the CLJ multi-arity version
coexisting with the Janet core-partition-all ([n & rest]) during migration — the
shadowing confused multi-arity dispatch; removing the Janet version resolved it.
The letfn in partition-all stays purely for minimal realization (jolt-yo3).
* compiler: compile set?/disj as plain fns (close the last Stage-1 fallback gap)
Stage 1 jolt-g3h. set? and disj were special-cased in all three "can't compile"
lists (host_iface special-names, compiler.janet uncompilable-heads, evaluator
special-symbol? + handlers) — but they're pure value-production with callable
core vars (core-set?/core-disj), and those vars are byte-for-byte equivalent to
the evaluator handlers. Removed them from all three lists + dropped the now-dead
evaluator handler arms, so they're ordinary clojure.core fns everywhere: the
analyzer compiles (set? x)/(disj s x) as normal var calls instead of punting to
the interpreter.
Verified identical results in default AND JOLT_MUTABLE builds (no representation
sensitivity — sets are phs in both, unlike vector?/list? which collapse).
With this, the self-hosted analyzer's compile-path fallback set equals the frozen
intentional stateful set (Task 2) — it's now a strict superset of the bootstrap
compiler's compilable surface, so the Janet bootstrap is retireable (Stage 2).
fallback-zero: set?/disj moved to must-compile (31 now), set! into must-punt.
Gate: conformance 267x3 (+5 set?/disj cases), lazy-infinite 44/44, suite 4034/67,
fixpoint, self-host, sci, specs+unit green.
adds self-hosted compiler is functionally:
- The default compile path is the portable pipeline using jolt.analyzer (Clojure) → host-neutral IR → backend.janet.
- The analyzer is itself Clojure, compiled by jolt for true self-hosting.
- bootstrap-fixpoint passes (stage1 == stage2 == stage3): rebuilding the compiler on its own output.
- clojure.core is now self-hosted in the overlay.
- Stateful forms (defmacro/ns/deftype/defmulti/require/in-ns) are interpreted by design.
- A map entry is a 2-element tuple (Jolt produces tuples only from map iteration;
vector literals are pvecs, lists are arrays). key/val/map-entry? now accept a
2-tuple and reject a plain vector, matching Clojure's MapEntry-vs-vector
distinction — no metadata needed, the representations already differ.
- min-key/max-key reproduce Clojure's NaN-aware folding (2-arg strict </>, then
<=/>=) and require numeric keys (NaN allowed, strings throw).
- subvec coerces float/NaN indices like (int ...) (truncate, NaN->0) then
bounds-checks, instead of throwing on non-integers.
min_key 35/14 -> 49/0 (clean); key/val recover the 2-vector cases; subvec floats
fixed. clojure-test-suite pass 3898->3921. Updated conformance-test (key/val now
needs a real entry). spec: map/map-entry-&-key-ordering (14). jpm test green.
Reorganize the flat 49-file test/ into three layers (jpm test recurses, so all
are still discovered):
- test/unit/ white-box component tests (reader, evaluator, types,
persistent-map, lazy-seq, macro, interop, compiler)
- test/integration/ cross-cutting + regression batteries (conformance, jank,
sci-bootstrap/runtime, features, systematic-coverage, api,
core, namespaces, ported clojure suites) and
.../ports/ ported clojure/cljs test batches pending consolidation
- test/spec/ the behavioral contract (built out in following commits)
- test/support/harness.janet shared defspec table runner (cases compared via
Jolt's own =, with a :throws sentinel) + expect= helpers
Files moved with git mv (history preserved) and import paths fixed for depth.
jpm test green. README Test section updated.
Next: build out test/spec/ to cover the public API area-by-area, mining the
integration batteries and filling gaps.
