deftype becomes a macro (30-macros) over make-deftype-ctor (a ctx-capturing
clojure.core fn that bakes the ns-qualified type tag at def time) plus
extend-type for any inline protocol methods — so it compiles as a plain (do …).
Mirrors defrecord's existing field-let/protocol-grouping pattern.
- make-deftype-ctor-impl (evaluator) builds the ctor; interned as a closure.
- removed the deftype special-symbol? entry + eval-list arm; dropped deftype/
defrecord from host_iface special-names + loader stateful-head?.
- defrecord no longer redefines ->name via (Name. …) interop (frozen) — deftype
already provides ->name, so defrecord compiles too (map->name builds via it).
- field-kws spliced into a vector LITERAL ([~@…]) so the analyzer sees a vector
form, not a runtime pvec; type name + fields are unwrapped of ^meta (the reader
yields (with-meta sym m) forms, e.g. sci's (deftype ^{:doc …} Var …)).
With tier 5a, all of deftype/defrecord/defmulti/defmethod compile. The loader's
interpret-only set is now just the frozen host-coupled forms: defmacro/set!/./
new/eval.
Tests: evaluator-test deftype case uses init (deftype is an overlay macro now);
fallback-zero moves deftype off must-punt, adds deftype/defrecord/defmulti/
defmethod to must-compile (43/3).
Gate green (full jpm build + jpm test): conformance 269x3, fallback-zero 43/3,
bootstrap-fixpoint stage1==2==3, self-host, staged-bootstrap, sci-bootstrap
422/0, clojure-test-suite >=4034/67, all unit + spec.
* core: Stage 2 Task 2 tier 4b — compile ns (+ use/import/refer-clojure)
ns becomes a macro (00-syntax) expanding to (do (in-ns 'name) (require …)
(use …) (import …) (refer-clojure …)) — matching Clojure, where require is
a fn and ns expands to require calls. use/import/refer-clojure join the
namespace ops as ctx-capturing clojure.core fns (use-impl/import-impl/
refer-clojure-impl, interned by install-stateful-fns!). Removed the ns
interpreter special arm; dropped ns/use/import from host_iface special-names
and loader stateful-head?.
Placement matters: ns lives in the FIRST overlay tier (00-syntax), because
the self-hosted analyzer build is triggered while 10-seq loads and processes
jolt.analyzer's own (ns …) form — ns must exist by then. Its body resolves
fn/map/reduce/cond at expansion time, by which point all of 00-syntax has
loaded. The bootstrap compiler (compiler.janet) still PUNTS ns/in-ns/require/
use/import (it compiles analyzer.clj/ir.clj, whose ns forms must fall back to
the interpreter that expands the macro) — only the self-hosted analyzer
compiles them.
use-impl fixes a latent bug: it now refers the used ns's vars into the
CURRENT ns (the old special interned a ns into itself, a no-op).
ns/use/import now compile + interpret as plain invokes. fallback-zero: ns off
must-punt, onto must-compile.
Gate green: conformance 267x3, fallback-zero 38/4, bootstrap-fixpoint
stage1==2==3, self-host, staged-bootstrap, sci-bootstrap, clojure-test-suite
>=4034/67, namespace, deps-loader, cli, aot, embedded-stdlib, uberscript,
features 78/78, all unit + spec.
* test/core: cover ns form + :use; fix use-impl spec parsing
Checking test coverage for the namespace tiers surfaced a real bug nothing
exercised: use-impl checked (array? s) to find a spec's ns symbol, but a
vector spec like [src.x] is a pvec/tuple, not a Janet array — so (:use ...)
and standalone (use ...) errored. Fixed to coerce (pvec->array) then take the
head when the spec is indexed (matching require-impl).
Added coverage that was missing:
- conformance (x3 modes): the (ns …) form itself + :use refers.
- namespaces-spec: (ns …) form, :use, standalone use.
The :use/(use …) path had ZERO tests before, which is why the bug slipped
through earlier tiers.
EBNF needs no change — it's a reader/syntax grammar with no special-forms
enumeration; ns/require/protocol syntax is unchanged (the destructuring note
was already updated in jolt-f79).
Gate green: conformance 269x3, fallback-zero 38/4, self-host, sci-bootstrap,
bootstrap-fixpoint, staged-bootstrap, clojure-test-suite >=4034/67, namespace,
all unit + spec (namespaces 27/27).
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
require/in-ns take quoted (already-evaluated) args, so they become ordinary
ctx-capturing clojure.core fns (install-stateful-fns!):
- require-impl (varargs over specs -> eval-require) / in-ns-impl, extracted
from the eval-list special arms.
- removed their eval-list match arms (they fall through to the function-call
default, resolving the interned closures) and dropped them from
host_iface special-names + loader stateful-head? + compiler
uncompilable-heads.
require+alias+refer and in-ns now compile + interpret as plain invokes.
Tests: namespace-test uses bare make-ctx, so it now installs the stateful
fns via a fresh-ctx helper (api/init does this automatically). fallback-zero
moves require off must-punt and adds require/in-ns/defprotocol/extend-type/
reify/(var map) to must-compile (37 now); deftype takes require's must-punt
slot.
Gate green: conformance 267x3, fallback-zero 37/5, bootstrap-fixpoint
stage1==2==3, self-host, staged-bootstrap, clojure-test-suite >=4034/67,
features 78/78, deps-loader, all unit + spec (namespaces 24/24).
Co-authored-by: Yogthos <yogthos@gmail.com>
* compiler: self-hosted analyzer compiles set literals (#{…})
Stage 1 Task 1. analyzer.clj punted set literals to the interpreter
((form-set? form) (uncompilable "set literal")); now it builds the set-node IR
(already defined in ir.clj) from (form-set-items form), and backend.janet emits
(make-phs e1 e2 …) — each element evaluated then the persistent set built,
mirroring compiler.janet's emit-set-expr and the interpreter's :jolt/set path.
Closes a self-hosted-analyzer vs bootstrap-compiler parity gap: #{…} no longer
forces interpreter fallback on the compile path.
Gate: conformance 262x3 (+4 set-literal cases incl computed elements / empty /
in-let), fixpoint, self-host, sci, suite 3981/66, specs+unit green; core-bench
neutral (A/B). set?/disj-as-fns remain deliberately interpreted (in-sync across
all three lists) — adjudicated in Task 2.
* test: fallback-zero harness — assert non-stateful forms compile (not interpret)
Stage 1 Task 3. self-host-test checks results but not which path ran. This runs
the portable analyzer (backend/analyze-form) on a corpus of non-stateful forms
and asserts NONE raise :jolt/uncompilable — i.e. the self-hosted analyzer
compiled them, not the interpreter fallback. Inverse sanity list confirms a few
intentional-interpret forms (ns/defmacro/require/set?/letfn) still punt, so the
harness can't pass by compiling everything.
29 must-compile (incl set literals from Task 1) + 5 must-punt, 0 failures. As
Stage 1 parity grows, forms move from the punt list into must-compile; when the
fallback set equals the frozen intentional stateful set, the bootstrap is
retireable.
* core: migrate 7 lazy seq fns from the Janet seed to the Clojure overlay (40-lazy)
Finishes a port the prior team started and reverted (bb4a3e0): the 40-lazy.clj
tier moved lazy seq fns Janet→Clojure but regressed the suite to 849 because
lazy-seq's expansion leaked as data in compile mode — that was jolt-r81, since
root-fixed (lazy-seq/lazy-cat moved to 00-syntax). With the wall gone, the port
works. This shrinks the Janet seed toward the north star (self-hosted
clojure-in-clojure on a minimal host bootstrap).
Moved to core/40-lazy.clj (wired as a loaded tier after 30-macros):
distinct keep keep-indexed map-indexed cycle repeat iterate
40-lazy.clj completed to full parity: distinct gains its transducer arity;
keep/keep-indexed/map-indexed already had both arities.
Removed from the Janet seed (core.janet): the 7 core-* fns + their core-bindings
entries, the now-dead td-keep/td-map-indexed transducer helpers (the CLJ versions
carry their own), and the already-dead core-partition-by/core-xml-seq (shadowed by
10-seq/20-coll). Net: core.janet −131 lines.
Deferred (kept in Janet, separate follow-ups): partition-all (a CLJ port via
take/drop realizes a non-minimal element count, tripping the §6.3 laziness
counters + a suite file) and repeatedly (canonical CLJ doesn't validate args, so
the repeatedly.cljc throw cases regress). Both need behavior-matching first.
Gate: conformance 262x3, lazy-infinite 44/44, clojure-test-suite 4004/66 (UP from
3981 — the CLJ versions add coverage, e.g. distinct value-equality), fixpoint,
self-host, sci 422/0, fallback-zero, specs+unit, core-bench all green.
* test: raise clojure-test-suite baseline 3981 -> 4004 (lazy-fn migration coverage)
* core: migrate partition-all to the Clojure overlay (minimal realization)
Resolves the deferred partition-all port (jolt-yo3). The earlier CLJ attempt via
lazy take/drop over-realized vs the Janet pstep, tripping the §6.3 laziness
counter. The collection arities now realize EXACTLY n per chunk with a first/rest
loop and continue from the advanced cursor (no re-drop), so (take 3 (partition-all
2 (map counting (range)))) realizes exactly 6 — matching minimal realization in
both interpret and compile modes. Keeps transducer + [n coll] + [n step coll]
arities. letfn-bound recursion sidesteps the compile-mode multi-arity closure bug
(jolt-zxw), like keep-indexed/map-indexed.
Removed from the Janet seed: core-partition-all + its binding + the now-dead
td-partition-all helper (the CLJ version carries its own transducer arity).
Gate: conformance 262x3, lazy-infinite 44/44 (incl the §6.3 partition-all
counter), clojure-test-suite 4004/66, fixpoint, self-host, specs+unit green.
* core: migrate repeatedly to Clojure + fix char-not-callable / take count validation
Resolves the deferred repeatedly port (jolt-8qx). The blockers were two jolt
leniencies vs Clojure, now fixed (and correct beyond repeatedly):
- A char (a :jolt/type-tagged struct) fell into the struct-as-map branch of both
jolt-call (compile path) and the interpreter's apply dispatch, so (\a) returned
nil instead of throwing. Now only an UNtagged struct (a map literal) — or a
record — is callable as a key lookup; tagged structs fall through to "Cannot
call … as a function". Symbols are still handled (keyword-style get).
- core-take didn't validate its count, letting Janet's >= silently compare an int
to a char/string. It now rejects a non-number n like Clojure.
With those, the canonical CLJ repeatedly matches: (first (repeatedly non-fn)) and
(repeatedly non-number f) throw. Moved repeatedly to core/40-lazy.clj; removed
core-repeatedly + its binding from the seed.
These correctness fixes help broadly: repeatedly.cljc goes clean (19/10 -> 29/0),
and the suite rises 4004 -> 4034 pass / 66 -> 67 clean. Baseline raised.
Gate: conformance 262x3, lazy-infinite 44/44, clojure-test-suite 4034/67,
fixpoint, self-host, sci, fallback-zero, specs+unit green.
* test: document the 4004 -> 4034 baseline raise (partition-all/repeatedly + char/take fixes)
* docs: partition-all letfn is for minimal realization, not jolt-zxw
jolt-zxw (multi-arity arity-param mis-capture in a nested lazy-seq under :compile?)
is no longer reproducible: an arity-direct partition-all now compiles correctly in
the overlay. The original failure was an artifact of the CLJ multi-arity version
coexisting with the Janet core-partition-all ([n & rest]) during migration — the
shadowing confused multi-arity dispatch; removing the Janet version resolved it.
The letfn in partition-all stays purely for minimal realization (jolt-yo3).
* compiler: compile set?/disj as plain fns (close the last Stage-1 fallback gap)
Stage 1 jolt-g3h. set? and disj were special-cased in all three "can't compile"
lists (host_iface special-names, compiler.janet uncompilable-heads, evaluator
special-symbol? + handlers) — but they're pure value-production with callable
core vars (core-set?/core-disj), and those vars are byte-for-byte equivalent to
the evaluator handlers. Removed them from all three lists + dropped the now-dead
evaluator handler arms, so they're ordinary clojure.core fns everywhere: the
analyzer compiles (set? x)/(disj s x) as normal var calls instead of punting to
the interpreter.
Verified identical results in default AND JOLT_MUTABLE builds (no representation
sensitivity — sets are phs in both, unlike vector?/list? which collapse).
With this, the self-hosted analyzer's compile-path fallback set equals the frozen
intentional stateful set (Task 2) — it's now a strict superset of the bootstrap
compiler's compilable surface, so the Janet bootstrap is retireable (Stage 2).
fallback-zero: set?/disj moved to must-compile (31 now), set! into must-punt.
Gate: conformance 267x3 (+5 set?/disj cases), lazy-infinite 44/44, suite 4034/67,
fixpoint, self-host, sci, specs+unit green.
Yogthos
Dmitri Sotnikov