make-hierarchy/derive/underive/isa?/parents/ancestors/descendants are now
Clojure in the overlay — Clojure's own pure-map implementation: a hierarchy
is {:parents {tag #{..}} :ancestors {..} :descendants {..}}, the 3-arity
forms are PURE (derive returns a new hierarchy), and the 1/2-arity forms swap
a private global-hierarchy atom.
This fixes three correctness gaps the Janet kernel had: multi-parent derive
(the kernel's :parents held a single parent per tag), TRANSITIVE descendants
(the kernel tracked direct children only), and vector-pair isa?
((isa? [child1 child2] [parent1 parent2])). Cyclic and duplicate derives now
behave like Clojure (throw / no-op).
Multimethod dispatch was the kernel's only internal caller: defmulti-setup's
dispatch closure now calls the overlay's isa? through a lazily-resolved var
(cached per multimethod); a :hierarchy option is an atom (deref per dispatch,
matching Clojure's var semantics) or a plain hierarchy map. The Janet kernel
(types.janet) and the core-* wrappers are deleted.
20 new hierarchy spec rows (pure 3-arity incl. cycle/duplicate edges, global
+ dispatch incl. custom :hierarchy atoms). Gate green: conformance 326x3,
suite 4566 >= 4540, all batteries, bench in the session band.
The representation predicates (sequential?/associative?/counted?/indexed?/
reversible?/seqable?, boolean?/double?/float?/infinite?, the qualified/simple
ident predicates), the realization boundaries doall/dorun, list*, find,
realized?/force, pop, the print-str family, and rand-nth/random-sample are
now Clojure in 20-coll, expressed over the overlay's own predicates and
primitives — no Janet representations referenced.
Several got MORE correct in the move: dorun honors its bounded-n arity (the
seed ignored n); find works on vectors by index (contains? gives it free);
indexed? no longer claims seq results; seqable? no longer claims arbitrary
tagged tables (atoms were "seqable"); realized? reads the tagged slots via
the established get pattern.
Seed: 3427 -> ~3200 lines, 371 -> ~345 core-* fns. Gate green (conformance
326x3, suite 4569 >= 4540, stdlib battery, all specs+unit); bench in noise.
*ns* is interned in clojure.core holding the current NAMESPACE OBJECT, kept
in sync by ctx-set-current-ns through a var table cached on the env (one
table put on the hot path; core-bench A/B neutral). A thread binding
(binding [*ns* ...]) shadows the root through var-get as usual. in-ns now
returns the namespace object (Clojure); str renders a namespace as its name
and pr-str as #namespace[name]. The ns-designator helper accepts namespace
objects (they are tagged STRUCTS — the old table?-based check never matched
them), so (ns-aliases *ns*) / (ns-unalias *ns* 'a) work — SCI and the jank
syntax-quote corpus use exactly that shape.
Known divergence (documented): inside an interpreted fn, *ns* reflects the
fn's defining ns (jolt's resolution model rebinds current-ns per call);
top-level and load-time reads match Clojure.
Gate green (conformance 326x3, suite 4572 >= 4540, all batteries,
specs+unit +8 *ns* rows); bench neutral.
The names turn 2a's leak removal exposed as honestly missing, now proper:
- slurp/spit/flush (host-classified): path-based IO over Janet files; spit
takes :append; flush flushes *out*. printf prints formatted (no newline)
over the existing format. file-seq walks paths via two host dir primitives
through the overlay's tree-seq.
- ns-map / ns-unmap / ns-refers (ctx fns). ns-refers required fixing the
refer MODEL: refer/use/:refer now map the SOURCE VAR into the target ns
(the Clojure model) instead of copying its value into a new var — so
source-ns redefinitions propagate, the :macro flag travels for free, and
refers are identifiable by the var's home :ns.
- Thread-binding family: with-bindings*/with-bindings, bound-fn*/bound-fn,
bound?, thread-bound?, get-thread-bindings. The captured binding map is a
Janet struct keyed by the var tables — the exact frame representation
var-get reads — so it re-pushes correctly (a phm frame is invisible to
var lookup).
- load-string and eval interned as VALUES at the api layer (they need the
loader's compile-or-interpret routing); the eval special form still
handles direct calls.
Suite 4532 -> 4572 (baseline floor 4540 across timeout variance, clean 86),
conformance 326x3, stdlib battery, all specs+unit (+21 turn-2b rows).
Coverage: missing-portable 27 -> 10 (left: the *in*-model readers, the
with-local-vars/with-precision/extenders tail).
resolve-sym's last resort silently resolved any unknown Clojure symbol
against Janet's root environment — leaking Janet builtins with JANET
semantics into Clojure code: (type 1) was Janet's :number, (gensym) returned
Janet symbols (the long-documented (symbol (str (gensym))) macro landmine
existed BECAUSE of this), compare/slurp/int?/any? likewise. The explicit
janet/ prefix is the deliberate interop channel; the implicit fallback is
gone — an unresolved symbol is an error.
What the leak was masking, now proper interned vars:
- gensym: jolt's own (already existed, never interned) — returns real jolt
symbols; the macro landmine is dead
- compare: full Clojure total order (nil-first, numbers, strings, keywords,
symbols by ns/name, booleans, chars, uuid/inst, vectors by length then
elementwise; cross-type throws)
- type: :type metadata override, deftype/record tag as symbol, else a
taxonomy keyword (host-classified)
- int?: core-integer? — which had a latent bug the leak hid: (integer?
##Inf) was true (floor of inf is inf); NaN/infinities now excluded
- any?: constantly true (Clojure 1.9; SCI's namespaces.cljc needs it)
- jolt.interop/janet-type now uses the explicit (janet/type x) channel
- evaluator-test uses init (a bare make-ctx resolved EVERYTHING via the leak)
Suite 4470 -> 4532+ pass / 86-87 clean (proper compare unlocks the sort
files); baselines raised. Conformance 326x3 (+5 rows), +22 predicate spec
rows, stdlib battery green, all specs+unit. Coverage dashboard now counts
previously-leak-resolvable names honestly (missing-portable 19 -> 27).
CI caught what the local per-change gate missed (clojure-stdlib-suite-test
wasn't in the habitual gate list): strict map? (PR #28) correctly stopped
treating tagged structs as maps, which silently disabled edn->value's
(and (map? x) (= :jolt/set ...)) set-form branch — edn/read-string returned
raw reader forms for #{...}. Reader FORMS are now detected by :jolt/type
directly, never via map?.
While here, EDN's built-in tagged elements are applied properly: a
:jolt/tagged form routes through the registered data reader (__read-tagged,
a ctx-capturing core fn over :data-readers), so (edn/read-string "#uuid ...")
yields a real UUID value and #inst an instant — per the EDN spec. The
read_string battery goes 47 -> 50 (above the old 49 baseline: the uuid
asserts pass now); baseline raised to 50.
#inst (jolt-rnh): an instant is an immutable tagged struct
{:jolt/type :jolt/inst :ms <epoch-millis>} — equality and map-key hashing by
INSTANT, so different offsets denoting the same moment are =. The reader
parses RFC3339 with Clojure's partial-timestamp defaults (#inst "2020" is
2020-01-01T00:00:00.000Z) and errors on malformed input; inst?/inst-ms in
the overlay; pr-str prints the canonical
#inst "yyyy-MM-ddThh:mm:ss.fff-00:00" and round-trips; str gives the bare
RFC3339 string. Self-evaluating in the evaluator (like uuid/chars).
Syntax-quote (jolt-l2a): a syntax-quoted self-evaluating literal (string,
number, boolean, nil, keyword, char) collapses to the literal at READ time,
matching Clojure's reader — so nested/adjacent backticks over literals are
inert: (= "meow" ```"meow") is true (jank pass-adjacent). Symbols still
qualify and collections still template. General nested syntax-quote over
non-literals remains UNVERIFIED in spec S25.
Tests: inst-spec (18 cases incl. partial defaults, offsets, round-trip),
+10 literal-collapse reader rows, +5 conformance rows (321x3). Spec
02-reader S20/S25 updated to normative. Suite stable 4470/86.
One documentation root: the prose docs (building-and-deps, self-hosting
architecture/compiler, tools-deps, grammar.ebnf) join the spec and RFCs under
docs/. References in README and deps-conformance-test updated.
jolt no longer satisfies :clj in reader conditionals. The shortcut was a
measured net liability: :clj branches carry JVM interop and JVM-specific
test expectations jolt fails, and they shadowed :default branches jolt
passes. A/B over the suite: clj,default = 4967 assertions / 4324 pass / 119
errors; jolt,default = 5069 / 4470 / 81 (+146 pass, -38 errors, +8 clean
files). Baselines raised to 4470/86.
Matching is now by CLAUSE order like Clojure — the first clause whose key is
in the feature set wins (#?(:default 5 :clj 6) is 5 everywhere); the old code
scanned for :clj first, then :default, regardless of position.
Foreign clj-targeted libraries are a property of the LOADING CONTEXT, not the
platform: reader-features-set! opts a load into a compatibility set, and the
SCI bootstrap/runtime tests load SCI under ["jolt" "clj" "default"] (its
.cljc selects implementations via :clj with no :jolt branches).
JOLT_FEATURES remains the process-wide override.
RFC 0002 records the decision with the measured data; spec 02-reader S18 is
now normative (clause order, documented feature set, per-context override).
Reader tests updated to the portable set + an opt-in round-trip.
The lexical-syntax chapter, granularity modeled on jank's 62-file
per-construct reader corpus: token grammar (whitespace/comments, collections
with read-time duplicate checks, numbers incl. the N/M tower question,
symbols/keywords incl. ::auto-resolution, strings/chars), the quote-family
sugars, the full #-dispatch catalog with normative entries (anonymous fn
%-derivation, discard composition, reader conditionals, symbolic floats,
tagged literals), and the syntax-quote contract (core/alias/current-ns
qualification, template-stable gensyms, ~' idiom, distribution through
collections).
Adapting the corpus surfaced and filed three findings, recorded as labeled
divergences/UNVERIFIED in the chapter: nested syntax-quote doesn't collapse
(S25, (= "meow" ```"meow") is false), #inst reads as a bare string (identity
data reader, no instant type), and jolt satisfies :clj in reader
conditionals (feature-key policy under review).
reader-forms-spec gains 11 chapter-cited rows (discard stacking, ##Inf/
##-Inf/##NaN, :default conditionals, qualified var-quote identity, gensym
stability within vs across templates) — all passing.
Fifteen vars from the spec coverage gap (docs/spec/coverage.md):
parse-long/parse-double/parse-boolean (strict validation; scan-number alone
accepts 0x10), newline, current-time-ms (host clock for time), update-keys/
update-vals (PHM base, collisions last-wins), partitionv/partitionv-all/
splitv-at (lazy seqs of vectors; splitv-at's tail stays a seq, matching the
reference), with-redefs/with-redefs-fn (roots restored on throw), time,
macroexpand (expand-1 to fixpoint), alias/ns-unalias (write BOTH alias stores
— require :as uses string-keyed :imports while ns-aliases reads :aliases;
split filed), ns-publics (symbol-keyed map; publics == interns, no privacy).
Three pre-existing bugs fixed along the way:
- (partition n step pad coll) misparsed pad as the coll and returned ()
- (require 'bare.symbol) rejected — only vector specs were accepted
- analyze-form leaked the interpreted analyzer's ns on a punt: a throw out of
the analyzer left current-ns=jolt.analyzer and :compile-ns set, so the
fallback interpretation resolved user vars against the wrong namespace
(bit (var user-sym) under compile mode)
And one overlay-authoring landmine documented in-code: a 20-coll fn must not
use 30-macros macros (with-redefs-fn's dotimes compiled as a forward ref that
resolved to the macro fn at runtime) — loop/recur instead.
Gate: conformance 316x3 (+14 rows), suite 4324 pass / 78 clean (was 4081/72;
parse_*/update_* files now contribute), baselines raised, all specs+unit,
fixpoint, self-host, sci, staged. Coverage: missing-portable 35 -> 20.
RFC 0001 proposes a normative, implementation-independent Clojure language
spec (the reader, evaluation model, special forms, data types, seq/laziness
contracts, namespaces/vars, and the portable clojure.core surface) to the
standard of R7RS/Racket — Clojure has none, and every alternative
implementation re-derives semantics from the reference and folklore. The
spec is executable-first: every numbered normative statement cites its
conformance test or is marked UNVERIFIED.
docs/spec/ carries the front matter (conformance terms, entry format, host
classification), the special-form catalog with worked normative entries for
if and let*, the core-library entry format with worked entries for first,
reduce, and parse-uuid, and a generated coverage dashboard over the 694-var
ClojureDocs inventory (tools/spec_coverage.py cross-references the surface
against jolt's interned+resolvable vars and the test suites).
Measured baseline: 380 implemented+tested, 154 implemented-untested, 35
portable-but-missing (filed), 22 resolvable-but-not-interned (filed — seed
fns invisible to resolve/ns-publics), rest classified host/JVM/concurrency.
map? treated ANY struct as a map, so (map? 'sym), (map? \a), and
(map? (random-uuid)) were all true; coll? had the same wart. Meanwhile both
were FALSE for sorted maps (and coll? for sorted sets and records), which are
collections in Clojure. Now: a map is a plain struct literal, a phm, a sorted
map, or a record; coll? additionally includes sorted sets. Tagged structs
(anything with :jolt/type) are values.
The loose-map? dependents (destructure's clause ordering, defn's attr-map
guard) already guarded with symbol? first, so nothing relied on the wart —
full gate green, and the suite gained: 4074 -> 4081 pass / 72 clean files
(map_qmark/coll_qmark assertions now correct); baselines raised. 22 new
strictness spec cases.
uuid support was broken end to end (jolt-6s2): random-uuid built malformed
strings ((string/format "%x") with no zero-padding, so hex groups came out
short, and no variant bits), uuid? was hardcoded false, the #uuid data reader
was identity (bare string), and parse-uuid didn't exist. Nothing tested it.
A UUID is now an immutable tagged struct {:jolt/type :jolt/uuid :str <lower>}
(make-uuid, types.janet) — struct value equality gives case-insensitive = and
map-key/set hashing for free, and the evaluator treats it as self-evaluating
(like chars). random-uuid emits a correct RFC 4122 v4 (zero-padded 8-4-4-4-12,
version nibble 4, variant 8-b). parse-uuid validates the canonical shape,
returns nil on a bad string, throws on a non-string (Clojure 1.11). uuid? is
an overlay tag predicate. str renders the bare string; pr-str renders
#uuid "..." and round-trips.
Tests: uuid-spec (30 cases: format, parse edge cases from the suite, value
semantics, reader literal), 6 conformance cases x3 modes. The suite's
parse_uuid/random_uuid/uuid_qmark files now contribute: 4049 -> 4074 pass,
71 clean files; baselines raised.
With tiers 6a-6c done, the interpreter punt surface is down to the frozen
curated set: defmacro, set!, letfn (needs letrec IR), eval, the host-interop
heads (./new/Foo./.method), and the JVM-compat stubs (gen-class,
monitor-enter/exit). must-punt now enumerates it exhaustively — growing the
list is a regression (lost compiler coverage); shrinking it is progress.
This is the Stage 2 definition of done for the special-names tail.
prefer-method/remove-method/remove-all-methods/get-method/methods become
overlay macros over ctx-capturing *-setup fns (a multimethod's method table
lives on its VAR, so the name passes quoted — the defmulti/defmethod shape).
instance? likewise (class names don't evaluate to values); satisfies? is a
plain ctx-capturing fn (evaluated args). locking and defonce become overlay
macros — locking now also evaluates the monitor expr (the old arm skipped it
and any body form past the first); defonce keeps the existing-root check.
read-string and macroexpand-1 are ctx-capturing fns.
Removed all eleven from the evaluator special arms + special-symbol?,
host_iface special-names, and compiler uncompilable-heads. evaluator-test's
locking/instance? cases use init now (overlay macros need the full env).
Surfaced pre-existing (filed): multimethod dispatch records prefer-method
preferences on the var but never consults them in ambiguous isa dispatch.
Gate: conformance 296x3 (+11 tier-6c cases), fallback-zero 73/3, fixpoint,
self-host, sci, staged, suite 4049>=4034, all specs+unit.
create-ns/remove-ns/find-ns/all-ns/the-ns/ns-interns/ns-aliases/ns-imports/
ns-resolve/resolve/refer become ctx-capturing clojure.core fns
(install-stateful-fns!) with evaluated-arg Clojure semantics, replacing
interpreter special arms that were loose: the-ns/ns-interns/ns-aliases/
ns-imports ignored their argument (always current ns), create-ns/remove-ns/
ns-resolve took theirs unevaluated. The optional-arg forms still default to
the current ns, preserving the prior 0-arg behavior. refer previously had a
special-names entry but NO interpreter arm — it errored everywhere; it now
refers the named ns's public vars (use-impl), :only/:exclude filters not yet
honored. ns-resolve does its lookup directly: types/ns-resolve keys ns-find
with the symbol struct instead of its name string and never finds anything.
Removed all of them from the evaluator arms, host_iface special-names, and
compiler uncompilable-heads (use/ns/require/in-ns stay punted in the bootstrap
compiler — it builds analyzer.clj, whose ns forms need the interpreter).
Gate: conformance 285x3 (+8 ns-fn cases), fallback-zero 62/3, fixpoint,
self-host, sci, staged, suite 4049>=4034 (+2 passes), all specs+unit.
var-get/var-set/var?/alter-var-root/alter-meta!/reset-meta! already had plain
core-bindings wrappers — the interpreter special arms shadowed them (and the
arm's alter-var-root silently dropped rest args, which the core wrapper
handles). find-var/intern need the ctx, so they join install-stateful-fns! as
ctx-capturing clojure.core fns; the old core-intern binding was a do-nothing
stub. Removed the eight from the evaluator special arms + special-symbol?,
host_iface special-names, and compiler uncompilable-heads.
Gate: conformance 277x3 (+6 var-fn cases incl. alter-var-root rest args),
fallback-zero 51/3, fixpoint, self-host, sci, staged, suite 4047>=4034 (one
new pass), all specs+unit.
deftype with inline protocol methods failed in compile mode with "Unable to
resolve symbol: jolt.analyzer/extend-type". deftype's expander template emits
(extend-type ...), but extend-type is defined AFTER deftype in 30-macros — so
when the expander compiles at defmacro time, syntax-quote lowering can't
resolve it and falls through to current-ns qualification. The analyzer runs
interpreted in jolt.analyzer, so ctx-current-ns mid-compile is jolt.analyzer,
baking the wrong qualification into the compiled template.
Same bug class jolt-265 fixed for resolve-var: read :compile-ns (the namespace
being compiled) when set, falling back to ctx-current-ns. Interpret and
self-host modes never hit it (interpreted expanders resolve at use time, when
extend-type exists) — which is also why no gate test caught it: nothing
EVALUATED a deftype-with-inline-methods in compile mode. Two 3-mode
conformance cases added.
Gate: conformance 271x3, fallback-zero, fixpoint, self-host, sci, staged,
suite 4046>=4034, all specs+unit.
The gate spent almost all its time rebuilding identical contexts: init is
~50 ms interpreted / ~900 ms compiled (tier loading, analyzer build, macro
recompilation), and both the conformance harness and the spec harness built a
fresh ctx PER CASE — 269 cases x 3 modes for conformance alone (~285 s), and
~1500 spec cases (~90 s). The suite battery additionally ran its 234 worker
subprocesses sequentially (~100 s incl. 5 x 6 s timeout files).
- api: snapshot/fork — marshal a fully-built ctx once (reverse-lookup dicts
from root-env, built at module load before any ctx exists), unmarshal cheap
fully-isolated deep copies (~2 ms). A fork shares nothing mutable with its
siblings, so per-case isolation is preserved exactly.
- conformance: one init per mode + fork per case (self-host pre-builds the
analyzer before snapshotting). 285 s -> 5 s, same 269x3 results.
- spec harness: jeval/run-spec/expect-throws fork from one lazy module-level
snapshot. Spec sweep ~90 s -> 9 s, all pass.
- clojure-test-suite: run the per-file worker subprocesses through a
token-channel worker pool (default 4, JOLT_SUITE_WORKERS to override,
capped at 8). 17.5 s with counts identical to the sequential run
(4046/520/116, 5 timeouts).
Full gate wall-clock: ~8 min -> 43 s, everything green (conformance 269x3,
fallback-zero, fixpoint, self-host, sci, staged, suite >= 4034/67, all
specs+unit).
compile-and-eval wrapped the compile step in a blanket protect — any failure,
including a genuine compiler bug, silently fell back to the interpreter and
hid behind a correct-looking result. Now only the analyzer's deliberate punt
signal (jolt/uncompilable: …) falls back; any other compile-step error
propagates.
Tightening this surfaced one accidental dependency: pre-kernel overlay forms
(00-syntax's destructure defn) trigger a compile while ensure-analyzer is
still gated, and the missing analyzer crashed var-get with a cryptic indexing
error that the blanket catch happened to convert into the designed interpret
fallback. That path now punts explicitly ("jolt/uncompilable: analyzer not
built (pre-kernel bootstrap)").
New unit test stubs jolt.analyzer/analyze to prove a non-punt error
propagates while the punt channel still falls back.
Gate: conformance 269x3, fallback-zero, fixpoint stage1==2==3, self-host,
sci-bootstrap, staged-bootstrap, all specs+unit, suite 4046>=4034 (5 timeouts);
core-bench A/B neutral (4562 vs 4572 ms).
* core: fix jolt-265 — syntax-quote fully-qualifies core syms to clojure.core/
Re-attempt of the deferred gap, now unblocked. The earlier uberscript break
wasn't qualification itself but an aliased-ref resolution bug it exposed:
resolve-var looked up ns-aliases (e.g. g/foo) against the analyzer's REBOUND
ctx-current-ns (jolt.analyzer, since the analyzer runs interpreted in its own
ns) instead of the namespace being compiled. A user's aliased refs failed
mid-compile (g/hello -> nil in the bundled standalone).
- resolve-var resolves aliases against :compile-ns when set (same ns
h-current-ns uses), falling back to ctx-current-ns — correct for both modes.
- sq-symbol qualifies a resolved clojure.core name to clojure.core/<name>
(Clojure hygiene); special forms stay bare; unresolved syms -> current ns.
Tests updated to the qualified behavior (features-test, reader-forms-spec).
* test: comprehensive spec — regex + sorted colls + random/predicate gaps
Filling the biggest untested clojure.core areas found in a coverage audit
(168 of 506 provided fns had no spec). New + expanded suites:
- regex-spec.janet (20): #"…" literals, regex?, re-find/re-matches/re-seq
(match/no-match/groups), re-pattern, and clojure.string split/replace with
regex (incl $1 backrefs). Whole area was previously unspecced.
- sorted-spec.janet (14): sorted-map/sorted-set construction + ordering, sorted?,
subseq/rsubseq. Pins the working subset — get/conj/assoc/keys/vals on sorted
colls and the by-comparator ctors are not yet first-class (jolt-ti9).
- predicates-spec +14: seqable?, integer?, reduced?/unreduced, not-empty.
- numbers-spec +5: rand/rand-int/rand-nth invariants.
Fix: sorted? was bound to core-sorted-map? so it returned false for sorted-sets;
now true for both sorted maps and sets (core-sorted?).
Filed: jolt-ti9 (sorted collections incomplete: get/conj/assoc/keys/vals don't
operate on the sorted wrapper; sorted-*-by ignore the comparator).
Gate green incl full jpm build + jpm test.
* core: close surfaced gaps — first-class sorted colls, with-out-str, rand arity, deref reduced
jolt-ti9: sorted-map/sorted-set are now first-class across the collection fns —
get/assoc/dissoc/conj/contains?/keys/vals/disj and call-as-fn all operate on the
wrapper and preserve sort order. The by-comparator constructors (sorted-map-by/
sorted-set-by) now thread the user comparator (numeric or boolean-predicate) through
all derived colls. Sorted predicates/ctors/ops moved above core-conj so the
collection fns can branch on them; jolt-invoke (interpreter) gets inline branches.
jolt-rfw: add with-out-str (binds output to a string buffer) + the macro.
jolt-ek3: (rand n) arity and deref-of-reduced (uuid? still deferred).
Specs: new io-spec.janet; sorted-spec expanded to pin the now-working map/set ops
and by-comparator ordering; predicate/number spec restorations.
* remove old doc
* core: fix stale comment — by-comparator sorted ctors are implemented
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Re-attempt of the deferred gap, now unblocked. The earlier uberscript break
wasn't qualification itself but an aliased-ref resolution bug it exposed:
resolve-var looked up ns-aliases (e.g. g/foo) against the analyzer's REBOUND
ctx-current-ns (jolt.analyzer, since the analyzer runs interpreted in its own
ns) instead of the namespace being compiled. A user's aliased refs failed
mid-compile (g/hello -> nil in the bundled standalone).
- resolve-var resolves aliases against :compile-ns when set (same ns
h-current-ns uses), falling back to ctx-current-ns — correct for both modes.
- sq-symbol qualifies a resolved clojure.core name to clojure.core/<name>
(Clojure hygiene); special forms stay bare; unresolved syms -> current ns.
Tests updated to the qualified behavior (features-test, reader-forms-spec).
Co-authored-by: Yogthos <yogthos@gmail.com>
* test: adapt jank's form/reader tests into spec suites; fix case no-match
Vendoring jank's behavior (not the project): we base our own copies on the
jank test corpus to close coverage gaps, but maintain them ourselves since
jank may diverge. Two new spec batteries (jank-isms translated to Jolt:
letfn* -> letfn, jank catch types -> :default; platform-specific bigdec/
biginteger/ratio/uuid/##Inf/unicode cases omitted):
- test/spec/forms-spec.janet (52): case, fn (arity/variadic/closure/recur/
named), let, letfn, loop, try, if/do/def/call — incl :throws regression
cases (no-match, bad params, nil call).
- test/spec/reader-forms-spec.janet (22): #() (% %N %&), #' var-quote,
^metadata, syntax-quote (gensym/unquote/splice).
Fix surfaced by adaptation: case with no matching clause and no default now
throws 'No matching clause' (Clojure semantics) instead of returning nil
(00-syntax). Gate green incl full jpm build+test.
Other gaps the adaptation surfaced are filed (tests adjusted to jolt's
current behavior + a comment, not silently dropped):
jolt-vdo case duplicate test constants not rejected
jolt-w2v loop bindings not sequential (later init can't see earlier)
jolt-6x1 #() %& miscomputes arity with a higher positional (%2 …)
jolt-xl0 ^meta not attached to collection literals ({}/[]/#{})
jolt-265 syntax-quote doesn't fully-qualify core syms to clojure.core/
jolt-edb syntax-quote ~/~@ not processed inside set literals
* core: fix 5 Clojure-conformance gaps surfaced by jank tests
All from adapting jank's form/reader tests; each fix verified in interpret +
compile modes and the spec tests now assert the correct behavior.
- jolt-vdo: case now rejects duplicate test constants at expansion (Clojure
compile error), via bootstrap-safe duplicate detection (00-syntax; analyzer.clj
uses case during its own build, so seed-only fns).
- jolt-w2v: loop bindings are now sequential like let — a later init can
reference an earlier binding. Fixed the interpreter loop* (accumulating scope)
and the back end emit-loop (bind initial inits in a sequential Janet let before
entering the recur target).
- jolt-6x1: #() reader computes the fixed arity from the MAX positional (%2 ->
[p1 p2 & rest]); % and %1 unify; unused lower slots get placeholder params.
- jolt-xl0: ^meta on collection literals ({}/[]/#{}) now attaches — read-meta
passes the NORMALIZED metadata map to with-meta (was the raw meta-form).
- jolt-edb: syntax-quote processes ~/~@ inside set literals (new __sqset builder
in core + set branches in syntax-quote* and syntax-quote-lower).
Deferred: jolt-265 (fully-qualify core syms to clojure.core/ in syntax-quote) —
it passes conformance but breaks the standalone uberscript (the ns macro emits
unqualified require/in-ns, which then qualify and break bundled require/alias).
Reverted to bare (functionally resolves); re-opened with the finding.
Gate green incl full jpm build + jpm test: conformance 269x3, suite >=4034/67,
fixpoint, self-host, sci 422/0, uberscript, all unit + spec (forms 55, reader 31).
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
* core: Stage 2 Task 2 tier 5a — compile defmulti + defmethod
defmulti/defmethod become macros (30-macros) over ctx-capturing
clojure.core fns (defmulti-setup/defmethod-setup, interned by
install-stateful-fns!):
- defmulti: (defmulti name dispatch & opts) -> (defmulti-setup 'name
dispatch ~@opts). name quoted; dispatch + opts (:default/:hierarchy)
evaluated. defmulti-setup builds the dispatch closure over the method
table and interns the var (same hierarchy/default/cache behavior).
- defmethod: (defmethod mm dval & fn-tail) -> (defmethod-setup 'mm dval
(fn ~@fn-tail)). The method impl is now a COMPILED (fn …) (was an
interpreted fn* eval). Auto-creates the multimethod if missing.
- removed their special-symbol? entries + eval-list arms, and dropped them
from host_iface special-names + loader stateful-head?.
Both compile + interpret as plain invokes; dispatch incl. :default and
derive/hierarchy works in both modes.
Tests: evaluator-test (defmulti case) + namespace-test now use init (these
forms are overlay macros now, so a bare make-ctx lacks them).
Gate green: conformance 269x3, fallback-zero 38/4, bootstrap-fixpoint
stage1==2==3, self-host, staged-bootstrap, sci-bootstrap, clojure-test-suite
>=4034/67, features 78/78, all unit + spec (multimethods 16/16).
* core: Stage 2 Task 2 tier 5b — compile deftype + defrecord
deftype becomes a macro (30-macros) over make-deftype-ctor (a ctx-capturing
clojure.core fn that bakes the ns-qualified type tag at def time) plus
extend-type for any inline protocol methods — so it compiles as a plain (do …).
Mirrors defrecord's existing field-let/protocol-grouping pattern.
- make-deftype-ctor-impl (evaluator) builds the ctor; interned as a closure.
- removed the deftype special-symbol? entry + eval-list arm; dropped deftype/
defrecord from host_iface special-names + loader stateful-head?.
- defrecord no longer redefines ->name via (Name. …) interop (frozen) — deftype
already provides ->name, so defrecord compiles too (map->name builds via it).
- field-kws spliced into a vector LITERAL ([~@…]) so the analyzer sees a vector
form, not a runtime pvec; type name + fields are unwrapped of ^meta (the reader
yields (with-meta sym m) forms, e.g. sci's (deftype ^{:doc …} Var …)).
With tier 5a, all of deftype/defrecord/defmulti/defmethod compile. The loader's
interpret-only set is now just the frozen host-coupled forms: defmacro/set!/./
new/eval.
Tests: evaluator-test deftype case uses init (deftype is an overlay macro now);
fallback-zero moves deftype off must-punt, adds deftype/defrecord/defmulti/
defmethod to must-compile (43/3).
Gate green (full jpm build + jpm test): conformance 269x3, fallback-zero 43/3,
bootstrap-fixpoint stage1==2==3, self-host, staged-bootstrap, sci-bootstrap
422/0, clojure-test-suite >=4034/67, all unit + spec.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
* core: Stage 2 Task 2 tier 4b — compile ns (+ use/import/refer-clojure)
ns becomes a macro (00-syntax) expanding to (do (in-ns 'name) (require …)
(use …) (import …) (refer-clojure …)) — matching Clojure, where require is
a fn and ns expands to require calls. use/import/refer-clojure join the
namespace ops as ctx-capturing clojure.core fns (use-impl/import-impl/
refer-clojure-impl, interned by install-stateful-fns!). Removed the ns
interpreter special arm; dropped ns/use/import from host_iface special-names
and loader stateful-head?.
Placement matters: ns lives in the FIRST overlay tier (00-syntax), because
the self-hosted analyzer build is triggered while 10-seq loads and processes
jolt.analyzer's own (ns …) form — ns must exist by then. Its body resolves
fn/map/reduce/cond at expansion time, by which point all of 00-syntax has
loaded. The bootstrap compiler (compiler.janet) still PUNTS ns/in-ns/require/
use/import (it compiles analyzer.clj/ir.clj, whose ns forms must fall back to
the interpreter that expands the macro) — only the self-hosted analyzer
compiles them.
use-impl fixes a latent bug: it now refers the used ns's vars into the
CURRENT ns (the old special interned a ns into itself, a no-op).
ns/use/import now compile + interpret as plain invokes. fallback-zero: ns off
must-punt, onto must-compile.
Gate green: conformance 267x3, fallback-zero 38/4, bootstrap-fixpoint
stage1==2==3, self-host, staged-bootstrap, sci-bootstrap, clojure-test-suite
>=4034/67, namespace, deps-loader, cli, aot, embedded-stdlib, uberscript,
features 78/78, all unit + spec.
* test/core: cover ns form + :use; fix use-impl spec parsing
Checking test coverage for the namespace tiers surfaced a real bug nothing
exercised: use-impl checked (array? s) to find a spec's ns symbol, but a
vector spec like [src.x] is a pvec/tuple, not a Janet array — so (:use ...)
and standalone (use ...) errored. Fixed to coerce (pvec->array) then take the
head when the spec is indexed (matching require-impl).
Added coverage that was missing:
- conformance (x3 modes): the (ns …) form itself + :use refers.
- namespaces-spec: (ns …) form, :use, standalone use.
The :use/(use …) path had ZERO tests before, which is why the bug slipped
through earlier tiers.
EBNF needs no change — it's a reader/syntax grammar with no special-forms
enumeration; ns/require/protocol syntax is unchanged (the destructuring note
was already updated in jolt-f79).
Gate green: conformance 269x3, fallback-zero 38/4, self-host, sci-bootstrap,
bootstrap-fixpoint, staged-bootstrap, clojure-test-suite >=4034/67, namespace,
all unit + spec (namespaces 27/27).
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
require/in-ns take quoted (already-evaluated) args, so they become ordinary
ctx-capturing clojure.core fns (install-stateful-fns!):
- require-impl (varargs over specs -> eval-require) / in-ns-impl, extracted
from the eval-list special arms.
- removed their eval-list match arms (they fall through to the function-call
default, resolving the interned closures) and dropped them from
host_iface special-names + loader stateful-head? + compiler
uncompilable-heads.
require+alias+refer and in-ns now compile + interpret as plain invokes.
Tests: namespace-test uses bare make-ctx, so it now installs the stateful
fns via a fresh-ctx helper (api/init does this automatically). fallback-zero
moves require off must-punt and adds require/in-ns/defprotocol/extend-type/
reify/(var map) to must-compile (37 now); deftype takes require's must-punt
slot.
Gate green: conformance 267x3, fallback-zero 37/5, bootstrap-fixpoint
stage1==2==3, self-host, staged-bootstrap, clojure-test-suite >=4034/67,
features 78/78, deps-loader, all unit + spec (namespaces 24/24).
Co-authored-by: Yogthos <yogthos@gmail.com>
* core: Stage 2 Task 2 tier 2a — compile defprotocol/extend-type/extend-protocol
Applies the proven enabler: stateful primitives become per-ctx closures
captured over ctx, interned in clojure.core (install-stateful-fns!), so
they resolve + compile as plain :var invokes and work for deferred calls.
- protocol-dispatch / register-method: extracted from the interpreter
special handlers into ctx-taking impls (protocol-dispatch-impl /
register-method-impl) + interned as ctx-capturing clojure.core fns.
Removed their special-symbol? entries + handler arms, and dropped them
from host_iface special-names + compiler uncompilable-heads.
- defprotocol/extend-type/extend-protocol macros now pass the protocol/
method/type NAMES as strings (not symbols), so the emitted calls compile
as ordinary invokes; removed the three macros from special-names so the
analyzer expands+compiles them instead of punting to the interpreter.
- Both interpreter and compiled paths now call the same ctx-capturing
closures (one dispatch implementation, no special-form duplication).
reify/make-reified deferred to tier 2b (map-eval shape); defrecord waits
on deftype (tier 5).
Gate green: conformance 267x3, fallback-zero 31/5, bootstrap-fixpoint
stage1==2==3, self-host, staged-bootstrap, clojure-test-suite >=4034/67,
features 78/78, all unit + spec (protocols 7/7, multimethods 9/9).
* core: Stage 2 Task 2 tier 2b — compile reify (make-reified as a fn)
Completes the protocol machinery: make-reified joins protocol-dispatch/
register-method as a ctx-capturing clojure.core fn (install-stateful-fns!).
- make-reified-impl takes the EVALUATED {keyword fn} method map (a phm when
compiled, struct/table when interpreted) and builds the reified object.
- reify macro passes the protocol NAME as a string; method map is an ordinary
map literal evaluating to {keyword fn}.
- Removed make-reified's special-symbol? entry + handler arm, and dropped
make-reified + reify from host_iface special-names + compiler
uncompilable-heads.
reify now compiles and dispatches in both modes (single- and multi-method).
With tier 2a, the full protocol surface (defprotocol/extend-type/
extend-protocol/reify) compiles; defrecord still waits on deftype (tier 5).
Gate green: conformance 267x3, fallback-zero 31/5, bootstrap-fixpoint
stage1==2==3, self-host, staged-bootstrap, clojure-test-suite >=4034/67,
features 78/78, all unit + spec (protocols 7/7x3).
* core: Stage 2 Task 2 tier 3 — compile (var x) + binding
binding keys its thread-binding frame on var cells via (var x), so it
needed (var x) to compile. Added a the-var IR node that emits the embedded
var cell itself (vs var-ref, which derefs):
- ir.clj: the-var node.
- analyzer: 'var' added to handled; analyze-special resolves the symbol to
its var and emits the-var (uncompilable for a non-var, matching Clojure).
- backend: :the-var emits (quote cell) — the exact per-ctx cell var-get
keys on, so a compiled binding overrides + restores correctly.
- removed var from loader stateful-head? + host_iface special-names, and
binding from special-names so it expands+compiles.
Dynamic binding now compiles end-to-end (override/restore, and a compiled
fn reading the dynamic var under the binding) in both modes.
Gate green: conformance 267x3, fallback-zero 31/5, bootstrap-fixpoint
stage1==2==3, self-host, clojure-test-suite >=4034/67, features 78/78,
all unit + spec (state/metadata).
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
