Commit graph

25 commits

Author SHA1 Message Date
Dmitri Sotnikov
8180c85393
Source locations: reader positions, error locations, native stack traces (#218)
* Reader records source line/column on list forms

The reader stamps 1-based :line/:column metadata on every list form (plus
:file when load-jolt-file is reading a file), and jolt.host/form-position
reads it back so the analyzer's :pos scaffold finally gets real data. A
left-to-right cursor counts newlines over the delta between successive forms,
so it stays O(n). Vector/map/set literals are untouched (their metadata is a
runtime value the analyzer would have to wrap in with-meta); empty () can't
carry meta. ^meta now merges onto the position keys instead of clobbering them.

Re-mint is byte-identical (the backend doesn't emit :pos), so this is a pure
scaffold for the error-location work that follows.

* Report source location on uncaught errors

Each top-level form records its source position (thread-local) before it
compiles+evals, and cli.ss jolt-report-uncaught appends 'at file:line:col'
when an error propagates out. Covers joltc -e, joltc run <file>, and
load-string — every interpreted path. Top-level granularity, one set per
form; deeper frames come from the Phase 2 frame walk.

Runtime .ss only, no re-mint.

* Clojure stack traces via source registry + native frame walk

A direct-link build emits (jolt-register-source! short-name ns name file line)
once per fn def — at definition time, so zero per-call cost. On an uncaught
error the reporter walks Chez's native continuation frames (jolt-throw captures
the live continuation via call/cc; host conditions carry their own
&continuation), maps each frame's procedure name through the registry, and
prints a Clojure backtrace 'ns/name (file:line)'. Wired into both the cli and a
built binary's launcher.

Frames are keyed by the short munged fn name Chez actually reports (emit-fn's
letrec self-binding), not jv$ns$name; a cross-namespace collision degrades to
the bare frame name rather than a wrong attribution. The analyzer carries the
original form's position through defn macroexpansion onto the def node.

Calling a non-fn now throws a catchable ClassCastException (via jolt-throw)
naming the operator, instead of a raw Chez error.

Caveats (documented in source-registry.ss): names map only in direct-link/AOT
closed-world builds — the open-world -e/repl/run path falls back to the
top-level location; and pervasive TCO erases tail-call frames, so a mapped
trace shows only the non-tail spine. JOLT_DEBUG_FRAMES dumps raw frame names.

Re-mint (analyzer + backend); prelude byte-identical (direct-link off during
mint). Corpus rows certified, build-smoke asserts the trace.

* Propagate source position through macroexpansion

hc-expand-1 now carries the macro call form's :line/:column onto the top of a
list expansion that has none of its own (merged under any meta the macro set),
so errors and stack traces in macro-generated code point at the call site —
Clojure parity. The analyze recursion re-expands inner macros, so each level's
top form picks it up, matching the reference compiler. (meta (macroexpand-1
'(when x y))) now reports the call-site line.

A direct-link fn defined through a user macro (build-app's defguarded) registers
with a real line, so build-smoke's trace assertion covers macro-defined fns.

Runtime .ss (host-contract.ss) — no re-mint; selfhost holds.

Phase 3's optional items are deferred: :line-in-ex-data has no clean consumer
(it would pollute ex-data, break = and printing, and positions already surface
via the trace + top-level location), and Chez source-object emission is a large
backend change the jv$-name registry already sidesteps.

* Review fixes: registration key, thread-locals, debug flag timing

- Register a fn under the name Chez actually reports for its frame, not the def
  name: a named fn literal whose name differs from the def (def foo (fn bar …))
  is framed as 'bar', and an anonymous fn def (def foo (fn …)) as jv$ns$foo.
  Both previously registered under the def name and so never appeared in traces.
- rdr-source-file / rdr-pos-cursor are thread parameters, so concurrent compiles
  (futures, core.async) don't clobber each other's file/line attribution.
- Read JOLT_DEBUG_FRAMES at call time: a built binary evaluates top-level forms
  at heap-build time, where a load-time getenv is always unset.

Re-mint (backend + reader); prelude byte-identical, selfhost holds.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 02:14:34 +00:00
Yogthos
67e642bdfb core.match: regex + array patterns (full support); library-conformance directive
Finishes core.match — its full test suite (115/115) now passes, including the
two patterns the earlier work left out:

- Regex-literal patterns. A #"…" now reads as a regex VALUE (Clojure parity: the
  reader constructs the Pattern, so a macro receives a regex, not jolt's tagged
  form), and the analyzer compiles a regex value to the same :regex IR leaf via
  its source. emit-quoted handles a quoted regex; a regex value carries the
  java.util.regex.Pattern host tag so extend-protocol/instance? dispatch on it.
- Primitive-array patterns. A ^Type hint's :tag is now the SYMBOL (e.g. `ints`),
  matching the JVM, so core.match's array-tag lookup engages the array
  specialization (alength/aget). jolt's :tag consumers already tolerate a symbol
  (hc-cell-num-ret normalizes; tag->nkind/def-meta handle both).

Also: a library-conformance directive in CLAUDE.md, and the supported-libraries
list (docs + site) simplified to one-line entries — a listed library is assumed
to work fully, so no tallies or feature enumerations. core.match + transit-jolt
added to the list.

Seed change (reader/backend/30-macros) -> re-minted; the rest runtime. JVM-
certified corpus rows; the stale `symbol hint -> :tag` divergence is dropped from
the allowlist (jolt now matches the JVM). make test + shakesmoke green.
2026-06-25 00:46:10 -04:00
Yogthos
f5455115a0 deftype/record: clojure.lang collection interfaces + protocol identity
Running clojure.core.match (a macro-heavy library that builds its compiler out
of deftypes implementing clojure.lang interfaces) shook out a cluster of general
gaps. Its own suite goes from not-loading to 111/115 assertions.

- deftype/defrecord implementing a clojure.lang collection interface now drives
  the core fns: Indexed -> nth, Counted -> count, Associative -> assoc, ILookup
  -> get/valAt (non-field keys only, so a method's own field bindings don't
  recurse), ISeq -> seq/first/rest, IPersistentCollection -> conj, IFn -> the
  value is callable. A jrec is still a map of fields by default; the interface
  method wins when declared.

- Multi-arity inline methods are grouped into one fn (a type with (nth [_ i]) and
  (nth [_ i x]) kept only the last before). Built as data, not a nested
  syntax-quote, so a `(= ~ocr ~l) method body keeps its unquotes.

- instance?/satisfies? recognize a protocol a type implements, including a MARKER
  protocol with no methods (core.match's IPseudoPattern) — deftype/defrecord now
  record protocol satisfaction even with zero methods. Added ILookup/Indexed/
  Counted to the instance? taxonomy for the built-in collections.

- Syntax-quote: a fully-qualified class name (clojure.lang.ILookup) stays bare
  instead of being namespace-qualified; (unquote x) is detected in a lazy seq
  (a macro that builds its template with map, e.g. deftype's rewrite-set).

- clojure.set union/intersection/difference are variadic (& sets) + union 0-arity.
- java map view methods: keySet/values/entrySet/size/isEmpty.
- deprecated java.util.Date getters (getYear/getMonth/...) + the multi-arg
  (Date. year-1900 month0 date hrs min) constructor.

Seed change (deftype/defrecord macros + clojure.set) -> re-minted; the rest are
runtime. 11 JVM-certified corpus rows; make test + shakesmoke green.
2026-06-25 00:14:19 -04:00
Yogthos
7b1ec9a1d3 java.time DST + data readers: make tick pass fully
Shaking out tick's api and alpha.interval suites (api 353->359, interval
0->103 passing) cleared a set of general gaps:

- Named-zone DST. Zones resolved to a fixed representative offset, so
  America/New_York in August read -05:00 not -04:00. Add US/EU DST rules
  (compact transition-date math) and make instant<->zoned, the zone rules'
  getOffset, and the zoned equality arm DST-aware.

- Nanosecond zoned/offset times. Instant is nanos but atZone/atOffset/
  toInstant/withZoneSameInstant and Instant/parse went through epoch-ms,
  truncating sub-ms. Route them through nanos.

- Locale month/day names. A formatter dropped its Locale; carry it and add
  French names so MMM under Locale/FRENCH renders "mai".

- Callable records. A defrecord implementing clojure.lang.IFn (tick's
  GeneralRelation) is now invokable: jolt-invoke dispatches to its inline
  invoke method. Also give collections the Iterable host tag so a protocol
  extended to Iterable matches vectors/seqs.

- Imported class short names. (:import [java.time ZonedDateTime]) then
  (. ZonedDateTime parse s) resolved to nil; an otherwise-unresolved bare
  Capitalized name that's a registered host class now resolves as a class.

- Data readers. A project's data_readers.{clj,cljc} is loaded into
  *data-readers* (reader namespaces required eagerly); registered #tag
  literals in source rewrite to (reader-fn 'form). clojure.core/read-string
  now applies #inst/#uuid/#"regex" and *data-readers* like Clojure.

- Duration/between accepts zoned/offset date-times.

All runtime shims, no re-mint. docs/libraries.md: tick full pass + aero.
2026-06-24 21:30:05 -04:00
Yogthos
21895cb932 Class-name symbols self-evaluate; extends? matches host classes; ISO_INSTANT
A slash-free dotted symbol with a Capitalized final segment (java.util.Map,
clojure.lang.Named, java.time.Instant) now self-evaluates to its name string
instead of resolving to nil — jolt models a class as its name, so a library
can extend a protocol to, or instance?-check, a host class jolt has no shim
for. hc-resolve-global classifies these as :class; the analyzer emits a const.

extends? now matches when either the query or the registered tag is a dotted
suffix of the other, so (extends? P java.util.Collection) finds the impl
extend registered under the canonical short tag.

Add DateTimeFormatter/ISO_INSTANT (UTC, trailing Z).

These unblock loading clojure.data.json, which dispatches JSONWriter on
java.util.Map/Collection/CharSequence/Instant and defaults a formatter to
ISO_INSTANT.
2026-06-24 14:17:34 -04:00
Yogthos
473d1002b7 Reader attaches collection metadata to data, not a with-meta form
The reader lowered ^meta on a vector/map/set literal to a runtime
(with-meta form meta) list, so read-string/edn of data with metadata
returned the form and lost the metadata. Attach it to the value instead,
as Clojure does; the analyzer re-emits (with-meta coll meta) for a
meta-carrying collection literal in code, so a literal still carries its
metadata at runtime and ^Type/^long arglist hints (consumed by
analyze-arity directly) are unaffected.

Also: pr honors *print-meta*, and clojure.walk/clojure.edn re-attach
metadata to the collections they rebuild (matches Clojure; a
metadata-driven config lib like aero relies on it).
2026-06-24 12:04:36 -04:00
Yogthos
79fa22eeab Enable IR inlining: splice small defns at call sites (lever 1/4)
jolt.passes.inline was fully written but dormant — it fetched bodies via the
inline-ir host hook, which was a stub returning nil. Wire it up: run-passes stashes
each inline-eligible defn (single fixed arity) as its form is optimized, and
inline-ir hands the body back at call sites under --opt.

The catch was the ^double/^long coercion: an inlined fn drops its param-entry and
return coercion, so (work 3 4) on a ^double fn would return 25 instead of 25.0. New
:coerce IR node carries the coercion inside the spliced body — the inline pass wraps
a hinted param's arg and the return in :coerce, the back end lowers it
(exact->inexact / jolt->fx), and jolt.passes.numeric reads its :kind. So an inlined
call matches the called one and the body's fl*/fx* fast path still fires.

Only under --opt (closed world); the seed mint and -e don't inline, so selfhost and
the corpus are unaffected. test/chez/inline-test.ss 12/12 (make inline); full make
test green, 0 new corpus divergences.

Bench (hot loop, body is a ^double helper call): direct-link 500ms -> --opt
(inlined) 184ms = 2.7x, by eliminating the call + coercion wrappers and letting Chez
fuse the fl-ops unboxed. ~26x over the default dispatched build.
2026-06-23 17:43:13 -04:00
Yogthos
5a9acd3cf4 Numeric return-type hints: ^double/^long on a defn (round 3)
A ^double/^long return hint on a fn's name now (a) coerces the fn's value on the
way out — exact->inexact / jolt->fx, like a JVM primitive return — and (b) types a
call to it, so an accumulator over the result specializes:

  (defn ^double work [^double x ^double y] (+ (* x x) (* y y)))
  (loop [acc 0.0] (recur (+ acc (work a b))))   ; (+ acc (work ..)) -> fl+

The analyzer pushes the name's numeric tag onto each arity (:ret-nhint) for the
back-end coercion, and resolve-global surfaces the callee's declared return
(:num-ret, read from var meta) onto the :var node so jolt.passes.numeric types the
call. defn carries the name hint through.

This unblocks the accumulator-over-fn-result pattern that round 2 had to demote.
The win is bounded by call overhead in an open/dispatched build (~1.15x on a hot
loop whose body is a helper call); it compounds with direct-linking and, later,
inlining. A numeric return hint is a contract, like ^long — redefining the var to
return another type in an open build breaks it.

Not yet: per-arity arglist return hints, (defn f (^double [x] ..)). Gate:
test/chez/numeric-test.ss 39/39; full make test green, 0 new corpus divergences.
2026-06-23 17:21:53 -04:00
Yogthos
f66925d3a8 jolt build: --opt mode turns on the inference passes (Phase 4 stage 4a)
Wire the optimization gate to build mode. inline-enabled? (which gates the
inference + flatten-lets + scalar-replace passes in jolt.passes/run-passes) was
hardwired off, so those passes had never run on Chez at all. host-contract now
exposes a settable hc-optimize? flag; `jolt build --opt` flips it on during app
emission.

Kept off for the default release build for now: the passes are sound by design
(RFC 0005/0006) but unexercised on Chez, so release stays on the proven
var-deref codegen until they're validated against the corpus. --opt is the
opt-in fast path. buildsmoke checks both modes produce the same result.

This does not yet deliver direct call binding — the backend has no direct-link
emission path (every :var call still routes through jolt-invoke/var-deref) and
the inline-ir host stash is still a stub. Those are the remaining stage-4 levers.
2026-06-22 23:07:04 -04:00
Yogthos
33eff7c7d8 Clean up codebase: rename stdlib layer, strip porting residue, fix tooling
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.

Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.

Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.

Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
2026-06-22 22:18:00 -04:00
Yogthos
ccf93c896a Resolve :use/:only refers, refer-aware syntax-quote, alias-aware resolve
Three related namespace-resolution fixes surfaced porting the clojure.tools.logging
library, all general:

- chez-register-spec! treats a :use :only vector like :require :refer, so a
  (:use [ns :only [names]]) clause actually brings those names in. Before, a bare
  reference to an :only'd name resolved to nil.
- syntax-quote (hc-sq-symbol) qualifies a referred name to its SOURCE namespace,
  not the compile namespace — so a macro that syntax-quotes a referred var (e.g.
  clojure.tools.logging/spy reaching clojure.pprint's pprint) expands correctly.
- resolve consults :as aliases and :refers like ns-resolve does; (resolve
  'alias/name) was returning nil because the alias wasn't expanded.
2026-06-22 10:25:35 -04:00
Yogthos
6ab65a30e3 Fix arg evaluation order + host interop gaps so reitit/selmer/honeysql run
Shaking the ring-app example's real library stack out against jolt surfaced a
batch of divergences from JVM Clojure, the biggest being evaluation order.

backend_scheme: call and recur arguments were emitted as bare Scheme operands,
so Chez's unspecified (right-to-left) order won out. Clojure evaluates left to
right, which selmer's reader loop relies on: (recur (add-node ... rdr) (read-char
rdr)) consumed a char early and dropped the first chars of every {{tag}}. Bind
operands to fresh temps in a let* (only when two or more can have side effects,
so hot calls over locals/consts stay un-wrapped). emit-ordered already did this
for collection literals; generalize it.

host-contract: syntax-quote now resolves the alias part of a qualified symbol
(impl/foo -> clojure.tools.logging.impl/foo) instead of leaving it bare, which
limped along via short-name matching until two loaded namespaces (reitit.impl,
clojure.tools.logging.impl) shared the short name and it broke.

collections: key-hash masks with bitwise-and, not fxand — jolt-hash is set!-
decorated per type (records return their own hash) and Chez's equal-hash can be a
bignum, so a key's hash isn't always a fixnum.

seq: even?/odd? handle bignums (JVM accepts any integer; the fxand crashed).

records: Keyword/Symbol .sym/.getName/.toString (honeysql's :clj branch reads
(.sym k)); Throwable .getMessage/.toString over a Chez condition.

host-static: __register-class-ctor!/__register-class-statics! so a host shim
(reitit.trie-jolt) can mirror a Java class.

natives-str: String.intern returns the string.

sqlite: jdbc.core fetch/fetch-one kebab-case column keys (the jolt-lang/db
convention; created_at -> :created-at).

io: a relative io/file path resolves against JOLT_PWD (the user's cwd), not the
repo root the launcher cd'd to — matches JVM cwd semantics, so config.edn loads.

cli: render an uncaught jolt throw (ex-info message + ex-data, or a condition)
instead of Chez's opaque "non-condition value" dump.
2026-06-22 05:26:09 -04:00
Yogthos
ab96650fbb real BigDecimal type (bigdec, M literals)
bigdec / 1.5M / 0.0M silently produced doubles. Add a jbigdec value type
{unscaled, scale} over Chez exact integers (host/chez/bigdec.ss): value =
unscaled * 10^-scale. An M-suffix literal reads to a :bigdec form that the back
end lowers to jolt-bigdec-from-string (same IR-leaf path as #inst/#uuid); bigdec
coerces a number/string. Equality is by value (1.0M = 1.00M true, 3M = 3 false),
str drops the M and pr keeps it, class is java.math.BigDecimal, decimal? is true.
Arithmetic contagion isn't modelled (out of scope). The old corpus cases passed
spuriously as doubles; they now exercise a genuine BigDecimal.
2026-06-22 00:01:01 -04:00
Yogthos
8ce00d29fd embed a namespace value spliced into a form (~*ns*)
A macro like (defmacro cur-ns [] `(str ~*ns*)) splices the live *ns* value
into its expansion, leaving an opaque jns object as a list element. The
analyzer had no way to carry a runtime value and threw uncompilable — the last
remaining corpus crash. Recognize a jns via the host contract (form-ns-value?)
and emit a :the-ns leaf that reconstructs it by name (intern-ns!) at the call
site, the same IR-leaf pattern as regex/inst/uuid. Closes unquote-*ns*-in-
template; corpus crash count -> 0.

A namespace fast path rather than a general constant pool: it's the only
embedded-value case in the corpus and the common real-world one (libs splice
~*ns*). A general pool can come later if other value types appear.
2026-06-21 23:40:59 -04:00
Yogthos
518683ccd6 syntax-quote leaves interop forms unqualified (jolt-z1zu)
A macro that syntax-quoted interop — `(.. (StringBuilder.) (.append x)) — had
its .method / Class. / .-field heads qualified to the compile ns (user/.append,
user/StringBuilder.), so they read as 'Unknown class user' at expansion. Like
Clojure, leave interop-head symbols bare in syntax-quote. Fixes any macro
templating interop, not just the one corpus case. Corpus 2694->2695.
2026-06-21 17:49:11 -04:00
Yogthos
31a453d492 (. obj :kw) is a keyword lookup (JVM parity)
The . special form rejected a non-symbol member; a keyword member now lowers to
an invoke of the keyword on the target ((. {:value 41} :value) => 41, as on the
JVM). Added a form-keyword? contract seam. Corpus 2692->2693. Re-minted.
2026-06-21 17:13:40 -04:00
Yogthos
547a8c6d17 The .. threading macro analyzes as a macro, not interop (jolt-c2l1 tail)
(.. x m ...) failed: the analyzer classified the .. head as a .method interop
call (method-head? matched any "."-prefixed name) and form-special?
(hc-interop-head?) also flagged it, so it never reached the macro check. Exclude
".." from both (the char after "." being "." means the threading macro, not
.method). Corpus 2690->2691. Re-minted.
2026-06-21 17:08:11 -04:00
Yogthos
48e2ef5910 Scrub dangling Janet references; drop dead Janet-coupled files
Rephrase comments that pointed at deleted Janet files (emit.janet, the seed
sources, 'the Janet back end punts ...') to present-tense descriptions of the
Chez behavior. Comment/docstring-only; the self-host fixpoint is unchanged
(comments don't affect the compiled seed).

Delete five files that were Janet-host shims with no Chez path: clojure.java.io
(provided natively by host/chez/io.ss), and jolt.{nrepl,png,interop,shell}
(the janet.* bridge, os/shell, janet.net — none exist on Chez).

jolt-cf1q.6
2026-06-21 12:01:04 -04:00
Yogthos
52a1ab5970 Chez parity: analyze any seq as a list form (macro/eval-built forms) (jolt-cf1q.7)
hc-list? required cseq-list? (a reader-built list), so a form built at runtime via
concat/map/cons — a lazy cseq with list?=#f — was rejected as "unsupported form".
In Clojure any seq is a valid call form, so accept any cseq. Lights up macros that
build their expansion with concat/list and (eval seq-form).

zero-Janet 2692->2694, 0 new divergences; self-host + Janet gate + JVM cert green.

jolt-cf1q.7
2026-06-20 18:15:30 -04:00
Yogthos
a23095b502 Chez: runtime eval / load-string / defmacro on the zero-Janet spine
The compiler image is already resident at runtime on the Chez spine, so eval
and load-string are just wiring: make them clojure.core functions instead of
analyzer special forms.

- eval / load-string are now functions, not special forms. Dropped "eval" from
  the host-contract special-symbol lists so it resolves as an ordinary var, and
  def-var! both in compile-eval.ss. eval takes an already-read form (e.g. from
  quote/list) and compiles+evals it in the current ns; load-string reads every
  form from a source string and evals each, returning the last.
- Runtime defmacro: jolt-compile-eval-form intercepts a (defmacro ...) form
  before analysis, defs the expander fn + mark-macro!s the var, exactly as
  emit-image.ss does at build time. The two helpers (macro-form? / defmacro->fn)
  move to compile-eval.ss and emit-image.ss reuses them.
- Top-level (do ...) is now unrolled form-by-form, like Clojure, so a defmacro
  or def in an earlier subform is visible (macro flag set / var interned) before
  a later subform is analyzed. This is what makes multi-form -e with a macro work.

Seed is byte-identical (no source references eval), so no re-mint; bootstrap-test
still passes. Zero-Janet corpus 2534 -> 2544 (eval/load-string cases now run),
0 new divergences; floor raised. Prelude corpus, JVM cert, full Janet gate green.

jolt-r8ku
2026-06-20 12:14:08 -04:00
Yogthos
24ef2b8d4b Chez inc7: :refer support — Chez analyzer reaches parity with the oracle
(require '[clojure.string :refer [blank?]]) then an unqualified blank? now
resolves. chez-register-spec! registers :refer names (in addition to :as) into a
refer table; hc-resolve-cell's unqualified branch consults it before clojure.core.

Zero-Janet corpus parity 2293 -> 2295 = the Janet-hosted oracle's exact pass
count. The self-hosted Chez compiler (read -> analyze -> emit -> eval, no Janet)
now compiles every corpus case the Janet-hosted compiler does, with 0
divergences. Remaining failures are shared runtime breadth (host interop,
futures, runtime eval) deferred to Phase 4 / jolt-r8ku. Floor 2295.
2026-06-20 02:06:27 -04:00
Yogthos
011e5d6337 Chez inc7: namespace :as aliases on the Chez-hosted analyzer
(require '[clojure.string :as s]) then s/foo crashed "Unknown class s": the
Chez analyzer resolved a qualified symbol's ns literally, and there was no
alias table (ns.ss jolt-ns-aliases was a stub, require a no-op). Add an alias
table + chez-register-spec! (parses [ns :as a]) in ns.ss; hc-resolve-cell now
resolves a qualified ns through it; compile-eval pre-registers a form's
require/use :as aliases before analysis (analysis precedes the runtime require,
so the runtime require staying a no-op is fine). The batched gate runner clears
the alias table between cases.

Zero-Janet corpus parity 2159 -> 2240, 0 divergences. spine-test 35/35.
2026-06-20 01:25:01 -04:00
Yogthos
8b86174b91 Chez Phase 3 inc7: corpus on the Chez-hosted analyzer + a 50x-faster gate
Point the Chez-HOSTED analyzer at the full parity corpus (read -> analyze ->
emit -> eval, all on Chez, no Janet) and close the divergences so the
self-hosted compiler is faithful: 0 divergences, 2159/2494 pass.

Keystone: the on-Chez emitter ran with prelude-mode off, so every call to a
non-native clojure.core fn tripped the "unsupported stdlib fn" out-of-subset
guard. The zero-Janet spine always has the full prelude loaded, so turn
prelude mode on in compile-eval.ss (22% -> 84% pass on a sample).

Faithfulness fixes (each was the Chez host/reader diverging from the Janet
analyzer; fixed in the keeper, not the seed):
- emit-const read a char's codepoint via (get v :ch) — the Janet rep; on Chez a
  char is native. Route through a new form-char-code host-contract fn (41 cases).
- next over a lazy seq returned the empty-list terminator (truthy), not nil, so
  butlast and other (if (next s) ...) loops ran one step too far — broke
  some->/some->>/cond->>.
- reader: radix literals (2r1010/16rFF/36rZ), #^ deprecated metadata, ^meta on
  collections (lowers to a runtime with-meta form like the Janet reader),
  map-literal source order (values eval left-to-right), and nested syntax-quote
  over a literal collapses at read time.
- keyword "a/b" splits into ns/name like the seed (destructure {:keys [x/y]}).
- form-syntax-quote-lower implemented on Chez (was a throwing stub).

7 divergences allowlisted: the same print-method-multimethod / host-class set
the prelude gate defers. 328 crashes remain = shared runtime breadth (host
interop, missing core fns, eval/load-string) deferred to Phase 4 / jolt-r8ku,
not compiler faithfulness.

Gate + speedup: test/chez/run-corpus-zero-janet.janet (floor 2159). Its batched
runner (driver/eval-corpus-zero-janet) runs every case in ONE chez process —
load the runtime once, guard + reset the user namespace per case — instead of a
fresh process per case: 1379s -> 1.6s.

spine-test 35/35; Janet gate 151/0; prelude parity 2295/2494 unchanged, 0 new
divergences.
2026-06-20 01:11:54 -04:00
Yogthos
d165198969 Chez Phase 3 inc6b: runtime macros on the zero-Janet spine
The on-Chez analyzer (inc6a) skipped macros, so let/when/->/defn didn't
expand from source. Each core/stdlib defmacro now emits into the prelude as
(def-var! ns name <expander fn>) + (mark-macro! ns name); form-macro?/
form-expand-1 on Chez look up the macro flag (rt.ss var-macro-table) and
apply the expander to the unevaluated arg forms, and the analyzer re-analyzes
the result. The expander's syntax-quote template was lowered to construction
code at cross-compile time, so it builds the expansion via __sqcat/__sqvec/
__sqmap/__sqset/__sq1 (new host/chez/syntax-quote.ss) as Chez reader forms.

Emit the bare (fn ...), not (def NAME (fn ...)): analyzing a def would
host-intern! NAME as a non-macro stub in the build ctx, and that stub makes a
later (require '[stdlib-ns]) skip loading the real macro — with-pprint-dispatch
then resolved as a fn and returned its unexpanded template. Wrapping the
lambda in def-var! manually never interns NAME. Fuller build-ctx isolation
(so stdlib cases pass instead of crash) tracked in jolt-lpvi.

__sqset builds a real set VALUE, not the reader's tagged-set form — a runtime
`#{~@xs} must be a set, not a map. form-set? additionally recognizes a pset so
a macro template's #{...} expansion still re-analyzes as a set literal.

spine-test 35/35 (20 macro cases: when/when-not/let/->/->>/and/or/cond/if-not/
defn run zero-Janet from source). Prelude parity 2280->2295, 0 new divergences.
Full Janet gate green.
2026-06-19 23:21:13 -04:00
Yogthos
c2d977cadf Chez Phase 3 inc6a: zero-Janet spine (analyzer + emitter on Chez), macro-free
Cross-compile jolt.ir + jolt.analyzer + jolt.backend-scheme to Scheme def-var!
forms via the existing Janet emit pipeline (driver/emit-compiler-image) and run
them ON CHEZ over a Scheme jolt.host impl. A macro-free Clojure expression now
compiles and runs with no Janet in the loop: read (reader.ss) -> analyze
(jolt.analyzer on Chez) -> IR -> emit (jolt.backend-scheme on Chez) -> eval.

host/chez/host-contract.ss is the jolt.host contract on Chez (the portable seam
the cross-compiled analyzer/emitter call): form-* over the Chez reader's forms,
resolve-global/compile-ns/host-intern!/late-bind? over the var-cell registry. ctx
is an opaque record carrying the compile ns. Native-op names are declare-var!'d
into clojure.core so +, *, <, ... classify as :var and the emitter's native-op
path lowers them. form-macro? is a #f stub and macro expansion / syntax-quote /
record hints are stubs for inc6b (runtime macros, jolt-r8ku).

host/chez/compile-eval.ss is the spine entry (read-string -> analyze -> emit ->
eval). driver gains emit-compiler-image / ensure-compiler-image (image caching)
and program-zero-janet / eval-zero-janet.

Two bugs fixed in the keeper, not reproduced:
- the Chez reader stores an unqualified symbol's ns as #f, but the analyzer tests
  (nil? ns); hc-sym-ns normalizes #f/'() -> jolt-nil. Without it every handled
  special (if/do/fn*) misanalyzed as a plain invoke.
- char (int->char) was missing from clojure.core on Chez; the emitter's
  chez-str-lit needs it for keyword/string consts. Added jolt-char to converters.ss.

Gate: test/chez/spine-test.janet 15/15 (Chez-hosted analyzer value == Janet-hosted
oracle through the same emitter/RT; only the analysis host differs). Full Janet
gate green (150 files). driver.janet is in the prelude fingerprint so the cache key
moved; prelude content is unchanged. jolt-chez fingerprint/ensure-prelude made
public for the test harness.
2026-06-19 21:16:24 -04:00