The on-Chez analyzer (inc6a) skipped macros, so let/when/->/defn didn't
expand from source. Each core/stdlib defmacro now emits into the prelude as
(def-var! ns name <expander fn>) + (mark-macro! ns name); form-macro?/
form-expand-1 on Chez look up the macro flag (rt.ss var-macro-table) and
apply the expander to the unevaluated arg forms, and the analyzer re-analyzes
the result. The expander's syntax-quote template was lowered to construction
code at cross-compile time, so it builds the expansion via __sqcat/__sqvec/
__sqmap/__sqset/__sq1 (new host/chez/syntax-quote.ss) as Chez reader forms.
Emit the bare (fn ...), not (def NAME (fn ...)): analyzing a def would
host-intern! NAME as a non-macro stub in the build ctx, and that stub makes a
later (require '[stdlib-ns]) skip loading the real macro — with-pprint-dispatch
then resolved as a fn and returned its unexpanded template. Wrapping the
lambda in def-var! manually never interns NAME. Fuller build-ctx isolation
(so stdlib cases pass instead of crash) tracked in jolt-lpvi.
__sqset builds a real set VALUE, not the reader's tagged-set form — a runtime
`#{~@xs} must be a set, not a map. form-set? additionally recognizes a pset so
a macro template's #{...} expansion still re-analyzes as a set literal.
spine-test 35/35 (20 macro cases: when/when-not/let/->/->>/and/or/cond/if-not/
defn run zero-Janet from source). Prelude parity 2280->2295, 0 new divergences.
Full Janet gate green.
Cross-compile jolt.ir + jolt.analyzer + jolt.backend-scheme to Scheme def-var!
forms via the existing Janet emit pipeline (driver/emit-compiler-image) and run
them ON CHEZ over a Scheme jolt.host impl. A macro-free Clojure expression now
compiles and runs with no Janet in the loop: read (reader.ss) -> analyze
(jolt.analyzer on Chez) -> IR -> emit (jolt.backend-scheme on Chez) -> eval.
host/chez/host-contract.ss is the jolt.host contract on Chez (the portable seam
the cross-compiled analyzer/emitter call): form-* over the Chez reader's forms,
resolve-global/compile-ns/host-intern!/late-bind? over the var-cell registry. ctx
is an opaque record carrying the compile ns. Native-op names are declare-var!'d
into clojure.core so +, *, <, ... classify as :var and the emitter's native-op
path lowers them. form-macro? is a #f stub and macro expansion / syntax-quote /
record hints are stubs for inc6b (runtime macros, jolt-r8ku).
host/chez/compile-eval.ss is the spine entry (read-string -> analyze -> emit ->
eval). driver gains emit-compiler-image / ensure-compiler-image (image caching)
and program-zero-janet / eval-zero-janet.
Two bugs fixed in the keeper, not reproduced:
- the Chez reader stores an unqualified symbol's ns as #f, but the analyzer tests
(nil? ns); hc-sym-ns normalizes #f/'() -> jolt-nil. Without it every handled
special (if/do/fn*) misanalyzed as a plain invoke.
- char (int->char) was missing from clojure.core on Chez; the emitter's
chez-str-lit needs it for keyword/string consts. Added jolt-char to converters.ss.
Gate: test/chez/spine-test.janet 15/15 (Chez-hosted analyzer value == Janet-hosted
oracle through the same emitter/RT; only the analysis host differs). Full Janet
gate green (150 files). driver.janet is in the prelude fingerprint so the cache key
moved; prelude content is unchanged. jolt-chez fingerprint/ensure-prelude made
public for the test harness.
driver.janet now compiles IR via the portable Clojure emitter (jolt.backend-
scheme) instead of emit.janet, at every entry point (compile-program, emit-core-
prelude, eval-e-with-prelude). The emitter is loaded into the ctx and called like
the analyzer. emit.janet stays only as the emit/program string-wrapper until
program assembly ports to Clojure with compile-from-source; its emit fn is no
longer called anywhere (emit-test's truthy-elision helper now uses the new
d/scheme-emit too). This takes the IR->Scheme emitter off Janet.
A form-by-form diff of the two emitters over the whole prelude found one gap:
emit-const missed char literals because a :jolt/type-tagged struct is not a plain
jolt map? — switched to the form-char? host contract. Diff then 0.
jolt-chez prelude fingerprint now includes backend_scheme.clj + host_iface.janet.
Gate: full prelude corpus 2280/2494, NEW divergence 0, same buckets as the Phase-2
emit.janet floor (36 emit-fail, 170 crash) — the Clojure emitter is byte-for-
behavior identical. emit-test 331/331 (now via the Clojure emitter), emit-parity
58/58. jolt-duot.
Close the remaining Phase-2 stdlib parity gaps.
clojure.set: pure Clojure over core, so just added to the prelude stdlib tier
(driver.janet stdlib-ns-files + jolt-chez fingerprint), same as clojure.edn.
clojure.math: not a .clj on the seed (native math/ bindings via jolt-h79), so
Chez gets its own host/chez/math.ss def-var!ing each fn over Chez native flonum
math. The analyzer already knows the ns (api.janet install-clojure-math!), so
refs lower to var-deref. Also adds the missing 'long' coercion to converters.ss
(int's sibling; several math cases wrap in long).
clojure.pprint: dropped the 2-arity's (binding [*out* writer] ...). *out* isn't
a bindable var in jolt — printing routes through the host (dyn :out) seam, so
the binding never redirected anything; it only made the defn uncompilable, which
the seed tolerated via interpreter fallback. Chez has no fallback, so the whole
pprint defn died. Dropping it is behavior-preserving (writer was always ignored)
and lets pprint compile cleanly. Both corpus cases pass.
Corpus parity 2259 -> 2280, crashes 191 -> 170, 0 new divergences. Floor raised.
New unit test test/chez/_stdlib.janet (27/27). Full Janet gate green (147 files).
Chez-side recursive-descent Clojure reader (host/chez/reader.ss) producing the
same jolt forms the Janet reader yields, behind the read-string / __parse-next /
__read-tagged seams the seed registers in eval_runtime.janet. That lights up the
whole *in* read family — read, read+string, with-in-str (read) — plus read-string
and read-string metadata, none of which needed an analyzer change (read-string is
a clojure.core seam, jolt-nil on the prelude until now).
Reader output is pinned to the Janet reader's shapes: numbers coerce to flonum
(the all-double model emit-const uses, else a read int isn't = a source int),
sets read as the {:jolt/type :jolt/set :value [...]} FORM, #tag/#inst/#uuid/#regex
as tagged forms (no data reader applied — read-string never evaluates), ^meta on
a symbol, and the ' ` ~ ~@ @ reader macros. clojure.edn is added to the prelude
tier; its edn->value builds the real set/tagged values and __read-tagged reuses
the inc X #inst/#uuid constructors. The reader-arity edn/read stays a lazy gap
(drain-reader is Janet-coupled) — read-string is the live path.
eval / load-string / runtime defmacro are still out: they need the compiler at
runtime, which is Phase 3 (self-host). Chez-only change, no Janet gate.
Parity 2238 -> 2259, 0 new divergences. _reader 47/47; all chez unit tests green;
emit-test 331/331.
The analyzer lowers a #inst/#uuid tagged form to a :inst/:uuid IR leaf, mirroring
the existing :regex node: the Janet back end punts to the interpreter (its
data-readers parse the literal, so seed behavior is unchanged), the Chez back end
emits jolt-inst-from-string / jolt-uuid-from-string.
host/chez/inst-time.ss is the Chez-native value layer: a jinst record holding
epoch ms (RFC3339 parsed via Hinnant civil/days math, with Clojure's partial
defaults and +/-hh:mm offsets), wired into jolt-get (so the overlay inst?/inst-ms
read it), jolt= / jolt-hash (instant identity as a map key), pr-str (#inst
"...-00:00"), str, type, and instance? java.util.Date. The java.time surface
(DateTimeFormatter ofPattern/ISO_LOCAL_DATE_TIME/ofLocalized*, the pattern engine,
Instant, ZoneId, LocalDateTime, FormatStyle, Locale, Date) ports java_base.janet
over host-static.ss's registries.
Corpus 2202->2238, 0 new divergences; clears the whole 'unsupported form'
emit-fail bucket. Full Janet gate green (analyzer/backend changes are
behaviour-preserving — #inst still parses through the interpreter's data-readers
on the seed).
A File is a path-backed jfile record: (instance? java.io.File f) is true,
str/slurp coerce it to its path, and the File method surface (getName/
getPath/exists/isDirectory/isFile/listFiles/getParent) dispatches through
record-method-dispatch. slurp/spit/flush run over Chez's filesystem
primitives; file-seq's dir primitives (__file?/__dir?/__list-dir) and the
overlay's File branch (.isDirectory/.listFiles, which emit to jolt-host-call)
are jfile-aware. clojure.java.io/file + as-file are def-var!'d natively.
New host/chez/io.ss, a Chez-native implementation -- the seed's
clojure.java.io (io.clj) is a Janet-backed shim over janet.*/janet.file, so
it can't be reused. The analyzer resolves io/file because the seed ctx has
clojure.java.io loaded; only a runtime def-var! is needed. type/instance-
check/str-render/jolt-host-call are extended via the set!-wrap pattern (type
also re-def-var!'d since the var cell captured the old value).
Reader/StringReader-coupled io (io/reader, line-seq over a file, .toURL,
slurp over a reader) deferred to jolt-at0a.
Parity 2176 -> 2191, 0 new divergences. New test/chez/_io.janet 20/20.
list? was nil on Chez because one cseq record backs both lists and lazy/
realized seqs. Add a list? marker field (cseq v2) set only on the HEAD cell
of a list -- (list ...), quoted list literals, cons, reverse, conj onto a
list. rest/next/seq/map therefore yield unmarked seq cells, so they are
seqs and not list?, matching the seed (where rest-of-a-list is a non-list
seq). Empty () is treated as a list.
vector?: drop the map-entry exclusion. Clojure's MapEntry implements
IPersistentVector and the seed agrees -- (vector? (first {:a 1})) is true.
Only dot-forms' coll dispatch read jolt-vector?, where a 2-vector entry is
correct.
clojure.walk + clojure.template join the prelude stdlib tier. The driver
now evals each stdlib ns's requires -- and the ns form's (:require ...)
clause -- so an aliased ref (template's walk/postwalk-replace) resolves at
emit time instead of lowering to an Unknown class host-static. ns forms are
evaled for that side effect but not emitted, so the runtime *ns* doesn't
leak to the last stdlib ns.
Parity 2163 -> 2176, 0 new divergences. New test/chez/_walk.janet 39/39.
Bare class names (String, Keyword, File...) evaluate to their JVM
canonical-name string, the same value (class x) returns, so
(= String (class "x")) holds and (defmethod m String ...) keys match a
(class ...) dispatch. New host/chez/host-class.ss ports
eval_resolve.janet's class-canonical-names + core_refs.janet's core-class
(scalar arms; collections/seqs are host-taxonomy-dependent and not class-
compared in the corpus).
The analyzer already resolves these names to clojure.core vars (the seed
ctx interns them via setup-class-ctors), so the back end emits
(var-deref "clojure.core" "String") and a runtime def-var! is all that's
needed -- no analyzer change, Janet path untouched. The class native MUST
land together with token resolution: alone it turns the bare-token corpus
cases (562/564) into divergences (this bit last session).
Parity 2154 -> 2163 (cases 560/562/563/564/2500-2503), 0 new divergences.
New test/chez/_class.janet 19/19.
The analyzer lowers the `.` special form (. target member arg*) and the
.-field field-access head to a :host-call instead of leaving them
uncompilable. Janet behaviour is unchanged — its back end punts :host-call
to the interpreter, which re-runs the original `.` form via eval-dot.
The Chez back end routes a non-shimmed :host-call through
record-method-dispatch, extended by a new host/chez/dot-forms.ss with the
arms dispatch-member covers but the record/string base did not, mirroring
src/jolt/interop/collections.janet precedence:
- collection interop first (count/seq/nth/get/valAt/containsKey on a
vector/map/set), so (. {:count 9} count) is the entry count like the seed
- field access for a "-name" member (records and maps)
- the seed's universal object-methods (getMessage/getCause/toString/
hashCode/equals) on a non-record map, winning over a field lookup
- non-record map member: a stored fn is a method called with self, else
the field value
Raw seqs are excluded from coll interop — the seed's behaviour there is
representation-dependent (plain (seq v) vs a lazy-seq) and a normalized cseq
can't mirror it. Also added getMessage/getLocalizedMessage/equals to the
string method surface so a thrown string / Exception. ctor (which keeps the
message string) answers .getMessage.
Parity 2134 -> 2150, 0 new divergences. New test/chez/_dotform.janet 26/26;
emit-test 331/331.
Lower host class interop on the Chez back end. The analyzer now turns a
non-var qualified ref `Class/member` into a :host-static node and a
`(Class. ...)` / `(new Class ...)` form into a :host-new node (ir.clj
gains both, with walker support). The Janet back end punts both to the
interpreter, so its behavior is unchanged (verified: dot-form, `..`
threading, shadowed `new`, and all interop still resolve via fallback).
The Chez emit lowers a value ref to host-static-ref, a call head to
host-static-call, and a constructor to host-new. host/chez/host-static.ss
is the runtime registry these resolve against — the Chez port of the
seed's class-statics / class-ctors / tagged-methods (java_base.janet +
host_io.janet), restricted to the java.lang/util/net/io surface portable
cljc code calls: Math, System (getenv/getProperty/exit/currentTimeMillis),
Long, Integer, Boolean, Character, String, Thread, Class, Pattern
(compile/quote/MULTILINE), URLEncoder/Decoder, Base64, the Number method
surface (byteValue/intValue/...), plus the StringBuilder, StringWriter,
StringReader, PushbackReader, HashMap, StringTokenizer, BigInteger,
String, MapEntry, and exception constructors. Constructed objects are
jhost records dispatched through record-method-dispatch.
Also: emit now evaluates collection-literal elements left-to-right
(emit-ordered) — Chez evaluates call args right-to-left, which had been
swapping side-effecting elements in [(read r) (read r)] and map literals.
This un-allowlisted the 6 eval-order corpus cases (the read-line trio +
the three map-construction cases). Removed `.write` from the
jolt-host-call fast-path so a StringWriter routes through dispatch.
java.time formatting, edn/read-over-readers, and slurp/with-open over
readers are deferred to a follow-up.
Corpus parity 2078 -> 2134 (floor raised), 0 new divergences; the
print-method builtin-override case is allowlisted (same multimethod gap,
newly reachable now that StringWriter constructs). emit-test 326/326,
_javastatic 51/51, conformance 355x3, full jpm test green.
Bring the clojure.string namespace up on Chez so aliased refs like s/split,
s/upper-case, s/join, s/replace resolve and run.
Three pieces. (1) The Chez AOT driver analyzes the whole user form before any
require runs, so a (require '[clojure.string :as s]) never registered the
alias in time; eval-e-with-prelude now recursively pre-evals require/use forms
against the ctx, which loads the aliased ns and registers the alias so the
analyzer resolves s/X to a clojure.string var. (2) emit-core-prelude emits
stdlib namespaces (clojure.string) as their own def-var! tier through the same
analyze->emit pipeline, so the runtime var-deref resolves. (3) natives-str.ss
def-var!s the str-* primitives clojure.string.clj is written over (upper/lower/
trim/triml/trimr/find/reverse-b/join/split/replace/replace-all), plus no-op
require/use. Regex split keeps interior empties and honors the limit (ported
the seed re-split); regex replace does $N backref expansion and fn replacement
(ported replacement-for). new RT/clj files added to the prelude fingerprint.
Corpus prelude floor 2026 -> 2078 (+52), 0 new divergences. _strns 28/28 vs
build/jolt. Four previously-CRASHING cases now emit+run and surface pre-existing
gaps (read-line vector eval-order x3, instance? clojure.lang.Atom) — allowlisted
with notes. full jpm test + conformance x3 green.
Port the java.lang.String/CharSequence method surface to the Chez RT so
(.toUpperCase s), (.substring s a b), (.indexOf s x), the regex methods
(.matches/.replaceAll/.replaceFirst/.split), etc. run on a string target.
natives-str.ss holds jolt-string-method, ported from the seed's surface in
eval_resolve.janet: ASCII case mapping (byte-oriented like the seed), -1 on
indexOf miss, flonum numeric returns to match jolt's number model, Scheme
chars for charAt, and the regex methods over the irregex compiled via
jolt-re-pattern. record-method-dispatch gains a string? arm that falls
through to it (unsupported methods still throw).
Corpus prelude floor 2002 -> 2026 (+24), 0 new divergences. _str 27/27 vs
build/jolt; full jpm test + conformance x3 green.
The (. x m) dot-form (the . special form, distinct from .method sugar) and
the clojure.string namespace (needs prelude plumbing + Pattern) stay deferred.
Capture a def's reader metadata on the Chez var. The :def emit now lowers a def
with non-empty metadata to def-var-with-meta!, which stores the user meta
(^:private / ^Type tag / docstring -> {:doc}) in an eq side-table keyed by the
var-cell. jolt-meta of a var-cell merges that onto {:ns :name} derived from the
cell, so every var reports {:ns :name} like Clojure with the def-time meta
layered on. (^{:map} metadata on a def name stays uncompilable for the compiler
generally — analyzer rejects it, the Janet back end punts to its interpreter,
which Chez lacks — so it's out of subset, not a meta-capture gap.)
Added natives-meta.ss to the prelude-cache fingerprint. Prelude parity
1969 -> 1972, 0 new divergences; the three var-metadata allowlist entries
(^:private / ^Type tag / docstring) dropped. New focused gate
test/chez/_var_meta.janet.
Reimplement the ctx-coupled seed ns natives over the rt.ss var-table, since
Chez has no ctx. host/chez/ns.ss adds a jns namespace value + a registry and
binds find-ns/the-ns/create-ns/in-ns/all-ns/ns-publics/ns-map/ns-interns/
ns-aliases/resolve/find-var/ns-unmap/*ns* into clojure.core.
The resolve friction: native-ops (+, map, …) are inlined at emit so they have
no var-cell, and (resolve '+) was nil — diverging from Clojure where it's a
var. Added a defined? flag to the var-cell record (set by def-var!/declare-var!,
left false on a lazily-materialised forward ref) and def-var!'d every native-op
name to its value-position proc, so resolve returns the cell iff genuinely
defined. ns-unmap clears the flag. resolve never interns an empty cell
(var-cell-lookup is non-creating).
ns-name is overridden natively in post-prelude (the overlay reads
(get ns :name), nil on a jns record); the printers render a namespace as its
name. *ns* binds to the user ns; in-ns re-binds it. use/require cross-ns
switching stays deferred to Phase 3 (the analyzer bakes a def's target ns at
compile time).
Prelude parity 1951 -> 1969, 0 new divergences; four now-passing allowlist
entries dropped (ns *ns* cases + str-of-a-var). New focused gate test/chez/
_ns.janet (19 cases, expectations from the JVM-canonical build/jolt).
host/chez/multimethods.ss implements the multimethod runtime: defmulti/defmethod
expand to defmulti-setup/defmethod-setup calls (+ get-method/methods/
remove-method/prefer-method/prefers). A jolt-multifn record carries its dispatch
fn and a jolt=-keyed method table; jolt-invoke dispatches it (exact match, then
isa?/hierarchy with prefer-method, then :default), reusing the overlay's
isa?/derive/make-hierarchy. The multifn's ns comes from a runtime chez-current-ns
(default user; the prelude load sets clojure.core for print-method/print-dup).
Two emit-side changes were needed:
- late-bind (:late-bind-unresolved? ctx flag, default OFF): defmulti expands to a
bare-symbol setup call, so the analyzer doesn't intern the name and a forward
reference '(area ...)' after '(defmulti area ...)' in one form was 'Unable to
resolve symbol'. The strict compiler punts these to the interpreter; the Chez
back end has none, so the flag lowers an unresolved symbol to a var-ref against
the compile ns (open-world -e semantics). Set only by the Chez make-ctx /
jolt-chez; the main compiler keeps strict resolution (host_iface late-bind?
defaults nil).
- a :var call head now routes through jolt-invoke, since a late-bound var can hold
a multifn (or keyword/coll IFn), not just a procedure. Transparent for
procedures; the hot self-recursive call is a :local known-proc, stays direct.
Class-based dispatch ((class x)/String) deferred (needs deftype/class subsystem).
Parity 1506 -> 1530/2497, 0 new divergences. emit-test 302/302. Full janet gate
green (the analyzer flag is off there; suite flakiness under parallel load only).
Emit every non-macro clojure.core form through the live analyzer -> Chez emit
pipeline as a def-var! prelude (prelude mode, tier dependency order), load it
before a user expression, and you get an -e-capable jolt-chez: analysis on Janet,
execution on Chez. driver/emit-core-prelude assembles it (each form behind a
silent load guard so the Phase-2 multimethod forms don't break the rest);
bin/jolt-chez is the -e CLI, caching the prelude on disk keyed by source hash.
run-corpus-prelude.janet is the full parity gate this opens, the prelude-backed
sibling of run-corpus-chez. First baseline: 1220/2497 evaluated cases pass, 0 new
divergences (10 allowlisted: dynamic vars, class names, eval-order — deferred
Phase 2). The rest is the punch-list: ~360 emit-fail (real host interop, out of
the analyzer subset) and ~900 runtime crashes, mostly core fns calling
host-coupled seed natives with no Chez shim yet (str/format/vec, transients).
Follow-ups jolt-t6cr/kl2l/q3w8/9ls5.
Two shims landed to get the prelude to load and run. atoms.ss: atom/deref/swap!/
reset! (+ the compare/vals kernel) — needed at load time for
global-hierarchy = (atom (make-hierarchy)). predicates.ss: the type predicates +
name/namespace/boolean the overlay assumes are seed natives, matching the seed's
strict semantics. post-prelude.ss re-asserts char?/atom? after the prelude: the
overlay implements those by reading :jolt/type, which is false for Chez-native
chars/atoms, so its def-var! would clobber the correct native versions.
Per-case Scheme files are PID-unique so a foreground -e never reads a half-written
file while the gate runs.
