Compare commits

...
Sign in to create a new pull request.

854 commits

Author SHA1 Message Date
0598f8e7c5 Allowed the boot script to check multiple names for the chez executable.
Some checks failed
tests / test (push) Has been cancelled
2026-07-06 17:09:51 +01:00
Dmitri Sotnikov
855fbc4794
Merge pull request #306 from jolt-lang/trace-source-lines
JOLT_TRACE: map tail-frame history to ns/name (file:line)
2026-07-04 21:20:16 +00:00
Yogthos
6c88198115 JOLT_TRACE: map tail-frame history to ns/name (file:line)
The eval path recorded only a frame's munged name, so a JOLT_TRACE backtrace was
a list of bare names. Register source for a runtime-compiled fn def when tracing
is on (keyed by the same munged name the entry push records), reusing the
source-registry the renderer already maps to "ns/name (file:line)". Direct-link
builds already registered via emit-def-cached; this covers the open-world eval
path. trace-off output is byte-identical (returns "" — seed mint / `jolt build`
unchanged), seed re-minted. A name shared across namespaces (e.g. -main) stays
bare, the existing ambiguity guard.

smoke asserts a file-backed project run maps a frame to ns/name (file:line).
2026-07-04 17:09:44 -04:00
Dmitri Sotnikov
e297a74501
Merge pull request #305 from jolt-lang/fix-jolt-trace-aot-binary
JOLT_TRACE: honor the env at runtime in a built joltc
2026-07-04 20:51:03 +00:00
Yogthos
c8167e1c05 JOLT_TRACE: honor the env at runtime in a built joltc
The JOLT_TRACE opt-in was a top-level form in compile-eval.ss, so in a
self-contained joltc it ran at heap-build time — where JOLT_TRACE is always
unset — and never at runtime. `JOLT_TRACE=1 joltc -M:run` therefore produced no
trace from the distributed binary (it worked only under the source-loaded dev
launcher). REPL/nREPL tracing was unaffected (those enable at runtime).

Make it a jolt-trace-init-from-env! fn called from the runtime entrypoints — the
cli.ss dispatch and the built-joltc launcher — before any app namespace compiles,
so the app's own code is traced. While here, drop a redundant trace print in the
joltc launcher (jolt-report-throwable already emits it) that double-printed the
block once tracing actually produced one.

joltc-selfbuild-smoke asserts JOLT_TRACE=1 through the built binary yields exactly
one tail-frame trace.
2026-07-04 16:40:14 -04:00
Dmitri Sotnikov
bff1c288b0
Merge pull request #304 from jolt-lang/tail-frame-history
Recover TCO-elided frames in uncaught-error stack traces
2026-07-04 20:01:51 +00:00
Yogthos
94d3bcca20 AOT: run -main with *ns* = user, matching clojure.main
A built binary loaded each namespace with (set-chez-ns! <ns>) and no restore, so
-main ran with *ns* left at the entry ns. clojure.main (and interpreted joltc)
run -main with *ns* = user, where a runtime (resolve 'alias/sym) is nil because
the alias lives in the entry ns, not user. Reset the current ns to user in the
launcher before -main so a compiled binary matches. build-smoke asserts it via a
separate two-namespace app (kept apart from the tree-shake app — a `resolve`
defeats tree-shaking).
2026-07-04 15:51:13 -04:00
Yogthos
79002526bb JOLT_TRACE: one case-insensitive off-check for both enable paths
Review turned up that the disable vocabulary was the exact lowercase strings
"0"/"false"/"no", so JOLT_TRACE=off (or FALSE, No, n) fell through and ENABLED
tracing — the opposite of intent — and the whole-run and dev-mode checks
disagreed on the empty string. Fold both into one jolt-trace-env-off? predicate
(case-insensitive, incl. off/n); empty/unset carries no signal (dev still traces,
a whole run still doesn't).
2026-07-04 15:51:13 -04:00
Yogthos
7167af4830 Trace by default in REPL-driven development
A repl or nREPL session now turns tail-frame tracing on, so an uncaught error in
evaluated/reloaded code shows a tail-frame backtrace with no JOLT_TRACE set. The
REPL and nREPL catch errors themselves rather than going through the uncaught
reporter, so they now print the history backtrace via a new jolt.host/backtrace-
string (history-only — the live continuation in a REPL is just REPL machinery).

Because the recording is baked in at compile time, only code compiled while a
session is live is traced; reload a namespace to trace already-loaded code.
JOLT_TRACE=1 still forces it on for a whole run (a plain -M:run traces its own
load); JOLT_TRACE=0 forces it off even in a session.

No seed change — jolt.main/jolt.nrepl are runtime-loaded and compile-eval.ss /
source-registry.ss are host files.
2026-07-04 15:23:17 -04:00
Yogthos
a3e2365217 Recover TCO-elided frames in uncaught-error stack traces
On the eval path nothing registers a source map, so jolt-backtrace-string
dropped every walkable frame and printed no trace at all. Keep any named,
non-plumbing continuation frame (rendered as a bare name when unmapped) so a
runtime error shows the surviving non-tail spine — "print what is available".

Add an opt-in tail-frame history behind JOLT_TRACE for the frames TCO erases.
Each compiled fn records itself on entry into a bounded ring-of-rings, MIT
Scheme's "history" shape: the outer ring holds one rib per non-tail subproblem,
each rib a small inner ring of the tail-calls made at that level. A tight tail
loop churns one rib instead of flushing the spine, so the non-tail caller
context survives and total space stays bounded. The reporter prefers this
history over the continuation when it's present, and resets it per top-level
form so an error's trace isn't padded with earlier REPL frames.

The emitter marks a tail call with (jolt-trace-mark! #t) so the runtime routes
the callee into the current rib vs a fresh one; a *tail?* dynamic var tracks
tail position (cleared by default, passed through if/do/let/loop/fn-body). It's
all gated on trace-frames?, which compile-eval turns on for JOLT_TRACE and
emit-image/`jolt build` force off — so non-trace emitted output is byte-identical
(prelude unchanged, seed re-minted), and a built binary carries no per-call cost.
2026-07-04 15:00:52 -04:00
Dmitri Sotnikov
773e647b4a
Merge pull request #303 from jolt-lang/clojure-1.13-parity
Clojure 1.13 parity + no-main build fix
2026-07-04 14:56:22 +00:00
Yogthos
dbc5afac32 build: a no-main entry namespace runs as a script instead of crashing
jolt build -m ns on a namespace with only top-level side effects and no
-main produced a binary that printed its output, then crashed calling a nil
-main ("nil cannot be cast to IFn"). The launcher now calls -main only when
the entry ns actually defines one; otherwise the top-level forms (already run
at heap build) are the whole program and it exits cleanly. Regression case
added to build-smoke.sh.
2026-07-04 10:18:57 -04:00
Yogthos
403c3f302f Clojure 1.13 parity: req!, checked-keys destructuring, keyword array maps
Bring the language up to the 1.13.0-alpha1 changes that apply off the JVM:

- req! (CLJ-2949): a get-variant that throws "Expected key: k" on a missing
  key, without nil-punning. The primitive behind checked destructuring.
- Checked-keys destructuring (CLJ-2961): :keys!/:syms!/:strs! bind and throw
  when a key is absent; keys after & are declared-only (required for the !
  variants, accepted otherwise) and create no binding.
- & is no longer a legal local binding in let/loop (CLJ-2954).
- Keyword-only array maps grow to 64 entries before going hash (was 8),
  across the literal, assoc, and transient paths, so the common keyword map
  keeps insertion order up to 64.

Skipped CLJ-2891 (JVM __init bytecode, JVM-only). 1.13 is still alpha, so
this tracks alpha1 and may shift. Regression tests in test/chez/unit.edn
(ahead of the JVM 1.12.5 the corpus certifies against). Seed re-minted.
2026-07-04 10:18:51 -04:00
Dmitri Sotnikov
1ed9656a0c
Merge pull request #302 from jolt-lang/windows-nrepl-and-version
Fix nREPL on Windows; add a version string
2026-07-04 04:36:47 +00:00
Yogthos
f9fcbc37fd Skip org.clojure/clojure in deps.edn without warning
jolt is Clojure, so a dep on org.clojure/clojure is always satisfied
intrinsically — the "skipping unsupported coordinate" warning on its
:mvn/version coordinate was just noise. Other unsupported mvn coords
still warn.
2026-07-04 00:26:07 -04:00
Yogthos
4398e2cf6c Fix nREPL on Windows; add a version string
The nREPL server bound its loopback socket through libc process symbols,
which don't carry the socket entry points on Windows — they live in
ws2_32.dll and aren't in joltc.exe's export table, so --nrepl-server died
with "no entry for socket". Load ws2_32 before the foreign bindings there,
call WSAStartup once, and use Winsock's closesocket and int-typed
recv/send. Also fix SOL_SOCKET/SO_REUSEADDR, which the old macos-only
check got wrong on Windows.

Bake a version string into the self-contained binary at build time (from
$JOLT_VERSION, else git describe) and expose it via jolt.host/jolt-version:
--version / -V print it, and it shows in --help, the repl banner, and the
nREPL startup line. Dev runs off bin/joltc read it from git describe.

Add -e to the help output.
2026-07-04 00:11:25 -04:00
Yogthos
5467c1d98d Fail actionably when vendor submodules are missing
A user downloaded the auto-generated 'Source code' zip from the release
(no submodules) and hit the raw 'load failed for vendor/irregex/irregex.scm'.
cli.ss and make now check for vendor/irregex up front and print the fix
(clone --recurse-submodules / git submodule update --init --recursive);
README documents both and warns that GitHub's source archives can't build.
Release notes updated with the same pointer.
2026-07-02 19:08:41 -04:00
Yogthos
dbc4298c0a Export joltc.exe symbols so foreign-entry finds the embedded bundles
The -e path worked but jolt build died: jolt_petite_boot_len wasn't
foreign-entry-visible. -rdynamic's Windows equivalent for an exe is an
export table; -Wl,--export-all-symbols provides one.
2026-07-02 18:47:40 -04:00
Yogthos
af12f77dcd Resolve optional libc entries at runtime, not boot load
A literal (foreign-procedure "chmod" ...) in compiled code becomes a fasl
relocation resolved when the boot loads — on Windows (no chmod/sigemptyset/
sigaddset in the CRT) that killed joltc.exe before any guard could run
(msvcrt abort, exit 3). jolt-foreign-proc-safe defers the lookup through
eval at evaluation time, where the guard works and a missing entry just
yields the fallback. chmod also skips the /bin/sh fallback on nt (execute
is by extension).
2026-07-02 18:36:40 -04:00
Yogthos
225073a11b Windows: static link (single-file exe) + a binary inspection step
The built joltc.exe exited 3 (msvcrt abort) with no output on the -e smoke.
Link -static so the exe carries no libwinpthread/libgcc/lz4 DLL deps (needed
for distribution regardless), and add an ntldd + direct-run debug step.
2026-07-02 18:22:53 -04:00
Yogthos
9382c67e48 Windows link set gains -luuid (FOLDERID_* GUIDs) 2026-07-02 15:43:27 -04:00
Yogthos
24c2280246 Route build shell commands through sh on Windows
Chez's system/process use cmd.exe on nt; every build command here is
written for sh. bld-sh-wrap spills the command to a temp script and runs
sh on it (no cmd quoting), identity on other platforms.
2026-07-02 15:33:11 -04:00
Yogthos
0afd2095e3 msys2 inherits the runner PATH (GITHUB_PATH additions were invisible) 2026-07-02 15:21:32 -04:00
Yogthos
c63a18aae1 Windows Chez: skip make install, assemble the csv layout from the build tree
zuo's install target shells the unix installsh through cmd and dies. The
build tree already has everything: scheme.exe (boot files beside it, where
the Windows kernel looks), and scheme.h/libkernel.a/boots into a csv dir
that JOLT_CHEZ_CSV points the jolt build at.
2026-07-02 15:10:36 -04:00
Yogthos
80f3206a6e Fix release.yml: literal newlines inside printf broke the YAML
The Windows PATH step embedded real newlines in its printf strings, which
broke the block scalar and invalidated the whole workflow file (the push
run failed at parse; workflow_dispatch refused). The wrappers are plain
echo pairs now.
2026-07-02 14:59:51 -04:00
Dmitri Sotnikov
7add315394
Merge pull request #301 from jolt-lang/release/windows-v011
Windows release binaries (x86_64) via MSYS2/MinGW
2026-07-02 18:57:46 +00:00
Yogthos
5ca7437826 Restore the :portability tags lost in the rebase 2026-07-02 14:46:13 -04:00
Yogthos
a67dbdb93d rand-nth follows the reference shape; refresh doc counts and the corpus floor
rand-nth vec'd its argument, so (rand-nth nil) hit index-out-of-bounds
through the empty vector where the reference's (nth coll (rand-int (count
coll))) returns nil (and a set throws) — the last genuinely-fixable row in
the suite baseline; rand-nth is fully clean now. Corpus/README counts
updated to the current ~3570 rows, the run-corpus regression floor raised
from 2730 to 3390 to match current parity, and the stale traceability line
dropped.
2026-07-02 14:46:13 -04:00
Yogthos
ab10e68218 Retry the Clojure installer download in CI
A transient CDN timeout handed bash a 2-minute HTML error page instead of
linux-install.sh and failed the run. curl now fails on HTTP errors and
retries, and the script is sanity-checked before running.
2026-07-02 14:46:00 -04:00
Yogthos
b2aa757af2 Windows release binaries (x86_64) via MSYS2/MinGW
Adds a windows-latest job to the release matrix: MSYS2/MinGW-w64 toolchain,
Chez 10.4.1 built from source (ta6nt), the same build-joltc flow, packaged
as a zip of joltc.exe. The whole job runs in the msys2 shell so cc/xxd/paths
behave; the produced binary is a plain Windows executable.

Platform seams: bld-nt? with the winsock/COM/registry link set, no -rdynamic
under MinGW, GetModuleFileName as the launcher's self-path on _WIN32, and
built binaries (joltc itself and jolt-build outputs) normalize to a .exe
suffix. workflow_dispatch added so the matrix can be dry-run without tagging;
the release upload step only fires on tags.

For #205.
2026-07-02 14:46:00 -04:00
Dmitri Sotnikov
58ef0c8fa1
Merge pull request #300 from jolt-lang/corpus/portability-key
Tag every corpus row with :portability (:common vs :jvm)
2026-07-02 18:15:01 +00:00
Yogthos
d6d11d5748 Tag every corpus row with :portability (:common vs :jvm)
Dialects without JVM interop can now filter the conformance corpus: :jvm
marks rows exercising host interop (java.*/clojure.lang.* class references,
dot forms, ctors, statics, arrays, proxy/bean), :common is portable Clojure
any dialect must satisfy. 3565 rows tagged (3175 common / 390 jvm), the key
documented in the row schema, and regen-corpus preserves it on rewrite.

Closes #289.
2026-07-02 14:03:47 -04:00
Dmitri Sotnikov
7b4369145d
Merge pull request #299 from jolt-lang/docs/superset-traceability
Contract fixes from the baseline audit; every residual suite failure traced
2026-07-02 18:03:15 +00:00
Yogthos
ce8e89ca86 Contract fixes from the baseline audit; every residual suite failure traced
Auditing the remaining cts baseline for R7 exposed real contract gaps hiding
among the model residue — all fixed to reference behavior:

- stale no-ratio-era stubs: numerator/denominator now work over jolt's exact
  rationals (non-ratio is the Ratio cast failure); rational? includes decimals
- casts and pending: peek/pop demand an IPersistentStack (pop nil is nil),
  realized? demands an IPending (a plain list/range throws), transient demands
  an editable COLLECTION (non-colls throw; the RFC 0003 sorted/list/seq
  superset keeps the copy-on-write fallback), empty on a plain record throws
- nil and empties: (nth nil i) is nil, (nth nil i d) is d, a nil index is NPE,
  keys/vals of anything empty are nil, (conj nil) is nil
- lookups: contains? on a string is index-only (other keys IAE), get on an
  array is lenient (nth still throws), a VECTOR invocation has nth semantics
  (([1 2] 5) throws — call position and jolt-invoke both)
- into only transients editable collections; a PersistentQueue/sorted target
  folds through conj (RT's IEditableCollection split)
- numbers: number?/num accept BigDecimal, quot/rem throw on an Infinite/NaN
  quotient, even?/odd? demand integers
- ordering: keywords compare namespace-first with nil first (Symbol.compareTo)
- misc: run! honors reduced, eval self-evaluates non-form values, intern
  demands an existing namespace, counted? excludes strings, seqable? includes
  arrays, shuffle rejects maps, sort-by rejects a collection comparator,
  when-let demands one binding pair, case*/deftype*/letfn*/reify*/& are
  special symbols

Two mis-certified corpus rows fixed (they threw on the JVM too and hid in the
tolerated bucket): a raw \d string escape and duplicate literal map keys.

SPEC.md gains the baseline-traceability section: every one of the 146
remaining suite failures maps to a documented divergence (integer-box,
no-single-float, RFC 0003 transients, seq/chunking model, stm-refs,
parse-uuid strictness, vec-array adoption). cts baseline 5955 -> 6042 pass,
5 errors, 30 namespaces. 9 JVM-certified corpus rows.
2026-07-02 13:52:59 -04:00
Dmitri Sotnikov
3fb8082802
Merge pull request #298 from jolt-lang/edn/strict-reader
Strict reader tokens; edn mode with the reference's error contracts
2026-07-02 17:22:54 +00:00
Yogthos
44d4875a24 Strict reader tokens; edn mode with the reference's error contracts
The reader now rejects what the JVM reader rejects: a token that starts
like a number but doesn't parse is NumberFormatException (1a, 08, 0x2g,
2r2 — never a symbol); ratio parts are digit runs (1/-1 invalid) with a
zero denominator throwing ArithmeticException; empty ns/name parts are
invalid tokens (:, ::, foo/, /foo) while /, ns//, and :/ stay valid;
duplicate map keys and set elements throw at read; unsupported string
escapes and octal escapes past \377 throw; a stray close delimiter is
'Unmatched delimiter'; \r ends line comments. #inst validates its
calendar fields progressively (leap years included) and #uuid demands
canonical hex. 1-arg symbol splits its ns at the FIRST slash
(Symbol.intern): (symbol "foo/bar/baz") is foo/"bar/baz".

clojure.edn gets its own strict seam (__read-form-edn): auto-resolved
keywords are invalid there, every #_ discarded form validates through the
same :readers/:default pipeline (an unreadable tagged element throws even
when discarded), built-in tags win over :default, M literals construct
BigDecimals, lists satisfy list?, and EOF honors :eof — an opts map
without :eof makes end-of-input an error.

clojure.edn-test.read-string goes 246 pass / 46 fail / 5 errors -> 297/0/0
(fully clean). cts baseline 5904 -> 5955 pass, 23 errors, 56 baselined
namespaces. 9 JVM-certified corpus rows; reader spec section.
2026-07-02 12:00:13 -04:00
Dmitri Sotnikov
95186a6782
Merge pull request #297 from jolt-lang/string/tostring-coercion
clojure.string toString coercion; some-fn/ifn? reference semantics; misc host gaps
2026-07-02 15:48:52 +00:00
Yogthos
e17bcfd0af clojure.string toString coercion; some-fn/ifn? reference semantics; misc host gaps
The clojure.string case fns and searches now take any Object s through its
toString like the reference's ^CharSequence signatures ((upper-case :kw) is
":KW", (capitalize 1) is "1"); nil throws, and a nil substr in
starts-with?/ends-with? throws. some-fn re-ported with the reference
arities: (some-fn) is an arity error and a no-match result is the last
predicate's own falsy value (false, not nil). ifn? covers multimethods,
promises (which are now invocable — calling one delivers, via a cold-path
invoke-arm registry that costs the hot dispatch nothing), and deftypes
implementing IFn's invoke.

One structural find on the way: defmulti/defmethod deferred inside a fn
body (the deftest pattern) interned/resolved in whatever namespace was
current when they RAN, not the one they were written in — the macros now
bake their expansion ns and the setups honor it.

Also: Boolean/Integer/Double wrapper ctors, primitive TYPE statics
(Integer/TYPE etc.), .reduce on collections (IReduce), and Long/TYPE.

cts baseline 5857 -> 5904 pass, 58 -> 28 errors, 57 baselined namespaces —
the string cluster, some-fn, ifn-qmark, boolean-qmark, and reduce
namespaces are all fully clean. 7 JVM-certified corpus rows; spec entry.
2026-07-02 11:38:37 -04:00
Dmitri Sotnikov
a9542077fc
Merge pull request #296 from jolt-lang/casts/checked-narrow
Checked narrow casts; fix runtime require in self-contained-built binaries
2026-07-02 13:52:34 +00:00
Yogthos
d0e1a11934 Checked narrow casts; fix runtime require in self-contained-built binaries
byte/short/int/long/char silently wrapped or passed out-of-range values
through; the JVM range-checks (RT.byteCast family). One checked-cast
helper now carries the ranges: a double range-checks ITSELF before
truncating ((byte 1.1) is 1, (byte 127.000001) throws), NaN casts to 0,
ratios and bigdecs truncate, a non-number is CCE, and the throw carries
the JVM message. float range-checks against Float/MAX_VALUE. The
unchecked-* casts now genuinely wrap and sign-fold ((unchecked-byte 200)
is -56 — the old bit-and lost the sign) with doubles saturating like
Java's conversions; unchecked-long/int are host natives. double/float of
a bigdec convert instead of crashing. The no-single-float residue stays
accepted (SPEC.md).

Also fixes #290: a binary built by the SELF-CONTAINED joltc died with
'variable var-deref is not bound' when a namespace loaded at runtime.
The in-process build compiled flat.ss against a clean copy-environment,
which orphans every top-level define in locations the binary's runtime
eval can't see. It now compiles against the default interaction
environment (defines land in the real symbol cells, same as the legacy
fresh-Chez path) and a generated prologue pre-binds each kernel name the
runtime redefines to its kernel value, so the earliest boot reads match
the legacy path's primitive references. requiring-resolve is implemented
(the issue's dynamic-require pattern), and the release workflow smokes a
runtime require in a built binary.

Cast namespaces byte/short/int/long/char now fully clean; cts baseline
5805 -> 5857 pass, 67 baselined namespaces. 7 JVM-certified corpus rows.
2026-07-02 09:42:06 -04:00
Dmitri Sotnikov
2610cb3ac3
Merge pull request #295 from jolt-lang/iref/watches-validators-meta
One IRef seam: watches/validators/meta over atom, var, and agent
2026-07-02 13:17:53 +00:00
Yogthos
f80f9aab4b One IRef seam: watches/validators/meta over atom, var, and agent
add-watch/remove-watch/set-validator!/get-validator were atom-only; the
atom ctor ignored :meta and :validator; watching a var crashed. Now the
ARef contract is one seam: atoms keep their record slots (hot path
unchanged), every other reference type registers a predicate and stores
watches/validators in identity-keyed side tables, and notifies at its
mutation points. Vars notify on root changes (def on a watched var,
var-set outside a thread binding, alter-var-root — thread-binding sets
don't notify, like the JVM); agents notify per action. The def-var! wrap
costs two weak-table probes per def and does IRef work only on a watched
var.

Ctor options follow ARef: the validator gates the initial value
(IllegalStateException 'Invalid reference state' — also the class for
rejected swap!/reset!), :meta must be a map (else ClassCastException),
nil allowed. meta reads any reference through the identity side-table
(the type-gated fall-through is gone); alter-meta!/reset-meta! work on
non-var references.

Runtime-only (no re-mint). 9 JVM-certified corpus rows; spec entry; cts
baseline 5781 -> 5805 pass, 73 baselined namespaces (the residual error
in the watch namespaces is their STM ref section — refs stay out of
scope).
2026-07-02 09:07:00 -04:00
Dmitri Sotnikov
257f822825
Merge pull request #294 from jolt-lang/hierarchy/reference-contracts
Hierarchy fns follow the reference contracts; deftype classes join the class graph
2026-07-02 12:58:33 +00:00
Yogthos
6e333b3020 Hierarchy fns follow the reference contracts; deftype classes join the class graph
derive/underive/ancestors/descendants/parents/isa? re-ported from
clojure.core with the argument assertions and throw contracts intact:
derive asserts tag/parent shapes (AssertionError) and throws on redundant
or cyclic derivation; underive/derive on a non-hierarchy value throw at the
parents lookup (the map is called as a function, like the reference);
(descendants h SomeClass) throws UnsupportedOperationException. isa? gains
the reference's supers arm (a relationship derived on a class's super
applies to the class).

The class arms now answer fully through the one class graph: parents of a
class are its direct supers (bases), ancestors are the transitive set
rooted at java.lang.Object for concrete classes (interfaces are marked and
don't root at Object, matching getSuperclass semantics). deftype/defrecord
classes register into the graph at definition — protocol interfaces they
implement appear as supers (JVM-munged ns spelling), records carry the
record interfaces (IRecord/IPersistentMap/... whose closure supplies
Associative/Seqable), bare deftypes carry IType. The type NAME var still
holds the ctor (a jolt-ism); class-key maps it back to the class so
(ancestors TypeName)/(isa? x TypeName) work. canonical-host-tag learned to
NOT canonicalize deftype names through the graph arm (extend-type on a
deftype was registering under the bare segment its values never report).

Five old corpus rows used non-namespaced derive tags that throw on the JVM
too; now namespaced. 8 new JVM-certified corpus rows; spec entries for the
hierarchy family; cts baseline 5730 -> 5781 pass (ancestors/derive/
descendants/parents/underive namespaces fully clean), 74 baselined
namespaces.
2026-07-02 08:48:30 -04:00
Dmitri Sotnikov
7e1df2c600
Merge pull request #293 from jolt-lang/numeric/ops-dispatch
Numbers-style category dispatch for binary numeric ops
2026-07-02 12:21:22 +00:00
Yogthos
e66a91750e Numbers-style category dispatch for binary numeric ops
Arithmetic and comparisons lowered to raw Chez ops, so an operand outside
Chez's tower (BigDecimal) crashed with a raw condition, and Chez contagion
leaked: (* 1.0 0) gave exact 0 where the JVM gives 0.0, (* ##Inf 0) gave 0
instead of ##NaN, (/ 1 0) raised an untyped error.

One seam now (host/chez/seq.ss): call position emits jolt-n* macros with the
both-Chez-numbers fast path open-coded; value position folds through the same
binary ops. Anything outside the tower falls to per-op slow hooks that
java/bigdec.ss extends, so bigdec arithmetic works in every position (the old
static-only :bigdec typing limitation is gone). JVM rules patched into the
fast path: a double operand wins, an exact zero divisor throws
ArithmeticException while a double zero divisor yields Inf/NaN, quot/rem/mod
cover ratios and doubles, min/max return the original operand with NaN
winning, a nil operand is NPE and a non-number CCE, zero-arg -// throw
ArityException at runtime instead of failing expansion.

Also: with-precision now binds *math-context* and bigdec results round with
real RoundingMode semantics (UNNECESSARY throws; division rounds to precision
instead of throwing); rationalize goes through the shortest decimal print
like BigDecimal.valueOf (the identity stub is gone); ratios coerce to bigdec
like Numbers.toBigDecimal; min/max int-literal operands no longer coerce to
flonum in the numeric pass.

Perf neutral: fib and seq benches unchanged (the fast path is two type checks
the optimizer folds); hinted fl/fx paths untouched. 19 JVM-certified corpus
rows; cts baseline 5614->5730 pass, 192->88 errors, 84->79 baselined
namespaces.
2026-07-02 06:41:45 -04:00
Dmitri Sotnikov
38abc1be84
Merge pull request #292 from jolt-lang/fix/are-clojure-template
clojure.test/are substitutes via clojure.template
2026-07-02 10:08:48 +00:00
Yogthos
86e36e8bee clojure.test/are substitutes via clojure.template
are let-bound its template vars, so a var inside quote never substituted:
(are [x] (special-symbol? 'x) if def) tested the literal symbol x twice.
Rebuild are on clojure.template/do-template (postwalk substitution), the
same architecture as upstream, with the same arg-count check.

This un-aborts every suite namespace whose are rows need substitution:
cts baseline moves 5302->5614 pass, 236->192 errors, 88->84 baselined
namespaces. The newly-reachable assertions also surface real divergences
now baselined and filed (edn reader strictness, Boolean ctor).
2026-07-02 05:57:57 -04:00
Dmitri Sotnikov
53a8aac2d0
Merge pull request #291 from jolt-lang/conformance/clojure-test-suite
Vendor clojure-test-suite as a standing gate (make cts)
2026-07-02 09:28:58 +00:00
Yogthos
edfd67a322 Vendor clojure-test-suite as a standing gate (make cts)
jank-lang/clojure-test-suite (per-core-fn clojure.test suites shared across
Clojure dialects) joins the default gate as vendor/clojure-test-suite, run by
host/chez/cts.sh: one joltc process per test namespace (a hang or crash is
contained by a per-process timeout), through the test/chez/cts-app project and
its cts-run runner, parallel workers.

Gating is exact per namespace against test/chez/cts-known-failures.txt, like
certify's allowlist: a namespace doing worse than the baseline fails, and one
doing better also fails as stale until the baseline is updated in the same
change. JOLT_CTS_WRITE_BASELINE=1 regenerates it; JOLT_CTS_NS runs a subset
verbosely.

Current standing: 243 namespaces, 5302 assertions pass, 340 fail + 236 error
across 88 namespaces pinned in the baseline (dominant clusters: BigDecimal
arithmetic operands, derive/ancestors hierarchy, transients, special-symbol?,
clojure.string case fns, the accepted narrow-int and seq-type-model
divergences). Two consecutive full runs produce identical counts. Wired into
make ci; skips cleanly when the submodule isn't checked out.
2026-07-02 03:46:57 -04:00
Yogthos
8ffc2f68c5 Fix general divergences surfaced by clojure-test-suite
Five fixes shaken out by running jank-lang/clojure-test-suite:

- = short-circuits on identity like Util.equiv's k1 == k2, so (= s s) on an
  infinite lazy seq answers true instead of walking forever. Numbers keep the
  exactness-aware arm ((= ##NaN ##NaN) stays false like the JVM's).
- Calling a non-fn names the operator's CLASS in the ClassCastException, like
  the JVM — never the value, whose printed form may be unbounded: ((range))
  must throw, not hang rendering an infinite seq.
- realized? on a seq cell answers by its forced flag (the rest of a realized
  lazy chain is a cseq under jolt's seq model), and the overlay's unsupported-
  type error names the class, not the (possibly infinite) value.
- clojure.test/is dispatches a REGISTERED assert-expr method before its by-name
  inline paths, like clojure.test where the built-ins are just pre-registered
  methods — so an alias-qualified p/thrown? (the suite's portability helper)
  isn't captured by the built-in thrown? path, which read its body as a class.
- clojure.test tracks tests and fixtures per namespace: deftest records its
  defining ns, use-fixtures registers under the calling ns (no more cross-ns
  clobbering), (run-tests 'ns ...) runs only those namespaces like clojure.test,
  and each run-tests call prints/returns its own summary (global counters stay
  cumulative for the n-pass/n-fail harness API).

Re-mint (20-coll.clj is seed; prelude only). +2 JVM-certified corpus rows;
the clojure-test fixture pins the alias-qualified assert-expr, per-call
summaries, and ns filtering.
2026-07-02 03:46:57 -04:00
Yogthos
51dec5fd2c Drop x86_64-macos from releases (GitHub retired the Intel runner)
The macos-13 Intel runner no longer gets allocated, so the x86_64-macos release
job queues forever. Ship prebuilt binaries for x86_64-linux and aarch64-macos;
Intel Macs build from source. The install script now says so instead of 404ing
on a missing asset.
2026-07-01 18:13:57 -04:00
Yogthos
eb768b13c1 docs: delist next.jdbc (JVM/JDBC-driver dependent)
next.jdbc's own source needs clojure.datafy + clojure.java.data and its tests
need JVM JDBC drivers, so it doesn't run on jolt. Keep clojure.jdbc (via
jolt-lang/db's jdbc.core over FFI SQLite) as the supported JDBC surface; point
migratus at jolt-lang/db.
2026-07-01 17:49:35 -04:00
Yogthos
802fb29b07 Link joltc with -rdynamic so build can spill its boots on Linux
The self-contained build reads the bundled Chez petite/scheme boots from the
joltc binary via foreign-entry on the embedded jolt_* symbols. On Linux dlsym
can't see an executable's symbols unless they're in the dynamic symbol table, so
'build' died with 'foreign-entry: no entry for jolt_petite_boot_len'. -rdynamic
exports them (macOS already resolves them). The new release self-contained-build
smoke caught this on the Linux runner.
2026-07-01 17:31:18 -04:00
Yogthos
db08ecc1bc Embed runtime source so a self-contained joltc can build apps
`joltc build` inlines the runtime (host/chez/rt.ss and everything it loads, the
seed, compile-eval, loader, ffi, the vendored irregex) into each app binary by
reading those files off disk. That works from a jolt checkout but not from the
installed self-contained binary, which has no source tree:

  joltc build -m app.core
  => Exception in call-with-input-file: failed for host/chez/rt.ss: no such file

build-joltc now bakes the exact transitive closure of files the build inlines
into the binary as embedded resources (keyed by the path the `(load "…")` forms
use), and build.ss/dce.ss read runtime source through bld-source-string, which
takes the embedded copy when present and falls back to disk otherwise. So the
same joltc builds apps both from a checkout and standalone.

The release workflow now smoke-tests a self-contained build (compile a tiny app
from an isolated dir, run it) — this is exactly what shipped broken, so it now
gates the release. buildsmoke/shakesmoke/staticnativesmoke unchanged and green.

Build tooling only — no re-mint, no runtime change.
2026-07-01 17:24:50 -04:00
Yogthos
f6bd2c6de5 readme: note the Homebrew install option 2026-07-01 16:52:02 -04:00
Yogthos
b460875772 Add an install script for the prebuilt joltc binary
install (root) downloads the self-contained joltc release asset for the host
platform, verifies its sha256, and drops the binary in /usr/local/bin (--dir /
--version override). Resolves the latest release via the GitHub API, clears the
macOS quarantine flag, and backs up an existing joltc. Modeled on babashka's
installer. README gets a one-line curl|bash install.
2026-07-01 16:38:43 -04:00
Yogthos
b5998f4b4a docs: register-class-supers! for library class hierarchies
Document the register-class-supers! seam next to the other host-class hooks:
when a library class belongs to a hierarchy (a custom exception caught as
IOException, a value matching instance? across its supertypes and dispatching a
protocol extended to any of them), declare its supers once and instance?/isa?/
supers/extend-protocol all derive.
2026-07-01 16:31:43 -04:00
Dmitri Sotnikov
21445375fa
Merge pull request #287 from jolt-lang/conformance/reader-literal-registry
Read an unknown #tag as a tagged-literal value
2026-07-01 20:24:52 +00:00
Dmitri Sotnikov
6500f968ce
Merge pull request #286 from jolt-lang/conformance/typed-throwables
Throw typed exceptions; one exception hierarchy
2026-07-01 20:24:16 +00:00
Dmitri Sotnikov
a59a32a0b0
Merge pull request #288 from jolt-lang/conformance/deftype-method-seam
Route deftype/reify interface dispatch through one seam
2026-07-01 20:23:26 +00:00
Dmitri Sotnikov
bd089f0845
Merge pull request #284 from jolt-lang/conformance/dispatch-arm-registry
Resolve .method calls through a priority arm registry
2026-07-01 20:20:54 +00:00
Yogthos
b4d9eaa527 Read an unknown #tag as a tagged-literal value
An unknown reader tag produced the reader's internal form
{:jolt/type :jolt/tagged :tag :#foo :form bar}, which tagged-literal? didn't
recognize and which leaked as a raw map when printed:

  (tagged-literal? (read-string "#foo bar"))  => false   ; want true
  (pr-str (quote [#foo bar]))                 => "[{:jolt/type :jolt/tagged ...}]"

Both the data path (rdr-construct-tag) and the compile path (emit-quoted) now
build a real tagged-literal for a tag with no registered reader, like Clojure's
*default-data-reader-fn*, so tagged-literal? / :tag / :form / printing all work.
clojure.edn reads raw forms through a separate __read-form-raw path and applies
:readers/:default itself, so it is unaffected.

Re-mint (backend + reader are seed sources); prelude byte-identical, image only.
make test green (selfhost holds, 0 new/stale), +2 unit rows.
2026-07-01 16:17:52 -04:00
Yogthos
f856c16f06 Throw typed exceptions; one exception hierarchy
jolt's own throw sites raised untyped Chez conditions with the class name buried
in an English message, so (class e) reported the opaque :object and only a broad
catch worked:

  (class (try (Long/parseLong "xyz") (catch Throwable e e)))  =>  :object
                                                              ; JVM: java.lang.NumberFormatException

Raise typed throwables (jolt-host-throwable) at the Long/Double parse and
StringTokenizer sites so (class e) / .getMessage / a specific catch all reflect
the real class. And fold the exception supertype table (exception-parent) into
the one class graph: exception-isa? now resolves the simple name to its graph key
and asks jch-isa?, so exceptions and every other class share a single hierarchy.

Runtime only, no re-mint. make test green (0 new/stale), +2 corpus rows.
2026-07-01 16:06:00 -04:00
Yogthos
01f98c2e89 Route deftype/reify interface dispatch through one seam
The "does value V declare method M; if so call it" decision was re-derived in a
dozen places with two different lookup helpers — records.ss jrec-cl and
collections.ss rec-coll-method were a byte-for-byte duplicate — and reduce only
honored a reify's own reduce method, not a deftype's:

  (reduce + 100 (->Rng 5))  ; Rng deftype implementing IReduceInit
  => "not seqable"          ; JVM: 110

Add iface-method / iface-call: one lookup that resolves a method for a deftype OR
a reify, with arity, and is the seam a core fn's interface arm collapses to.
jrec-cl now aliases rec-coll-method (the duplicate is gone). reduce routes its
IReduceInit arm through iface-method, so a deftype's reduce drives the reduction
like a reify's, reduced short-circuit included.

Runtime only, no re-mint. make test green (0 new/stale), +2 corpus rows.
2026-07-01 15:59:01 -04:00
Yogthos
0b07b376bb Resolve .method calls through a priority arm registry
record-method-dispatch was rebound with (set! record-method-dispatch ...) in six
files, each wrapping the previous binding, so precedence was whatever the rt.ss
load order happened to be — the true outermost arm was inst-time's Date arm, not
the one you'd guess. A type-gated wrapper that only whitelists its own methods
then errored on everything else, stealing universal Object methods from the arms
beneath it: (.getClass (java.util.Date.)) threw "No method getClass on Date",
same for File, while (class ...) and (.getClass "s") worked.

Replace the wrapper stack with an ordered list of arms (register-method-arm!,
ascending priority), each returning 'pass to defer. getClass is now one arm at
the top reached by every value, so it can't be shadowed; the three duplicate
getClass checks (dot-forms, host-static, base) collapse into it. Each former
wrapper is an arm at an explicit priority instead of an implicit load-order slot.
A library can register its own arm rather than set!-wrapping the dispatcher.

Runtime only, no re-mint. make test green (0 new/stale divergences), +1 corpus
row for getClass on Date/File.
2026-07-01 15:52:24 -04:00
Yogthos
d4acd69a73 Derive class identity from one hierarchy graph
instance?, extend-protocol dispatch, isa?/supers/ancestors, and the exception
hierarchy each read their own hand-kept table, and those tables had drifted:
(instance? clojure.lang.Associative [1 2]) was true but a protocol extended to
Associative wouldn't dispatch to a vector; keyword/IFn and seq/Seqable had the
same split; (isa? ExceptionInfo RuntimeException) was false and
(supers NumberFormatException) was empty.

Add one FQN -> direct-supers graph (class-hierarchy.ss) and derive the views
from it. value-host-tags builds on the graph closure so a vector reports
Associative/Indexed/ILookup/Counted/Seqable, a keyword reports IFn, a seq
reports Seqable/List/Counted, etc. instance? now tests membership in that same
list, so it can't disagree with dispatch. canonical-host-tag recognizes any
modeled class (was a separate literal set missing Seqable/ILookup/...).
class-direct-supers unions the graph edges and class-supers returns the
transitive closure, so the exception hierarchy answers isa?/supers/ancestors.

The graph is open: jolt.host/register-class-supers! lets a library graft its
own classes on and get every view for free.

Runtime only, no re-mint. make test green (0 new/stale divergences), +3
JVM-certified corpus rows.
2026-07-01 15:38:04 -04:00
Dmitri Sotnikov
d7dad2b450
Merge pull request #283 from jolt-lang/rewrite-clj-full-suite
Fix seven more JVM divergences (rewrite-clj full suite)
2026-07-01 18:35:14 +00:00
Dmitri Sotnikov
20d88324f4
Merge pull request #282 from jolt-lang/windows-sigint-fix
Don't abort startup on Windows resolving POSIX signal fns
2026-07-01 18:28:59 +00:00
Yogthos
cb03e36088 Don't abort startup on Windows resolving POSIX signal fns
joltc failed to start on Windows — "Exception in foreign-procedure: no entry for
pthread_sigmask". concurrency.ss resolves pthread_sigmask/sigemptyset/sigaddset at
load with a top-level (foreign-procedure …), which resolves its symbol eagerly;
those POSIX signal fns don't exist on Windows, so the whole runtime aborted.

Guard the three resolutions (like sched_yield/chmod already are) so a non-POSIX
host yields #f, and make jolt-set-sigint-blocked a no-op when they're unavailable.
The per-thread SIGINT mask is a POSIX-only optimization for the nREPL accept loop;
Windows delivers ^C through the console, and park-until-interrupt still parks on a
condition variable. macOS/Linux resolve the symbols as before — unchanged.
2026-07-01 14:23:19 -04:00
Yogthos
9bcac13fd2 Fix seven more JVM divergences (rewrite-clj full suite)
Running the whole rewrite-clj test suite (159 tests) surfaced seven more bugs;
with these it passes 3377/0/0. Each is a general jolt/JVM divergence:

- *out* was pinned to the startup stdout port, so (.write *out* …) escaped a
  with-out-str capture (z/print writes via *out*). It now resolves the live
  current-output-port, like print/__write, so a redirect is seen.
- nth / assoc past the end of a vector or seq threw a bare Chez error (class
  :object). Throw IndexOutOfBoundsException, matching the JVM.
- A number's .toString(radix) ignored the base. Render in the base, lowercase
  (rewrite-clj rebuilds 0xff / 0377 / 2r1001 through it).
- A required namespace's own :as aliases leaked into its requirer: the loaded ns
  form compiles while (chez-current-ns) is still the requirer, so ce-scan-requires!
  registered the loaded ns's aliases under the wrong ns and clobbered a same-named
  alias there. Register an (ns NAME …) form's aliases under NAME.
- A quoted collection dropped its metadata; now it keeps USER metadata (drops the
  reader's :line/:column/:file), like a Clojure quoted constant.
- enumeration-seq only did (seq e); it now drives a java.util.Enumeration through
  hasMoreElements/nextElement, and StringTokenizer implements them.

Regressions: corpus rows (with-out-str/*out*, nth/assoc bounds, toString radix,
quote metadata, enumeration-seq) certified against JVM; a smoke fixture for the
alias leak (a required ns's alias must not leak). tools.reader + rewrite-clj added
to docs/libraries.md. make test green.
2026-07-01 14:17:03 -04:00
Dmitri Sotnikov
53112d06fb
Merge pull request #281 from jolt-lang/irregex-submatch-clear
Compile capturing regexes with the backtracking matcher
2026-07-01 16:54:06 +00:00
Yogthos
7c4f9bb974 Compile capturing regexes with the backtracking matcher
irregex builds a POSIX leftmost-longest DFA for a pattern when it can, and jolt
used it for everything. For a pattern with an alternation whose branches have
capturing groups, that DFA leaks a non-participating branch's group: e.g.
#"(?:([0-9])|([0-9])r([0-9]+))" on "2r11" left group 1 = "2" instead of nil, so
tools.reader (rewrite-clj's dep) misread 2r1100 as 2 and 16rFF as 16.

java.util.regex is itself a leftmost-first backtracking engine, so compile a
capturing pattern with irregex's backtracking matcher ('backtrack): its submatch
semantics match the JVM and it clears a losing branch's group. Non-capturing
patterns keep the DFA — with no groups to read, its whole-match result is all a
caller sees, and it avoids backtracking's worst case. The submatch count comes
from a first cheap compile; a capturing pattern recompiles once and caches.

This clears the last rewrite-clj parser-test failure (now 772/0/0). Corpus rows
for the alternation-group case and the radix read. make test green.
2026-07-01 12:48:12 -04:00
Dmitri Sotnikov
e2d842b073
Merge pull request #280 from jolt-lang/rewrite-clj-conformance-fixes
Fix six JVM divergences surfaced by rewrite-clj
2026-07-01 16:31:06 +00:00
Yogthos
77e80dab9c Fix six JVM divergences surfaced by rewrite-clj
Running the rewrite-clj test suite under jolt exposed six bugs, each fixed here:

- `for`/`doseq` `:let` bindings never went through `destructure`, so a
  destructuring pattern (`:let [{:keys [y]} x]`) hit `let*` raw and failed to
  compile. Emit `let`, like Clojure.
- `with-open` couldn't close a deftype/defrecord that implements a `close` method
  (java.io.Closeable / AutoCloseable, e.g. tools.reader's readers) — `__close`
  only knew jhost readers and map `:close` fns. Dispatch a record's `close`.
- A deftype/defrecord method param named like a field didn't shadow the field
  (the field's let-binding wrapped the params). Params now shadow, as in Clojure.
- A deftype whose simple name collided with a built-in host class clobbered it in
  the global ctor table, so `(java.io.PushbackReader. …)` built tools.reader's
  same-named deftype. Register deftypes/built-ins by FQN, don't let a deftype
  overwrite a built-in's simple name, and qualify a bare `(Name. …)` to the
  deftype's FQN only in the ns that defined it.
- `clojure.walk` was lazy over a non-list seq (missing `doall`), so a walk whose
  fn has side effects read stale state. Make it eager, like Clojure.
- `Character/isWhitespace` used an ASCII-only check that missed U+2028 and other
  Unicode whitespace. Use the JVM's Unicode set (minus the no-break spaces it
  excludes).

Regressions: corpus rows (for-let destructure, method-param shadow, walk eager,
isWhitespace), a unit row (with-open closes a record), and smoke checks (the
class-name collision, run in a fresh -e process so the deftype doesn't leak).

One divergence remains unfixed: a submatch from a losing regex alternation branch
leaks when the winning branch has a quantified group (a bug in the vendored
irregex engine, not jolt) — tracked separately.
2026-07-01 12:25:05 -04:00
Dmitri Sotnikov
0bad467372
Merge pull request #279 from jolt-lang/data-reader-code-forms
Compile data readers that return code forms
2026-07-01 15:03:43 +00:00
Yogthos
908ad63caa Compile data readers that return code forms
A registered #tag data reader whose fn returns a FORM (borkdude/html's #html
expands to (->Html (str …))) was rewritten to a runtime call (reader-fn 'inner),
so the returned code became a runtime list value instead of being compiled —
(str #html [:div]) rendered the code, not "<div>". Clojure applies a data reader
at read time and substitutes its result as code.

loader.ss now applies the reader at load time: a code form (a list) is spliced in
to be compiled, a value (time-literals #time/date -> a Date) keeps the runtime
call, which also keeps a non-serializable constant out of an AOT build. The build
emit path never applied data readers at all (a #tag literal failed a `jolt build`
with "unsupported form"); emit-image.ss gets an ei-emit-form-hook the build sets
to the same rewrite, left as a no-op elsewhere so the seed mint (which doesn't
load loader.ss) is unaffected and the self-host byte-fixpoint holds.

Also make clojure.test report the actual values of a failing (is (= a b)) — it
printed only the form. Restricted to the common pure predicates so a macro head
still takes the plain path.

Fixture test/chez/datareader-app + a smoke check (interpreted) and a build-smoke
check (AOT). make test green, no corpus change.
2026-07-01 10:57:55 -04:00
Dmitri Sotnikov
1008e922d8
Merge pull request #278 from jolt-lang/dynamic-c-linking
Static-link :jolt/native C libraries into built binaries by default
2026-07-01 13:57:40 +00:00
Yogthos
d79ad6dc6a Static-link :jolt/native C libraries into built binaries by default
A :jolt/native spec can now carry a :static archive; `jolt build` links it
into the executable, so the app calls the C code with no shared object on the
target. --dynamic (or :jolt/build {:dynamic-natives true}) keeps the old
runtime load-shared-object behavior; a spec with no :static is unchanged.

The cc link force-loads the archive (-force_load on macOS, --whole-archive on
Linux) and exports the executable's symbols (-rdynamic on Linux) so the baked-in
symbols resolve via (load-shared-object #f) + foreign-procedure at startup. Build
step 1 evaluates the app's foreign-procedure forms in-process, so a static
archive is preloaded there as a throwaway shared object to resolve them.

The distributed self-contained joltc has no external cc/Chez but must build these
apps, so it now bundles the Chez kernel (libkernel.a + scheme.h) and the launcher
source and re-links a custom stub with the archives baked in — needing only a
system cc, no Chez. run/repl skip static-only specs (nothing to load); keep a
:darwin/:linux candidate to use such a lib interpreted.

Adds static-native-smoke (cc path) to ci and a static phase to the joltc
self-build smoke (distributed path).
2026-07-01 09:52:00 -04:00
Dmitri Sotnikov
a2e99fff45
Merge pull request #277 from jolt-lang/improved-error-handling
Make the REPL read multi-line forms and render real error messages
2026-07-01 04:16:36 +00:00
Yogthos
e4cbbb8912 fix REPL treating a regex literal as an unbalanced form
repl-form-complete? entered the :regex state on '#' but only consumed the
'#', so the opening '"' was then read by the :regex handler as the CLOSING
quote. The regex body got scanned in :code state, and any delimiter or quote
inside it (a group like #"(a)", a char class #"[0-9]+") threw off the
paren/string count — so a one-line regex form was judged incomplete and the
REPL hung waiting for continuation lines. Consume the '#"' together.

Adds a self-checking predicate test (test/chez/repl-reader-test.clj, run via
joltc so jolt.main resolves) and an end-to-end regex REPL case in smoke.sh.
2026-06-30 23:44:22 -04:00
Yogthos
4889505204 fix corpus crash on 'replace on a seq is lazy'
The :expected was a bare list "(0 :a 2)", but run-corpus evals :expected as
source, so it applied 0 as a fn -> "0 cannot be cast to IFn". Every other
list-valued :expected is self-evaluating; this one slipped in unquoted.
Vectorize it to [0 :a 2], matching what regen-corpus.clj produces.
2026-06-30 23:24:47 -04:00
Yogthos
f625099ddf fix clojure.core/max shadowed by a local 2026-06-30 23:05:04 -04:00
Yogthos
4a1dec277e fix tests 2026-06-30 21:25:33 -04:00
Yogthos
240458d994 Make the REPL read multi-line forms and render real error messages
The REPL evaluated one line at a time, so a form split across lines
(e.g. `(+` then `1 2)`) raised instead of waiting. The read loop now
accumulates lines until delimiters are balanced — skipping string,
char, regex and comment context — printing a `... ` continuation prompt
for each extra line.

Reader/runtime errors rendered as Chez's "attempt to apply
non-procedure #[chez-pmap...]" instead of their real message. Two causes:

jolt-throw raised the thrown value raw. When a throw crossed the host
`eval` boundary, Chez re-wrapped the non-condition into a compound
condition whose message extraction applies the value, losing the message
and crashing on ex-info's empty-map :data. jolt-throw now raises a
&jolt-throw condition wrapping the value; catch (lowered to `guard`),
jolt-report-uncaught and jolt-render-throwable unwrap it back via
jolt-unwrap-throw, so ex-data/ex-message and the backtrace tag survive.

Every reader/post-prelude EOF-throw site used `(empty-pmap)` (with
parens), applying the empty-map value as a procedure and crashing during
ex-info construction before jolt-throw ran. Fixed to `empty-pmap`.

Re-minted the seed; smoke 23/23, unit 574/574.
2026-06-30 20:36:06 -04:00
Dmitri Sotnikov
a58bca3bee
Merge pull request #276 from jolt-lang/clean-nrepl-exit
Fix nREPL server ^C shutdown crash
2026-06-30 23:15:35 +00:00
Yogthos
8c7553fe55 update readme 2026-06-30 19:14:53 -04:00
Yogthos
46c9c7b4d9 Fix nREPL server ^C shutdown crash
^C to a running `joltc --nrepl-server` aborted with "thread does not
own mutex" because the accept-loop thread absorbed SIGINT in its foreign
accept() call, where Chez can't run the keyboard-interrupt handler, and
run-main-pump's tight condition-wait loop wasn't interruptible anyway.

Block SIGINT in the primordial thread before starting the server so the
accept loop inherits a blocked mask, park in a single interruptible
condition-wait via the new park-until-interrupt, and run registered
shutdown hooks (newest-first, each isolated) from the keyboard-interrupt
handler before (exit 0). The stop fn now drops .nrepl-port via the new
jolt.host/delete-file seam — clojure.java.io/delete-file doesn't exist
in Jolt and silently no-ops, so .nrepl-port was never removed.
2026-06-30 19:08:13 -04:00
Yogthos
8c2bd60257 cleanup 2026-06-30 17:14:44 -04:00
Dmitri Sotnikov
7275eb54a5
Merge pull request #275 from jolt-lang/repl-quit-command
Add :repl/quit and :exit gestures to the REPL
2026-06-30 19:10:16 +00:00
Yogthos
649e33fe3b Add :repl/quit and :exit gestures to the REPL
^D (EOF) exits cleanly in canonical mode but some terminals and editors
don't deliver it, leaving the user stuck. Accepting :repl/quit or :exit
as the first form of a line gives a reliable keyword exit that works
everywhere. The check parses the line with read-string rather than
checking the evaluated value, so a nested value that happens to print
as the keyword can't trigger an exit.
2026-06-30 14:33:26 -04:00
Dmitri Sotnikov
9e53ba4248
Merge pull request #274 from jolt-lang/clojure-lift
Clojure lift
2026-06-30 15:24:41 +00:00
Yogthos
bbca8bc0de Migrate list?/ratio?/rational? to the overlay; narrow jolt.host exposure
list?, ratio?, and rational? are the predicate-web members that are
genuinely safe to migrate: not extended at runtime, not on the compiler
emit/inference path, not reached by the kernel tier. They now live in the
overlay (clojure/core/20-coll.clj) built on the jolt.host tower/rep tests,
lowering to the same code the native shims did. Removed their native
definitions (predicates.ss) and, for ratio?/rational?, the now-redundant
post-prelude re-assertions. Also dropped the dead all-flonum overlay
ratio?/rational?/decimal? stubs.

The rest of the web stays native and is documented as such: map?/set?/
seq?/coll? are extended with sorted/record/lazy arms, decimal? is extended
by the optional bigdec module, integer?/float? are on the emit/inference
path, vector? is reached by the kernel-tier peek. jolt.host exposure is
therefore narrowed to just the tests these three consume (exact?,
rational-type?, cseq?, cseq-list?, empty-list?).

Numeric probe is byte-identical to pre-migration; list? correct across
list/vector/lazy/empty/cons/rest cases. Selfhost fixpoint holds, values/
unit/smoke/corpus green, bench flat within noise.
2026-06-30 11:10:36 -04:00
Yogthos
12058d2dcf Expose raw host type-test primitives under jolt.host
The clojure.core type predicates bottom out at host tests that overlay
Clojure can't reach. Expose them under jolt.host so the predicate web can
be built as pure compositions that lower to exactly these calls:

  numeric tower: exact? flonum? integer-type? rational-type?
  collection reps: pvec? pmap? pset? cseq? empty-list? cseq-list? lazyseq?

exact? is wrapped to be total (Chez's raw exact? errors on a non-number;
the others return #f for a non-match). lazyseq? is exposed in
lazy-bridge.ss because jolt-lazyseq? is defined there, after predicates.ss.

map?/set?/seq? are deliberately not reduced to a single rep test: they are
extended at runtime with sorted-collection/record/lazy arms, so only the
rep predicates are exposed, not those unions. Additive only (new bindings,
nothing references them yet); bench unchanged within noise.
2026-06-30 10:58:44 -04:00
Yogthos
1481a806b7 Document why reader-conditional stays a native shim
Attempting to migrate the reader-conditional constructor to the overlay
revealed that an overlay defn returning a :jolt/type-tagged map literal
silently fails to bind during the seed mint: the guard around each
prelude form swallows the load-time error, leaving the var unbound. This
is the same reason every other tagged-value constructor (atom,
volatile!, tagged-literal) is native, so reader-conditional is
reclassified STAY-PRIMITIVE rather than a safe migration.
2026-06-30 10:42:49 -04:00
Yogthos
d77b4e6420 Migrate clojure.core/set from a native shim to the kernel overlay tier
set was a native shim (apply jolt-hash-set (seq->list coll)). It is a
pure composition, so the Clojure version (apply hash-set (seq coll))
lowers to the same code. The compiler uses set, but only off the emit
path (the backend's bare-native-names def and type inference), so it can
live in the kernel tier: compiling that tier never calls set, and by the
time those callers run the tier is already bound.

This is distinct from boolean, which the backend calls for every :if
node on the emit path. Moving boolean even to the kernel tier deadlocks
(compiling the tier that defines boolean needs boolean), so boolean stays
native. Added a comment in predicates.ss recording that.

Re-mint converges in 3 passes and the benchmark suite is unchanged
within noise (collections 43.3 vs 43.1, binary-trees 367 vs 367, the
rest flat).
2026-06-30 10:35:57 -04:00
Yogthos
3d0cbed3c5 Remove dead native transduce shim (overlay already provides it)
The overlay defines transduce in clojure/core/22-coll.clj as a pure
composition (xf (reduce xf init coll)), and it shadows the native
jolt-transduce by load order. The compiled overlay version is already
what gets baked into the seed, so the native binding in
natives-transduce.ss was dead weight.

transduce is not used by the self-hosted compiler and no overlay tier
before 22-coll references it, so removing the native binding is safe.
Re-minting produces a byte-identical seed, which proves the runtime is
unchanged. sequence stays native (its transformer iterator drives the
reduced box and lazy realization directly).
2026-06-30 10:27:27 -04:00
Dmitri Sotnikov
7f163faf2e
Merge pull request #273 from jolt-lang/proper-chunking
Chunk range/map/filter to match JVM Clojure
2026-06-30 02:09:29 +00:00
Yogthos
bd33d605ef Chunk range/map/filter to match JVM Clojure
range, map, and filter were fully element-by-element lazy, so
(map f (range 1 50)) realized one element per first/nth where JVM
Clojure realizes a whole 32-element chunk. range is a chunked
LongRange on the JVM and map/filter are chunk-preserving, so the
observable side-effect timing differed.

Following clojure.lang.LongRange, ChunkedCons, ChunkBuffer and
core.clj, this adds a crest field to the cseq record and a
cseq-chunked constructor modeling ChunkedCons (a standalone chunk
pvec, an offset, and the after-chunk seq). The chunk accessors move
to seq.ss next to the representation they read. map/filter/remove
take a chunked branch when the source is chunked, realizing the whole
chunk and chunk-cons'ing it onto a lazy rest, so their output is
itself chunked and chained transforms each batch by 32. Bounded range
is now an eager chunked seq, and the reduce fast path flows through a
ChunkedCons rest. The chunk-buffer/chunk/chunk-cons builder API in
natives-array.ss now produces a real ChunkedCons.

Single-arg (range), multi-coll map, and plain lazy seqs stay
element-by-element, like the JVM.

Adds a lazy / chunking suite to the corpus that observes realization
timing via an atom counter: first over a chunked map realizes 32,
crossing a chunk boundary realizes 49, chained maps batch [32 32],
filter applies the predicate to the whole first block, and a plain
lazy seq still realizes one element at a time. Two cases that
documented the old over-laziness now assert the JVM value of 32 and
were dropped from the allowlist. certify against JVM Clojure 1.12.3
reports 0 new and 0 stale divergences.
2026-06-29 22:02:06 -04:00
Dmitri Sotnikov
c0a0ec98ee
Merge pull request #272 from jolt-lang/feature/self-contained-joltc
Self-contained joltc binary + release workflow
2026-06-30 01:25:23 +00:00
Yogthos
df4653e57f Add release workflow: build joltc binaries on a v* tag
On a pushed v* tag, build the self-contained joltc (make joltc-release) for
x86_64-linux, x86_64-macos, and aarch64-macos, package each as a tar.gz plus a
SHA256, and attach them to the GitHub Release. Linux builds Chez from source like
tests.yml (the apt package lacks the kernel dev files build-joltc cc-links
against); macOS uses Homebrew chezscheme, which ships chez and the csv kernel
files. No notarization, matching dirge — macOS tarball users de-quarantine once
or install via a Homebrew tap.

The Homebrew tap update job is a separate follow-on; this covers building and
publishing the release assets.
2026-06-29 21:16:26 -04:00
Yogthos
242eeac5c6 Build joltc as a self-contained binary (make joltc-release / joltc-debug)
host/chez/build-joltc.ss builds joltc into target/<profile>/joltc: it emits a
flat source of the full runtime + compiler image + inlined build.ss + every
jolt-core/stdlib file as a baked string literal + a cli.ss-style launcher, then
(in a fresh Chez, so the inlined runtime's redefinition of error doesn't strand
early references and runtime eval still sees the runtime's top-level procedures)
compiles it and cc-links it with the Chez petite/scheme boots and the launcher
stub embedded as C arrays. The launcher reads those arrays via FFI on
(jolt-materialize-bundles!) and registers them so build-self-contained can spill
them. joltc itself is cc-linked (clean signature for Homebrew); only the apps it
later builds use the appended-stub path.

build.ss: skip the csv toolchain check on the self-contained path and create the
build dir with a subprocess-free bld-mkdir-p, so a  from the
distributed binary shells out to nothing.

release = optimize-level 3 + no inspector info + compressed; debug =
optimize-level 0 + inspector + procedure source + debug-on-exception.

joltc-selfbuild-smoke.sh (make joltcsmoke) builds joltc and, with an empty
environment (no chez/cc/PATH), drives it through the build-app fixture, asserting
the produced binary's output. .gitignore ignores target/.
2026-06-29 21:04:23 -04:00
Yogthos
0420cd4d79 Self-contained build foundation: embedded-bytes helpers, launcher stub, in-process app link
Adds the pieces a toolchain-free joltc needs to compile apps with no external
Chez or cc:

- host/chez/java/io.ss: register-embedded-bytes!/jolt-embedded-bytes,
  read-file-bytes, jolt-spill-embedded!, jolt-append-payload! (frames an app
  boot onto the stub as [stub][boot][len:le64]["JOLTBOOT"]), and jolt-chmod-755
  via load-shared-object #f (no subprocess).
- host/chez/stub/launcher.c: a native stub that locates its own executable,
  reads the trailing frame, and hands the appended boot to the Chez kernel.
- host/chez/loader.ss: resolve-on-roots consults the embedded source store before
  disk; ldr-read-source reads baked source. Dev (empty store) is unaffected.
- host/chez/build.ss: build-binary step 4 splits into build-self-contained
  (in-process compile-file/make-boot-file with the system error restored, then
  append the boot to a copy of the embedded stub) and build-with-cc (the existing
  dev path). The self-contained path is taken only when the stub is embedded.

The legacy cc path is unchanged behaviorally; make buildsmoke still passes.
2026-06-29 20:48:44 -04:00
Yogthos
0abb958955 Merge fix/chunked-seq-stub-comment: correct the chunked-seq? stub comment 2026-06-29 14:36:25 -04:00
Yogthos
f82568281e Correct the misleading chunked-seq? stub comment
The overlay comment claimed Jolt has no chunked seqs and that chunked-seq?
is always false. That is no longer true: a vector's seq is a real chunked
seq, and post-prelude.ss rebinds chunked-seq? to na-chunked-seq?, which
returns true for a vector seq. The defn here is only a placeholder so
references compile during overlay load. Updated the comment to say so.
2026-06-29 14:33:31 -04:00
Dmitri Sotnikov
2a8783649e
Merge pull request #271 from jolt-lang/fix/nrepl-startup-and-shutdown
nREPL: surface startup failures and close the listen socket on shutdown
2026-06-29 18:17:26 +00:00
Dmitri Sotnikov
baf78c63bd
Merge pull request #270 from jolt-lang/fix/graceful-main-pump-shutdown
Make main-pump shutdown graceful and stop-main-pump race-free
2026-06-29 18:17:24 +00:00
Yogthos
ad5affe89f nREPL: surface startup failures and close the listen socket on shutdown
Two pre-existing issues in the nrepl command, exposed when #269 moved the accept
loop into a future.

jolt.nrepl/start was invoked inside (future ...), so binding the socket happened
on a background thread. A bind failure such as the port already being in use was
captured into a future that nothing derefs and silently swallowed, leaving the
main thread parked in run-main-pump forever with no server and no error. start
now binds the socket synchronously before returning, so that failure propagates
to the caller and the process exits with a visible message. Only the blocking
accept loop runs on a worker thread.

There was also no shutdown path: the accept loop ran forever and the listen
socket was never closed. start now returns a stop fn that breaks the accept
loop, closes the socket to free the port, and removes .nrepl-port. main.clj runs
run-main-pump and then calls the stop fn, so a stop-main-pump (from a glimmer app
on quit, or from nrepl-evaluated code) shuts the server down cleanly and the
process exits.
2026-06-29 14:09:05 -04:00
Yogthos
e76816d9fc Reset main-pump active flag and make stop-main-pump work race-free
PR #269 added the main-thread executor (call-on-main-thread, run-main-pump,
stop-main-pump) so the nREPL accept loop can run on a worker thread while the
primordial thread owns the GUI main loop. Two problems made stop-main-pump
unusable as a graceful-shutdown or external API.

run-main-pump set jolt-main-pump-active to #t on entry but never cleared it on
exit, so after the pump returned the flag stayed #t. call-on-main-thread also
read that flag outside the queue mutex, so even with a reset there was a window
where a job could be enqueued just as the pump left, then block forever on a
pump that was gone.

Both are now decided under jolt-main-queue-mu. The pump clears active in the
same critical section where it sees the stop flag and an empty queue, and
call-on-main-thread reads active and enqueues atomically under that lock. A
caller that loses the race sees the pump inactive and runs the thunk inline,
the same fallback used when no pump is running, rather than blocking. A
dynamic-wind around the loop also clears active on an abnormal exit so a later
run-main-pump starts clean.
2026-06-29 13:49:50 -04:00
Dmitri Sotnikov
d21feba486
Merge pull request #269 from jolt-lang/nrepl-thread
update nrepl to run in a thread
2026-06-29 03:33:30 +00:00
Yogthos
8bba526c8c update nrepl to run in a thread 2026-06-28 20:02:56 -04:00
Dmitri Sotnikov
28ee005855
Merge pull request #268 from jolt-lang/cli/nrepl-server-flag
cli: rename nrepl command to --nrepl-server flag
2026-06-28 21:33:21 +00:00
Dmitri Sotnikov
4b29594eff
Merge pull request #267 from jolt-lang/docs/bench-arith-numbers
bench: document 64-bit arithmetic + generator numbers vs JVM
2026-06-28 21:33:02 +00:00
Yogthos
823bc5bcc6 cli: rename nrepl command to --nrepl-server flag
Match babashka's spelling: the nREPL server now starts with
`bin/joltc --nrepl-server [port]` instead of `bin/joltc nrepl`. Port
parsing and JOLT_NREPL_PORT are unchanged.

Also wire up --help/-h to print usage (previously only the no-arg
invocation did), and fix the usage listing to show the real flag.
Smoke now asserts --help mentions --nrepl-server. Docs updated to match.
2026-06-28 17:26:03 -04:00
Yogthos
ba58d7ec85 bench: document 64-bit arithmetic + generator numbers vs JVM
The AOT suite doesn't cover 64-bit integer arithmetic (Chez fixnums are 61-bit,
so genuine 64-bit values are bignums) — the SplitMix PRNG behind test.check is
the worst case. Add the measured jolt-vs-JVM numbers for the PRNG/mix-64 and the
generator workload: the bitwise native-ops + var-cell caching took mix-64 from
~18x to ~3.2x JVM and the PRNG from ~30x to ~12x; the residual is the open-world
generator dispatch/allocation and the bignum floor, not arithmetic.
2026-06-28 15:39:17 -04:00
Dmitri Sotnikov
1375a59568
Merge pull request #266 from jolt-lang/conformance/laziness-semantics
Conformance hardening + perf: seq semantics, chunked-seq O(n^2), bitwise/var-cache codegen
2026-06-28 16:40:50 +00:00
Yogthos
04180c1e4e backend: cache resolved var cells per reference site (run-path ~5x)
Profiling jolt-i5if showed <=60-bit arithmetic is already native-fast; the real
general overhead in the run/-e/-m path is var resolution. Every var reference
compiled to (var-deref ns name), which builds + hashes a fresh "ns/name" string
and does a hashtable lookup per access (~45ns). The var cell is interned and
def-var! mutates it in place, so caching the resolved cell is sound under
redefinition.

Generalize the devirt per-site cache-cell mechanism to var value references: a
ref inside a fn resolves its cell once into the def's closure, then reads it via
var-cell-deref (a field read after the first). var-cell-deref is the cell-based
var-deref — binding-aware (dynamic vars + *ns* still resolve) and lenient on an
unbound root (a forward-declared var doesn't throw, unlike jolt-var-get).

Gated by a runtime flag: ON for runtime-compiled code (compile-eval.ss), OFF for
the seed mint and AOT build (emit-image.ss) so the seed stays a byte-fixpoint --
prelude.ss is unchanged, only image.ss picks up the new backend. ~5x on a
var-ref-heavy loop (1058ms->205ms); ~1.2x on test.check (its generators are more
deftype/dispatch-bound than var-deref-bound). No C/FFI.

Corpus rows pin redefinition / dynamic binding / forward ref through a cached
ref. make test + shakesmoke green, selfhost holds, SCI 211/218, certify 0-new.
2026-06-28 12:36:35 -04:00
Yogthos
f17b68ccfe backend: emit bitwise ops as native ops (test.check PRNG ~2.4x)
Profiling the test.check distribution/large-sample slowness (jolt-i5if): the
hot path is the SplitMix PRNG, dominated by 64-bit mix arithmetic, and the
bitwise ops (bit-and/or/xor/not, shifts) were NOT in the backend native-ops
table — so (bit-xor a b) compiled to a var-deref through the variadic overlay
(__bit-xor) instead of a direct call, the way +/-/* already emit.

Map bit-and/or/xor/not to the Chez bitwise-and/ior/xor/not primitives (inlined
to native code; a non-integer operand now errors like the JVM instead of being
silently truncated) and the shifts to a direct helper call. bit-and-not stays on
its overlay — its only Scheme impl is 2-arg, so a value-position arity-3 use
would mis-emit.

mix-64 arithmetic 2.7x faster, raw split+rand-long 2.4x, gen/vector ~1.4x. The
remaining gap is the bignum-vs-native-long floor (~20x, substrate) plus the
generator machinery (deftype/fn dispatch, separate). Corpus rows added for value
position, bit-not, apply, and a full-64-bit unsigned shift.
2026-06-28 11:25:52 -04:00
Yogthos
b5ea06c5c2 clojure.test: assert-expr / do-report / report extension points
jolt's `is` was a fixed macro with no assert-expr multimethod, and the runner
bypassed the report multimethod, so libraries couldn't register custom
assertions or custom report types (e.g. test.check's ::trial/::shrunk).

Add assert-expr (2-arg [msg form], dispatch on the form's first symbol /
:default / :always-fail), do-report routing through report, and report
:pass/:fail/:error methods that feed the counters. `is` dispatches to an
explicitly-registered assert-expr method before its inline path, so thrown?/
thrown-with-msg?/= and every built-in form stay byte-identical.

Runtime stdlib only, no re-mint. test/chez/clojure-test.clj self-checks the
extension points + full is/are/testing/thrown?/use-fixtures surface; smoke gate
runs it.
2026-06-28 10:37:59 -04:00
Yogthos
4a72897dfd conformance: document narrow-int unification (byte/short/int -> Long)
jolt unifies every integer as one exact-integer type, so (byte/short/int n)
report Long not Byte/Short/Integer and instance? Byte is false. Confirmed
substrate-inherent: (byte 5) is a Chez immediate identical? to 5 (nothing to
tag, numbers carry no metadata), and arithmetic compiles to a raw Chez + that a
boxed narrow type would crash. Value/arithmetic/equality are correct.

Certify the value-correctness (= to plain int, arithmetic promotes, is a Number)
and pin the class/instance? divergence under a new :integer-box-model category.
Data/doc only.
2026-06-28 10:28:10 -04:00
Yogthos
59cfa5f53f conformance: audit + pin seq semantics (laziness, eagerness, chunking, type)
A 62-case jolt-vs-JVM probe across seq type identity, chunking
granularity, eagerness, and realization timing. Findings: the whole
producer family is lazy at construction (no eager bugs remain), and the
26 divergences fall into two classes that diverge by representation, not
value.

Lock in the laziness contract as certified corpus rows: construction=0
for keep/keep-indexed/map-indexed/distinct/partition-by/partition-all/
interpose/interleave/take-nth/reductions/tree-seq/replace, sequence
realizes 1, next realizes 2, rest realizes 1.

Pin the two accepted divergence classes (allowlisted, gate-guarded):
- seq-type-model: jolt reifies seqs as PersistentList/LazySeq vs JVM's
  Cons/Iterate/LongRange/Repeat/Cycle/ChunkedSeq/StringSeq/KeySeq/RSeq/
  ArraySeq/SubVector (jolt-aei7)
- chunking-model: unchunked, realizes one where JVM realizes a 32-chunk;
  mapcat/dedupe fully lazy at construction (jolt-mm6v)

known-divergences.edn gains both categories; SPEC.md documents the seq
semantics contract. Data/doc only, no re-mint. certify 0 new / 0 stale.
2026-06-28 03:22:47 -04:00
Yogthos
6d441e2d00 chunk-first: pull the trie leaf instead of flattening the whole vector
A pvec is a 32-way trie, but na-chunk-first built each block by calling
pvec-v on the full backing vector — materializing all n elements to a
flat Scheme vector — then copying the 32-wide window out of it. That made
chunk-first O(n), so walking a vector chunk-by-chunk (Clojure's real
chunked map/filter fast path) was O(n^2): a ported chunked map over 500K
elements took 39s, superlinear to ~700s at 2M.

na-chunk-size equals pv-width and blocks are 32-aligned, so a block is
exactly one trie leaf — pv-chunk-for hands it back in O(log n). Copy that
leaf directly; fall back to per-index reads for the rare window that
crosses a leaf boundary. Chunked map is now linear, ~133x faster at 500K
(293ms) and within ~2.3x of the native seq loop, which makes a
clojure-in-clojure seq tier viable.

Corpus rows pin chunk-first window contents + chunk-rest boundaries
against JVM; fixed a stale 'always false' chunked-seq? label.
2026-06-28 01:56:26 -04:00
Yogthos
6940b2c7f5 corpus: certify seq realization order, count, and memoization
The corpus compares values, so eager-vs-lazy was invisible (identical
values). Add rows that reduce laziness to a value via a side-effect
counter: realization order (map/filter left-to-right), exact realization
count under take/nth/drop (no over-realization), and lazy-seq
memoization (realize-once across repeated walks). Sourced through
unchunked producers (iterate, lists) so jolt's unchunked model matches
the JVM. All certify against Clojure 1.12.5.
2026-06-28 01:40:51 -04:00
Dmitri Sotnikov
83ff96c3c8
Merge pull request #265 from jolt-lang/conformance/lazy-map
seq fns are lazy by default (LazySeq), like Clojure
2026-06-28 05:31:25 +00:00
Dmitri Sotnikov
f921e97c90
Merge pull request #264 from jolt-lang/numeric/unchecked-ratio
unchecked arithmetic: ratio correctness + in-range fast path
2026-06-28 05:30:59 +00:00
Yogthos
b879430618 seq fns are lazy by default, like Clojure (LazySeq, not eager-headed)
map/filter/remove/take/drop/concat/take-while/drop-while/mapcat/partition
built an eager-headed cseq: the first element (and the fn application) ran
at construction, so a side-effecting (map f coll) fired f immediately and
(class (map …)) was PersistentList instead of LazySeq. This diverged from
Clojure, which wraps the whole body in lazy-seq. It went unnoticed because
the conformance gate certifies values, not realization — eager and lazy
heads produce identical values — and unit.edn even baked PersistentList in
as expected. test.check's for-all-takes-multiple-expressions (which counts
side effects in a for-all body) exposed it.

Wrap each native producer's result in a lazy-seq node so the body, incl.
the first element, defers until forced — the forced cseq still has eager
heads, so reduce/count/dorun/etc. force on walk and there's no per-element
cost. dedupe's (seq coll) is moved inside its lazy-seq. A jolt LazySeq is
now recognized by coll?/empty, the analyzer's form predicates (a macro can
build its expansion with map), value-host-tags + instance? (LazySeq/ISeq/
Sequential), and reports clojure.lang.LazySeq.

Kept the native Scheme implementations rather than porting Clojure's: a
straight lazy-seq+cons port is 3x slower and Clojure's chunked fast path is
288x slower because jolt's chunk machinery is unoptimized (filed jolt-j9dz);
the wrapped natives are Clojure-lazy at native speed.

+12 corpus rows (laziness at construction, LazySeq type, both JVM-certified).
make test + shakesmoke green, selfhost holds, 0 new divergences.
2026-06-28 00:16:47 -04:00
Yogthos
a49ca3b5ea jolt-wrap64 fast path: skip the mask when already in signed-64 range
Chez fixnums are 61-bit, so the bignum bitwise-and mask allocates for any
value past 2^60 — and unchecked-* ran it on every result, even small
in-range ones. An exact integer already in [-2^63, 2^63) is its own wrap,
so return it directly; only an out-of-range result (a multiply overflowing
into 128 bits) needs the mask. ~30% on in-range unchecked-add loops,
neutral on full-64-bit multiply.

Note: the 64-bit arithmetic floor on Chez stays ~31ns/multiply (bignum, no
native 64-bit int); the test.check distribution hangs are dominated by
generator/dispatch overhead, not arithmetic — this is a general win for
long-heavy code, not a fix for those.
2026-06-27 23:12:16 -04:00
Yogthos
253d64b1e7 unchecked-* on a ratio (or any non-long) shouldn't wrap to 64-bit
jolt-unc{add,sub,mul,inc,dec,neg}2 wrapped every non-flonum result to a
signed 64-bit integer, so (unchecked-add 2/3 2/3) truncated to 1 instead
of 4/3. Under *unchecked-math* the analyzer rewrites +/-/* to unchecked-*,
so any ratio arithmetic in such a file silently floored. Clojure's
unchecked-add falls back to regular arithmetic for non-primitives; only
long math wraps. Wrap iff both operands are exact integers.

Shaken out by test.check's gen/ratio monoid property (the + and 0 monoid
held for small-integers but failed for ratios).
2026-06-27 22:49:42 -04:00
Dmitri Sotnikov
92368b49f1
Merge pull request #263 from jolt-lang/conformance/spec-alpha
spec.alpha: symbol-of-var, demunge, MultiFn methods, fn class names, namespaced maps
2026-06-28 02:01:27 +00:00
Dmitri Sotnikov
c75d698815
Merge pull request #262 from jolt-lang/conformance/test-check
test.check generators: unchecked-math/rand-double, ThreadLocal proxy, and host interop
2026-06-28 02:01:02 +00:00
Yogthos
522ff10d62 spec.alpha: reify ILookup get, NPE/CCE, quoted #inst/#uuid, anon-fn class, kwargs map
Close clojure.spec.alpha's remaining gaps — its conform/explain/describe/multi-spec
suite (clojure.test-clojure.spec, multi-spec) now passes fully.

- (get reify k) / (:k reify) routes to a reify's clojure.lang.ILookup valAt. spec
  reifies fspec/regex specs as ILookup and reads (:args spec) off them, so before
  this instrument never saw the args spec.
- A failed numeric comparison reports the JVM class: a nil operand is
  NullPointerException, a non-number is ClassCastException (was an opaque :object
  condition). conform-explain checks the thrown class.
- A quoted / macro-form #inst / #uuid literal constructs its Date/UUID value, like
  the JVM reader (which builds it at read time). emit-quoted was emitting the raw
  tagged form, so #inst "1939" and #inst "1939-01-01T00:00:00.000-00:00" weren't =.
- An anonymous fn reports class clojure.lang.AFunction$fn (the $fn marker), so
  spec's fn-sym returns ::s/unknown for it, matching the JVM's ns$fn__N.
- A fn with & {:as m} kwargs accepts a trailing map (Clojure 1.11): (f :a 1 {:b 2})
  and (f {:a 1}) both bind m, by merging an odd trailing map over the pairs.
- A thread responds to .getStackTrace (empty — jolt does TCO).

clojure.test-clojure.instr does not fully pass: its ::caller assertions need the
calling fn's stack frame, which TCO erases (an inherent host divergence, like the
JVM keeping tail frames).

make test green (+4 corpus rows, 0 new divergences), shakesmoke byte-identical.
Re-mint (backend emit-quoted + the destructure macro).
2026-06-27 21:56:04 -04:00
Yogthos
219d1e52c9 reader: #:ns{...} namespaced map literals
jolt's reader had no case for #: , so #:event{:type :search} died as an unknown
tagged literal. Now #:ns{...} qualifies each bare keyword/symbol key with ns
(:_/x stays unqualified, an already-qualified key is left alone); #::{...} uses
the current ns and #::alias{...} resolves the alias — matching Clojure.

clojure.spec.alpha's multi-spec test (which builds #:event{...} event maps) now
passes.

make test green (+1 corpus row, 0 new divergences), shakesmoke byte-identical.
One re-mint (the reader is a seed source).
2026-06-27 21:12:27 -04:00
Yogthos
0becba7f93 A fn def'd into a var reports a JVM-style class name (clojure.core$odd_QMARK_)
jolt fns reported (class f) = clojure.lang.IFn, so they carried no defining
symbol — clojure.spec.alpha's fn-sym (which reads a fn's class name to recover its
symbol) produced garbage, so explain-data's :pred for a bare-fn predicate was `/`
instead of e.g. clojure.core/keyword?.

Now def-var! records proc -> (ns . name) (first def of a proc wins, so an alias
like (def inc' inc) doesn't rename inc), and jolt-class-name returns "ns$munged"
for a known fn — matching the JVM, where (class odd?) is clojure.core$odd_QMARK_.
A munged fn class's ancestors include clojure.lang.AFunction's hierarchy
(IFn/AFn/Fn/Runnable/Callable), so (ancestors (class f)) still holds. Anonymous /
unregistered fns stay clojure.lang.IFn (fn-sym yields :unknown, as on the JVM).

This fixes explain-data / s/form / s/describe of bare-fn predicates in
clojure.spec.alpha (and unblocks parts of its suite + test.check's reporter test).

make test green (+1 corpus row, the (type inc) unit row updated to the JVM value),
shakesmoke byte-identical, runtime only (no re-mint).
2026-06-27 21:03:12 -04:00
Yogthos
10592fa746 drop (symbol var) corpus row — certify harness mis-evals (var ...); value is correct 2026-06-27 20:48:53 -04:00
Yogthos
48908f3a9b spec.alpha: (symbol var), Compiler/demunge, MultiFn .dispatchFn/.getMethod, fn .applyTo
General fixes from clojure.spec.alpha's test suite.

- (symbol a-var) returns the var's qualified symbol (clojure.spec.alpha/->sym).
- clojure.lang.Compiler/demunge reverses Clojure's name munging
  ("clojure.core$odd_QMARK_" -> clojure.core/odd?); spec's fn-sym uses it.
- clojure.lang.MultiFn .dispatchFn / .getMethod — spec's multi-spec walks a
  multimethod through them.
- (.applyTo f args) applies a fn to a seq of args (spec instrument).

Most of spec.alpha's conform/explain/describe suite passes. Remaining gaps:
explain-data's :pred for a BARE fn predicate (jolt fns don't carry their defining
symbol, so fn-sym can't recover it), #inst form rendering, and instrument — follow-up.

make test green (+3 corpus rows, 0 new divergences), runtime only (no re-mint).
2026-06-27 20:46:33 -04:00
Yogthos
4d61145e9c proxy [ThreadLocal] via thread-parameter; clojure.test/*testing-vars*
- (proxy [ThreadLocal] [] (initialValue [] body)) now builds a real per-thread
  store backed by a Chez thread-parameter, with a lazy initialValue; .get/.set/
  .remove work. Other proxies stay nil. test.check's no-seed PRNG (next-rng) uses
  one, so gen/sample and gen/generate (and everything built on them) now work.
- clojure.test/*testing-vars* (+ *report-counters*) are bound vars now, so a
  defspec run through its :test metadata / default reporter doesn't hit an unbound
  var.

make test green (+1 corpus row), shakesmoke byte-identical. One re-mint (proxy).
2026-06-27 19:51:49 -04:00
Yogthos
f32bd335e3 test.check generators: rand-double, take +Inf, UUID/Long/shiftLeft, transient
More general fixes from clojure.test.check's own suite.

- *unchecked-math* on doubles: unchecked-* only wrap integer math; on a flonum
  operand they're an ordinary float op (Clojure: (unchecked-multiply 1.5 2.0) =>
  3.0). test.check's rand-double is (* double-unit shifted) under *unchecked-math*
  and was truncating to a long 0, so every distribution-driven generator (choose,
  vector, …) collapsed to its lower bound.
- (take Double/POSITIVE_INFINITY coll) takes the whole coll instead of throwing
  on the infinite count coercion (rose-tree unchunk relies on it).
- (java.util.UUID. msb lsb) 2-long constructor (the uuid generator), formatted as
  the canonical lowercase 8-4-4-4-12 string; (Long. n) constructor; BigInteger
  .shiftLeft / .shiftRight (size-bounded-bigint); number methods now receive args.
- A transient (ITransientSet) responds to .contains / .valAt / .count
  (distinct-collection generators).

make test green (+3 corpus rows, 0 new divergences), runtime only (no re-mint).
2026-06-27 19:08:34 -04:00
Yogthos
992fc0af34 *unchecked-math* on macro-emitted arithmetic + local shadowing a bare native op
Two general fixes shaken out by clojure.test.check's own suite (its splittable
PRNG mixes 64-bit longs and binds locals named min/max).

- *unchecked-math* now wraps arithmetic a macro emits. The analyzer rewrote a
  bare (+/-/*) to its wrapping unchecked-* under *unchecked-math*, but a macro's
  syntax-quote produces clojure.core/* (qualified), which was skipped — so e.g.
  test.check's mix-64 multiply grew to a bignum instead of a 64-bit long. The
  rewrite now also fires on the clojure.core-qualified form.
- A local binding named like a bare-emitted native op no longer shadows it. ops
  where native-ops maps the name to itself (+ - * / < > min max …) emit as the
  bare Scheme name; a local `max` emitted the same token, so
  (fn [max] (clojure.core/max …)) called the param. munge-name now prefixes such
  locals, like reserved words (derived from native-ops so they can't drift).

make test green (+1 corpus row, 0 new divergences), shakesmoke byte-identical.
One re-mint (analyzer + backend).
2026-06-27 18:19:14 -04:00
Dmitri Sotnikov
75652de1ad
Merge pull request #261 from jolt-lang/conformance/algo-monads
algo.monads: a seq reports IPersistentList for protocol dispatch
2026-06-27 21:43:15 +00:00
Yogthos
d38402eb57 algo.monads: a seq reports IPersistentList for protocol dispatch
algo.monads' writer monad extends a protocol to clojure.lang.IPersistentList,
but jolt's lists only reported ASeq/ISeq in value-host-tags, so writer-m-add
didn't dispatch ("No method writer-m-add"). jolt models every seq as a list (no
distinct LazySeq — (class (map inc xs)) is PersistentList), so a seq now also
reports PersistentList / IPersistentList / IPersistentStack, in value-host-tags
and host-type-set. extend-protocol clojure.lang.IPersistentList then dispatches
on a list.

algo.monads passes its whole suite (11/11) over tools.macro. Listed in docs +
site. Runtime only, no re-mint. make test green (+1 corpus row, 0 new
divergences), shakesmoke byte-identical.
2026-06-27 17:38:48 -04:00
Dmitri Sotnikov
65cf6ac3d4
Merge pull request #260 from jolt-lang/conformance/letfn-star
letfn is a macro over a letfn* special form (Clojure semantics)
2026-06-27 21:30:58 +00:00
Yogthos
21cd88deee letfn is a macro over a letfn* special form (Clojure semantics)
jolt modelled letfn as a special form directly, so (macroexpand-1 '(letfn …))
returned the form unchanged. Clojure's letfn is a macro that expands to letfn*,
and macroexpansion tooling (tools.macro, tools.analyzer) depends on that — its
special-form handlers key on letfn*, not letfn.

Split it the Clojure way:
- letfn* is now the special form (analyzer), taking flat name/fn-form pairs
  [name1 fn1 name2 fn2 …] — the letrec :let lowering is unchanged.
- letfn is a macro (00-syntax) turning each (name [params] body*) spec into a
  name + (fn name [params] body*) binding, so it expands to letfn*.

So (macroexpand-1 '(letfn [(f [x] x)] (f 1))) now yields
(letfn* [f (fn f [x] x)] (f 1)), and clojure.tools.macro passes its whole suite
(macrolet / symbol-macrolet / mexpand-all). Listed in docs + site.

make test green (+1 corpus row, 0 new divergences), shakesmoke byte-identical.
One re-mint (analyzer + the letfn macro); selfhost holds.
2026-06-27 17:26:18 -04:00
Dmitri Sotnikov
7891fa0d55
Merge pull request #259 from jolt-lang/conformance/ns-vector-clauses
ns: accept vector reference clauses; add Compiler/specials
2026-06-27 21:14:01 +00:00
Yogthos
192ef66e7e ns: accept vector reference clauses; add Compiler/specials
Two general fixes shaken out by clojure/tools.macro.

- The ns macro now accepts a vector reference clause [:require …] / [:use …],
  not just the list form (:require …). Clojure dispatches on (first clause) and
  accepts both; jolt silently dropped vector clauses, so a ns written with them
  loaded with nothing required/used (tools.macro's test ns uses [:use …]).
- clojure.lang.Compiler/specials is now a static whose keys are the special-form
  symbols (matching Clojure 1.2/1.3). Macroexpansion tooling reads
  (keys Compiler/specials) to know which heads not to expand.

tools.macro itself isn't fully passing yet — its mexpand-all works, but the
macrolet/symbol-macrolet tests need letfn to macroexpand to letfn* (jolt models
letfn as a special form, not a macro over letfn*), so it stays off the list.

make test green (+1 corpus row, 0 new divergences), shakesmoke byte-identical.
One re-mint (the ns macro).
2026-06-27 17:08:50 -04:00
Dmitri Sotnikov
1ba79aa223
Merge pull request #258 from jolt-lang/conformance/data-priority-map
data.priority-map: deftype interop fixes (rseq, arity-overload, empty, Sorted)
2026-06-27 20:52:54 +00:00
Yogthos
75f6bc79d1 data.priority-map: deftype interop fixes (rseq, arity-overload, empty, Sorted)
data.priority-map's whole suite passes (4/4). It leans on deftype/collection
interop jolt got wrong; four general fixes:

- rseq dispatches to a deftype's clojure.lang.Reversible.rseq method instead of
  always demanding a vector/sorted-coll (natives-seq.ss).
- a deftype method declared at two arities from two interfaces now dispatches by
  arity: the priority-map has seq[this] (Seqable) and seq[this ascending]
  (Sorted), so (.seq pm false) must reach the 2-arg one. find-method-any-protocol
  now matches the call's arg count via procedure-arity-mask, and a deftype's own
  declared method wins over the generic collection interop in dot-forms.
- (empty x) on a deftype/record with its own empty method uses it rather than
  returning {} (jolt.host/jrec-method? gate in clojure.core/empty).
- clojure.lang.Sorted (comparator / entryKey / seqFrom) works on jolt's
  sorted-map/set, so subseq/rsubseq run — including the priority-map delegating
  .comparator to its backing sorted-map (dot-forms.ss + host-static.ss).

Listed in docs/libraries.md + the site. One re-mint (clojure.core/empty);
everything else runtime. make test green (0 new divergences), shakesmoke
byte-identical.
2026-06-27 16:48:14 -04:00
Dmitri Sotnikov
8a4df7b204
Merge pull request #257 from jolt-lang/numeric/unchecked-math-wrap
Long compatibility: *unchecked-math* wrapping + ^long is 64-bit (unblocks test.check)
2026-06-27 20:10:07 +00:00
Yogthos
3340635714 ^long is a 64-bit long: fast-path-with-fallback ops + logical unsigned shift
Completes the JVM long-compatibility gap so clojure.test.check (and the
property-based suites built on it, e.g. data.codec) run on jolt.

A ^long is 64-bit but a Chez fixnum is only 61-bit, so the backend's fast fx
comparison / quot / min / max / inc / dec ops raised on a full-width long (one
from the PRNG or wrapping arithmetic). They now go through the jolt-l* macros
(host/chez/seq.ss): the fx fast path when the operands ARE fixnums, the generic
op otherwise — so e.g. ((fn [^long a ^long b] (< a b)) Long/MAX 1) is false, not
an error. Arithmetic +/-/* keep the raw fx ops (under *unchecked-math* they're
already the wrapping unchecked-*).

Also fixes unsigned-bit-shift-right: it was an arithmetic (sign-propagating)
shift, now a logical shift over the 64-bit two's-complement window, so
(unsigned-bit-shift-right -1 1) is 2^63-1 like the JVM.

Result: test.check 1.1.3 loads and runs (generators, quick-check, shrinking);
data.codec's base64 property suite passes (12/12 defspecs; the 2 deftests check
clojure.lang.IFn$OLLOL, a JVM primitive-fn interface, N/A). Both added to
docs/libraries.md + the site.

re-mint (backend/seed). make test green (+3 corpus rows, 0 new divergences,
numeric gate updated to the jolt-l* ops), shakesmoke byte-identical.
2026-06-27 16:04:19 -04:00
Yogthos
a028cab04f Unchecked / *unchecked-math* arithmetic wraps to signed 64-bit
clojure.core's unchecked-* (and +/-/*/inc/dec under *unchecked-math*) are long
ops that WRAP on overflow; jolt's checked arithmetic is arbitrary-precision and
its unchecked-* were plain non-wrapping (+ x y), diverging from the JVM. Now they
truncate to the low 64 bits as a signed long, matching Clojure:

  (unchecked-add 9223372036854775807 1)        => -9223372036854775808
  (unchecked-multiply 9223372036854775807 …)   => 1

- host/chez/seq.ss: jolt-wrap64 + binary jolt-unc{add,sub,mul,inc,dec,neg}2 and
  the variadic clojure.core/unchecked-* fns (def-var!'d in natives-seq.ss, where
  def-var! is bound). The overlay's plain unchecked-* defns are removed.
- backend lng-ops: unchecked-+/-/* emit the wrapping jolt-unc* helpers (the
  raising fx ops can't wrap on Chez's 61-bit fixnums); unchecked-inc/dec too.
- *unchecked-math* is honored: the analyzer reads it (jolt.host/unchecked-math?)
  and rewrites +/-/*/inc/dec to their unchecked-* for the rest of a file that
  (set!)s it, like the JVM.
- jolt->fx: a ^long value that overflows the 61-bit fixnum range passes through
  as an exact integer instead of erroring (a full-width long from wrapping math).

Also adds Long/bitCount / numberOfLeadingZeros / reverse and Math/getExponent /
scalb (test.check's splittable PRNG uses them).

This lets clojure.test.check load and run quick-check on jolt. re-mint (analyzer/
backend/overlay are seed sources). make test green (+6 corpus rows, 0 new
divergences, numeric gate updated), shakesmoke byte-identical.
2026-06-27 15:41:35 -04:00
Dmitri Sotnikov
86dd9650b6
Merge pull request #256 from jolt-lang/conformance/tools-reader
conformance: general fixes from tools.reader, core.contracts, data.zip, data.csv (+ math.combinatorics)
2026-06-27 19:06:19 +00:00
Yogthos
44837f01ab data.csv: fully passes, three general fixes
clojure.data.csv runs its whole suite on jolt (4/4 reading/writing/eof/line-
endings). Three general gaps fixed, all runtime, no re-mint, JVM-certified:

- The prefix-list form of :require/:use — (:require (clojure [string :as str]))
  means clojure.string :as str — now expands (loader.ss). It silently failed
  before, trying to load a "clojure" namespace.
- extend-protocol to java.io.Reader / Writer / StringReader / PushbackReader now
  dispatches: those reader/writer host tags carry the right class names in
  value-host-tags AND are in host-type-set, so extend-protocol registers under
  the canonical tag instead of a local ns tag (records.ss). data.csv's
  Read-CSV-From protocol extends to String / Reader / PushbackReader.
- (str StringWriter) returns its accumulated content (register-str-render for the
  "writer" jhost), not the opaque host object — data.csv writes CSV to one and
  reads it back.

Listed in docs/libraries.md + the site.

make test green (+2 corpus rows, 0 new divergences), shakesmoke byte-identical.
2026-06-27 15:02:32 -04:00
Yogthos
745d22260f data.zip: add clojure.zip/xml-zip; clojure.xml lives in jolt-lang/xml
clojure.zip was missing xml-zip — a zipper over xml {:tag :content} elements,
which clojure.data.zip and any xml-zipper code needs. Added (runtime, loaded on
require). clojure.data.zip's whole xml suite (9/9) then passes, once XML parsing
is provided: clojure.xml/parse now ships in jolt-lang/xml over its
javax.xml.stream pull parser (committed there).

Listed in docs/libraries.md + the site.
2026-06-27 14:49:49 -04:00
Yogthos
a83ff6ce40 core.contracts: fully passes, two general fixes
clojure.core.contracts (over core.unify) now runs its whole suite on jolt —
14/14 across contracts/constraints/with-constraints/provide tests. Two general
gaps fixed:

- Symbol and Keyword now report IFn (and Fn/Runnable/Callable) in the modeled
  class hierarchy, so a (class x)-dispatched multimethod with an IFn method
  matches a symbol or keyword, like the JVM (both implement IFn — they're
  callable). core.contracts' funcify* dispatches on (class constraint) and a
  bare predicate symbol must hit the IFn arm. Runtime, no re-mint.
- A live Var value spliced into a form by a macro (defcurry-from resolves a var
  and emits (~v l r)) now compiles: analyze treats a var-cell form as a
  :the-var reference by ns+name, the same node as (var ns/name), mirroring the
  existing spliced-namespace (~*ns*) case. analyzer.clj + host-contract.ss,
  re-mint (prelude stays byte-identical; only the analyzer image changes).

Listed in docs/libraries.md + the site.

make test green (+2 corpus rows, 0 new divergences), shakesmoke byte-identical.
2026-06-27 14:32:57 -04:00
Yogthos
2c5b7dd918 libraries: add math.combinatorics
Its full suite (18 deftests) passes on jolt unchanged — pure Clojure over
seqs, no host interop.
2026-06-27 14:18:05 -04:00
Yogthos
e16085402b General fixes shaken out by clojure/tools.reader
Running clojure.tools.reader's own suite on jolt surfaced a batch of general
gaps (all runtime, JVM-certified, no re-mint — reader.ss is loaded at runtime
and jolt-core has no octal literals, so selfhost holds):

Reader:
- (load "rel") resolves a non-/ path against the current namespace's directory,
  like Clojure — (load "common_tests") from clojure.tools.reader-test loads
  clojure/tools/common_tests.clj. Was resolved against the roots directly.
- Octal integer literals: 042 reads as 34, not decimal 42; octal string escapes
  (\377 is one char, not \0 + "00"). \oNNN char octal already worked.
- (symbol nil name) now equals (symbol name) and the reader literal — a nil
  namespace is the #f no-ns sentinel, not jolt-nil (jolt= compares ns by equal?).

clojure.test:
- thrown-with-msg? honors the class hierarchy (instance?) before falling back to
  a simple-name match, so (thrown-with-msg? RuntimeException ...) matches an
  ExceptionInfo, like thrown? already did.

Host interop (java layer):
- java.util.regex: Pattern.matcher / Matcher.matches / .group / .groupCount /
  .find, and Pattern/compile.
- clojure.lang: RT/map, PersistentList/create, PersistentHashSet/createWithCheck.
- java.lang.Character: digit / isDigit / isWhitespace / valueOf.
- java.util.LinkedList (Deque surface over the ArrayList backing); ArrayList /
  LinkedList are now seqable.
- BigInteger 2-arg ctor (string, radix) + .negate / .bitLength / .signum / .abs;
  BigInt/fromBigInteger and Numbers/reduceBigInt (identity on jolt's exact ints).

Suite: reader_test 22/30, reader-edn_test 13/16. The remaining failures are
fundamental numeric-model differences (no BigDecimal type; BigInt and Long are
one exact-integer type) or need JVM reflection (record/ctor tagged literals via
getConstructors) — out of scope.

make test green (+8 corpus rows, 0 new divergences), shakesmoke byte-identical.
2026-06-27 14:11:02 -04:00
Dmitri Sotnikov
850a84c272
Merge pull request #255 from jolt-lang/conformance/core-typed
core.typed runtime contracts: instance? Object + Compiler/LINE
2026-06-27 17:37:18 +00:00
Yogthos
720734a481 Two general fixes shaken out by core.typed's runtime contract suite
Running clojure.core.typed's runtime contract tests (typed/runtime.jvm,
test_contract — 5/5 pass) surfaced two general jolt gaps, both runtime, both
JVM-certified:

- instance? Object / java.lang.Object returned false for everything. Object is
  the root of the type hierarchy: every non-nil value is an instance of Object,
  nil is not. core.typed's (instance-c Object) contract depends on this; many
  libraries do.
- @Compiler/LINE and @Compiler/COLUMN (clojure.lang.Compiler statics — Vars on
  the JVM holding the line/column of the form being compiled) were unresolved.
  Macros read @Compiler/LINE as a fallback when &form carries no position. Now
  backed by derefable cells updated per top-level form, like *current-source*.

The core.typed type checker itself (tools.analyzer.jvm + ASM bytecode +
clojure.lang.Compiler internals) and the cljs runtime are not portable, so the
checker/check-ns surface is out of scope; this is the runtime contract layer.

make test green (+4 corpus rows, 0 new divergences), shakesmoke byte-identical.
2026-06-27 13:33:30 -04:00
Dmitri Sotnikov
9805620997
Merge pull request #254 from jolt-lang/license/epl-2.0
Relicense under EPL-2.0
2026-06-27 17:21:02 +00:00
Yogthos
5c99ff8c79 Relicense under EPL-2.0
EPL-1.0 is superseded by EPL-2.0 (clearer jurisdiction/patent terms). Updates
the LICENSE file and README. Clojure-derived files under stdlib/ and the
vendored sci keep their original EPL-1.0 headers per the weak-copyleft terms.

Closes #206
2026-06-27 13:17:19 -04:00
Dmitri Sotnikov
c479010536
Merge pull request #253 from jolt-lang/conformance/core-async
core.async: higher-level API over native channels + general fixes
2026-06-27 17:09:20 +00:00
Yogthos
4cf95dc27c core.async: higher-level API over native channels + two general fixes
Adds clojure.core.async's higher-level dataflow API as a Clojure overlay
(stdlib/clojure/core/async.clj) over jolt's native channel primitives, plus
clojure.core.async.lab. The native layer (host/chez/java/async.ss) gains
offer!/poll!, put specs and :priority/:default in alts!, a transducer
ex-handler arg to chan, unblocking-buffer?, promise-buffer, and on-caller?
handling for put!/take!. The overlay covers alts!/pipe/pipeline/split/
reduce/transduce/into/take/mult/mix/pub-sub/map/merge/onto-chan/to-chan and
the deprecated map</map>/filter>/... family (rewritten as go-loops since the
JVM versions reify the impl handler protocol jolt doesn't expose).

Loading: the native primitives pre-seed clojure.core.async, so the loader now
drops it from the loaded set and a require pulls the overlay from the source
roots like clojure.test (AOT-bundled into built binaries).

Running clojure/core.async's own suite shook out two general bugs:
- :refer with a list form, (:require [ns :refer (a b c)]), dropped the names
  (only the vector form was handled) — chez-register-spec! now accepts both.
- (range 0) / (range 5 5) returned nil instead of the empty seq () — empty
  ranges now match Clojure, so (= () (range 0)) holds.

Suite: async_test 15/20, pipeline_test 7/7, timers_test 2/2, lab_test 2/2.
The five non-passing async_test cases all assert JVM go-machine limitations
jolt's thread-based model is a superset of (the 1024 pending-op cap, parking
ops that must throw outside a go block, expanding-transducer buffer
backpressure) or dispatch-thread identity, not data semantics.

make test green (0 new divergences, +4 range corpus rows), shakesmoke
byte-identical.
2026-06-27 13:05:19 -04:00
Dmitri Sotnikov
4007af8d6a
Merge pull request #252 from jolt-lang/conformance/core-logic-fd
core.logic fd + &env: instance? boundary, ~ unquote, macro &form/&env
2026-06-27 16:06:04 +00:00
Yogthos
438742702a Macros receive &form and &env
A macro body can now read &form (the call form) and &env (a map of the in-scope
local symbols), like Clojure. This is what core.logic's matche/defne use to tell
a pattern symbol that names an enclosing local from a fresh pattern var — so
locals-membero and the recursive checko in `matches` now compute correctly. The
suite reaches 535/2/0 (the last two are constraint reification ORDER, where the
constraint set is right but it is spliced from a set whose iteration order differs
from the JVM — a host set-ordering divergence, not a bug).

&form/&env are clojure.core dynamic vars bound around each expander call rather
than prepended params, so the macro calling convention is unchanged and the mint
stays consistent (the seed prelude is byte-identical; only the analyzer carries
the env into form-expand-1). macroexpand-1 passes an empty env.

corpus.edn: the ~@ unquote row is now a boolean compare (a bare clojure.core/
unquote-splicing symbol evaluates to an unbound var, not the symbol).
2026-06-27 11:54:47 -04:00
Yogthos
f46772d576 fd subsystem: instance? name-boundary + ~ reads as clojure.core/unquote
Two general fixes that clear core.logic's finite-domain -difference, safefd, and
the defne quoted-list patterns (form->ast), taking the suite to 532/5/0.

- instance? on a deftype matched a simple type name against the qualified tag by
  raw string suffix, so "a.b.MultiIntervalFD" tested true for IntervalFD. The
  suffix must land on a "." boundary. core.logic's fd dispatches on
  (interval? x) = (instance? IntervalFD x), and a MultiIntervalFD wrongly counted
  as an interval, so -difference/safefd computed the wrong set.
- the reader reads ~ / ~@ as clojure.core/unquote(-splicing), like the JVM reader,
  instead of a bare unquote. Code that inspects quoted pattern/template data —
  core.logic's defne checks (= f 'clojure.core/unquote) — now sees the symbol it
  expects, so '(fn ~args . ~body) patterns compile. hc-head-is? accepts the
  qualified head in syntax-quote lowering; the value-preserving change leaves the
  minted seed byte-identical.

corpus.edn: 2 JVM-certified unquote rows. unit.edn: two reader rows updated to the
qualified unquote. make test + shakesmoke green, 0 new divergences, self-host holds.
2026-06-27 11:22:01 -04:00
Dmitri Sotnikov
42c163bacb
Merge pull request #251 from jolt-lang/conformance/core-logic-clp
core.logic constraint layer: CLP/unifier fixes
2026-06-27 15:07:32 +00:00
Yogthos
580e3a1407 A defrecord exposes its clojure.lang interfaces for protocol dispatch
value-host-tags returned only ("Object") for a record, so a protocol extended to
clojure.lang.IRecord / IPersistentMap / Associative / Seqable / … (and not to the
record's own type) dispatched to the Object default. core.logic extends IWalkTerm
to IRecord, and walking a record value hit Object's (walk-term [v f] (f v)) — which
re-enters walk* and loops forever (the test-53-lossy-records hang).

A defrecord now carries the map/record interface tags it genuinely satisfies. The
record's own type is still tried first (jrec-tag, before these tags), so a direct
extend to the record type wins, and record equality / map? / record? are unchanged.
A bare deftype stays opaque (its tag + Object; declared interfaces dispatch via its
inline methods). Runtime only, no re-mint.
2026-06-27 10:57:37 -04:00
Yogthos
5411db3729 Track :refer-clojure :exclude so syntax-quote qualifies an excluded name to the current ns
A name a namespace excludes from clojure.core (:refer-clojure :exclude) is not
clojure.core/name even before the ns defines its own — syntax-quote must qualify
it to the current ns, like Clojure. refer-clojure was a no-op, so a syntax-quoted
excluded name (core.logic.fd's `==`, referenced by a constraint's -rator before fd
defines ==) resolved to clojure.core/==.

jolt-refer-clojure now records the :exclude set per ns; hc-sq-symbol consults it
before falling back to clojure.core. Fixes core.logic's fd constraint -rator names.
Runtime only, no re-mint.
2026-06-27 10:46:52 -04:00
Yogthos
e6aa2aace7 core.logic constraint layer: fixes for the CLP/unifier failures
Follow-on to the core.logic relational-engine work. These clear every crash in
core.logic's constraint-logic-programming and unifier layers (33 errors -> 0) and
most of the value mismatches; the suite goes 504 -> 523 passing assertions. All
are general gaps, not core.logic-specific.

- symbols intern their ns/name strings (JVM Symbol.intern .intern()s them): two
  separately-read `?a` symbols now share one name-string object. core.logic's
  non-unique lvars compare names by identity (via (str sym)), so without this a
  term's lvar and a constraint's lvar built from different `?a` reads never matched
  and constraints silently never fired.
- (str x) of a single arg returns its rendering directly instead of copying through
  string-append, and a symbol stringifies to its (interned) name — JVM (str x) is
  x.toString(). Needed for the identity comparison above.
- a clojure.core-qualified special form dispatches correctly: syntax-quote
  namespace-qualifies a macro like letfn to clojure.core/letfn (matching Clojure,
  where it's a macro), and the analyzer now maps that back to the special form
  instead of treating it as an invoke of a nil var. core.logic's fnc/defnc emit
  (clojure.core/letfn ...). Re-mint.
- (disj nil ...) is nil (JVM), instead of crashing in the set path — core.logic's
  constraint store does (disj (get km v) id) where the get can be nil.

corpus.edn: 4 JVM-certified rows. make test + shakesmoke green, 0 new divergences,
self-host fixpoint holds.
2026-06-27 10:37:32 -04:00
Dmitri Sotnikov
36105ba702
Merge pull request #250 from jolt-lang/conformance/core-logic
General fixes shaken out by running core.logic
2026-06-27 14:00:15 +00:00
Yogthos
9dbfd7e5c1 General fixes shaken out by running core.logic's test suite
Running clojure/core.logic's own suite surfaced a batch of general jolt gaps.
None are core.logic-specific; each is a language/host behavior that was wrong or
missing. With these, the core relational engine (unify, run/fresh/conde,
conso/membero/appendo, reification to _0/_1, lcons) runs; the remaining failures
are in core.logic's constraint-logic-programming and finite-domain layers
(tracked separately).

- analyzer: accept the list-member dot form (. target (method args)), sugar for
  (. target method args). Re-mint.
- identical? is reference identity (eq?), not value equality. It was aliased to =,
  which infinite-loops when a deftype's .equals short-circuits on (identical? this o)
  (core.logic's Substitutions) and is wrong for distinct equal collections.
- jrecs use a deftype's declared hashCode/equals/equiv for map/set keying instead
  of structural field comparison, so metadata-wrapped keys still match (core.logic
  keys substitutions on lvar id, ignoring metadata).
- meta/with-meta dispatch to a deftype's clojure.lang.IObj meta/withMeta methods
  when present, so metadata threaded through the type's own assoc/withMeta survives
  (previously kept in an identity side-table the reconstructed instances didn't share).
- coll?/seqable? on a deftype require IPersistentCollection (cons) or ISeq (first);
  ILookup(valAt)/Indexed(nth)/Counted(count)/Seqable(seq) alone no longer qualify,
  matching the JVM.
- syntax-quote resolves a bare symbol to the compile ns's own def before
  clojure.core, so a name the ns excluded and redefined (core.logic's == after
  :refer-clojure :exclude) qualifies correctly in macro output.
- reader: record literals #ns.Type{...} / #ns.Type[...] expand to the map->/->
  factory call.
- structmap API: defstruct/create-struct/struct-map/struct/accessor (map-backed,
  insertion-ordered). Re-mint.
- .hashCode on strings/symbols (Java String.hashCode, Symbol Util.hashCombine);
  Class.isInstance; java.util.Collection.contains over vector/list/set;
  clojure.lang.RT/nextID and clojure.lang.Util hash/hasheq/equiv/identical statics.

corpus.edn: 8 JVM-certified rows. unit.edn: a Counted+Seqable deftype is coll?=false
(was a stale expectation encoding the old behavior).
2026-06-27 09:20:11 -04:00
Dmitri Sotnikov
af91dbbaa6
Merge pull request #249 from jolt-lang/conformance/map-insertion-order
Small maps preserve insertion order
2026-06-27 12:14:26 +00:00
Yogthos
bfa2cbf49d Small maps preserve insertion order
jolt maps were HAMTs with hash iteration order; Clojure keeps small maps as
PersistentArrayMap (insertion order), converting to PersistentHashMap past a
threshold. Map literals, array-map, assoc, into/transient, merge, zipmap,
select-keys, update-keys/vals, frequencies and group-by now iterate in insertion
order for <=8 entries, matching the JVM. hash-map and >8-entry maps stay hash
order; sets stay hash order.

The pmap record gains an order field (the insertion-order key list, or #f once
hashed); the HAMT still backs the values so equality/hash/lookup are unchanged.
pmap-fold visits an array-mode map last-to-first so the runtime's cons-accumulate
idiom reconstructs insertion order without touching its many call sites, and
hash-mode output stays byte-identical; pmap-fold-fwd visits in order for the few
sites that build a value directly. Transient maps track insertion order and
promote to hash past max(8, source-count), matching TransientArrayMap.

The hash-map native-op retargets to a hash-order builder so (hash-map ...) stays
hash-ordered while {...} literals are ordered; syntax-quote builds maps via the
hash builder (Clojure expands `{...} to apply hash-map). The core overlay map
builders seed from {} instead of (hash-map) to keep order.

Threshold is 8 for any key (the keyword exception in newer Clojure isn't in
1.12.5). honeysql now passes 832/0/0; 19 JVM-certified corpus rows added.
2026-06-27 05:48:17 -04:00
Dmitri Sotnikov
e2efff6c8e
Merge pull request #248 from jolt-lang/conformance/reitit-aero-honeysql
Conformance: reitit 327/0/0, aero 59/0/0, honeysql 638/6
2026-06-27 08:57:08 +00:00
Yogthos
a99991a818 defn- marks :private; ns-publics drops private vars
defn- now adds :private to the var metadata (like Clojure), and ns-publics
filters those out while ns-interns/ns-map keep them — they were all the same
unfiltered scan before. A lib that introspects ns-publics (honeysql asserts
every public helper has a docstring, and that the clause set matches the public
helpers) saw the private defn- helpers and failed; now honeysql 636/8 -> 638/6
(the rest are map key-order).
2026-06-27 01:27:47 -04:00
Yogthos
4df3d0fa34 Add a java.util.Locale shim (no-op default locale)
jolt's case ops are codepoint-based and locale-independent, so the default
locale is a no-op token: getDefault/setDefault/forLanguageTag + ROOT/US/ENGLISH.
honeysql sets and restores the locale around formatting to assert output is
locale-stable (its Turkish-İ regression guard) — that test errored on the
missing Locale/setDefault static, now passes (honeysql 635/8/1 -> 636/8/0).
2026-06-27 01:21:42 -04:00
Yogthos
135bad9d3a edn: read raw forms so a #tag goes through :readers/:default
clojure.edn/read built the built-in #inst/#uuid eagerly (via read-string), so a
:readers override couldn't win and #inst applied to a non-string form (aero's
#inst ^:ref […]) threw. Read the raw form instead and let edn->value route every
tag through :readers then :default then the built-in — matching clojure.edn,
where a reader from opts wins. edn->value now also converts the (recursively
converted) metadata, since the raw path skips the read-string data seam. aero
suite: 59/0/0 (full pass). clojure.edn baked, re-minted.
2026-06-27 01:15:33 -04:00
Yogthos
3491312ca1 with-meta on a list/seq returns a fresh copy, not the original
meta-copy keyed metadata on the SAME cseq/lazyseq object (the else branch), so
(with-meta xs m) mutated the original list in place — Clojure's PersistentList
is immutable and withMeta returns a new list. (with-meta xs {:k xs}) thus built
a self-referential cycle (the list's metadata pointed at the list), which looped
*print-meta* printing forever — the root of aero's meta-preservation hang. Now
copies the cseq/lazyseq node like the other collections. aero suite completes:
58/0/1 (was hanging).
2026-06-27 01:10:34 -04:00
Yogthos
afc733a439 edn: apply tag readers inside a set literal
clojure.edn/read with :readers/:default recursed into vectors/maps/seqs but
not a constructed set, so a tagged literal in #{…} (aero's #ref in a set) kept
its raw form. edn->value now recurses into a set. clojure.edn is baked into the
seed, re-minted. Fixes aero #ref-in-set + falsey-user-return.
2026-06-27 00:07:49 -04:00
Yogthos
eb64240e29 Read metadata as data, consistently (sets, empty lists)
Clojure's reader reads a ^{…} map with the same read() as any value, so a set/
tagged literal in metadata is a value, not a form. jolt's data seam converted a
set-form to a set in the VALUE but left it as the tagged form inside the
METADATA, and dropped metadata on an empty list entirely (a wrong 'interned, =
Clojure' special case in rdr-attach-meta — Clojure's MetaReader withMetas () via
IObj). rdr-form->data now always converts + carries the (recursively converted)
metadata, whether or not the value structure changed; rdr-attach-meta no longer
skips (). Fixes aero's meta-preservation (set/map/vector/empty-list ds round-
trip). All runtime .ss (data seam), no re-mint.
2026-06-26 23:52:22 -04:00
Yogthos
6b99591266 Fix [_ _] inline method field binding + Var protocol dispatch
Two gaps reitit-core surfaced (now 322/0/1 -> 327/0/0):

- A deftype/defrecord inline method with two _ params, (m [_ _] field), read
  the field as nil: mk-clause bound fields off (get _ :field) where _ was the
  first param, but the second _ shadowed it. Each _ param is now renamed to a
  fresh symbol so the instance is unambiguous.

- A var did not dispatch to a protocol's clojure.lang.Var extension (reitit
  extends Expand to Var for a #'handler route): value-host-tags gained a var arm
  (Var/clojure.lang.Var/IDeref/IFn) and host-type-set gained Var/IDeref so the
  extension keys under Var.

deftype/defrecord is a seed source, re-minted.
2026-06-26 23:22:22 -04:00
Dmitri Sotnikov
9404512b97
Merge pull request #247 from jolt-lang/conformance/lib-fixes
More conformance fixes: defn attr-map, set-in-macro, record interop, Byte/Short
2026-06-27 03:09:09 +00:00
Yogthos
2fd9763d94 Add java.lang.Byte / Short / Float class tokens + Byte/Short statics
jolt had Long/Integer/Double class tokens but not Byte/Short/Float, and no
Byte/Short MIN_VALUE/MAX_VALUE/valueOf/parse* statics. clojure.test.check (a
malli dependency) references Byte/MIN_VALUE and Byte/MAX_VALUE. The values are
plain integers on jolt; the statics expose the JVM ranges (127/-128, 32767/
-32768).
2026-06-26 23:04:55 -04:00
Yogthos
ed1ea46ca2 Records delegate their clojure.lang interface methods to the map fns
A defrecord is Associative/ILookup/IPersistentMap/Seqable/Counted on the JVM,
so (.assoc r k v) / (.valAt r k) / (.without r k) / (.containsKey r k) /
(.cons r x) / (.count r) / (.seq r) / (.equiv r o) / (.entryAt r k) now work
via Java interop, delegating to the map fns when not overridden by a declared
method. reitit's impl calls (.assoc match k v) directly. A bare deftype uses
its own declared methods (record-only branch). reitit-core 58/1/18 -> 321/1/1
with the router lib's Trie shim.
2026-06-26 22:49:51 -04:00
Yogthos
5cd8d15ae7 A set literal reaches a macro as a set value
#{...} reads as the tagged set-form for the analyzer, but a macro saw that
map instead of a set (set? false / map? true, unlike a vector). hc-expand-1
now converts a set-form argument to a real set before calling the expander, so
(set? arg)/conj/seq work — hiccup's compiler introspects a literal set this way
(str (html #{"<>"}) was empty, now #{&quot;&lt;&gt;&quot;}). Elements stay as
read; a deeply-nested set literal inside another form is left for the analyzer.
hiccup 382->383. Jolt-side unit guards (macro def+use in one form isn't
JVM-portable).
2026-06-26 22:42:19 -04:00
Yogthos
bd645a68d6 defn: support the attr-map form
(defn name docstring? {:k v} arglists...) and the multi-arity name+attr-map
now merge the attr-map into the var metadata like Clojure — jolt was parsing
the map out of the body and discarding it. The metadata (the name's own ^{},
the attr-map, and the docstring as :doc) is attached to the def name symbol,
which analyze-def reads and evaluates. defn is in the earliest tier, so the
macro uses only conj/assoc/meta/with-meta (not merge/last). The rare trailing
attr-map (after the last arity) is not yet handled. Fixes hiccup's defelem
meta + honeysql docstring tests.
2026-06-26 22:29:47 -04:00
Dmitri Sotnikov
ab63966ab6
Merge pull request #246 from jolt-lang/fix/conj-on-lazyseq
Fix two regressions from the lazy-seq / deftype work (#245)
2026-06-27 02:20:08 +00:00
Yogthos
b74dbfd2f0 Symbols are IFn (invoke as a map lookup)
('sym coll) / ('sym coll default) now do (get coll 'sym ...), like keywords —
a symbol is IFn on the JVM. jolt threw "cannot be cast to clojure.lang.IFn".
Pre-existing gap (not a regression), surfaced by honeysql's :checking mode,
which does ('where dsl) to look up a clause. honeysql 623/13/8 -> 635/8/1.
Corpus rows added.
2026-06-26 22:05:21 -04:00
Yogthos
3dc5de91e5 Fix map? for a deftype implementing IPersistentMap
The deftype-is-not-a-map change (#245) gated map? on jrec-record?, so only a
defrecord was map?. But a deftype that implements clojure.lang.IPersistentMap is
map? on the JVM — clojure.core.cache's caches are exactly that, and its TTL
factory asserts (map? base) on an LRUCache passed as the base (its suite went
1314 -> 2 errors). map? now also covers a deftype whose without/dissoc method is
registered — the IPersistentMap-distinctive op a vector or set lacks. An opaque
deftype (RawString) stays non-map?; a defrecord stays both. Guards added to
unit.edn (jolt-side: a full IPersistentMap impl will not compile on the JVM
corpus oracle).
2026-06-26 21:37:32 -04:00
Yogthos
271411b3e2 Fix conj on a lazy-seq, add lazy-seq interop regression rows
The rest = more() change made (rest coll) return a jolt-lazyseq, so the very
common (conj (rest xs) y) hit jolt-conj1's base case, which doesn't recognize a
lazyseq, and threw "conj: unsupported collection" (caught by core.match's
seq-pattern compiler). conj on a lazy-seq now prepends like conj on any seq.

The corpus had no row exercising a collection op on a rest-derived seq, so the
class slipped past the gate; add a seqs/lazy-seq-interop suite (conj/into/first/
count/nth/reduce/map/filter/apply/cons/=/empty?/seq over (rest …) and lazy-seq),
all JVM-certified.
2026-06-26 21:25:10 -04:00
Dmitri Sotnikov
1d55d9fa27
Merge pull request #245 from jolt-lang/conformance/close-gaps
Close conformance gaps: regex, exceptions, printer vars, lazy-seq model, deftype/str/print
2026-06-27 01:05:33 +00:00
Yogthos
28d938c396 Review fixes: print fast-path + regex zero-width advance
- with-deeper-print only parameterizes the print depth when *print-level* is
  set, so printing pays no parameterize on the common nil-default path.
- re-find (the matcher) and re-seq advance past a zero-width match relative to
  the match's own start, not the search origin — a zero-width match found past
  the origin (lookahead/boundary) no longer repeats. (re-seq #"a*" "aba") now
  matches the JVM.
2026-06-26 21:01:55 -04:00
Yogthos
cc26e3abba Expose jolt.host/throwable
A library that throws a typed host exception (an http client raising
java.net.ConnectException) needs (class e), instance?, .getMessage and
ex-message to all reflect the named class. jolt-host-throwable already builds
that value; expose it as a jolt.host seam so libraries stop hand-rolling a
:jolt/ex-info table that carries only the class (its .getMessage/ex-message
return nil).
2026-06-26 20:42:14 -04:00
Yogthos
3d80bdc10b Fix general gaps the hiccup suite shook out
Six correctness fixes, each a general gap (not hiccup-specific):

- deftype is not a map. jolt treated every deftype instance as a map
  (map?/record?/seqable over its fields); in Clojure only a defrecord is
  map-like, a bare deftype is an opaque object. defrecord now marks its type;
  map?/record?/coll?/seq/empty? gate on it, while a deftype implementing a
  collection interface still dispatches through its methods.

- cross-ns extend-protocol on an imported deftype. register-method built the
  type tag from the *calling* ns + bare name, so (extend-protocol P Raw …) in
  one ns missed a Raw value defined in another. A simple-name index resolves
  the bare name to the type's real tag (local ns still wins).

- str vs print. str of a collection is its readable form (nested strings
  quoted: (str ["x"]) => ["x"]); print leaves them raw. jolt defined print
  as str, conflating the two. Split via a __print1 seam.

- clojure.test thrown? now honors the exception hierarchy (instance?), so
  (thrown? IllegalArgumentException …) matches an ArityException subclass.

- java.net.URI is value-equal (= and hash by string form).

- clojure.walk/macroexpand-all was missing; an unresolved qualified var made
  the analyzer report "Unknown class walk".

deftype/defrecord + print are seed sources, re-minted. hiccup 365->381 of its
own suite; the rest are charset-encoding / var-meta niches.
2026-06-26 20:15:39 -04:00
Yogthos
448611a5df Match Clojure's lazy seq realization model
jolt's seq layer realized one element ahead of Clojure, so a side-effecting
lazy seq ran its producer too eagerly. Four changes bring it in line:

- rest is Clojure's more(): it returns the tail without realizing it. An
  unforced tail (vector / string / lazy-seq cell) comes back as a deferred
  seq, so (rest (iterate f x)) does not call f. next still realizes one.
- iterate applies f lazily, inside the tail thunk, so (first (iterate f x))
  is x with no call to f (clojure.lang.Iterate parity).
- take realizes exactly n: the last element terminates without touching the
  rest, instead of forcing one more element of the source.
- an empty realized lazy seq is still a sequence value, printing "()" not
  "nil" (a JVM LazySeq is never nil).

Also: the map transducer's step fn now takes multiple inputs
([result input & inputs]) so a multi-collection transduce applies f across
all of them. Fixes medley's join/window/sequence-padded laziness and
multi-input transducer tests (now 293/293). The rest change also fixed a
latent overrun in distinct/dedupe over a map's empty tail.

iterate is a seed source, re-minted.
2026-06-26 19:41:02 -04:00
Yogthos
331a41ee26 Honor *print-length* / *print-level* / *default-data-reader-fn*
Both printers (jolt-pr-str, jolt-pr-readable) now thread a print depth and
read the two limit vars. *print-length* truncates each collection to N
elements + "...", walking seqs lazily so an infinite seq prints under the
limit without realizing it. *print-level* renders a collection at depth >=
the level as "#". The reader consults *default-data-reader-fn* for an
unregistered #tag before falling back (tagged form on the data seam, throw
on the edn seam). All three interned with nil defaults.
2026-06-26 19:04:42 -04:00
Yogthos
a9ecae9a29 Map raw Chez runtime errors to their JVM exception classes
A wrong-arity or non-seqable error that Chez raises carried no jolt exception
class, so (class e) was :object and (thrown? ArityException …) / (thrown?
IllegalArgumentException …) never matched. Classify these by message:
incorrect-number-of-arguments -> clojure.lang.ArityException, not-seqable ->
java.lang.IllegalArgumentException, with ArityException modeled as a subclass of
IllegalArgumentException. (class e) and instance? now match the JVM.

All java-layer (records-interop.ss classifies, host-class.ss reports the class +
the ArityException token). medley 281 -> 283 passing.

jolt-o9dc
2026-06-26 17:55:30 -04:00
Yogthos
5f72ec9bcb Close portable clojure.core gaps: re-groups, letfn, REPL + dynamic vars
Spec coverage dashboard had 6 missing-portable and 24 dynamic-var entries. The
portable ones are now implemented (missing-portable -> 0, dynamic-var -> 14):

- Stateful matcher: re-matcher now returns a real mutable Matcher; re-find over
  it steps through matches and re-groups returns the last match's groups (was an
  inert tagged map). Closes re-groups.
- letfn is interned as a clojure.core var so (resolve 'letfn) matches the JVM. It
  stays a special form (the value is never invoked, not marked a macro).
- *1 *2 *3 *e interned (nil outside a REPL).
- Portable dynamic vars whose default already matches jolt's behaviour:
  *read-eval* *print-dup* *print-namespace-maps* *flush-on-newline*
  *compile-files* *math-context* *command-line-args* *file*.

The remaining 14 dynamic-var entries are host-internal (compile-path,
compiler-options, fn-loader, reader-resolver, repl, source-path, ...) or deferred
pending printer/reader support (*print-length* *print-level*
*default-data-reader-fn*). Corpus rows added for each closed gap; coverage.md
regenerated.
2026-06-26 17:48:21 -04:00
Yogthos
8a877662dc Regex: accept Java-compatible char-class dash and (X+)* quantifier
irregex rejected two patterns the JVM accepts, which blocked library loads:

- [\w-_] errored with bad char-set because a - after a shorthand class was
  read as a range start. Java reads it as a literal hyphen. Preprocess the
  pattern to escape such a dash.
- (X+)* errored with duplicate repetition because sre-repeater? recurses
  through submatch, treating a quantified group like a dangling a**. Override
  it to a bare leading * / + check, matching the JVM (which only rejects the
  dangling case).

Both in regex.ss (runtime). Unblocks cuerdas (was load-fail, now 292 passing)
and aws-api config-test. Also documents the host/chez/java source-layering rule
in host-interop.md.

jolt-l8so
2026-06-26 17:35:08 -04:00
Dmitri Sotnikov
687dc60af6
type returns the JVM class (Clojure semantics) (#244)
(type x) was jolt's internal taxonomy keyword (:string/:set/:jolt/inst), which
breaks any library dispatching a multimethod on [(type a) (type b)] against
java/clojure.lang classes (e.g. clojure.tools.logging.test's matchers). Make the
PUBLIC clojure.core/type Clojure's (or (:type meta) (class x)).

The taxonomy keyword stays the core model: natives-meta.ss keeps jolt-type and
exposes it as __type-tag, which print-method/print-dup dispatch on (so #uuid/#regex/
records still print). The JVM mapping lives in the java host layer — host-class.ss
defines the public type next to (class …), and a jinst now reports java.util.Date
(was :jolt/inst). So the core emits the taxonomy and the java layer remaps it in one
place. unit.edn's type suite updated to the class names. make test green.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 21:14:06 +00:00
Dmitri Sotnikov
6c03dffd00
Class/forName honesty + class/isa? conformance for builtins (#243)
Class/forName claimed every java.*/clojure.* name found (and any "x.y.Class"
matched the registered Class via a short-name fallback), so a library's
(class-found? "optional.Dep") feature-probe always said yes — tools.logging then
tried to build the java.util.logging / log4j backends jolt lacks and crashed.
Resolve forName by exact registry lookup + an honest prefix that excludes the
unbacked optional packages (java.util.logging, javax.management), so the probe
sees them absent and skips the backend.

class of a persistent collection / namespace now reports its JVM class name
(clojure.lang.PersistentHashSet, …Namespace, …) instead of jolt's internal :set/
:object tag, and isa? consults JVM class assignability — Object as every class's
root plus a modeled clojure.lang/java.util hierarchy — so (isa? (class x) C) and a
class-keyed multimethod dispatch like the JVM (e.g. (isa? Keyword Object) was
false). Adds the bare class tokens (Fn/Namespace/Set/…) these dispatch on.

(type x) is unchanged — it keeps jolt's documented internal-keyword form. Six
JVM-certified corpus rows. make test green, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 21:02:44 +00:00
Dmitri Sotnikov
283a0f0eec
instance? matches a thrown :class envelope by its class hierarchy (#242)
A library that throws an ex-info envelope carrying a JVM :class (jolt.host/
tagged-table with a "class" entry — e.g. http-client's UnknownHostException on a
DNS failure) was caught only by a broad (catch Throwable …): (class e) read the
class but (instance? C e) — which catch dispatch lowers to — always returned
false, so clj-http-lite's (catch UnknownHostException e …) for :ignore-unknown-host?
never matched and the condition propagated.

Add an instance? arm matching such an envelope against the carried class or any
ancestor (full name or last segment), and register the common exception hierarchy
(Throwable/Exception/RuntimeException/IOException + the java.net socket/host
exceptions) so (catch IOException e) / (instance? Throwable e) also match. A
non-throwable class (RuntimeException over an IOException, String) stays false.

Fixes http-client's :ignore-unknown-host? test (116/0/0, was 1 error). make test
green, 0 new divergences. Runtime .ss, no re-mint.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 20:18:27 +00:00
Dmitri Sotnikov
301cda2e46
Add java.util.Optional (#241)
A value class libraries return from getters (java.time, and the java.net.http
client shim that aws-api's java backend builds on). Statics of/ofNullable/empty,
methods isPresent/isEmpty/get/orElse/orElseGet/ifPresent/toString, value-equal so
(= (Optional/of x) (Optional/of x)). Five JVM-certified corpus rows. Runtime .ss,
no re-mint.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 20:08:23 +00:00
Dmitri Sotnikov
27d99db4bc
java.time: parse fractional seconds in formatter-based date parsing (#240)
parse-ms ignored sub-second fractions: a formatter parse of
"2020-07-06T10:59:13.417Z" dropped the .417 (and the ISO_OFFSET_DATE_TIME pattern
"…ssXXX" then mis-aligned, reading ".417Z" as the offset). java.time's ISO
formatters accept an optional fractional second, so jolt should too.

After parsing the seconds field, consume an optional .fff from the input (to
millis) when the pattern carries no fraction field — which is how the ISO_*
constants are modeled here (ss, no S). Also handle the S pattern letter for
explicit .SSS patterns. Carry the fraction into the result.

Fixes aws-api shape iso8601 parse (1 FAIL -> 0; cognitect.aws.util/parse-date now
returns the right Date). Two JVM-certified corpus rows. make test green, 0 new
divergences. Runtime .ss — no re-mint.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 19:49:49 +00:00
Dmitri Sotnikov
b3ffd357a2
java.time: complete LocalTime/LocalDate/Year/YearMonth ChronoField coverage (#239)
tick's fields-test walks every ChronoField a temporal supports and reads it,
which crashed on fields jolt didn't implement. Fill the gaps:

- LocalTime: CLOCK_HOUR_OF_DAY (1..24), HOUR_OF_AMPM, CLOCK_HOUR_OF_AMPM,
  MICRO_OF_DAY — both isSupported and getLong.
- LocalDate: the aligned-* group (ALIGNED_DAY_OF_WEEK_IN_MONTH/_YEAR,
  ALIGNED_WEEK_OF_MONTH/_YEAR).
- LocalDateTime field routing now asks which part supports the field instead of a
  hardcoded date list, so a date field never misroutes to the time part (the actual
  cause of "LocalTime has no field ALIGNED_DAY_OF_WEEK_IN_MONTH" — a ZonedDateTime's
  date field fell through to its time).
- Year / YearMonth gain isSupported / get / getLong.

tick api_test 344/0/1 -> 599/0/0. Seven JVM-certified corpus rows. make test green,
0 new divergences. Runtime .ss — no re-mint.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 19:12:40 +00:00
Dmitri Sotnikov
dcfa607dc2
bench: mono-dispatch 48x->15x after the devirt inline cache (#238)
The per-site inline cache (#237) resolves a statically-proven monomorphic devirt
once instead of per call, so mono-dispatch is no longer worse than megamorphic.
The remaining dispatch lever is the megamorphic case (a runtime receiver-type-keyed
cache).

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 18:36:52 +00:00
Dmitri Sotnikov
a31c1af8c4
Devirt: cache the resolved impl in a per-site cell (inline cache) (#237)
A devirtualized protocol call resolved its impl with devirt-resolve on EVERY call
— but the tag/proto/method are compile-time constants, so the resolved fn is a
runtime constant (closed world). That per-call find-protocol-method (three
hashtable lookups) was the cost: on mono-dispatch, dispatch was ~75% of the time
(ablation: same arithmetic direct-call 166ms vs dispatch 673ms).

Resolve once. When emitting a direct-link def, each devirt site gets a fresh cache
cell, bound to #f in a let wrapping the def (so it persists across calls and is
shared by every invocation); the site resolves into it on first use ((or cell
(let ((_f (devirt-resolve ..))) (set! cell _f) _f))) and reuses it after — the
inline cache the JVM gets for free. First call still passes the real receiver, so
the Object/host-tag fallback (devirt-resolve) is unchanged.

mono-dispatch 673ms -> 214ms (~3.15x), 47.5x -> ~15x JVM, near the 166ms
direct-call floor. run-devirt.ss gains the cached-path checks (cell present, 1st
call caches + 2nd reuses, both == dispatch). make test / shakesmoke green, selfhost
holds, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 18:34:13 +00:00
Dmitri Sotnikov
f923d52cad
bench: refresh the jolt/JVM scorecard (#236)
The type-proving / native-record / bare-field-read / inference work collapsed
the dispatch + allocation gaps by an order of magnitude (binary-trees ~140x->~10x,
mono-dispatch ~330x->~48x, dispatch ~130x->~12x) and brought compute to parity
(fib now beats JVM, collections ~4x, mandelbrot ~7.5x). mono-dispatch is now the
standout gap — a runtime-monomorphic call site the JVM inline-caches to near-free.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 18:20:11 +00:00
Dmitri Sotnikov
f920ff6ea2
Nilable record types + flow-sensitive nil narrowing (#235)
A record-or-nil (a protocol method whose impls return a record in one branch and
nil in another, or an `if` over a ctor and nil) now types as a NILABLE record
instead of widening to :any. A nilable record still bare-indexes its field reads
(jrec-field-at falls back to jolt-get on nil), but some?/nil? do NOT fold on it, so
a runtime guard is preserved — and inside (if (some? x) ..) / (if x ..) the then-
branch narrows x to the non-nil record, so its reads bare-index AND unbox there.

This is what lets the bounced ray type without a hint: scatter returns
ScatterResult-or-nil (Metal absorbs some rays), and the consumer reads
(:ray scattered) only under (if (some? scattered) ..). The narrowing proves
scattered non-nil there.

lattice: :nil type; :nil ∨ struct -> nilable struct, ∨ anything else -> :any;
nilability is contagious through a struct join, which also now preserves :type when
both sides agree (needed so a record ∨ its nilable self stays that record).
truthy-type?/field-type/pred-on treat a nilable struct as maybe-nil. types: nil
literal -> :nil; an `if` whose test is (some? x)/(nil? x)/x narrows the nilable
local x in the proven branch.

Ray tracer with NO hints: 38.4s -> 23.9s (~1.6x) — hit-sphere now types fully
(0 jolt-get, 57 jrec-field-at, 38 fl-ops), identical to the hand-hinted build.

run-narrow.ss gate, incl. the load-bearing check that the nil case still takes the
else branch (the guard is not folded away). make test / shakesmoke green, selfhost
holds, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 17:16:16 +00:00
Dmitri Sotnikov
f124701393
Infer monomorphic protocol-method return types (#234)
A protocol method whose impls all return the same record type has a monomorphic
return. collect-pm-rets! scans the unit's (register-(inline-)method ..) forms,
infers each impl fn's return type, and joins them per method; call-ret-type then
types a (method recv ..) call as that record, so a field read off the result
bare-indexes — e.g. (:ray (scatter m ..)) reads off a Ray. A disagreeing impl
joins to :any and keeps the generic path.

run-protoret.ss: a method with all-record impls bare-indexes + unboxes the field
read; a mixed-return method (one impl returns a number) stays generic. make test /
shakesmoke green, selfhost holds, 0 new divergences.

Foundation for auto-typing record values that flow through protocol dispatch. Does
not yet move the ray tracer: its scatter returns ScatterResult-or-nil (Metal
absorbs some rays), and the nil widens the join to :any — typing a nullable return
soundly needs flow-sensitive narrowing (a guarded (some? x) proves non-nil), filed
separately.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 16:59:35 +00:00
Dmitri Sotnikov
8c9cba44b3
WP: don't let a recursive pass-through poison a param to :any (#233)
The whole-program fixpoint collects a self-recursive call's arg types into the
fn's own params. When a recursive call threads a param straight through unchanged
(same arg, same position — e.g. ray-cast passing `hittables` to itself), that arg's
type is the param's own current type: :any until external callers determine it. And
:any is absorbing, so collecting it pinned the param at :any forever — the type a
caller supplied (a vector of records) was lost, and the fn's field reads stayed
generic.

Skip a same-position pass-through arg in the self-recursion collection (contribute
the join identity). It can't add information — param i ⊇ param i is trivial — so
dropping it is sound; the param is still constrained by every external caller and
by any non-pass-through recursive arg. Applies to both self-recursion paths: a
`defn` recursing through its var, and a named fn literal recursing via its
self-local.

This is why ray-cast's `ray` typed (its recursion passes a fresh ray) but
`hittables` didn't (passed through). With the fix, hittables keeps its
vec<Sphere> element type, so hit-all's reduce element — and hit-sphere's reads —
type without any hint: ray tracer 38.4s -> 31.3s (~1.23x) with no annotations.

run-wp.ss: a recursive fn threading a vec param through keeps its element type.
make test / shakesmoke green, selfhost holds, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 16:29:46 +00:00
Dmitri Sotnikov
09345f10c2
Make ^Record param hints work (resolve the record tag) (#232)
jolt.host/record-ctor-key and record-type? were stubs returning nil since the
rehost, so phint-of always produced nil — a ^Record param hint (^Sphere s) was
silently dead. The inference only ever typed a record param when its callers
passed a concrete ctor (whole-program), so a fn called with an untyped value (a
vector element, a value threaded through recursion) read its fields generically.

Resolve the tag against the record registry (records.ss): chez-find-ctor-key maps
a ^Type tag to the ctor-key "ns/->Name" the inference seeds with, preferring the
compile ns and falling back to any registered record of that simple name (cross-ns
/ imported). record-type? / record-ctor-key call it. Runtime .ss — no re-mint (the
analyzer reaches record-ctor-key through the host var).

With this, a ^Sphere/^Ray-hinted hit-sphere in the ray tracer types its params, so
its 48 generic jolt-get field reads + boxed arithmetic become bare-index reads +
fl-ops: ray tracer 38.4s -> 23.9s (~1.6x), make test / shakesmoke green, selfhost
holds, 0 new divergences. A wrong hint stays safe — jrec-field-at falls back to
jolt-get for a non-record.

run-fieldnum.ss: a ^V-hinted param (no inferable caller) bare-indexes + unboxes,
with no generic jolt-get left.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 16:03:57 +00:00
Dmitri Sotnikov
4671e1b67e
Unbox ^double record field reads (#231)
A record field tagged ^double now reads back as a flonum and feeds the numeric
pass, so hintless arithmetic over those fields lowers to fl-ops — the leaf-numeric
analog of the ^Vec3 nested-field hints. Combined with the whole-program :double
param inference, a vec3-dot over a ^double-fielded record unboxes end to end with
no per-fn hints.

records.ss: a ^double field tag passes through resolution, and the ctor (and a
mutable-field set!) coerce a ^double field to a flonum — JVM primitive-field
parity (jolt returned an exact 1, not 1.0, before), and what makes reading the
field back as :double sound for an fl-op.

types.clj: field-type-from-tag maps "double" -> :double, and a keyword/get lookup
whose result is :double annotates the node :num-read :double. numeric.clj reads
that annotation and classifies the field read as a :double operand, so the
enclosing arithmetic specializes — the read itself keeps its jrec-field-at/jolt-get
emit.

run-fieldnum.ss gate: ctor coercion (int field -> flonum), field-field arithmetic
emitting fl*/fl+, and an untagged field staying generic. make test / shakesmoke
green, selfhost holds, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 14:43:00 +00:00
Dmitri Sotnikov
8ae45057d6
Hintless whole-program double inference (#230)
The closed-world fixpoint (#226) flowed record types across fn boundaries; this
adds a numeric refinement so a hintless fn whose every call site passes a flonum
has its param unboxed to fl-ops, no ^double hint needed.

Lattice gains :double, a flonum refinement of :num: two doubles join to :double,
a double joined with anything else widens to :num — so a param is :double only
when every contributing value is a flonum, which is what makes the fl-op sound.
infer types a flonum literal and flonum arithmetic (+ - * / min max inc dec over
double/int-literal operands) as :double, and the fixpoint joins those across call
sites and return types like any other lattice value.

The bridge to the existing hint-directed pass is a synthetic [param :double]
nhint: wp-infer! stashes the :double params separately from the structural seeds,
and run-passes injects them as nhints before numeric/annotate, so the fl-op
emission and the exact->inexact entry coercion (a no-op on a proven flonum) apply
unchanged.

Sound subset only: :double, never :long — an untyped integer can be a bignum and
fx-ops would overflow/diverge from jolt's arbitrary precision. So an integer
caller leaves a param generic; an escaped fn (unknown callers) keeps :any.

run-numwp.ss gate: cross-fn :double propagation incl. through a flonum-returning
helper, the integer-caller and escape negatives, and the full run-passes path
emitting fl* + entry coercion. make test / shakesmoke green, selfhost holds, 0
new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 14:18:10 +00:00
Dmitri Sotnikov
e6e3612332
Devirt: fall back to dispatch when the static tag has no direct impl (#229)
The devirtualized protocol call emitted find-protocol-method on the inferred
record tag, but a record can satisfy a protocol via an Object/host-tag default
rather than a direct impl — find-protocol-method on its own tag misses that,
while protocol-resolve walks to the default. So a record relying on
(extend-protocol P Object ...) resolved under ordinary dispatch but applied #f
under devirt and crashed. Closed-world opt builds only; the gate previously
covered just direct inline/extend-type impls so it shipped green.

Emit devirt-resolve, which tries the static tag and falls back to
protocol-resolve on a miss — same fast path, correct regardless of how the
record satisfies the protocol. Mirrors jrec-field-at falling back to jolt-get.
The receiver binds to one temp so it feeds the resolve and the application
without double-evaluating a side-effecting arg 0.

Also widen the whole-program fixpoint to :any on hitting the iteration cap: a
non-converged pre-fixpoint is more specific than the least fixpoint, so seeding
it would be unsound. Not reached in practice (~2 passes); a defensive floor.

run-devirt.ss gains an Object-default case. make test / shakesmoke green,
selfhost holds, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 13:58:23 +00:00
Dmitri Sotnikov
de31221573
Bare-index field reads for statically-known records (#228)
When the inference types a keyword-lookup receiver as a record — it carries the
field-order :shape and :hint :struct from the whole-program fixpoint — the back
end reads the field by its declared slot via jrec-field-at instead of jolt-get.
That skips the jolt-get case-lambda, the dispatch fn, and the field-key
hashtable lookup, leaving a jrec? check + a static-index vector-ref.

jrec-field-at falls back to jolt-get when the receiver isn't the expected record
(a map downgraded by dissoc, or a value the inference mistyped), so it stays
correct if the static type is wrong. Only the no-default form takes the bare
path (a declared field is always present).

Sound only for non-nil records: a self-recursive param that can be nil (e.g.
binary-trees check-tree, whose untagged child is nil at leaves) types :any and
keeps jolt-get — the whole-program fixpoint demotes it. The target is non-nil
record params, like a Vec3 dot product (~5% there; boxed-flonum arithmetic
dominates the rest, a separate numeric lever).

run-fieldread.ss gate: emitted form uses jrec-field-at at the right slot and
matches jolt-get for each declared field; a non-field key and a default-arg form
keep the generic path. make test / shakesmoke green, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 11:29:14 +00:00
Dmitri Sotnikov
af11aaa7ff
Devirtualize monomorphic protocol calls (#227)
When the inference proves a protocol call's receiver is one record type, the
back end resolves the impl by that static tag (find-protocol-method) instead of
routing through the protocol var -> jolt-invoke -> protocol-resolve, which
re-derives the tag and walks the type table. Same table lookup, minus the
var-deref, the rest-cons, and the receiver-type computation.

Fires only on a monomorphic site: a megamorphic receiver joins to :any and
carries no :devirt-type, so it keeps ordinary dispatch (the dispatch bench is
unaffected). The annotation comes from the whole-program fixpoint typing a
reduce/HOF element or a ctor return as a specific record.

Modest on the dispatch benchmarks (~6% on mono-dispatch) — float boxing in the
reduce accumulator dominates there, a separate numeric lever — but it removes
the dispatch overhead wherever a typed receiver is known.

run-devirt.ss gate: emitted form uses find-protocol-method, and evaluating it
matches ordinary dispatch for an inline impl, an extend-type impl, and the
non-devirt path. make test / shakesmoke green, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 11:16:19 +00:00
Dmitri Sotnikov
09712ec575
Whole-program param-type inference (closed world) (#226)
Re-derive each app fn's param types from its call sites under --opt, so a
record type flows across fn boundaries: a ctor's return reaches a callee
param, and a typed vector's element reaches a HOF closure's param. The back
end can then bare-index field reads and devirtualize protocol calls at those
sites (it reads the resulting :hint/:devirt annotations; consuming them is
separate work).

This rebuilds the inter-procedural driver the Janet host had — the API
(infer-body/reinfer-def) survived the rehost but nothing drove it, and the
record-shapes/protocol-methods registries were empty stubs.

- records.ss: populate record-shapes (ctor key -> fields/tags/type, resolving
  nested record field tags) and protocol-methods (method var -> [proto method])
  registries at deftype/defprotocol load time; jolt.host accessors materialize
  them.
- passes/types.clj: wp-infer! runs a closed-world fixpoint joining call-site
  arg types into callee params; reinfer-def re-seeds each def at emit. Self-
  recursive calls and fn-level recur are collected so a recursive fn's params
  are constrained by its recursion, not just external callers — else a param
  the recursion widens (e.g. binary-trees check-tree, whose untagged child can
  be nil) would be unsoundly typed non-nil. A fn used in value position keeps
  :any params (callers unknown). Megamorphic sites join to :any.
- build.ss: analyze all app forms and run the fixpoint before per-form emit.
- run-wp.ss: gate (cross-fn propagation, escape soundness, self-recursion).

make test / shakesmoke green, 0 new divergences, selfhost holds.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 10:57:45 +00:00
Dmitri Sotnikov
32ef74b9b0
Persistent vector as a 32-way trie (#225)
The persistent vector was a flat Scheme vector with copy-on-write: every conj
copied the whole backing array, so building an n-element vector was O(n^2). On the
collections bench that's vec-sum building 7500 elements with 227MB of copies, 90%
of the bench's allocation.

Replace it with Clojure's PersistentVector — a 32-way trie plus a trailing tail
chunk. conj appends to the tail and, when it fills, path-copies it into the trie,
so conj is O(1) amortized and a linear build is O(n). nth/assoc/pop are
O(log32 n). make-pvec (build a trie from a flat vector) and pvec-v (materialize
back) stay as compatibility shims, so the ~14 callers that read the backing array
— all one-shot conversions in =, hash, seq, meta-copy, transients, the reader —
are untouched; only this file's internals change.

vec-sum 70ms/227MB -> 1ms/3MB; collections 10.4x -> 4.0x vs JVM, under the 5x
target. 5 new corpus rows plus boundary stress (level transitions at 32 and 1024,
pop-collapse, assoc at every index) cover the trie.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 06:32:18 +00:00
Dmitri Sotnikov
93be40f3fe
See through or/and in truthy-elision (#224)
A loop test like (or (>= i cap) (> ... 4.0)) desugars to
(let* [g (>= i cap)] (if (truthy? g) g (> ... 4.0))) and the whole thing was
wrapped in jolt-truthy? because returns-scheme-bool? only looked at :const and
:invoke nodes, not the let*/if an or/and expands to. The wrapper defeats Chez's
branch inlining on the hot loop edge.

Make returns-scheme-bool? recursive over :if (both branches bool), :let (body
bool, tracking which bound locals hold a Scheme boolean), and :local (in that
set). or/and over bool-returning ops then read as Scheme booleans and the outer
wrapper drops. Still sound: eliding only when the value is provably #t/#f — a
jolt-nil is a truthy record in Chez, so a false positive would be a real bug, and
the recursion only proves bool-ness through ops already known to return one.

No bench regression; the win lands on hinted float loops where the branch, not
boxed arithmetic, is the cost.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 06:11:35 +00:00
Dmitri Sotnikov
0a7e818700
Fixed-arity protocol dispatch shims (#223)
defprotocol emitted one variadic (fn [this & rest] (protocol-dispatch P m this
(list->cseq rest))) per method, so every protocol call — even a no-extra-arg one
like (area s) — consed a rest list, wrapped it in a cseq, var-deref'd
protocol-dispatch, and jolt-invoke'd it (consing again). On mono-dispatch that was
2.07GB of allocation, ~65% of the benchmark.

Emit one fixed-arity clause per declared arglist instead. The 1/2/3-param arities
call positional protocol-dispatch{1,2,3}, which resolve the impl (by record tag,
reify method, or host-tag extension — factored into protocol-resolve) and apply it
directly; no rest-list, no seq round-trip. The dispatchN entry points are in the
native-op table so the shim calls bind straight to the records.ss procedures
rather than var-deref. 4+ params fall back to the variadic protocol-dispatch.

mono-dispatch 1.5s/2.07GB -> 0.69s/280MB; dispatch 26x -> 12.2x, mono-dispatch
111x -> 51x vs JVM. 5 new corpus rows pin multi-arity methods, host-type args,
and protocol-method-as-value against JVM Clojure.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 05:57:42 +00:00
Dmitri Sotnikov
8bea1abe12
Native record representation + inline nil?/some? (#222)
Records were a jrec holding an alist of (kw . val) conses: ~113B/node, built
fresh per construction, field reads a list scan. Replace that with a shared
per-type descriptor (tag + field keywords + an eq?-keyed keyword->index table)
plus a flat per-instance value vector and an extension map for any non-field
keys assoc'd on (jolt-nil when there are none). Construction now allocates one
vector instead of a cons chain and a field read is an index lookup. binary-trees
construction allocation drops 2.085GB -> 1.19GB.

That alone barely moved binary-trees wall-time: profiling showed the read loop,
not allocation, dominates, and the read loop's own allocation came from (nil? l)
lowering to (jolt-invoke (var-deref "clojure.core" "nil?") l), which conses its
args every call. Add nil?/some? to the backend native-op table so they inline to
jolt-nil?/jolt-some? (and drop the truthy wrapper, like the other predicates).
check-tree's read loop goes from 1.476GB allocated to zero; binary-trees 18.9x
-> 9.7x vs JVM. The remaining gap is the field-read dispatch chain (jolt-c3mw).

Two JVM divergences fixed along the way, both certified:
- dissoc of a declared field downgrades a record to a plain map (was kept as a
  record); an extension key still drops cleanly.
- map->R keeps extension keys (was dropping anything outside the declared basis).

16 new corpus rows pin assoc/dissoc/count/keys/seq/=/hash/extension-field
behavior against JVM Clojure.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 05:42:24 +00:00
Dmitri Sotnikov
eacfa04e5b
Perf round 1: self-call, keyword interning, fast record field reads (#221)
* Make the benchmark harness build optimized binaries on Chez

bench/run.sh was Janet-era: it invoked a 'jolt' binary and set
JOLT_DIRECT_LINK/JOLT_WHOLE_PROGRAM, none of which exist on Chez, where
'joltc run -m' runs fully unoptimized (direct-link and inline default off). So
the suite was measuring jolt's unoptimized path.

run.sh now compiles each benchmark to an optimized AOT binary (joltc build
--direct-link --opt) and times it against JVM Clojure on the same portable
source, auto-detecting the Chez kernel dev files like build-smoke.sh. Adds
bench/deps.edn so joltc resolves the namespaces, NO_JVM to skip the reference.

mandelbrot.clj dropped its jolt.png require so the JVM reference can run it; the
picture demo moved to mandelbrot_png.clj (jolt-only). README scorecard refreshed
with current Chez numbers and the two-regime read (compute ~8-10x substrate floor;
dispatch/alloc ~120-330x architectural gaps the passes don't touch). Stale
'jolt -m' header lines point at bench/run.sh.

* Emit direct self-calls for named-fn self-recursion

A self-recursive call to a named fn compiled to (jolt-invoke fib ...) instead of
a direct (fib ...): emit-invoke handled a :local callee only when it was NOT a
known proc, so a :local that IS in *known-procs* (the letrec-bound self-name) fell
through to the :else jolt-invoke branch. Now a :local known proc emits a direct
Scheme call — no jolt-invoke, no per-call arg-list consing; case-lambda handles
arity.

fib 30: 63.3ms -> 4.7ms (faster than JVM Clojure's 7.1ms; was 9x slower). The win
is on every self-recursive non-loop fn, including the compiler's own. No semantic
change — selfhost holds, make test green, shakesmoke/buildsmoke byte-identical.

Re-mint (backend is seed). Corpus rows pin self-recursion across fixed/multi/
variadic arities.

* Intern no-ns keywords without per-call allocation

(keyword #f name) built a fresh combined-key string (string-append) on every
call just to do the intern-table lookup — ~80 bytes of garbage per (:kw x), map
literal, keyword arg, etc. A no-ns keyword now interns in a table keyed by the
name string directly, so a lookup of an already-interned keyword is one
hashtable-ref with no allocation. The ns table keeps the combined key; both share
the keyword-t khash (equal-hash of the combined key) so hash values are unchanged.

Small time win on its own (the field-read dispatch dominates hot record code —
see jolt-unx4) but removes per-call keyword allocation everywhere. Runtime .ss,
no re-mint; identity/=/hash unchanged, make test green.

* Fast record field reads: single eq? scan, skip the get-arm walk

(:field rec) / (get rec :field) lowers to (jolt-get rec kw), which walked the
get-arm list to reach the jrec arm, then did jrec-has? + jrec-lookup — TWO linear
scans, each comparing keys through the generic jolt=2 equality dispatcher. Field
keys are interned keywords, so:

- jrec-key=? compares a keyword query by eq? (jolt=2 only for non-keyword keys),
- jrec-ref does ONE scan (vs has?+lookup) and runs a deftype's ILookup valAt only
  when the field is genuinely absent (present-nil still returns nil, not default),
- jolt-get-dispatch checks jrec? first, skipping the get-arm walk for the hottest
  get target. jrec-lookup/jrec-has? (used by =, contains?, etc.) get the fast
  compare too.

binary-trees 135x->18.9x, dispatch 121x->26.4x, mono-dispatch 327x->108x vs JVM.
Runtime .ss (collections.ss + records.ss), no re-mint; make test + shakesmoke +
buildsmoke green, record get/assoc/keys/=/count semantics unchanged.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 05:00:28 +00:00
Dmitri Sotnikov
93ddf2c85a
Make the benchmark harness build optimized binaries on Chez (#220)
bench/run.sh was Janet-era: it invoked a 'jolt' binary and set
JOLT_DIRECT_LINK/JOLT_WHOLE_PROGRAM, none of which exist on Chez, where
'joltc run -m' runs fully unoptimized (direct-link and inline default off). So
the suite was measuring jolt's unoptimized path.

run.sh now compiles each benchmark to an optimized AOT binary (joltc build
--direct-link --opt) and times it against JVM Clojure on the same portable
source, auto-detecting the Chez kernel dev files like build-smoke.sh. Adds
bench/deps.edn so joltc resolves the namespaces, NO_JVM to skip the reference.

mandelbrot.clj dropped its jolt.png require so the JVM reference can run it; the
picture demo moved to mandelbrot_png.clj (jolt-only). README scorecard refreshed
with current Chez numbers and the two-regime read (compute ~8-10x substrate floor;
dispatch/alloc ~120-330x architectural gaps the passes don't touch). Stale
'jolt -m' header lines point at bench/run.sh.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 04:59:52 +00:00
Dmitri Sotnikov
f3084f8043
Collection fns: JVM-faithful return types + laziness (#219)
A type-aware audit (~190 collection expressions vs reference Clojure) found four
divergences the corpus missed — value-equality (= [0 1] '(0 1)) hides type and
laziness differences. Fixed, with type-predicate + over-infinite corpus rows that
pin them.

- partition-all [n coll] built vector chunks; JVM chunks are seqs. (The [n step
  coll] arity was already correct, as is the partition-all transducer, whose
  chunks are vectors in JVM too.) Now builds seq chunks.
- replace always returned a vector (mapv) and was eager; JVM is type-preserving —
  a vector maps to a vector, any other seqable to a lazy seq.
- sequence eagerly realized its source (into-xform), so (first (sequence (map inc)
  (range))) hung. Rewrote as a transformer iterator: pull one input at a time,
  buffer the step outputs, emit lazily, run the completion to flush a stateful
  xform. eduction builds on it (lazy, no longer an eager vector).
- mapcat and (apply concat coll-of-colls) hung over an infinite source because
  jolt-apply seq->lists the trailing arg and mapcat seq->lists the map result.
  Added lazy-concat-seq (lazily flatten a seq of colls); mapcat uses it directly,
  and apply special-cases concat (its result is lazy) to route through it.

Docs: a cross-cutting return-type + laziness contract in docs/spec/09-core-library;
SPEC.md notes that = masks type/laziness so they need predicate / over-infinite
rows. EBNF is reader syntax only — unaffected.

Seed change (partition-all/replace/eduction are clojure.core overlay) -> re-mint;
selfhost holds. make test + shakesmoke + buildsmoke green, 0 new divergences.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 03:01:36 +00:00
Dmitri Sotnikov
8180c85393
Source locations: reader positions, error locations, native stack traces (#218)
* Reader records source line/column on list forms

The reader stamps 1-based :line/:column metadata on every list form (plus
:file when load-jolt-file is reading a file), and jolt.host/form-position
reads it back so the analyzer's :pos scaffold finally gets real data. A
left-to-right cursor counts newlines over the delta between successive forms,
so it stays O(n). Vector/map/set literals are untouched (their metadata is a
runtime value the analyzer would have to wrap in with-meta); empty () can't
carry meta. ^meta now merges onto the position keys instead of clobbering them.

Re-mint is byte-identical (the backend doesn't emit :pos), so this is a pure
scaffold for the error-location work that follows.

* Report source location on uncaught errors

Each top-level form records its source position (thread-local) before it
compiles+evals, and cli.ss jolt-report-uncaught appends 'at file:line:col'
when an error propagates out. Covers joltc -e, joltc run <file>, and
load-string — every interpreted path. Top-level granularity, one set per
form; deeper frames come from the Phase 2 frame walk.

Runtime .ss only, no re-mint.

* Clojure stack traces via source registry + native frame walk

A direct-link build emits (jolt-register-source! short-name ns name file line)
once per fn def — at definition time, so zero per-call cost. On an uncaught
error the reporter walks Chez's native continuation frames (jolt-throw captures
the live continuation via call/cc; host conditions carry their own
&continuation), maps each frame's procedure name through the registry, and
prints a Clojure backtrace 'ns/name (file:line)'. Wired into both the cli and a
built binary's launcher.

Frames are keyed by the short munged fn name Chez actually reports (emit-fn's
letrec self-binding), not jv$ns$name; a cross-namespace collision degrades to
the bare frame name rather than a wrong attribution. The analyzer carries the
original form's position through defn macroexpansion onto the def node.

Calling a non-fn now throws a catchable ClassCastException (via jolt-throw)
naming the operator, instead of a raw Chez error.

Caveats (documented in source-registry.ss): names map only in direct-link/AOT
closed-world builds — the open-world -e/repl/run path falls back to the
top-level location; and pervasive TCO erases tail-call frames, so a mapped
trace shows only the non-tail spine. JOLT_DEBUG_FRAMES dumps raw frame names.

Re-mint (analyzer + backend); prelude byte-identical (direct-link off during
mint). Corpus rows certified, build-smoke asserts the trace.

* Propagate source position through macroexpansion

hc-expand-1 now carries the macro call form's :line/:column onto the top of a
list expansion that has none of its own (merged under any meta the macro set),
so errors and stack traces in macro-generated code point at the call site —
Clojure parity. The analyze recursion re-expands inner macros, so each level's
top form picks it up, matching the reference compiler. (meta (macroexpand-1
'(when x y))) now reports the call-site line.

A direct-link fn defined through a user macro (build-app's defguarded) registers
with a real line, so build-smoke's trace assertion covers macro-defined fns.

Runtime .ss (host-contract.ss) — no re-mint; selfhost holds.

Phase 3's optional items are deferred: :line-in-ex-data has no clean consumer
(it would pollute ex-data, break = and printing, and positions already surface
via the trace + top-level location), and Chez source-object emission is a large
backend change the jv$-name registry already sidesteps.

* Review fixes: registration key, thread-locals, debug flag timing

- Register a fn under the name Chez actually reports for its frame, not the def
  name: a named fn literal whose name differs from the def (def foo (fn bar …))
  is framed as 'bar', and an anonymous fn def (def foo (fn …)) as jv$ns$foo.
  Both previously registered under the def name and so never appeared in traces.
- rdr-source-file / rdr-pos-cursor are thread parameters, so concurrent compiles
  (futures, core.async) don't clobber each other's file/line attribution.
- Read JOLT_DEBUG_FRAMES at call time: a built binary evaluates top-level forms
  at heap-build time, where a load-time getenv is always unset.

Re-mint (backend + reader); prelude byte-identical, selfhost holds.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-26 02:14:34 +00:00
Dmitri Sotnikov
bdf436e242
Merge pull request #217 from jolt-lang/feat/bigdec-arithmetic
BigDecimal arithmetic (value-position + type-directed call-position)
2026-06-26 00:26:18 +00:00
Dmitri Sotnikov
838ff37207
Merge pull request #216 from jolt-lang/refactor/host-chez-java-folder
Group the JVM interop shims under host/chez/java/
2026-06-26 00:23:30 +00:00
Yogthos
09a9ad8c75 Add bigdec min/max (review follow-up)
Review found (< 1M 2M) worked but (min 1M 2M) threw — incoherent. Wire min/max
the same way as the other ops: value-position jolt-min/jolt-max shims (new in
seq.ss, added to core-value-procs) and call-position via bd-spec/bd-ops ->
jbd-min/jbd-max.

min/max return the original operand by value, not a coerced copy, matching
Clojure: (min 1M 2.0) -> 1M, (max 1M 2.0) -> 2.0, (min 1.50M 2M) -> 1.50M; a tie
keeps the second operand ((max 1.5M 1.50M) -> 1.50M). bigdec mixed with a flonum
in call position stays in the documented :any/contagion gap (value position
handles it). Re-mint; 6 more JVM-certified rows.
2026-06-25 20:22:26 -04:00
Yogthos
6fcc9fa8e6 BigDecimal call-position arithmetic via :bigdec type (Phase 2)
A direct (+ 1.5M 2.5M) emits a raw Chez + that rejects the bigdec record. Rather
than guard every arithmetic call site (measured 2-4x on unhinted fixnum loops),
let the analyzer dispatch where it can prove the type.

jolt.passes.numeric seeds a :bigdec kind from the M-literal and flows it through
let/loop/if like the existing :double/:long kinds; an arithmetic/comparison invoke
whose operands are all bigdec (integer literals allowed) gets :num-kind :bigdec.
The back end (bd-ops + emit-numeric) lowers those to the bigdec.ss engine
(jbd-add/-sub/-mul/-div, jbd-lt?/…, jbd-zero?/-pos?/-neg?, jbd-quot/-rem).

Zero cost on non-bigdec code: with no bigdec literals present the kind never
arises, so emission is byte-identical — the re-mint leaves prelude.ss unchanged,
only image.ss (the compiler) moves. Gaps (filed): a bigdec mixed with a flonum in
call position, and a bigdec the analyzer types :any, still hit the raw op and
throw; use value position or a literal-typed let.

Re-mint (numeric/backend are seed sources). 16 JVM-certified corpus rows.
2026-06-25 19:49:17 -04:00
Yogthos
bd7b75fb5d BigDecimal arithmetic: value-position + compare (Phase 1)
bigdec values existed but +,-,*,/ and compare threw — the header even said
"arithmetic contagion is not modelled". Add the scale-aware engine on the
{unscaled, scale} pair (jbd-add/-sub/-mul/-div + comparison helpers) following
java.math.BigDecimal's rules: add/sub align to the larger scale, multiply adds
scales, divide gives the exact quotient at minimal scale or throws
ArithmeticException on a non-terminating expansion. Clojure contagion: a bigdec
mixed with an integer stays bigdec, a flonum operand wins (result is a double).

Wire it into the value-position shims only — jolt-add/-sub/-mul/-div (what
(reduce + bigs)/(apply * bigs) lower to) and compare — so the inlined native hot
path is untouched. A call-position (+ 1.5M 2.5M) still reaches the raw Chez op;
that needs the analyzer's :bigdec type (next).

Runtime .ss only, no re-mint. 13 JVM-certified corpus rows.
2026-06-25 19:42:12 -04:00
Yogthos
ec9fde9e7e Group the JVM interop shims under host/chez/java/
The host/chez directory mixed jolt's own runtime (value model, seq, reader,
vars, ns, multimethods) with the shims that emulate the JVM: java.* / javax.*
classes, clojure.lang interfaces, and the host-class registry they hang off.
Move that JVM-emulation layer into host/chez/java/ so it reads as a distinct
unit instead of being interleaved with the platform runtime.

Moved (content unchanged): host-static, host-static-methods,
host-static-classes, host-class, dot-forms, records-interop, byte-buffer,
io, io-streams, inst-time, java-time, bigdec, natives-queue, natives-str,
natives-array, math, concurrency, async, ffi.

The load paths in rt.ss/cli.ss and the build.ss runtime manifest are updated
to point at java/; the build inliner follows the (load ...) strings, so the
AOT path needs no other change. All runtime shims, no seed source touched
(the three .clj edits are doc comments), so no re-mint.

Gate green: make test (selfhost fixpoint, certify 0-new, sci 211, infer),
shakesmoke (4 apps byte-identical).
2026-06-25 18:35:44 -04:00
Dmitri Sotnikov
5b77efa499
Merge pull request #215 from jolt-lang/cleanup/dedup-fresh-sym
Drop duplicate fresh-sym; clarify group-by-head vs parse-extend-impls
2026-06-25 21:38:23 +00:00
Yogthos
d77fd22bfe Drop the duplicate fresh-sym; clarify group-by-head vs parse-extend-impls
A post-conformance review (chiasmus) flagged fresh-sym defined byte-identically
in 00-syntax and 30-macros; 00-syntax loads first, so the second is redundant.
Also note why deftype uses group-by-head while extend-protocol uses
parse-extend-impls (the latter must treat a computed class type in head position).
No behavior change.
2026-06-25 17:35:18 -04:00
Dmitri Sotnikov
dcfe205a61
Merge pull request #214 from jolt-lang/fix/defn-docstring-assert-seqable
defn docstrings, assert throws AssertionError, Seqable covers collections
2026-06-25 21:26:41 +00:00
Yogthos
14ce46fb2a defn docstrings, assert throws AssertionError, Seqable covers collections
Conformance gaps surfaced re-running the library suites:

- defn now keeps a leading docstring as :doc metadata — it was dropped, so
  (:doc (meta #'f)) was always nil. Rides the def docstring slot.
- assert (and :pre/:post) throw a real AssertionError instead of an ex-info, so
  (catch AssertionError …) / (thrown? AssertionError …) match, with Clojure's
  "Assert failed: <msg>\n<form>" message.
- instance? clojure.lang.Seqable was conflated with ISeq, so a vector/map read
  as not-Seqable. Split them: Seqable covers every persistent collection, ISeq
  only seqs.
2026-06-25 17:23:24 -04:00
Dmitri Sotnikov
60e129a95c
Merge pull request #213 from jolt-lang/conformance/aws-api-host-shims
Host shims and protocol fixes shaken out by aws-api
2026-06-25 21:03:37 +00:00
Yogthos
829c251bca Host shims and protocol fixes shaken out by aws-api
Running cognitect aws-api's pure test namespaces (signing/shapes/protocols/
util/retry/endpoints) surfaced general gaps:

- extend-protocol/extend-type accept a computed class type, e.g.
  (Class/forName "[B") for the byte-array class — the byte-array idiom data.json
  and aws-api use. The macro grouping handled only symbol/nil heads (it crashed on
  a list type); type->name resolves a Class value via .getName; a byte-array
  dispatches on the "[B" host tag.
- java.nio.ByteBuffer over a jolt byte-array (wrap/allocate/get/put/array/
  remaining/position/limit/duplicate/flip), plus extend-protocol to it.
- java.util.Arrays (equals/copyOf/copyOfRange/fill) and java.util.Random
  (nextBytes/nextInt/…).
- java.net.URI/create and clojure.lang.RT/baseLoader statics.
- clojure.core.async/promise-chan (deliver-once, peek-don't-pop).
- a failed java.time parse throws DateTimeParseException (typed), so
  (catch DateTimeParseException …) matches it instead of leaking an untyped
  condition.

The XML side lives in the jolt-lang/xml library (libxml2 over jolt.ffi); ByteBuffer
stays in core as a generic java.nio primitive.

Gate: make test green (corpus +6 JVM-certified rows, 0 NEW divergence; unit
553/553; SCI 211).
2026-06-25 16:56:48 -04:00
Dmitri Sotnikov
05f29d1bcb
Merge pull request #212 from jolt-lang/conformance/core-memoize
Run core.memoize's test suite on jolt
2026-06-25 17:26:22 +00:00
Yogthos
d21ab77e7e Run core.memoize's test suite on jolt
Shaking out clojure.core.memoize (207 assertions, 0 fail) cleared several
general gaps:

- deref/@ on a deftype or reify implementing clojure.lang.IDeref dispatches to
  its deref method (RetryingDelay / make-derefable).
- deftype mutable fields (^:unsynchronized-mutable / ^:volatile-mutable) are
  read live: a set! within a method is observed by a later read in the same
  invocation, not the entry-time capture. Needed for double-checked locking.
  Immutable fields stay let-bound. Field reads rewrite to (.-field inst) with
  lexical-shadow tracking.
- def metadata values are evaluated, like Clojure: ^{:k (f)} stores (f)'s
  result and ^{:af some-fn} the fn. :tag stays a literal hint.
- try dispatches catch clauses by class in order via the exception supertype
  hierarchy; a non-matching value re-throws, an untyped host condition is caught
  by a RuntimeException/Exception/Throwable clause. Previously the last clause
  won and the class was ignored.
- locking takes a real per-object monitor (recursive Chez mutex) now that
  futures/agents/threads share one heap; it was a no-op.
- supers/ancestors reflect a small modeled JVM interface hierarchy, so
  (ancestors (class f)) yields Runnable/Callable (core.memoize's arg check).
- AssertionError / Error constructors.

JOLT_FEATURES is gone from the docs: it isn't read anywhere on Chez, and the
reader already includes :clj in its default feature set. RFC 0002's
{:jolt :default} design was reverted in the reader; docs now match the code.

Raises the SCI floor 205 -> 210.
2026-06-25 13:23:05 -04:00
Dmitri Sotnikov
3dde290f1a
Merge pull request #211 from jolt-lang/docs/host-interop-concurrency-refs
docs: host-interop for the new concurrency + reference shims
2026-06-25 15:46:26 +00:00
Yogthos
d06c7a0acc docs: host-interop — Thread/CountDownLatch, Soft/WeakReference + ReferenceQueue, ConcurrentHashMap, System/gc, Class/forName 2026-06-25 11:42:44 -04:00
Dmitri Sotnikov
ec792d28a0
Merge pull request #210 from jolt-lang/feat/gc-weak-references
Soft/WeakReference: real GC eviction via Chez weak pairs + guardians
2026-06-25 15:36:00 +00:00
Yogthos
80b2dfa9f9 Soft/WeakReference: real GC eviction via Chez weak pairs + guardians
Replace the strong-ref stub with genuine reclamation. The referent is held
through a weak-cons, so Chez's generational collector reclaims it once it is
otherwise unreachable (the pair's car becomes the bwp object, and .get returns
nil). A guardian registered on the referent makes the reference itself available
the instant its referent is collected, which ReferenceQueue.poll surfaces as
enqueued — the same hook clojure.core.cache's clear-soft-cache! drains.

Chez has no softer-than-weak reference, so a SoftReference clears on
unreachability rather than under memory pressure: a SoftCache evicts more eagerly
than the JVM's but is now real GC eviction, not an unbounded strong cache.
WeakReference gets the same (faithful) semantics. Added System/gc -> a full
collect so callers (and the queue) can force the cycle.

core.cache stays 1314/0/0 (its test values are immortal literals). Corpus row for
System/gc; make test + shakesmoke green.
2026-06-25 11:32:32 -04:00
Dmitri Sotnikov
e955b2072a
Merge pull request #209 from jolt-lang/fix/delay-exn-deftype-method-merge
core.cache: full conformance (delay/deftype/dispatch fixes + Thread/CountDownLatch/SoftReference)
2026-06-25 15:19:27 +00:00
Yogthos
774c6c0795 docs: list core.cache 2026-06-25 11:16:17 -04:00
Yogthos
9312ad0937 Thread/CountDownLatch + SoftReference/ConcurrentHashMap so core.cache fully passes
Closes the last clojure.core.cache gaps (now 1314/0/0, including the 1000-thread
cache-stampede):

- java.lang.Thread over Chez fork-thread (shared heap): (Thread. thunk) + start/
  join/run/isAlive, on a "user-thread" tag distinct from Thread/currentThread's
  interrupt shim. java.util.concurrent.CountDownLatch (count/mutex/condition).
- java.util.concurrent.ConcurrentHashMap = the mutable HashMap shim; get / count /
  contains? read it (clojure.core), which SoftCache uses on its backing map.
- java.lang.ref.SoftReference / ReferenceQueue: no JVM GC reference semantics, so
  the referent is held strongly (a SoftCache is unbounded rather than GC-evicting),
  but enqueue / poll work so clear-soft-cache! drains the queue.

JVM-certified corpus rows. make test + shakesmoke green.
2026-06-25 11:15:12 -04:00
Yogthos
5d0989a860 delay exception memoization, deftype cross-protocol method merge, more map-like dispatch
Further clojure.core.cache fixes (198 -> 257 of its assertions):

- delay: a throwing body re-ran on every force and never became realized?. Run it
  once like Clojure's Delay — cache the exception, mark realized, re-throw the same
  on each deref. Fixes value-fn memoization / cache-stampede protection.
- deftype/defrecord: a method name appearing in two protocols with different
  arities (data.priority-map's seq is in IPersistentMap [this] AND Sorted
  [this asc]) registered per-protocol and shadowed; merge clauses by name across
  all protocols into one multi-arity fn.
- empty?/peek/pop (IPersistentStack) dispatch through a deftype's methods; (= a-
  deftype other) uses its equiv method (so caches compare to their backing map);
  seq handles a host iterator (iterator-seq over .iterator).
- pop of an empty PersistentQueue returns it, like the JVM (was an error).

JVM-certified corpus rows. make test + shakesmoke green.
2026-06-25 10:57:31 -04:00
Dmitri Sotnikov
c445aaeaa2
Merge pull request #208 from jolt-lang/fix/destructure-or-prepost-deftype-dispatch
Destructuring :or, fn :pre/:post, deftype field access + map-like dispatch
2026-06-25 14:09:39 +00:00
Yogthos
b21b99b275 destructuring :or, fn :pre/:post, deftype field access + map-like dispatch
General fixes shaken out running clojure.core.cache (66 -> 198 of its assertions):

- Map destructuring applied an :or default only for :keys/:strs/:syms, not a
  direct {x :x} binding — so {x :x :or {x 9}} (and the & {…} kwargs form) ignored
  the default. Apply it for the direct binding too.
- fn didn't implement :pre/:post: a leading conditions map was evaluated as a body
  literal (so % was unbound and (.q %) blew up). Recognize it and assert pre
  before the body, bind % to the result, assert post, return %.
- (.q inst) on a deftype field with no matching method reads the field, like the
  JVM (was "No method q").
- A deftype implementing the clojure.lang collection interfaces now dispatches
  dissoc (without), contains? (containsKey), peek/pop (IPersistentStack), and
  keys/vals (via its Seqable seq) through its methods — they were field-only, so
  core.cache's caches and data.priority-map didn't behave as maps.

JVM-certified corpus rows for each. make test + shakesmoke green.
2026-06-25 10:06:33 -04:00
Dmitri Sotnikov
93b4d101a1
Merge pull request #207 from jolt-lang/fix/io-copy-htable-input-stream
io/copy: drain a byte-input-stream shim source
2026-06-25 13:04:21 +00:00
Yogthos
7952b1fe03 io/copy: drain a byte-input-stream shim source
input-bytes handled in-stream/bytevector/byte-array sources but not a host
byte-input-stream table (:jolt/input-stream — http-client's ByteArrayInputStream),
so io/copy fell through to rendering it as text (#[chez-htable …]) instead of its
bytes. Drain it like slurp does. Fixes http-client's response-body handling
(jolt-bjbi): its suite goes 100/16-fail -> 116/0, and the ring adapter's.
2026-06-25 09:01:02 -04:00
Dmitri Sotnikov
19b19fb83f
Merge pull request #204 from jolt-lang/feat/ring-defaults-host-interop
Host interop + deps fixes for ring-defaults on jolt
2026-06-25 10:40:46 +00:00
Yogthos
b2f671989d docs: list ring-defaults (via jolt-crypto) 2026-06-25 06:37:27 -04:00
Yogthos
65c8072ec8 io/copy: write to a byte-output-stream shim
io/copy handled file/stream/writer targets but not a host byte-output-stream
table (jolt-lang/http-client's ByteArrayOutputStream shim, :jolt/output-stream),
erroring 'don't know how to write to'. Dispatch through the shim's .write method,
byte-exact for a byte source — the JVM's io/copy writes to any OutputStream.
2026-06-25 06:29:57 -04:00
Yogthos
635cab0e49 host interop + deps fixes for running ring-defaults on jolt
Shaken out getting ring-defaults (and its ring-core/anti-forgery/session stack)
to load and serve static resources on jolt. All general fixes, all runtime:

- Class/forName throws a catchable ClassNotFoundException for a class jolt can't
  back (it returned a broken truthy value for any name, and crashed on use). Lets
  the common (try (Class/forName "optional.Dep") (catch ...)) probe libraries use
  to detect an absent dependency work — e.g. ring's joda-time check.
- deps: reconcile native libs (and source roots) in one step, deduped by library
  identity, instead of the ad-hoc distinct at each call site. An app pulling two
  libs that declare the same shared object (libcrypto via both jolt-crypto and
  http-client) now includes and loads it once.
- io: a File answers getProtocol ("file") / getFile so resource-serving
  middleware that expects io/resource to hand back a file: URL works; the
  classloader gains getResources (every source root holding the resource).
- clojure.string/replace accepts a char match/replacement, like the JVM.

JVM-certified corpus rows for the Class/forName and string/replace behavior.
2026-06-25 04:42:35 -04:00
Dmitri Sotnikov
16528e8637
Merge pull request #203 from jolt-lang/docs/libraries-conformance-directive
core.match: full support (regex + array patterns) + library-conformance directive
2026-06-25 04:50:41 +00:00
Yogthos
47b4971367 remove agent files 2026-06-25 00:47:23 -04:00
Yogthos
67e642bdfb core.match: regex + array patterns (full support); library-conformance directive
Finishes core.match — its full test suite (115/115) now passes, including the
two patterns the earlier work left out:

- Regex-literal patterns. A #"…" now reads as a regex VALUE (Clojure parity: the
  reader constructs the Pattern, so a macro receives a regex, not jolt's tagged
  form), and the analyzer compiles a regex value to the same :regex IR leaf via
  its source. emit-quoted handles a quoted regex; a regex value carries the
  java.util.regex.Pattern host tag so extend-protocol/instance? dispatch on it.
- Primitive-array patterns. A ^Type hint's :tag is now the SYMBOL (e.g. `ints`),
  matching the JVM, so core.match's array-tag lookup engages the array
  specialization (alength/aget). jolt's :tag consumers already tolerate a symbol
  (hc-cell-num-ret normalizes; tag->nkind/def-meta handle both).

Also: a library-conformance directive in CLAUDE.md, and the supported-libraries
list (docs + site) simplified to one-line entries — a listed library is assumed
to work fully, so no tallies or feature enumerations. core.match + transit-jolt
added to the list.

Seed change (reader/backend/30-macros) -> re-minted; the rest runtime. JVM-
certified corpus rows; the stale `symbol hint -> :tag` divergence is dropped from
the allowlist (jolt now matches the JVM). make test + shakesmoke green.
2026-06-25 00:46:10 -04:00
Yogthos
5737a39b7c docs: list core.match; add library-conformance directive to CLAUDE.md 2026-06-25 00:19:59 -04:00
Dmitri Sotnikov
d8683b0598
Merge pull request #202 from jolt-lang/feat/deftype-clojure-lang-interfaces
deftype/record: clojure.lang collection interfaces + protocol identity
2026-06-25 04:17:58 +00:00
Yogthos
f5455115a0 deftype/record: clojure.lang collection interfaces + protocol identity
Running clojure.core.match (a macro-heavy library that builds its compiler out
of deftypes implementing clojure.lang interfaces) shook out a cluster of general
gaps. Its own suite goes from not-loading to 111/115 assertions.

- deftype/defrecord implementing a clojure.lang collection interface now drives
  the core fns: Indexed -> nth, Counted -> count, Associative -> assoc, ILookup
  -> get/valAt (non-field keys only, so a method's own field bindings don't
  recurse), ISeq -> seq/first/rest, IPersistentCollection -> conj, IFn -> the
  value is callable. A jrec is still a map of fields by default; the interface
  method wins when declared.

- Multi-arity inline methods are grouped into one fn (a type with (nth [_ i]) and
  (nth [_ i x]) kept only the last before). Built as data, not a nested
  syntax-quote, so a `(= ~ocr ~l) method body keeps its unquotes.

- instance?/satisfies? recognize a protocol a type implements, including a MARKER
  protocol with no methods (core.match's IPseudoPattern) — deftype/defrecord now
  record protocol satisfaction even with zero methods. Added ILookup/Indexed/
  Counted to the instance? taxonomy for the built-in collections.

- Syntax-quote: a fully-qualified class name (clojure.lang.ILookup) stays bare
  instead of being namespace-qualified; (unquote x) is detected in a lazy seq
  (a macro that builds its template with map, e.g. deftype's rewrite-set).

- clojure.set union/intersection/difference are variadic (& sets) + union 0-arity.
- java map view methods: keySet/values/entrySet/size/isEmpty.
- deprecated java.util.Date getters (getYear/getMonth/...) + the multi-arg
  (Date. year-1900 month0 date hrs min) constructor.

Seed change (deftype/defrecord macros + clojure.set) -> re-minted; the rest are
runtime. 11 JVM-certified corpus rows; make test + shakesmoke green.
2026-06-25 00:14:19 -04:00
Dmitri Sotnikov
3cbfa8719c
Merge pull request #201 from jolt-lang/fix/edn-unknown-tag
edn: clean exception for an unknown reader tag
2026-06-25 03:13:35 +00:00
Yogthos
fb2749ac4c edn: clean exception for an unknown reader tag
clojure.edn's __read-tagged seam called (empty-pmap) — applying the empty-pmap
VALUE as a procedure — so an unknown tag (e.g. #object[...] from a JVM-printed
object, or any unregistered #foo) crashed the Chez VM with "attempt to apply
non-procedure" and surfaced a malformed condition (class :object, nil message)
instead of a catchable error.

Throw a clean ex-info naming the tag, matching the JVM's "No reader function
for tag <tag>". A reader port over edn (transit-jolt's read-conformance skip
path, aero, etc.) now catches a real exception instead of aborting.

clojure.core/read-string stays lenient (returns the tagged form) so clojure.edn
can apply :readers / :default before falling through to this throw.
2026-06-24 23:09:07 -04:00
Dmitri Sotnikov
9c48440a43
Merge pull request #200 from jolt-lang/feat/java-io-streams
java.io: full File API + byte/char streams over Chez ports
2026-06-25 02:17:02 +00:00
Yogthos
1853d827bd java.io: full File API + byte/char streams over Chez ports
Expand java.io so libraries that touch the filesystem work unchanged.

File: the full method surface — length, lastModified, can{Read,Write,Execute},
isHidden, list, mkdir(s), delete, createNewFile, renameTo, getParentFile,
get{Absolute,Canonical}File, compareTo/equals/hashCode — plus the statics
separator / pathSeparator / createTempFile / listRoots. A File now keeps the
path as given (new File("rel").getPath() is "rel", .isAbsolute false); a
relative path resolves against JOLT_PWD only when the filesystem is touched,
matching the JVM. slurp/spit and the dir helpers go through the same
resolution, fixing a spit-vs-slurp inconsistency.

Streams (host/chez/io-streams.ss) — each a jhost wrapping a Chez port, so
buffering, EOF and binary<->char transcoding come from Chez:
- FileInputStream / FileOutputStream / ByteArrayInputStream /
  ByteArrayOutputStream / BufferedInputStream / BufferedOutputStream
- FileReader / FileWriter / InputStreamReader / OutputStreamWriter /
  BufferedReader / BufferedWriter
Buffered* return the wrapped stream (Chez ports are already buffered).

clojure.java.io: input-stream/output-stream now yield real byte streams (were
aliased to the char reader/writer); added copy (byte-exact for byte sources),
make-parents, delete-file. with-open also closes file-writer/port-writer/
print-writer (a pre-existing gap).

All runtime shims, no re-mint. 15 JVM-certified corpus rows; make test +
shakesmoke green.
2026-06-24 22:12:46 -04:00
Dmitri Sotnikov
7bc4288e98
Merge pull request #199 from jolt-lang/feat/tick-dst-nano-data-readers
java.time DST + data readers: make tick pass fully
2026-06-25 01:33:41 +00:00
Yogthos
7b1ec9a1d3 java.time DST + data readers: make tick pass fully
Shaking out tick's api and alpha.interval suites (api 353->359, interval
0->103 passing) cleared a set of general gaps:

- Named-zone DST. Zones resolved to a fixed representative offset, so
  America/New_York in August read -05:00 not -04:00. Add US/EU DST rules
  (compact transition-date math) and make instant<->zoned, the zone rules'
  getOffset, and the zoned equality arm DST-aware.

- Nanosecond zoned/offset times. Instant is nanos but atZone/atOffset/
  toInstant/withZoneSameInstant and Instant/parse went through epoch-ms,
  truncating sub-ms. Route them through nanos.

- Locale month/day names. A formatter dropped its Locale; carry it and add
  French names so MMM under Locale/FRENCH renders "mai".

- Callable records. A defrecord implementing clojure.lang.IFn (tick's
  GeneralRelation) is now invokable: jolt-invoke dispatches to its inline
  invoke method. Also give collections the Iterable host tag so a protocol
  extended to Iterable matches vectors/seqs.

- Imported class short names. (:import [java.time ZonedDateTime]) then
  (. ZonedDateTime parse s) resolved to nil; an otherwise-unresolved bare
  Capitalized name that's a registered host class now resolves as a class.

- Data readers. A project's data_readers.{clj,cljc} is loaded into
  *data-readers* (reader namespaces required eagerly); registered #tag
  literals in source rewrite to (reader-fn 'form). clojure.core/read-string
  now applies #inst/#uuid/#"regex" and *data-readers* like Clojure.

- Duration/between accepts zoned/offset date-times.

All runtime shims, no re-mint. docs/libraries.md: tick full pass + aero.
2026-06-24 21:30:05 -04:00
Dmitri Sotnikov
8d7d03bfbc
Merge pull request #198 from jolt-lang/docs/libraries-spec-tick-json
libraries: add data.json, spec.alpha, tick
2026-06-25 00:41:52 +00:00
Yogthos
d75f06980f libraries: add data.json, spec.alpha, tick 2026-06-24 20:38:43 -04:00
Dmitri Sotnikov
462d53a28e
Merge pull request #197 from jolt-lang/spike/special-form-precedence
Make clojure.spec.alpha load and run
2026-06-25 00:36:13 +00:00
Dmitri Sotnikov
866b8c47d4
Merge pull request #196 from jolt-lang/spike/instant-nanos
java.time.Instant: nanosecond precision
2026-06-25 00:35:43 +00:00
Yogthos
7a343351d6 Make clojure.spec.alpha load and run
Four general gaps, shaken out by loading clojure.spec.alpha:

- Special forms were shadowable by a same-named macro. analyze-list
  macroexpanded before checking special forms, so a ns that redefs def/and/or
  (spec excludes them via :refer-clojure :exclude) made a bare def resolve to
  the macro instead of the special form, breaking every defn after. Now a head
  in the special-form set is never macroexpanded, matching the reference
  macroexpand1 isSpecial check.

- reify dropped all but the last arity of a multi-arity protocol method (spec
  reifies (specize* [s]) and (specize* [s _])). The macro keyed methods by name
  and overwrote; now it groups arities into one multi-arity fn.

- reify instances did not implement IObj: with-meta threw and (instance?
  clojure.lang.IObj r) was false. Every Clojure reify carries metadata. with-meta
  now copies the reify to a fresh identity (shared method table) and keys its
  meta; instance? IObj/IMeta is true for any reify. This was the registry bug —
  spec's with-name returned nil for specs, so get-spec missed.

- (set! (. Class field) val) was rejected. spec toggles
  clojure.lang.RT/checkSpecAsserts this way; the analyzer now lowers it to a
  jolt.host/set-static-field! call over a mutable-statics table, and a plain
  Class/field read consults that table.

Also: .name/.getName on a Namespace and .ns/.sym on a Var (spec's ns-qualify /
->sym). analyzer + reify are seed sources (re-minted). spec.alpha now does
valid?/conform/cat/keys/explain-str/check-asserts. tick.alpha.interval-test still
needs time-literals data readers (separate).
2026-06-24 19:46:22 -04:00
Yogthos
f417516148 java.time.Instant: nanosecond precision
The Instant jhost stored epoch-ms, so plusNanos/getNano rounded to the
millisecond and two instants a nanosecond apart compared =. Store epoch-nanos
instead: mk-instant still takes ms (scales to nanos) for the ms-based zone/Date
callers, mk-instant-nanos/inst-nanos own the nano arithmetic, and inst-ms floors
nanos back to ms. plus/minus/getNano/truncatedTo/compare/equals and the ISO
renderer all run on nanos; toString shows the fraction in 3/6/9-digit groups like
ISO_INSTANT. Fixes tick's interval coincidence test (shift end by one nano).
2026-06-24 19:10:58 -04:00
Dmitri Sotnikov
dcfc764b69
Merge pull request #195 from jolt-lang/spike/java-time-phase1
java.time on Chez: LocalDate..ZonedDateTime + formatters — runs tick (api_test 352/7)
2026-06-24 22:51:33 +00:00
Yogthos
a05eeefb08 java.time Phase 3: zones, offsets, ZonedDateTime, formatters — tick runs
ZoneOffset/ZoneId (SHORT_IDS, fixed-offset + UTC + system; named zones via a
fixed-offset table), ZonedDateTime/OffsetDateTime/OffsetTime, Clock (fixed/
system, with now [clock] arity), and DateTimeFormatter integration (ofPattern
+ ISO_* constants, .format/.parse over the rich java.time values via the
inst-time.ss pattern engine). systemDefault resolves to UTC to keep the
#inst atZone/toInstant round-trip machine-tz-independent.

tick.core + tick.protocols + tick.locale-en-us load; tick's api_test runs
31 tests / 352 pass / 7 fail / 0 error. The 7 are host gaps: named-zone DST
(no tzdb), French locale month names (no locale DB), nanosecond Instant.

General fixes surfaced by tick: :ns/keys map destructuring ({:tick/keys [..]})
in 00-syntax.clj (re-minted), and extend-protocol to java.time classes
(records.ss host-type-set). 12 corpus rows certified vs JVM. make test +
shakesmoke green, selfhost holds, 0 new divergences, data.json stays 138/139.
2026-06-24 18:45:46 -04:00
Yogthos
e3c14e656c java.time Phase 2: Duration/Period, enums, ChronoUnit/Field machinery
Duration (ISO PT.. toString, between, full arithmetic), Period (between with
borrow, P.. toString, normalized), full Month/DayOfWeek enums (named constants,
print as their name — fixes the Phase-1 raw-jhost print), Year, YearMonth
(2020-02 toString, leap, atDay/atEndOfMonth), ChronoUnit (between/getDuration)
and ChronoField. The temporal machinery on the Phase-1 types now works with
ChronoUnit/ChronoField: (.plus t n DAYS), (.until t1 t2 unit), (.get/.getLong
t field), (.with t field v), (.isSupported ..), (.truncatedTo ..).

Analyzer: (. Class method args) with a class target lowers to a static call
(Class/method args) instead of mis-dispatching as an instance call on the arg
— matches JVM; needed by cljc.java-time.year. Seed re-minted; selfhost holds.

The Phase-2 cljc.java-time namespaces load; tick.core advances to a Phase-3
zone gap. 10 corpus rows certified vs JVM. make test + shakesmoke green, 0 new
divergences, data.json stays 138/139.
2026-06-24 18:10:40 -04:00
Yogthos
c0561a8d14 java.time Phase 1: LocalDate/LocalTime/LocalDateTime/Instant
Core java.time value types as jolt host objects backed by the inst-time.ss
calendar engine (days-from-civil/civil-from-days/inst-fields/format-ms), in a
new host/chez/java-time.ss. tz-free reps: LocalDate=epoch-day,
LocalTime=nano-of-day, LocalDateTime=(epoch-day,nano-of-day); Instant reuses
the ms-based instant jhost (ms granularity; nano is a documented gap). Each
type registers statics (of/parse/now/MIN/MAX/...), instance methods
(plus/minus/with/get/isBefore/until/toString/...), =/hash, compare,
ISO-8601 print, instance?, and value-host-tags for protocol dispatch.

Reconciles the old ms-based local-date/local-dt stubs into the rich types
(LocalDateTime now prints ISO; .toLocalDate/.atZone paths preserved). The
four cljc.java-time namespaces (local-date/local-time/local-date-time/instant)
load. Deep temporal-field/unit machinery (range/get-long/with-field/until/
adjust-into) stubbed for Phase 2.

12 corpus rows certified vs JVM 1.12.5. make test + shakesmoke green, 0 new
divergences, data.json stays 138/139, selfhost holds.
2026-06-24 17:44:32 -04:00
Dmitri Sotnikov
e94ddb2ffe
Merge pull request #194 from jolt-lang/spike/datajson-final
data.json: Calendar/java.time, java.sql.Date class, JVM-faithful Class value (suite 138/139)
2026-06-24 20:49:57 +00:00
Yogthos
8c7b98e5f2 (class x) returns a JVM-faithful Class value
class / .getClass now return a Class value that renders like the JVM —
(str c)/.toString -> "class <name>", pr -> "<name>", .getName/.getSimpleName/
.getCanonicalName work — but stays = and hash equal to its name string, so
(= (class x) String), class-keyed maps, multimethod dispatch on class, and
instance? keep working against the bare class-name tokens. instance? unwraps
a Class passed as the type arg.

clojure.test/class-match? no longer assumes (class e) is a string (a jolt-ism;
on the JVM a Class isn't a string either) — reads the name via .getName.

Matches JVM Class.toString, which libraries surface in error messages
(clojure.data.json DJSON-54 expects "...of class java.net.URI"). data.json
suite 139/139 bar the one UTF-16 surrogate test (Unicode-scalar char model).

5 corpus rows certified vs JVM; make test + shakesmoke green, 0 new divergences.
2026-06-24 16:46:09 -04:00
Yogthos
9092b30f8b java.util.Calendar + java.time round-trip + java.sql.Date class
java.util.Calendar (a UTC calendar over epoch-ms): getInstance, the int
field constants, set/get/setTime/getTime/getTimeInMillis. java.time pieces
for a zoned round-trip: DateTimeFormatter/ISO_ZONED_DATE_TIME, formatter
.withZone/.parse, LocalDateTime/parse (with formatter), Instant/from,
.toLocalDate/.atStartOfDay. java.sql.Date is now a distinct class (its own
host value + dispatch tags) so a protocol extended to both java.util.Date
and java.sql.Date routes a sql.Date to its own impl. All UTC-consistent.

data.json print-sql-date + print-time-supports-format pass (suite 137/139).
2026-06-24 16:31:59 -04:00
Dmitri Sotnikov
7e10904e8c
Merge pull request #193 from jolt-lang/spike/pprint-port
Port real clojure.pprint (pretty-printer + cl-format); data.json pprint passes
2026-06-24 20:17:38 +00:00
Yogthos
ce4bd45852 java.sql.Date 1-arg ctor (epoch millis) 2026-06-24 16:13:22 -04:00
Yogthos
f66cf486b0 Port real clojure.pprint (pretty-printer + cl-format)
Replace the 33-line pprint shim with a column-aware pretty printer and a
Common Lisp cl-format engine, ported from the ClojureScript implementation
(no STM — atom-backed fields) and adapted to JVM-Clojure interop. Provides
pprint/write/write-out/with-pprint-dispatch/formatter-out/cl-format and
simple/code dispatch.

core print routes column-aware into an active pretty-writer via a __write
hook (suppressed inside with-out-str captures); PrintWriter host class
forwards into the wrapped writer. Re-mint: pprint is baked into the seed.

Unblocks clojure.data.json/pprint (its pretty-printing test passes).
Directives ~R/~P/~C/~F/~E/~G/~$/~(~) not ported (unused by the targets).

make test + shakesmoke green, 0 new divergences, selfhost holds.
2026-06-24 16:09:09 -04:00
Dmitri Sotnikov
348d91c1d1
Merge pull request #192 from jolt-lang/spike/datajson-tests
data.json host interop: readers, number dispatch, dates, exceptions (suite 134/137)
2026-06-24 19:21:55 +00:00
Yogthos
31c8f56784 Host interop for clojure.data.json: readers, numbers, dates, exceptions
Shaking out clojure.data.json's own test suite (now 134/137):

- Reader.read(char[],off,len) bulk read + PushbackReader.unread(char[],off,len)
  on the string/pushback reader jhosts; instance? java.io.PushbackReader/
  Reader/StringReader (data.json re-wraps a reader unless it's already a
  PushbackReader, so this is load-bearing for repeated reads).
- number protocol dispatch by actual type: a flonum is Double (not Long),
  exact ratio is Ratio, exact integer is Long — value-host-tags split.
- Integer/toHexString|toOctalString|toBinaryString|toString; .isNaN/.isInfinite
  as instance methods on numbers.
- EOFException ctor/class; .isArray on a class-name string.
- dispatch tags for the uuid/bigdec/inst host values so a protocol extended to
  java.util.UUID / java.math.BigDecimal / java.util.Date / java.time.Instant
  reaches its impl; canonical-host-tag strips java.math./java.time.
- instant/zoned/local time values compare = by epoch-ms (two parsed Instants).
- java.time.Instant/parse, java.sql.Date ctor + valueOf, TimeZone/getDefault,
  DateTimeFormatter/ISO_INSTANT.

All runtime .ss (no re-mint). 9 corpus rows certified vs JVM; make test +
shakesmoke green, 0 new divergences.
2026-06-24 15:16:02 -04:00
Dmitri Sotnikov
18874ce4ab
Merge pull request #191 from jolt-lang/spike/json-conformance
JSON conformance: read-string sets, host-class dispatch, String/char interop, format %x, extend nil
2026-06-24 18:39:46 +00:00
Yogthos
8bd781e6a8 String/char-array interop: getChars, String(char[]), str of StringBuilder, append(csq,start,end)
- String.getChars copies into a char-array; String. builds from a char[]
  (whole or offset/count slice); subSequence returns the substring.
- str of a StringBuilder returns its content (was the opaque host object;
  .toString already worked).
- Appendable.append gains the 3-arg subsequence form append(csq,start,end)
  on StringBuilder/StringWriter/file/port writers.
- reader combines a \uXXXX surrogate pair into the one Unicode scalar
  (😃) instead of crashing on the lone high surrogate; a stray surrogate
  maps to U+FFFD. (A high-plane char re-escaped as \uXXXXX stays the
  irreducible UTF-16/scalar divergence.)
- TimeZone/getDefault.

These let clojure.data.json read and write JSON; its own suite goes from
not loading to 97/133 assertions passing (remainder: per-type writer
dispatch for uuid/bigdec/date, EOFException, niche date interop).
2026-06-24 14:36:13 -04:00
Yogthos
c6b8f31608 instance? recognizes common host interfaces
(instance? clojure.lang.Named :a), java.lang.CharSequence/Number,
java.util.Map/Set/List/Collection, clojure.lang.Associative now report
true for jolt's value model, matching the JVM (a Map is not a Collection;
a List excludes sets/maps). Libraries branch on these — data.json's
default-write-key-fn keys on clojure.lang.Named, so map keys serialized
with the leading colon before this.
2026-06-24 14:21:59 -04:00
Yogthos
21895cb932 Class-name symbols self-evaluate; extends? matches host classes; ISO_INSTANT
A slash-free dotted symbol with a Capitalized final segment (java.util.Map,
clojure.lang.Named, java.time.Instant) now self-evaluates to its name string
instead of resolving to nil — jolt models a class as its name, so a library
can extend a protocol to, or instance?-check, a host class jolt has no shim
for. hc-resolve-global classifies these as :class; the analyzer emits a const.

extends? now matches when either the query or the registered tag is a dotted
suffix of the other, so (extends? P java.util.Collection) finds the impl
extend registered under the canonical short tag.

Add DateTimeFormatter/ISO_INSTANT (UTC, trailing Z).

These unblock loading clojure.data.json, which dispatches JSONWriter on
java.util.Map/Collection/CharSequence/Instant and defaults a formatter to
ISO_INSTANT.
2026-06-24 14:17:34 -04:00
Yogthos
c26fd175f2 read-string constructs sets; format %x lowercase; extend/extends? on nil
read-string/read now return real sets for #{...} literals (top-level and
nested) instead of the reader's {:jolt/type :jolt/set} form — the data
seams convert set forms to sets (recursing, preserving metadata and source
map-key order); clojure.edn already did this. The compiler keeps reading
via the raw reader, so set literals in code stay forms the analyzer lowers.

format %x now emits lowercase hex (Chez number->string is uppercase); %X
unchanged.

extend and extends? handle a nil target type (host tag "nil"), matching
extend-type — protocols can be extended to nil via the function form, not
just the macro.

Found porting transit/data.json and shaking out aero.
2026-06-24 14:03:47 -04:00
Dmitri Sotnikov
e74b940db5
Merge pull request #190 from jolt-lang/spike/coll-ops-metadata
Collection ops carry the receiver's metadata
2026-06-24 17:52:34 +00:00
Yogthos
2a74f9652c corpus: collection ops preserve metadata rows 2026-06-24 13:46:58 -04:00
Yogthos
4ae3d3116e Collection ops carry the receiver's metadata
conj/assoc/dissoc/disj/pop/into and empty now thread the receiver's
metadata onto the result, matching Clojure (each op constructs a new
collection with meta() carried forward; coll.empty() is
EMPTY.withMeta(meta())). The metadata side-table is now weak so meta on
intermediate collections is reclaimed with them, and empty-list-t carries
an (unused) field so a metadata-bearing () is a distinct identity from the
shared singleton instead of leaking meta onto every ().

Unblocks metadata-driven walks (aero/integrant): (into (empty form) ...)
now preserves a vector/map/set's metadata, so a postwalk whose outer fn
reads (meta x) sees it.
2026-06-24 13:46:58 -04:00
Dmitri Sotnikov
5b55c7f7b0
Merge pull request #189 from jolt-lang/spike/auto-resolved-keywords
Auto-resolved keywords, edn reader opts, and related reader/runtime fixes
2026-06-24 16:07:52 +00:00
Yogthos
d19310cb7e corpus: metadata round-trip rows 2026-06-24 12:04:36 -04:00
Yogthos
473d1002b7 Reader attaches collection metadata to data, not a with-meta form
The reader lowered ^meta on a vector/map/set literal to a runtime
(with-meta form meta) list, so read-string/edn of data with metadata
returned the form and lost the metadata. Attach it to the value instead,
as Clojure does; the analyzer re-emits (with-meta coll meta) for a
meta-carrying collection literal in code, so a literal still carries its
metadata at runtime and ^Type/^long arglist hints (consumed by
analyze-arity directly) are unaffected.

Also: pr honors *print-meta*, and clojure.walk/clojure.edn re-attach
metadata to the collections they rebuild (matches Clojure; a
metadata-driven config lib like aero relies on it).
2026-06-24 12:04:36 -04:00
Yogthos
091d2c4b62 corpus: rows for the tagged-literal / interface / data-reader fixes 2026-06-24 11:03:34 -04:00
Yogthos
79d0084163 Readable messages for host conditions
clojure.test reported a host-condition crash with a blank message (ex-message is
nil for non-ex-info conditions); fall back to jolt.host/condition-message.
condition-message itself left a Chez format-template message (open-input-file's
'failed for ~a: ~(~a~)') unformatted; apply its irritants.
2026-06-24 11:03:34 -04:00
Yogthos
7947918919 Add File .isAbsolute and the default #inst/#uuid data readers
jfile lacked .isAbsolute (path starts with /). clojure.core/default-data-readers
and *data-readers* were nil, so a library merging them (aero's reader opts)
couldn't resolve #inst. Define them keyed by symbol, like Clojure.
2026-06-24 11:03:34 -04:00
Yogthos
b2394325d4 Recognize core clojure.lang interfaces in instance?
(instance? clojure.lang.IObj/IMapEntry/IRecord/IPersistentMap/... x) all returned
false, so libraries branching on these interfaces (e.g. a metadata-preserving
walk guarded by IObj) silently misbehaved. Register an instance-check arm mapping
the interface tokens to jolt predicates, matched by last dotted segment.
2026-06-24 11:03:34 -04:00
Yogthos
3412563a79 Tagged literals are value-equal
(tagged-literal t f) compared by identity, so two equal tagged literals were not
= and didn't dedup as map keys / set members; JVM's TaggedLiteral is value-equal.
Add a register-eq-arm! for jtagged (tag + recursive form); hash is already
structural. Fixes an infinite loop in any fixpoint that dedups structures
containing tagged literals (e.g. aero's #ref expansion).
2026-06-24 11:03:33 -04:00
Yogthos
b2bc52935d Resolve relative slurp/io-reader paths against JOLT_PWD
io/file already resolved a relative path against JOLT_PWD (the user's cwd before
the launcher cd's to the repo root), but slurp and io/reader on a relative string
path opened it against the process cwd — so a program run from its own directory
couldn't read its own relative files (e.g. aero's (read-config "config.edn")).
Route the string branches through project-relative, matching io/file. Internal
callers pass already-resolved paths through read-file-string and are unaffected.
2026-06-24 10:22:10 -04:00
Yogthos
d5deba2df8 Define *print-meta* as a bindable dynamic var
clojure.core/*print-meta* was missing, so (binding [*print-meta* true] …) failed
to compile ("var of non-var"). Add it (default false, like the JVM); corpus
covers binding it.
2026-06-24 10:22:10 -04:00
Yogthos
54a72498ce spec: note jolt's unknown-alias behavior; corpus rows for the reader/edn fixes
The EBNF and reader S7 already specified ::kw auto-resolution — the
implementation was out of spec, now aligned. S7 noted unknown ::alias/k MUST be a
read error; jolt is lenient (reads :alias/k), so record that as a deviation.
Corpus gains JVM-certified rows for ::kw resolution, clojure.edn :default
receiving a symbol tag, and with-meta on a lazy seq.
2026-06-24 09:33:45 -04:00
Yogthos
289e41943e Shim clojure.lang.LineNumberingPushbackReader
A pushback-reader (drainable, so clojure.edn/read over it works); getLineNumber
is a stub since jolt doesn't track line numbers. Config readers (aero) wrap their
input in one.
2026-06-24 09:13:28 -04:00
Yogthos
44b7f39562 defmethod on a referred multifn resolves to its home ns
A (defmethod m …) where m is :refer'd from another ns passed the bare symbol to
defmethod-setup, which resolved it in the current ns and created a shadow multifn
— the method never reached the real one. Resolve an unqualified name through the
refer table (then current ns) so it lands on the referred multifn.

The AOT build strips the ns form, so the refer table is empty in a binary; emit
chez-register-refer!/-refer-all! per app ns alongside the existing alias
registrations. build-app's fixture gains a defmethod on a referred multifn.
2026-06-24 09:13:28 -04:00
Yogthos
2c5b7fc27a Support metadata on lazy seqs
with-meta/meta only recognized the eager collections, so (with-meta (map-indexed
…) m) — a lazy seq — threw "value does not support metadata". A lazy seq is IObj
on the JVM; add it to the metadatable set (keyed in the side-table by identity,
like cseq/procedure).
2026-06-24 09:13:28 -04:00
Yogthos
6d56982abe Auto-resolve ::keywords; honor clojure.edn reader opts
The reader dropped the namespace on ::kw (read ::foo as :foo), so auto-resolved
keywords never matched their qualified form — code that round-trips them (spec
keys, aero's :aero.core/* expansion keys) silently broke. Resolve ::name against
the current ns and ::alias/name through the alias table, as Clojure does. The
runtime loader reads form-by-form with the ns set after the ns form; the
cross-compile reads all forms up front, so ei-emit-ns*/ei-emit-ns-records set the
ns before reading.

clojure.edn/read over a reader discarded its opts map — :readers/:default/:eof
were ignored, so a custom :default never saw the tag. Route the reader arity
through read-string so opts apply, and pass the tag to :default as a symbol (not
the internal :#name keyword), matching Clojure.

Seed re-minted (the ::halt transducer key in clojure.core now reads as
:clojure.core/halt). Corpus gains ::-keyword rows; the unit case that asserted the
old ns-dropping behavior now asserts the qualified result.
2026-06-24 09:13:14 -04:00
Dmitri Sotnikov
53b1c79d99
Merge pull request #188 from jolt-lang/spike/arch-refactor
Architecture refactor + AOT/runtime fixes
2026-06-24 05:45:09 +00:00
Yogthos
70d52ae704 Split the success-checker out of types.clj
types.clj held the inferencer, the success-type checker, and the driver in one
716-line namespace. Move the self-contained checker into jolt.passes.types.check:
the error-domain predicates (not-number?/not-seqable?/not-callable?), the op
tables, type-name, check-invoke, and the user-fn registry. These are pure over
inferred types and the run's env cells, with no inference, so a check-rule edit
can no longer perturb the inferencer.

The infer-coupled probes stay in types.clj — isolated-diag-count and
check-user-call re-run inference, so moving them would make check depend on the
inferencer and reintroduce the cycle. Verbatim move; new ns wired into
ei-compiler-ns-files; seed re-minted to the byte-fixpoint.
2026-06-24 01:39:39 -04:00
Yogthos
59e231e40d docs: list integrant under working libraries 2026-06-24 01:31:08 -04:00
Yogthos
66236628db Flush stderr in __eprint
The stderr seam wrote to current-error-port without flushing, so a process that
never returns from -main (a server loop) left its log lines in a buffer that only
drained at exit — they never appeared. Flush each write, like the JVM's
auto-flushing System.err.
2026-06-24 01:27:49 -04:00
Yogthos
66ad475722 AOT build: set per-ns ns context and register aliases
The source loader sets the current ns and registers :as aliases per file. The
build flattened every app namespace into one image with no such markers, so all
app forms ran under the last-set ns ("user"). Two breakages followed, both only
in a built binary:

- defmulti/defmethod resolve their target var through chez-current-ns, so they
  registered the multifn under "user" while compiled var-derefs used the baked
  ns — the multifn the app saw was uninitialized ("not a fn nil" on dispatch).
- a quoted alias-qualified symbol (a (defmethod ig/foo …) on an aliased multifn)
  resolves its ns through chez-resolve-alias, but the stripped (ns …) form left
  the alias table empty, so it landed in ns "ig".

bld-ns-prelude now emits (set-chez-ns! ns) plus chez-register-alias! for each
ns's :as aliases before that ns's forms, in both the normal and tree-shake emit
paths. The build-app fixture gains a :default multimethod and an aliased cross-ns
defmethod so buildsmoke covers both across all build modes.
2026-06-24 01:27:49 -04:00
Yogthos
ea609d72eb clojure.walk: preserve record types
walk treated a record as a plain map (record? implies map?), rebuilding it via
(into (empty form) ...) which yields a bare map and drops the type. Add a record
branch before the map branch that conj-es the walked entries back onto the
original, matching JVM clojure.walk's IRecord case. Type-dispatched walks need it
— integrant resolves #ig/ref by detecting its Ref record while postwalking the
config, so without this every ref silently fails to resolve.

clojure.walk is baked into the prelude, so the seed is re-minted. Corpus gains
five JVM-certified rows for record type/instance? survival through pre/postwalk.
2026-06-24 01:26:16 -04:00
Yogthos
54c3c6dd2b Decompose the type inferencer's :invoke arm
infer's :invoke case was ~120 lines of cond arms hand-coding eight call
patterns, all destructured positionally with (nth r 0)/(nth r 1) on the
[type node'] tuples infer returns. Split each pattern into a named helper
(infer-pred-fold/-kw-lookup/-get-lookup/-reduce-hof/-seq-hof/-conj-into/-call)
behind an infer-invoke dispatcher that keeps the cond guards verbatim, and add
ty/nd accessors for the tuple so a silent transposition can't hide.

The accessors are applied only to genuine infer results (the new helpers and
infer-fn-seeded); the :map/:let/:loop arms interleave non-infer pairs
(binding tuples, accumulator pairs) with infer results, so those keep nth.
Pure restructuring — the guards, order, and bodies are unchanged; seed
re-minted to the byte-fixpoint, gate green, 0 new corpus divergences.
2026-06-24 00:39:33 -04:00
Yogthos
a893e21111 Share the const-keyword-lookup head predicates
The inliner and the type inferencer each recognized (:k m) and (get m :k)
lookups with their own copy of the callee tests — the get-callee check was
duplicated verbatim across both. Lift kw-callee?/get-callee? into
jolt.passes.fold (alongside scalar-const?) and call them from both passes so
the head recognition can't drift.

Only the head predicates move. The deliberate differences stay: the inliner
still accepts any scalar key in the get-form (its scalar-replacement targets
can be string/number-keyed maps) while the inferencer requires keyword keys
for struct field typing, and the inferencer keeps its two arms separate so each
rebuilds args for its form. The backend's value-as-fn ifn-kind is left alone.
2026-06-24 00:30:06 -04:00
Yogthos
e9d56422bd docs: cross-link the numeric specializer + op tables (jolt-nzuo)
numeric.clj dbl-spec/lng-spec and backend_scheme.clj dbl-ops/lng-ops must agree —
a spec'd op with no table entry makes emit-numeric splice a nil op string. Document
the contract on both sides. Comment-only; seed re-mints byte-identical, gate green.

The other two ideas in this bead were rejected after inspecting the code:
- collapsing inline.clj local-escapes? onto reduce-ir-children would reintroduce the
  under-reporting hazard its docstring deliberately guards against (default-true is
  load-bearing for scalar-replacement soundness).
- folding numeric recur-kinds/recur-arg-lists into one walk loses the type-env
  threading recur-kinds needs through :let; the parallel split is justified.
2026-06-24 00:07:35 -04:00
Yogthos
9b4769f2e5 cleanup: drop dead form-char? refer, document an-invoke :wild rule (jolt-xkbo)
analyzer.clj referred jolt.host/form-char? but never called it (form-char? stays
live — backend_scheme.clj uses it). Promote numeric.clj an-invoke's :wild operand
rule (an integer literal is valid in either fl/fx kind) from an inline comment to the
function docstring. Both output-neutral: the seed re-mints byte-identical, gate green.
2026-06-24 00:03:34 -04:00
Yogthos
43a0da4dd0 refactor: rename dynamic-vars.ss, extract natives-format.ss (jolt-7dkx)
Two small clarity moves from the review:
- dynamic-vars.ss -> dynamic-var-defaults.ss. It holds the default VALUES of a few
  core dynamic vars (*clojure-version*, *assert*, …); the near-identical dyn-binding.ss
  holds the binding-stack machinery. The names were easy to confuse.
- Pull the ~60-line %-format engine out of the natives-misc.ss grab-bag into
  natives-format.ss (its ->long/pad-left/fmt-float helpers were local to it).
rt.ss loads + MODULES.md updated. Runtime .ss, no re-mint; make test green, format +
the dynamic vars verified.
2026-06-23 23:56:50 -04:00
Yogthos
960d8b6e24 refactor: consolidate transducers, dissolve natives-parity.ss (jolt-jcs7)
natives-parity.ss was a grab-bag (its own header called it 'parity'). Dissolve it
into homes that say what they hold:
  - cat            -> natives-transduce.ss (was natives-xform.ss, renamed for the
                      whole transducer surface: volatiles + cat + transduce/sequence)
  - transient?     -> transients.ss
  - rseq           -> natives-seq.ss
  - hash family    -> natives-misc.ss (the public hash API over jolt-hash)
The remainder — the #?() feature set, the reader-conditional + re-matcher tagged-map
ctors, and macroexpand — is a coherent reader/macro runtime-support unit, kept as
natives-reader.ss (np- prefix -> nr-). rt.ss loads + two comment refs updated.

Runtime .ss, no re-mint. make test green; hash/transient?/rseq/cat/macroexpand and
#?() reader features all verified.
2026-06-23 23:50:42 -04:00
Yogthos
a594c9deb4 refactor: rename host-static-{statics,objects}.ss for clarity (jolt-wn0u)
The trio split on a fine axis (registry core / statics / object classes) but the
names didn't say so — 'static-statics'/'static-objects' and headers that read
'Continues X'. Rename:
  host-static-statics.ss -> host-static-methods.ss  (Class/member statics + fields)
  host-static-objects.ss -> host-static-classes.ss  (instantiable object classes)
host-static.ss stays the registry core. Headers rewritten to state each file's role
and what it covers instead of chaining. rt.ss loads + the one comment reference +
MODULES.md updated. No code moved; runtime .ss, make test green.
2026-06-23 23:42:11 -04:00
Yogthos
4461179804 Fix direct-link crash on a non-fn var called as a function
Under --direct-link a top-level def binds jv$<fqn> and app->app calls bound directly
to it. emit-invoke raw-applied that binding for any var callee, but only a fn-valued
def is a Scheme procedure: (def cfg {...}) then (cfg :a) emitted (jv$cfg :a), applying
a pmap -> "attempt to apply non-procedure". Maps/sets/keywords are invokable in Clojure
via jolt-invoke, which the indirect path used, so this only bit closed-world builds.

Track which direct-linked vars hold fn literals (direct-link-fns, registered at the def
site when the init op is :fn) and only raw-apply those. A non-fn callee falls through to
the jolt-invoke branch, which still uses the direct jv$ binding as the invoke target —
so the var-deref is still skipped, just not the dispatch.

Seed source: re-minted. Regression in directlink-test.ss (jolt-cw1o).
2026-06-23 23:34:28 -04:00
Yogthos
9a21325972 refactor: registry pattern for jolt-pr-str/pr-readable + remaining arms (jolt-lmot)
The printer's two entry points (jolt-pr-str in rt.ss, jolt-pr-readable in printing.ss)
get register-pr-str-arm! / register-pr-readable-arm!, plus register-pr-arm! for the
types whose str and readable forms match (bigdec/inst/uuid/tagged/record/ns/var). The
normalize arms (sorted, lazy-seq, queue) and the uri readable arm register per-printer.
Also folds in the hash (dyn-binding var-cell), class (io uri/uuid/file), and get
(transients) arms missed earlier.

natives-array's get stays a case-lambda wrapper on purpose: its 2-arg path errors on
an out-of-bounds index while the 3-arg path returns the default, an arity distinction
the (coll k d) registry collapses — left as-is to preserve behaviour.

Completes jolt-lmot: all six dispatchers (hash/class/get/=/pr-str/pr-readable) off the
set!-rebind chains. make test green, 0 new corpus divergences; pr-str/str of inst,
uuid, bigdec, sorted-map, record-with-lazyseq, queue all verified.
2026-06-23 23:22:25 -04:00
Yogthos
acf3e1ffd3 refactor: registry pattern for jolt-get + jolt=2 (jolt-lmot)
jolt-get (4 sites: host-table/natives-misc/inst-time/records) and jolt=2 (6 sites:
records/vars/inst-time/natives-misc/bigdec/host-table) move off the set!-rebind
chains to register-get-arm! / register-eq-arm!. get's case-lambda becomes a stable
2/3-arg entry over a 3-arg dispatch; the equality arm pred is (a b) since either arg
may carry the type, and the host-table sorted arm normalizes-then-re-dispatches.

Behaviour-preserving (runtime .ss): var/inst/bigdec/record/uuid equality, record!=map,
sorted-map=plain-map, and all the get cases verified; make test green, 0 new corpus
divergences. Four of six dispatchers done; the printer (pr-str/pr-readable) remains.
2026-06-23 23:12:08 -04:00
Yogthos
d168d1195b refactor: registry pattern for jolt-hash + jolt-class (jolt-lmot)
Replace the set!-capture-and-rebind chains extending jolt-hash (3 sites: inst-time/
host-table/records) and jolt-class (3 sites: bigdec/natives-queue/host-table) with a
register-hash-arm! / register-class-arm! registry (the pattern already used by
register-str-render!). The base dispatcher walks its arms — disjoint types, so order
is immaterial — then falls to the base cases. The entry is stable, so the per-site
(def-var! "clojure.core" "class" …) re-points are gone.

Behaviour-preserving (runtime .ss, no re-mint): jinst hash, bigdec/queue/record
class, record hash, and a sorted-map hashing as its plain map all verified; make test
green, 0 new corpus divergences. First two of six dispatchers; get/=/pr-str follow.
2026-06-23 23:01:30 -04:00
Yogthos
bcd752717e Remove REFACTOR_PLAN.md (now tracked in beads epic jolt-g7t1) 2026-06-23 22:52:20 -04:00
Yogthos
0e7257ba2b refactor: dynamic-wind the build's compiler-global flags (tier 1)
set-optimize!/set-direct-link! are process-global flags in the back end, set then
reset around the emit. A strict-form build error (failing forms error the build by
design) skipped the reset, leaving the compiler in optimize/direct-link mode for any
later in-process caller. dynamic-wind guarantees the revert. Behaviour unchanged on
the success path; both --tree-shake and --opt --direct-link build + run identically.
2026-06-23 22:22:43 -04:00
Yogthos
e313e02c9d refactor: extract host/chez/dce.ss; tag the runtime manifest (tier 1)
Tree-shaking was split across emit-image.ss (the dce-* helpers + record producer)
and build.ss (bld-shake-all + the manifest splice), with the DCE record accessed by
raw (vector-ref r 0..3) in ~10 places and the manifest splice/drop driven by
substring-matching (load "…prelude.ss").

- New host/chez/dce.ss owns the whole DCE concern: a named record API
  (dce-rec/-keep?/-fqn/-refs/-str — no more positional vector indexing), the ref
  extraction + ref-set constants, dce-blob-records, and dce-shake decomposed into
  dce-build-graph / dce-reachable / dce-bail-scan / dce-partition (was one 50-line
  bld-shake-all doing five jobs with shared mutable state).
- emit-image.ss keeps only ei-emit-ns-records (it drives the ei-* compiler) and uses
  the dce-rec constructor.
- The runtime manifest is now tagged ('prelude/'image/'compile-eval); bld-emit-runtime
  dispatches on the tag instead of substring-matching file paths, so the core-splice
  and compiler-drop can't silently break on a rename/reorder.

Behaviour-preserving (runtime .ss, no re-mint): build-app shakes identically
(56/460, 8.12MB), make test green, make shakesmoke green (4 git-lib apps
byte-identical, same sizes).
2026-06-23 22:20:48 -04:00
Yogthos
d84c88f830 docs: module map, RFC index, refactor plan (arch-refactor tier 0)
Navigability groundwork from the architecture review — zero behaviour change.

- docs/MODULES.md: the repo map. Area -> directory -> key files -> re-mint?, plus
  per-feature touch points (tree-shaking, direct-linking, numeric fl/fx, inlining,
  multimethods, deps) and where a given clojure.core fn lives. Answers "where does
  X live / what's related to Y" in one read.
- docs/rfc/README.md: index the 7 RFCs; flags RFC 0007's stale "no code yet" status
  (direct-linking + tree-shaking shipped) and the undocumented inlining/numeric work.
- CLAUDE.md: document the var-deref calling convention (public defns reached from the
  .ss runtime by string lookup aren't dead), the def-var! native pattern, and the
  overlay shadowing rule; point at MODULES.md.
- REFACTOR_PLAN.md: the prioritized, risk-tiered plan (working doc for this branch).
2026-06-23 21:50:58 -04:00
Dmitri Sotnikov
7d1f0b058c
Merge pull request #187 from jolt-lang/spike/tree-shake-docs
docs: tree-shaking + runtime-resolution limitation
2026-06-24 01:30:32 +00:00
Yogthos
d5fea19a42 docs: document tree-shaking + the runtime-resolution limitation
README + tools-deps.md cover --tree-shake and --direct-link: what tree-shaking does
(whole-program reachability over app + libraries + clojure.core, drop unreachable,
drop the compiler for no-eval apps), and why it bails to keep-all when reachable code
resolves vars by name at runtime (eval/resolve/ns-resolve/...), with the diagnostic
output and how to make an app shakeable. Notes the Stalin soundness model.
2026-06-23 21:30:28 -04:00
Dmitri Sotnikov
51a1a173d5
Merge pull request #186 from jolt-lang/spike/bail-diagnostics
tree-shake: diagnose the resolve bail
2026-06-24 01:11:52 +00:00
Yogthos
86e77b322f tree-shake: report which reachable defs force the resolve bail
When --tree-shake keeps everything because reachable code resolves vars at runtime,
list the offending def -> bail-ref pairs (up to 6) instead of just saying it
skipped. Makes it actionable: e.g. ring-app shows
clojure.tools.logging/call-str -> ns-resolve and selmer.filters/generate-json ->
resolve, so you know which library (not your code) blocks shaking.
2026-06-23 21:11:47 -04:00
Dmitri Sotnikov
aa1fc18b5d
Merge pull request #185 from jolt-lang/spike/core-shake
Tree-shake the clojure.core prelude (whole-program DCE)
2026-06-24 00:42:42 +00:00
Yogthos
298ac66fb1 Tree-shake the clojure.core prelude too (whole-program DCE)
--tree-shake now shakes the clojure.core/stdlib prelude in the same reachability
graph as the app + libraries — only core fns actually reached from -main ship.

dce-blob-records reads prelude.ss with Chez `read`, unwraps each
(guard ... (def-var! "ns" "name" V)) and builds the core->core call graph from the
(var-deref/jolt-var "ns" "name") refs in V — the real emitted edges, no
re-analysis. bld-shake-all merges core + app records into one BFS; roots are -main,
side-effecting forms, and the clojure.core fns the runtime .ss shims reference by
name (enumerated in dce-runtime-core-roots). The shaken core is spliced where the
prelude.ss blob was, in original (tier) order, so load-time deps are preserved.
Bail (reachable runtime resolve) keeps the prelude whole.

Soundness follows Stalin's rule (any reference, value or call, keeps a def live):
dce-collect-refs counts :var and :the-var; non-def registration forms are always
kept (covers protocol/multimethod dispatch). Validated by make shakesmoke: the four
deps.edn git-lib apps build byte-identical output shaken vs not.

Wins (binary): build-app 9.84MB -> 8.12MB (dropped 403/457 core defs); malli-app
10.0MB -> 8.1MB; markdown 9.9 -> 8.3; commonmark 9.8 -> 8.1 — all output-identical.
build-smoke asserts an unused core fn (group-by) is dropped; full make test green.
2026-06-23 20:42:23 -04:00
Dmitri Sotnikov
f7fd73c448
Merge pull request #184 from jolt-lang/spike/shake-validation
Tree-shake: #'x references + multi-app soundness smoke
2026-06-24 00:27:39 +00:00
Yogthos
e0c1564ff9 Tree-shake: count #'x references; multi-app soundness smoke
Two things, both from studying Stalin's closed-world DCE:

1. Soundness fix: dce-collect-refs now counts a :the-var (#'x / (var x)) reference,
   not just a :var. Stalin's rule is that ANY reference — value position, not only a
   direct call — keeps the target live; a var referenced as #'x would otherwise be
   wrongly dropped. (My :var collection already covered value-position refs; this
   closes the the-var hole.)

2. host/chez/tree-shake-smoke.sh (make shakesmoke): builds example apps default vs
   --tree-shake and requires identical output — the real risk is shaking a binary
   that pulled libraries via deps.edn. Covers markdown/malli/commonmark/hiccup
   (git-lib apps). All produce byte-identical output shaken vs not, and drop
   ~0.8-1MB (malli 10.0MB -> 9.0MB) from the compiler-drop. Slow; not in the default
   gate. Skips without the examples repo.
2026-06-23 20:27:21 -04:00
Dmitri Sotnikov
709d152a1e
Merge pull request #183 from jolt-lang/spike/compiler-drop
Drop the compiler image from no-eval binaries (footprint)
2026-06-24 00:08:03 +00:00
Yogthos
3c5d548b70 Drop the compiler image from no-eval binaries (footprint)
An AOT-compiled app only needs the analyzer/back end at runtime to compile from
source — eval / load-string / load-file. Macros are expanded at build time and a
require of a baked namespace no-ops, so a closed app that never compiles at runtime
carries the compiler image (~0.8MB) as dead weight.

Under --tree-shake, when reachability is trustworthy (no bail) and no reachable code
references eval/load-string/load-file/load-reader/load, omit host/chez/seed/image.ss
+ compile-eval.ss from the runtime manifest. bld-tree-shake returns the flag
alongside the shaken forms; bld-emit-runtime filters the manifest.

Measured: build-app 9.84MB -> 9.05MB, still runs. Safety verified: an app that evals
keeps the compiler (eval is a bail + compile ref) and eval works at runtime.
build-smoke asserts the compiler is gone in the no-eval app; full make test green.
2026-06-23 20:07:46 -04:00
Dmitri Sotnikov
20c5b92ea8
Merge pull request #182 from jolt-lang/spike/tree-shaking
Tree-shaking: drop unreachable library code (footprint, opt-in)
2026-06-23 23:46:49 +00:00
Yogthos
75ac93689b Tree-shaking: drop library code unreachable from -main (lever 3/4)
`jolt build --tree-shake` (or deps.edn :jolt/build {:tree-shake true}) does
reachability DCE over the re-emitted app + library namespaces: keep -main, every
side-effecting (non-def) top-level form, and every def reachable from those; drop
the rest. A macro (expanded at AOT, never called at runtime) is prunable too.

Sound: bails (keeps everything) if REACHABLE code resolves vars by name at runtime
(eval/resolve/ns-resolve/requiring-resolve/find-var/intern/load-string/...), which a
static call graph can't follow. Unreached eval-using library code is simply shaken
away and never triggers the bail. clojure.core and the compiler image stay baked
(prelude + image blobs), so only re-emitted namespaces are shaken for now.

The reachability machinery is in emit-image.ss (records: keep?/fqn/refs/str via
reduce-ir-children) + build.ss (BFS + bail check). build-smoke covers it (drops the
unreachable `twice` macro, output unchanged). Opt-in; default builds are untouched.
full make test green.

Scope note: this shakes the re-emitted app/lib code only. Measurement shows jolt's
compiled code is ~5.8MB of a ~9.8MB binary, dominated by the clojure.core prelude
(~1.5-2MB) and the compiler image (~0.8MB) — both baked blobs this pass doesn't
touch. Those (shake-core, drop-compiler-when-no-eval) are the larger footprint wins,
filed as follow-ups.
2026-06-23 19:45:13 -04:00
Dmitri Sotnikov
3e0fb805da
Merge pull request #181 from jolt-lang/spike/perf-levers
Perf levers: IR inlining + loop-counter typing
2026-06-23 21:59:38 +00:00
Yogthos
d4ba87446a Type literal-init loop counters as fixnums (lever 2/4)
A loop var with an integer-literal init now types :long (fx ops) when every recur
arg in its slot is an increment-style step — the var unchanged, inc/dec, or (+/-
var <int-literal>). So (loop [i 0] (recur (inc i))) gets fx1+/fx<? without a hint,
matching how Clojure treats a primitive-long loop counter.

Soundness: only increment steps qualify. A multiplicative or large-growth
accumulator like (recur (* acc 2)) is never seeded, so it stays generic and keeps
arbitrary precision — a bignum-producing loop (e.g. a factorial) is unaffected.
counter-step? gates this; the existing fixpoint demotes anything inconsistent.

test/chez/numeric-test.ss 44/44 (incl. a factorial loop staying bignum-exact while
its counter is fx); full make test green, 0 new corpus divergences.
2026-06-23 17:57:39 -04:00
Yogthos
79fa22eeab Enable IR inlining: splice small defns at call sites (lever 1/4)
jolt.passes.inline was fully written but dormant — it fetched bodies via the
inline-ir host hook, which was a stub returning nil. Wire it up: run-passes stashes
each inline-eligible defn (single fixed arity) as its form is optimized, and
inline-ir hands the body back at call sites under --opt.

The catch was the ^double/^long coercion: an inlined fn drops its param-entry and
return coercion, so (work 3 4) on a ^double fn would return 25 instead of 25.0. New
:coerce IR node carries the coercion inside the spliced body — the inline pass wraps
a hinted param's arg and the return in :coerce, the back end lowers it
(exact->inexact / jolt->fx), and jolt.passes.numeric reads its :kind. So an inlined
call matches the called one and the body's fl*/fx* fast path still fires.

Only under --opt (closed world); the seed mint and -e don't inline, so selfhost and
the corpus are unaffected. test/chez/inline-test.ss 12/12 (make inline); full make
test green, 0 new corpus divergences.

Bench (hot loop, body is a ^double helper call): direct-link 500ms -> --opt
(inlined) 184ms = 2.7x, by eliminating the call + coercion wrappers and letting Chez
fuse the fl-ops unboxed. ~26x over the default dispatched build.
2026-06-23 17:43:13 -04:00
Dmitri Sotnikov
9927fd7f74
Merge pull request #180 from jolt-lang/spike/fast-arith-return-hints
Numeric return-type hints (round 3)
2026-06-23 21:22:15 +00:00
Yogthos
5a9acd3cf4 Numeric return-type hints: ^double/^long on a defn (round 3)
A ^double/^long return hint on a fn's name now (a) coerces the fn's value on the
way out — exact->inexact / jolt->fx, like a JVM primitive return — and (b) types a
call to it, so an accumulator over the result specializes:

  (defn ^double work [^double x ^double y] (+ (* x x) (* y y)))
  (loop [acc 0.0] (recur (+ acc (work a b))))   ; (+ acc (work ..)) -> fl+

The analyzer pushes the name's numeric tag onto each arity (:ret-nhint) for the
back-end coercion, and resolve-global surfaces the callee's declared return
(:num-ret, read from var meta) onto the :var node so jolt.passes.numeric types the
call. defn carries the name hint through.

This unblocks the accumulator-over-fn-result pattern that round 2 had to demote.
The win is bounded by call overhead in an open/dispatched build (~1.15x on a hot
loop whose body is a helper call); it compounds with direct-linking and, later,
inlining. A numeric return hint is a contract, like ^long — redefining the var to
return another type in an open build breaks it.

Not yet: per-arity arglist return hints, (defn f (^double [x] ..)). Gate:
test/chez/numeric-test.ss 39/39; full make test green, 0 new corpus divergences.
2026-06-23 17:21:53 -04:00
Dmitri Sotnikov
0db417a491
Merge pull request #179 from jolt-lang/spike/fast-arith-long-loops
Type :long loop-carried vars (complete round 2)
2026-06-23 21:05:06 +00:00
Yogthos
7d1b9e56d8 Type :long loop-carried vars too (complete round 2)
loop-kinds only typed :double accumulators; a ^long-seeded loop var (e.g.
(loop [acc start] ...) with a ^long start) stayed generic even though it's sound
to fx-type — :long only ever comes from an explicit hint, and a ^long value is
already coerced to a fixnum at fn entry. Keep the init's kind (:double or :long)
through the fixpoint, demoting only on a recur-arg mismatch.

Integer-literal-init loop vars (a bare (loop [i 0] ...)) still stay generic by
design: :long is never seeded from a literal, so a bignum-producing loop keeps
arbitrary precision.
2026-06-23 17:04:44 -04:00
Dmitri Sotnikov
eab18e363c
Merge pull request #178 from jolt-lang/spike/direct-linking
Direct-linking builds + hint-directed fast arithmetic
2026-06-23 20:53:24 +00:00
Yogthos
7d85b3892f Hint-directed fast arithmetic: loop-carried variable typing (round 2)
A loop binding whose init is double and whose every recur arg stays double (a
bounded monotone fixpoint) is typed :double, so its arithmetic — and the recur
args feeding it — emit fl-ops. Chez can then keep the accumulator unboxed in a
float register across the loop.

Integer loop vars stay untyped: a bare integer init never seeds :long (same rule
as round 1), so a bignum-producing loop keeps arbitrary precision rather than
overflowing a fixnum. recur-kinds walks only tail position (if/do-ret/let-body),
stopping at nested loop/fn so a loop sees only its own recur.

A/B on a loop-carried double accumulator: 735ms generic -> 500ms typed (1.47x),
closing the gap to the JVM from ~3.3x to ~2.2x. The integer counter stays generic,
which is most of the residual.
2026-06-23 16:49:59 -04:00
Yogthos
59905a71fd Hint-directed fast arithmetic: fl*/fx* from ^double/^long (round 1)
A ^double/^long param hint (or a float literal) now drives Chez flonum/fixnum
ops instead of generic arithmetic — JVM-style primitive hints, available in every
build and at -e (not gated on direct-linking or whole-program inference).

New pass jolt.passes.numeric: a local forward type-flow seeded from ^double/^long
fn-param hints (analyzer attaches :nhints per arity) and float literals,
propagated through let inits / arithmetic / if / do. It tags an arithmetic invoke
:num-kind :double|:long when every operand is that kind (an integer literal is a
wildcard, coerced to a flonum in a double op). The back end lowers a tagged node
to fl+/fl-/fl*/fl//fl<?/... or fx+/fx*/fx1+/fxquotient/... (unchecked-add etc.
join the fixnum path; == too). Runs last in run-passes, both branches.

Soundness: :long is seeded ONLY from an explicit ^long hint, never a bare integer
literal, so un-hinted integer code keeps jolt's arbitrary-precision numbers — no
fixnum-overflow surprise, no corpus divergence. :double comes from ^double hints
and float literals (flonum arithmetic is always flonum, matching the generic
result). A ^long hint is a promise the value is a fixnum: fx+ raises on overflow,
like a JVM fixed-width long.

Numeric-hinted params coerce at fn entry (exact->inexact / jolt->fx), the way the
JVM coerces a primitive parameter — so the body's fl*/fx* ops can rely on the
type even when a caller passes an exact int (e.g. Chez's (* 0 1.0) => exact 0).

Round 1 specializes hinted straight-line / fn-body arithmetic. fl-ops are ~4x
generic in a tight Chez loop, but realizing that on loop-carried accumulators
needs loop-var typing — round 2. Sound foundation, gated by test/chez/numeric-test.ss.
2026-06-23 16:43:55 -04:00
Yogthos
2c18fcdc61 Make direct-linking opt-in, not a release default
Release builds can legitimately want runtime dynamism (redefinition, eval,
load-string), so closed-world direct-linking shouldn't be forced on them. Gate it
behind an explicit --direct-link flag (or deps.edn :jolt/build {:direct-link
true}); off by default in every mode, including release and --opt.

build-binary takes an explicit direct-link? arg instead of deriving it from the
mode. build-smoke now covers the --direct-link path and asserts the cross-ns call
actually lowers to a jv$ binding; default release stays dynamically linked.
2026-06-23 16:02:18 -04:00
Yogthos
c908e996c3 docs: note direct-linking in release/optimized builds 2026-06-23 15:52:34 -04:00
Yogthos
7bc277b2e8 Direct-linking for closed-world builds (jolt build)
A release/optimized `jolt build` is a closed world: every app def is final, so
an app->app call can bind to the def's Scheme binding directly instead of going
through (jolt-invoke (var-deref ns name)).

The emitter gains a direct-link mode (off for the seed mint, runtime -e/repl, and
dev builds). With it on, a top-level app def also emits a binding jv$<ns>$<name>
that def-var! aliases; an app->app call or value-ref to a name already emitted in
the unit lowers to that binding, skipping both the var-table lookup and the
generic IFn dispatch. ^:dynamic/^:redef defs and nested defs (a defonce's inner
def) opt out and stay indirect. Off direct-link mode, emit-top-form is exactly
emit, so the seed and runtime eval are byte-unchanged (selfhost holds).

build.ss turns it on for release + optimized; the defined-set accumulates across
the dependency-ordered namespaces so a dep's defs are linkable by the time the
entry that calls them is emitted. App->core calls stay indirect for now (core is
the baked seed); that's a later stage.

~1.74x on a hot cross-namespace call loop (26.5s -> 15.2s).
2026-06-23 15:51:34 -04:00
Dmitri Sotnikov
9a68f96b6c
Merge pull request #177 from jolt-lang/build-target-output
jolt build: default output under target/{debug,release}
2026-06-23 17:46:12 +00:00
Yogthos
56d5707bfe jolt build: default output under target/{debug,release}, resolved against the project
Build output landed in the CLI's cwd (the jolt repo, since bin/joltc cd's
there), not the project — so a bare -o path or the default binary appeared
in the wrong place. Resolve output against JOLT_PWD, and default it cargo-
style under the project's target/: target/release for release/--opt,
target/debug for --dev, named after the project dir. The <name>.build scratch
dir sits beside the binary, so it lands under the same target dir. -o is
honored — absolute as-is, relative against the project.
2026-06-23 13:45:41 -04:00
Dmitri Sotnikov
d9d55fd533
Merge pull request #176 from jolt-lang/spike/foreign-callable
foreign-callable + standalone-binary bundling
2026-06-23 17:24:07 +00:00
Yogthos
1d345bfd0f jolt build: bundle native libs + resources into standalone binaries
A built binary dropped its deps.edn :jolt/native declarations and its
resource roots, so an FFI+resources app (ring-app) failed at runtime:
sockets/sqlite gave 'no entry for socket' and io/resource returned nil.
The buildsmoke fixture is pure compute, so neither path was exercised.

The launcher now loads required + :process native libs before the app's
top-level forms (a library's defcfn resolves its foreign-procedure symbols
at top-level eval during startup, so the libs must be loaded first);
optional libs load in the scheme-start launcher, where a missing lib is
caught rather than aborting the heap build.

deps.edn :jolt/build {:embed [dirs]} bakes those dirs' files into the heap
(register-embedded-resource! at heap build), so io/resource serves them with
no files on disk. Non-embedded resources resolve at runtime against JOLT_PWD,
and io/file reads (e.g. config.edn) stay external.

build-binary now takes the encoded natives, embed dirs, and project paths
from cmd-build; deps/resolve-project surfaces them. Buildsmoke fixture grows
an embedded resource + a :process native to cover both paths.
2026-06-23 13:19:33 -04:00
Yogthos
22c08d30f2 remove ffi_smoke.clj 2026-06-23 10:02:23 -04:00
Yogthos
c91b6092bc ffi: foreign-callable — receive callbacks from C
jolt could call C (foreign-fn -> foreign-procedure) but C could not call back
into jolt, which GTK signals (and any callback-taking C API) require. Add the
inverse: jolt.ffi/foreign-callable wraps a jolt fn as a C-callable function
pointer, mirroring the foreign-fn pipeline.

A new jolt.ffi/__ccallable special form carries the fn as a child expression
(analyzed + walked by the passes; ir.clj gains an :ffi-callable arm in both
child walks) plus literal arg/ret type keywords. The back end lowers it to a
locked Chez foreign-callable and returns its entry-point address as a jolt
pointer; host/chez/ffi.ss registers the code object so the collector keeps it,
and free-callable unlocks it. :collect-safe emits the convention that
reactivates the thread on entry, for callbacks fired while it is parked in a
:blocking call (a GTK main loop).

Test: ffi-binding-test.ss sorts an int array through libc qsort with a jolt
comparator (C -> jolt -> C). Re-minted seed.
2026-06-23 09:55:17 -04:00
Yogthos
91eed2b622 Merge architecture-cleanup epic (jolt-ogib)
Dead code removal, O(n^2) hot-path fixes (transients/queues/deps), wiring the
optimization pass pipeline into compile + build, deterministic seed emission,
splitting the oversized files (20-coll, host-static, records, types lattice),
and restructuring the two worst maintainability smells: the str-render /
instance-check set!-override chains became registries, and the type-inference
walk now threads an immutable env instead of ~14 module atoms.

Adds an inference gate (make infer, 26 cases) so the type pass — which the
corpus/unit gates don't exercise — has real coverage. Full gate green:
self-host fixpoint holds, corpus 2735 (0 new divergences).
2026-06-23 09:38:53 -04:00
Yogthos
7d8392e5ab infer gate: cover the run-inference / take-diags! checker path
Adds 3 cases for the opt-path checker (set-check-mode! -> run-inference ->
take-diags!): diagnostics drain once, the buffer empties on re-drain, and
check-mode off produces none. This is the public checker entry the other 23
cases didn't touch. Runtime-only.
2026-06-23 09:38:13 -04:00
Yogthos
4fdc9f165e Thread an immutable env through the type-inference walk
types.clj drove inference through ~14 module-level atoms; the infer walk was
non-reentrant and depended on hidden set-*! install order. Thread one immutable
env (mk-env) through infer instead: it snapshots the installed config
(rtenv/vtypes/record-shapes/protocol-methods/map-shapes?) and carries the
per-run flags and accumulator/guard cells (diags/calls/checking-set/diag-memo).
A fresh env per run makes the pass re-entrant — isolated-diag-count's probe now
runs under a sub-env with its own diags cell instead of save/restoring a shared
atom.

Only state whose lifecycle spans separate API calls stays module-level: a
config-box the set-*! API writes, the escapes/user-sig sweep registries, and a
bridge holding the last checking run's diags for take-diags!. record-type-from-
entry/field-type-from-tag now take the shapes map directly rather than reading a
global.

jolt-ogib.10. Behavior pinned by the new infer gate (23 cases) plus selfhost +
buildsmoke. Re-minted seed.
2026-06-23 09:19:24 -04:00
Yogthos
9c5e46e91b Add inference gate (jolt.passes.types)
The corpus/unit gates compile through run-passes' const-fold-only branch, so
the type-inference walk runs only under jolt build --opt — buildsmoke hit one
trivial app and checked stdout. run-infer.ss drives the pass directly: analyze
a source string, then call check-form / infer-body / the set-*! registries and
assert diagnostic counts and collected calls/escapes. Wired into make ci.

Gives the inference pass real behavior coverage so refactoring its internal
state is gate-validatable. jolt-ogib.10 groundwork.
2026-06-23 09:11:21 -04:00
Yogthos
e434356590 Replace str-render/instance-check set! chains with registries
jolt-str-render-one and instance-check were each extended by a chain of
set!-wrapping closures spread across ~10 and ~5 host files, so the real
behavior of either was scattered and load-order-dependent. Give each a
registry the base file owns: converters.ss/records-interop.ss define the
registry plus a register-* helper, and each extending file registers one arm
instead of capturing %prev and set!-ing the global.

str-render arms are type-disjoint; instance-check arms run newest-first (the
old outermost-wins order) and may return 'pass to defer. The string-token ->
symbol normalization the natives-array arm did for every inner arm moves to
the dispatcher head; array tokens stay strings for that arm to decide.

jolt-ogib.14. Runtime-only shims, no re-mint.
2026-06-23 09:05:36 -04:00
Yogthos
bc16513afd Sync re-minted seed for the types.clj lattice split
The lattice-split commit staged its seed before make remint ran, so image.ss
lagged the source. Commit the correct re-minted image (gensym renumbering only).
2026-06-23 04:35:32 -04:00
Yogthos
99a41d17b9 Extract the type lattice into jolt.passes.types.lattice
types.clj was 852 lines mixing the pure structural-type algebra with the
inference engine, checker, and driver. Move the lattice — scalar/struct/vec/set/
union types, join-t, depth-cap, shape, and the numeric/vector return-fn sets —
into jolt.passes.types.lattice (no inference state, no requires). types.clj
requires it; the engine is now ~720 lines. Compiled into the image before
jolt.passes.types. Re-minted seed differs only by gensym label renumbering.
2026-06-23 04:31:43 -04:00
Yogthos
864af8ef7e Split records.ss: lift the JVM-emulation taxonomy into records-interop.ss
records.ss mixed the record/protocol/reify model with JVM exception emulation.
Move the contiguous ex-info accessors, the exception supertype hierarchy,
instance-check, case-string, and the instance-check def-var into records-interop.ss,
loaded right after records.ss. Those are only called at runtime, so the relocated
forms resolve fine; records.ss is now the value model and protocol dispatch.
Runtime shim — no seed change.
2026-06-23 04:22:02 -04:00
Yogthos
23f6fee250 Split host-static.ss into base / statics / objects
The 929-line interop registry split three ways: host-static.ss keeps the
registries, the jhost record, the emit entry points and coercion helpers;
host-static-statics.ss holds the java.lang/util static-method registrations;
host-static-objects.ss holds the host object classes (ArrayList, HashMap, the
reader/writer/tokenizer shims, ctors, URL codecs) and the instance? hook. rt.ss
loads the three in order. Runtime shim — no seed change.
2026-06-23 04:16:43 -04:00
Yogthos
f7767706cf Split 20-coll.clj into three collection-tier files
The 1123-line collection tier is the largest source file. Cut it at two existing
section banners into 20-coll (predicates, printing, hierarchies, pure-over-core
leaves), 21-coll (rand/sort seams, the test runner, fn combinators), and 22-coll
(canonical Clojure ports, transduce/into, JVM-shape stubs). No macros in this tier,
so order is the only constraint; the emit-image manifest lists the three in
sequence. Re-minted seed is identical apart from gensym label renumbering.
2026-06-23 02:06:24 -04:00
Yogthos
0db08e7571 Memoize the success checker's per-fn body re-inference
check-user-call rebuilt the all-:any env once per parameter (O(params^2)) and
re-inferred a callee body at every call site. Build the env once and memoize each
probe by [key i argtype] (and the baseline by [:base key]), cleared per form in
check-form. The global type-env is stable within a form's check and the probe's
calls/escapes side effects aren't read there, so a skipped repeat is observably
identical. (The inline-side re-walk the audit flagged is moot: hc-inline-ir is a
no-op on Chez, so try-inline never reaches body-size/body-closed?.)
2026-06-23 01:54:39 -04:00
Yogthos
b8492ad81a Share one ns cross-compile loop between the seed minter and jolt build
ei-emit-ns (emit-image) and bld-emit-ns (build) were near-verbatim copies that had
drifted: the minter guard-wraps and skips failing forms, the build is strict, and
since the passes were wired the build also runs run-passes. Fold both into
ei-emit-ns* with optimize?/guard? flags; ei-emit-ns and bld-emit-ns become one-line
callers. Output is byte-identical (selfhost fixpoint and build smoke stay green).
2026-06-23 01:48:40 -04:00
Yogthos
1dcd4678e8 Deduplicate small host helpers
- str-join delegates to str-join-strs (it only adds the per-element render).
- loader: extract resolve-on-roots; find-ns-file and load both use it.
- NumberFormat registers its short + FQ names from one shared member list.
- inst-time's private floor-div/floor-mod renamed inst-floor-* so they don't read
  as the math.ss reals version.

Left the fold/inline/types pure-fn sets and the keyword/symbol ns builders alone:
those are file-local and semantically distinct (e.g. the intern key uses NUL, not
"/"), so merging them would be wrong.
2026-06-23 01:42:33 -04:00
Yogthos
14547bd1d5 JVM-semantics fixes and small cleanups
- take-last / drop-last return seqs, not vectors: take-last wraps in seq; drop-last
  is the JVM (map (fn [x _] x) coll (drop n coll)) form (lazy, () when empty).
- cycle is lazy ((lazy-seq (concat coll (cycle coll)))) so it no longer counts its
  argument and terminates on a lazy/infinite input.
- fold's foldable-call catch uses :default, matching the rest of jolt-core and
  also catching a raw host condition from a folding primitive.
- alts! rejects non-channel ports with a clear error (put specs / :default are
  unsupported) instead of crashing inside ac-poll!.
- Misc: drop the unreachable second getCause clause; jolt-nth on a string raises
  'nth "index out of bounds" like the vector branch; name the inline fixpoint cap;
  bld-sh-capture rejoins lines with newlines; clarify a couple of comments.
2026-06-23 01:36:51 -04:00
Yogthos
524d4cd8d1 Add reduce-ir-children; rebuild the read-only IR walks on it
map-ir-children single-sourced the child layout for rewrite passes; the read-only
analyses each re-enumerated ops by hand. Add a fold companion, reduce-ir-children,
and rebuild body-size, pure?, and body-closed? on it (each reduces to a leaf value
+ the special ops it actually needs). local-escapes? stays an explicit walk — its
default is conservatively true and it inspects node shape beyond child purity, so
folding an unhandled op over its children would be unsound for scalar replacement.
2026-06-23 01:28:32 -04:00
Yogthos
adf00a3b51 Extract analyze-def / analyze-set! from analyze-special
analyze-special inlined def (~35 lines) and set! while try/letfn/fn* were already
helpers. Pull both out and move field-head? above analyze-special so its set! arm
and analyze-list reach it without a forward reference — the file's "only analyze
is forward-declared" invariant holds again. Pure code motion.
2026-06-23 01:23:17 -04:00
Yogthos
2953320599 Wire the optimization pass pipeline into compile + build
The fold/inline/types passes and the jolt.passes façade were baked into neither
seed half and never invoked: compile-eval and build went analyze -> emit directly,
and `jolt build --opt` flipped an optimize flag that nothing consumed.

- Compile the passes into the image (emit-image manifest): fold, inline, types,
  then the jolt.passes façade, after jolt.ir.
- compile-eval and build.ss now run jolt.passes/run-passes between analyze and
  emit. Off the direct-link path it is a pure const-fold; `jolt build --opt`
  turns on inline + flatten + scalar-replace + type inference (it sets
  hc-optimize?, which inline-enabled? reads).
- The seed minter (emit-image) stays analyze -> emit, so the seed is built
  un-optimized and the self-host fixpoint is unaffected.

build-smoke already exercised --opt; it now actually optimizes and still matches
the release binary's output. Corpus floor and the fixpoint are green.
2026-06-23 01:14:14 -04:00
Yogthos
e93006b4be Dead-code removal, perf fixes, deterministic seed emission
Round 1 (correctness + dead code):
- Fix duplicate java.util.HashMap registration in host-static.ss: the alist
  impl shadowed the hashtable ctor while leaving the hashtable methods bound,
  so .keySet/.values/.remove/.clear crashed. Drop the alist version.
- Delete jolt-core/jolt/reader.clj: a 463-line dead duplicate reader, never
  required or compiled (the live reader is host/chez/reader.ss) and drifted.
- Remove dead defs: ir/rt + :rt op + unused ir/op; the Janet branch in
  clojure.edn/drain-reader; a shadowed first clojure.string/trim-newline;
  io.ss jolt-char-array + the reader def-var (both shadowed by natives-array);
  concurrency.ss jolt-future-done?*; compile-eval.ss jolt-analyze-emit.

Round 2 (perf + determinism):
- emit-quoted-map-value / quoted sets now emit sorted by emitted text instead
  of host-hash order, which isn't stable across Chez versions (jolt-8479).
- jolt-into folds through a transient, so into/vec/mapv/filterv onto a vector
  are O(n) instead of O(n^2).
- deps resolve-deps walks its queue with an index cursor (was subvec-per-pop).
- async channel and agent action queues use amortized-O(1) FIFOs; ArrayList is
  backed by a growable vector (O(1) add/get) instead of a list.
2026-06-23 01:05:45 -04:00
Dmitri Sotnikov
adafe04072
Merge pull request #175 from jolt-lang/spike/chez-bootstrap
Real Thread/yield + Thread/interrupted
2026-06-23 04:27:40 +00:00
Yogthos
980ec73933 Real Thread/yield + Thread/interrupted (jolt-l2gc)
Thread/yield was a no-op and Thread/interrupted always returned false. Now:

- yield calls libc sched_yield (resolved once via the process symbols), so a
  spin loop relinquishes the CPU. Falls back to a zero-length park if the symbol
  can't be resolved.
- each OS thread carries an interrupt flag (a box, thread-local). currentThread
  returns a handle wrapping the calling thread's flag, so .interrupt from another
  thread sets the target's flag. .isInterrupted reads without clearing; the static
  Thread/interrupted reads and clears — JVM semantics.

Consolidates the Thread surface: currentThread + the instance methods live in
io.ss (where the handle and its classloader are built), the flag box + yield +
the interrupted static in host-static.ss. Unit cases cover yield, the read/clear
split, and a cross-thread interrupt over a future.
2026-06-23 00:06:04 -04:00
Dmitri Sotnikov
b2eaef51cc
Merge pull request #174 from jolt-lang/spike/chez-bootstrap
ci: cover the jolt build pipeline on Linux; README binary docs
2026-06-23 03:50:03 +00:00
Yogthos
87aec859b8 ci: build Chez with --disable-x11; README: how to build a binary
The from-source Chez build failed on expeditor.c needing X11/Xlib.h — the
expression editor's clipboard. Configure with --disable-x11 (not needed in CI)
and bump the cache key. Add a "Compile a binary" section to the README.
2026-06-22 23:43:26 -04:00
Yogthos
339cd4b691 ci: build Chez from source so buildsmoke links a real binary
The apt chezscheme package ships petite+scheme only — no kernel dev files — so
the standalone-binary gate skipped on CI, leaving the whole jolt build pipeline
and the --opt inference passes uncovered on Linux. Build Chez v10.4.1 from
source (cached) to get libkernel.a + scheme.h, install the libs the kernel links
against, and set the Linux link flags. buildsmoke now runs for real in CI.
2026-06-22 23:37:15 -04:00
Dmitri Sotnikov
b59e5ecc42
Merge pull request #173 from jolt-lang/spike/chez-bootstrap
jolt build: standalone binaries (Phase 4 stages 1-2 + 4a)
2026-06-23 03:29:44 +00:00
Yogthos
a2146c0f0d buildsmoke: skip when the Chez build toolchain is absent
The standalone-binary build needs Chez's kernel dev files (libkernel.a,
scheme.h) and a C compiler. A distro chezscheme package ships neither, so the
gate failed on CI (apt installs petite+scheme only). Preflight for the csv dir
and cc and skip cleanly when they're missing — same pattern as certify skipping
without Clojure. Where the toolchain exists (dev machines), it runs as before;
the discovered csv dir is pinned via JOLT_CHEZ_CSV so the build uses exactly it.
2026-06-22 23:27:49 -04:00
Yogthos
35a854eca1 README: trim differences to actual differences
The list led with parity (numeric tower, persistent collections, future/agent/
pmap, core.async) framed as divergences. Keep the four real ones — no JVM/Java
interop, no BigDecimal, no STM, the irregex engine — plus the coverage caveat,
and state the parity as parity.
2026-06-22 23:22:42 -04:00
Yogthos
f66925d3a8 jolt build: --opt mode turns on the inference passes (Phase 4 stage 4a)
Wire the optimization gate to build mode. inline-enabled? (which gates the
inference + flatten-lets + scalar-replace passes in jolt.passes/run-passes) was
hardwired off, so those passes had never run on Chez at all. host-contract now
exposes a settable hc-optimize? flag; `jolt build --opt` flips it on during app
emission.

Kept off for the default release build for now: the passes are sound by design
(RFC 0005/0006) but unexercised on Chez, so release stays on the proven
var-deref codegen until they're validated against the corpus. --opt is the
opt-in fast path. buildsmoke checks both modes produce the same result.

This does not yet deliver direct call binding — the backend has no direct-link
emission path (every :var call still routes through jolt-invoke/var-deref) and
the inline-ir host stash is still a stub. Those are the remaining stage-4 levers.
2026-06-22 23:07:04 -04:00
Yogthos
43778eafd7 jolt build: compile an app to a standalone binary (Phase 4 stages 1-2)
Restores the standalone-binary capability the Janet host had. `bin/joltc build
-m NS -o OUT` AOT-compiles an app into a single self-contained executable — the
whole runtime, clojure.core, stdlib and compiler embedded, no Chez install or
jolt source needed at runtime.

Pipeline (host/chez/build.ss, host primitive jolt.host/build-binary driven by
jolt.main's build command): resolve deps, load the entry namespace recording the
app namespaces in dependency order, re-emit each to Scheme, textually inline the
cli.ss runtime load sequence into one flat source + the app + a launcher, then
compile-file -> make-boot-file -> embed the boot as C bytes -> cc-link against
libkernel.a.

Two non-obvious bits: the compile pass runs in a fresh Chez, not the loaded
runtime (regex.ss shadows top-level `error`, which otherwise bakes a broken
reference into the boot); and the launcher installs scheme-start rather than
running -main at top level, since boot top-level forms execute during heap build
before argv is set, so args only reach -main through scheme-start.

Loader: a require of an in-memory namespace with no source file now no-ops, so
AOT'd app namespaces satisfy require in a built binary.

Mode flags (--opt/--dev, default release) are plumbed; the optimization passes
they gate come in a later stage. RFC 0007 has the design. Gated by `make
buildsmoke`.
2026-06-22 23:01:36 -04:00
Yogthos
33eff7c7d8 Clean up codebase: rename stdlib layer, strip porting residue, fix tooling
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.

Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.

Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.

Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
2026-06-22 22:18:00 -04:00
Dmitri Sotnikov
2a07792757
Merge pull request #172 from jolt-lang/spike/chez-bootstrap
Conformance + nREPL fixes from the library sweep
2026-06-22 23:09:01 +00:00
Yogthos
c18f8087f0 Real nREPL interrupt + thread-local *ns* (jolt-amzy, jolt-6rld)
Two nREPL divergences the library shakeout surfaced — both have a real host
mechanism on Chez.

Interrupt (jolt-amzy): Chez's engine timer (set-timer + thread-local
timer-interrupt-handler) is polled at procedure-call / loop back-edges, so a
running computation — even a tight loop — can be aborted from another thread.
concurrency.ss adds jolt.host/{make-interrupt, interrupt!, run-interruptible}: an
interrupt token is a shared box; run-interruptible arms a periodic timer whose
handler escapes (call/cc) when the token is set, throwing {:jolt/interrupted true}.
The eval thread is reused, not abandoned. (A thread blocked in a __collect_safe
foreign call only sees it on return — like the JVM not killing native code.)

Thread-local *ns* (jolt-6rld): chez-current-ns is now a Chez thread-parameter, so
each session worker / future has its own current ns (vars stay global, only the
pointer is per-thread). *ns* reads derive from it (dyn-binding.ss), and a bound
*ns* drives chez-current-ns — so (binding [*ns* the-ns] (load-string code))
resolves against the-ns, and concurrent in-ns across threads don't clobber each
other. Single-threaded behaviour is unchanged. All runtime .ss — no re-mint.
2026-06-22 18:57:16 -04:00
Yogthos
f30a517cf7 Conformance: reify falls back to a protocol's default extension (jolt-az9a)
A reify that doesn't implement a given protocol method now dispatches to that
protocol's extended impls over the reify's host tags (e.g. an Object/default
extension) instead of erroring 'No reified method'. This is malli's pattern: it
reifies some protocols and relies on RegexSchema's default for the rest. A method
with neither a reify impl nor a default still errors.
2026-06-22 18:30:44 -04:00
Yogthos
47864403e8 Conformance: throwable chaining, URL ctor/getProtocol, ClassLoader/Thread shims
Surfaced running the DB libraries (migratus) on the jolt db library:
- java.sql.SQLException .getNextException / .getStackTrace / .printStackTrace on
  jolt throwables (conditions + ex-info) return nil/empty, so a library walking
  the exception chain doesn't crash.
- java.net.URL ctor + .getProtocol (file/http), alongside the existing url shim.
- A generic java.lang.ClassLoader: getSystemClassLoader / a thread's
  contextClassLoader resolve a named resource against the source roots (the same
  model as clojure.java.io/resource) — file: URL or nil. Thread/currentThread.
  These are generic host capabilities, not DB-specific.

The jolt-lang/db next.jdbc surface now runs migratus far enough to connect, build
the migrations table, and discover migrations; migratus's remaining dependency is
java.nio.file (FileSystems/Path/PathMatcher glob), a JVM filesystem API kept out
of core.
2026-06-22 18:27:27 -04:00
Yogthos
5c9c5ed6e1 Conformance: String/format static + java.text.NumberFormat
Part of the java.* host-class gap (jolt-1nnn). String/format delegates to the
core format engine; NumberFormat getInstance/getNumberInstance/getIntegerInstance
group the integer part and honor min/max fraction digits.
2026-06-22 18:04:51 -04:00
Yogthos
c7bbdea11d Conformance: SimpleDateFormat parse + read over host readers
- java.text.SimpleDateFormat.parse parses the RFC1123/1036/asctime patterns
  (day/month names, 2-digit-year sliding window, tz token), and .format renders
  z/Z/X timezone tokens (GMT/+0000/Z) instead of emitting them literally. Date
  gains toLocalDate/toLocalDateTime/before/after/equals. Fixes ring.util.time.
- read / read+string work over a host java.io reader (StringReader wrapped in a
  PushbackReader): drain, parse one form, push the tail back. Fixes cuerdas
  istr / << string interpolation (and selmer's <<), which read embedded forms
  from the template via (read pushback-reader).
2026-06-22 18:02:25 -04:00
Yogthos
d83175b8c2 Fix conformance gaps: exception types, byte/getBytes, host classes
Shake-out from the conformance-library sweep. Host-side fixes (runtime .ss,
no re-mint) plus one analyzer change (re-minted):

- Exception fidelity: ex-info and host-constructed throwables (RuntimeException.
  etc.) now carry their JVM class, so (class e), instance? across the exception
  hierarchy, .getMessage, and clojure.test thrown?/thrown-with-msg? all work.
- .getBytes returns a seqable/countable byte-array and honors UTF-16/UTF-32;
  String. decodes them. ->bytevector accepts byte-arrays (Base64).
- Universal .getClass / .toString / .indexOf / .lastIndexOf on any value/seq.
- record? uses the host jrec? predicate (the old (get x :jolt/deftype) crashed
  on a sorted-map by invoking its comparator).
- extend-protocol to abstract host types (clojure.lang.Fn/IFn/APersistentVector,
  java.net.URI) dispatches.
- New host classes: clojure.lang.PersistentQueue, java.util.ArrayList,
  java.net.URI, java.io.File / java.util.UUID ctors, Double/Float ctors+statics,
  regex instance? Pattern, System/setProperty.
- *assert* / *print-readably* are real settable/bindable vars.
- (symbol "ns/name") splits the namespace at the last slash.
- letfn fn params desugar destructuring (analyzer; re-minted).

unit.edn gains exinfo/hostobj/queue/hostctor/destructure regression rows.
2026-06-22 17:52:38 -04:00
Dmitri Sotnikov
b7fcf1ed8c
Merge pull request #171 from jolt-lang/spike/chez-bootstrap
nREPL: middleware-extensible server (seam for the nrepl library)
2026-06-22 19:58:21 +00:00
Yogthos
185b4fd3ca nREPL: make the built-in server middleware-extensible
The built-in nREPL stays minimal (clone/describe/eval/load-file/close) but now
composes a middleware stack so a library can add the heavier features (sessions,
interruptible-eval, completion, lookup) without bloating core.

- A middleware is (fn [handler] (fn [request] ...)); request carries :reply (a
  thread-safe send that adds id/session) plus the wire fields. List them in
  deps.edn :nrepl/middleware (symbols -> a middleware fn or a vector of them);
  jolt.nrepl composes them over the built-in handler.
- Public seam: respond, evaluate, register-ops!, new-session, err-msg.
- Per-connection send lock so middleware replying from other threads don't
  interleave bytes. describe advertises built-in + registered ops.

deps.clj surfaces :nrepl/middleware; jolt.main passes it to the server. Built-in
behavior unchanged when no middleware is declared. Runtime, no re-mint.
2026-06-22 15:37:14 -04:00
Dmitri Sotnikov
ad2d597816
Merge pull request #170 from jolt-lang/spike/chez-bootstrap
REPL fixes + nREPL server for editor-connected dev
2026-06-22 19:21:10 +00:00
Yogthos
d33277c0b2 REPL fixes + an nREPL server for editor-connected dev
The line REPL was broken (read-line called nil — the __stdin-read-line host seam
the clojure.core *in* reader drives was never implemented on Chez) and didn't
load the project, so (require '[some.lib]) failed. Now:

- __stdin-read-line reads a line from stdin (get-line); read-line / read / the
  REPL work.
- repl resolves the project first (deps on the roots, native libs loaded), so
  libraries are available — same context a run gets.
- jolt.nrepl: a jolt-native nREPL server (bencode over a loopback jolt.ffi
  socket) speaking the real protocol — clone / describe / eval / load-file /
  close, with stdout capture, :ns-scoped eval (in-ns; binding *ns* doesn't drive
  load-string resolution here), and real error text. 'joltc nrepl [port]' applies
  the project then serves; writes .nrepl-port. Editors (CIDER/Calva/Cursive)
  connect and develop live; project libraries load in the session.
- ex-message returns nil for raw Chez conditions, so jolt.host/condition-message
  exposes the condition text; the REPL and nREPL surface it instead of an opaque
  #<compound condition>.

Why native, not real nREPL: nrepl.server is welded to java.util.concurrent
executors, two compiled Java helper classes, a DynamicClassLoader, Compiler
internals and a JVMTI agent — not faithfully shimmable. The wire protocol, which
is what clients depend on, is small and implemented directly.

Runtime .ss + jolt-core, no re-mint. Full gate green.
2026-06-22 15:18:52 -04:00
Dmitri Sotnikov
74cdd3c1a0
Merge pull request #169 from jolt-lang/spike/chez-bootstrap
Library class-shim host hooks + FFI byte I/O; remove built-in http-client
2026-06-22 18:37:59 +00:00
Yogthos
9a60922d61 Remove the built-in jolt.http-client (the curl shim)
jolt-lang/http-client (clj-http-lite over jolt.ffi) replaces it, the same way
ring-janet-adapter replaced the built-in HTTP server. An HTTP client is a library
concern — jolt core no longer ships one or shells out to curl. Apps depend on the
library; (require '[jolt.http-client]) now resolves to its source.

Full gate green.
2026-06-22 13:55:49 -04:00
Yogthos
13aaf74c4b Host completeness for clj-http-lite: with-open/slurp on shims, charset, exc classes
Gaps the clj-http-lite suite hit running over jolt-lang/http-client:

- with-open now closes a tagged-table stream shim via its .close method (was a
  :close field lookup that only matched the Janet-era tables).
- slurp accepts a bytevector / jolt byte-array / a byte input-stream shim and
  honors :encoding — clj-http-lite slurps response bodies and :as :stream bodies.
- (.getBytes s charset) and Charset/forName respect the charset (ISO-8859-1 etc.,
  one byte per char), not always UTF-8; Charset/forName returns the name string.
- (class e) reads the JVM :class off a throwable tagged-table, so clojure.test's
  (thrown? Class …) / (= Class (class e)) match a library's typed exceptions.
- Registered the HTTP/IO exception class names (IOException, UnknownHostException,
  SocketTimeoutException, SSLException, …) so the FQ literals self-evaluate.

All runtime .ss shims, no re-mint. Full gate green.
2026-06-22 13:54:04 -04:00
Yogthos
5bcfc629fc clojure.java.io/copy + registry-aware io/as-url
clj-http-lite drives bytes through clojure.java.io/copy (response body into a
ByteArrayOutputStream, request body out) and resolves URLs via io/as-url. Neither
worked for a library shim:

- io/copy was absent. Add it: raw bytes / string / a jhost reader write in one
  shot; any other source drains via .read into a buffer and .write to the dest,
  both through method dispatch — so a library's tagged-table streams copy without
  the host knowing their layout.
- io/as-url ignored a library-registered URL class, so it and (URL. spec)
  disagreed (the file-only jhost has no getProtocol/getHost/...). It now honors a
  registered URL ctor, falling back to the jhost.

Runtime .ss shims, no re-mint.
2026-06-22 13:34:19 -04:00
Yogthos
3253df979a Byte-array <-> bytevector interop + charset-aware (String. bytes cs)
The host carries bytes two ways: Chez bytevectors (what String/.getBytes
produce) and jolt byte-arrays (what byte-array / the Java-array shims use). They
didn't interconvert, so code mixing the two — like clj-http-lite, which buffers
into (byte-array n) but encodes via .getBytes and decodes via (String. ^[B body
charset) — broke.

- byte-array now also accepts a bytevector or a string (UTF-8 bytes), so the two
  representations convert freely at interop seams.
- (String. bytes [charset]) decodes a bytevector OR a jolt byte-array with the
  named charset (UTF-8 default; ISO-8859-1/latin1/ascii = one byte/char). It
  previously only took a bytevector and ignored the charset.

Runtime .ss shims, no re-mint. Unit covers both directions + charset.
2026-06-22 13:20:27 -04:00
Yogthos
c5e1e0544a jolt.ffi: read-array/write-array for binary-faithful buffer I/O
read-bytes/write-bytes go through UTF-8 (with a latin1 fallback), which mangles
arbitrary binary — gzip payloads, TLS records, any non-text body. An HTTP client
moving bytes between jolt byte-arrays and foreign socket/zlib/OpenSSL buffers
needs byte-exact transfer. read-array returns a fresh byte-array of n bytes from
foreign memory; write-array copies a byte-array's bytes into a pointer. Test
covers a round-trip preserving high bytes (200, 255).
2026-06-22 13:11:08 -04:00
Yogthos
8c6623503f Host hooks for library class shims: __register-class-methods! + __register-instance-check!
clj-http-lite drives java.net URL/HttpURLConnection and java.io byte streams
through .method interop. The Chez host had __register-class-ctor!/-statics! (what
router/reitit needs) but no way to register instance methods on a shim object or
to extend instance?. Add both, plus jolt.host/table?:

- tagged-table .method dispatch: an htable-arm on record-method-dispatch routes
  (.m obj a*) through a per-tag method registry keyed off the table's jolt/type;
  unregistered methods fall through (sorted colls are htables too).
- __register-instance-check! installs (fn [class-name val] -> true|false|nil),
  nil = fall through; chained ahead of the base instance-check.

Runtime .ss shims, no re-mint. Unit covers dispatch, args, instance? both ways.
2026-06-22 13:06:12 -04:00
Dmitri Sotnikov
40216f415e
Merge pull request #168 from jolt-lang/spike/chez-bootstrap
deps.edn :jolt/native — declare native shared libs
2026-06-22 16:39:33 +00:00
Yogthos
21fbc50014 deps.edn :jolt/native — declare a library's native shared libraries
An FFI library declares the system shared objects it binds in its deps.edn
(:jolt/native), with per-platform candidate sonames, :optional for feature-gated
deps, and :process for libraries that use the running process's own symbols
(libc sockets). jolt.deps collects them transitively; jolt loads them before the
library's namespaces are required, so foreign-fn bindings resolve — and a missing
required lib fails early with a clear message instead of a cryptic symbol error.
Replaces hardcoded soname-probing inside library .clj files.
2026-06-22 12:37:18 -04:00
Dmitri Sotnikov
b39081d350
Merge pull request #167 from jolt-lang/spike/chez-bootstrap
Remove built-in jolt.http.server (adapter library replaces it)
2026-06-22 16:21:21 +00:00
Yogthos
b7bd144321 Remove built-in jolt.http.server (ring-janet-adapter library replaces it)
The HTTP server moves out of the host into the jolt-lang/ring-janet-adapter
library, which binds sockets itself via jolt.ffi and shuts down cleanly. Drop
host/chez/http-server.ss and the obsolete ffi-server-test (the FFI collect-safe
path is covered by ffi-binding-test; the server by the adapter's own CI).
2026-06-22 12:19:04 -04:00
Dmitri Sotnikov
8bbdc1273d
Merge pull request #166 from jolt-lang/spike/chez-bootstrap
System/getProperty os.name reflects the real platform
2026-06-22 16:16:35 +00:00
Yogthos
01748d2b17 System/getProperty os.name reflects the real platform (was hardcoded Mac OS X)
Derive os.name from Chez's machine-type (*osx -> Mac OS X, else Linux/Windows).
OS-branching code (socket sockaddr layout, etc.) needs the truth; a library
binding sockets via jolt.ffi reads os.name to pick the platform struct layout.
2026-06-22 12:15:14 -04:00
Dmitri Sotnikov
8b2344c0b8
Merge pull request #165 from jolt-lang/spike/chez-bootstrap
jolt.ffi: read-bytes/write-bytes buffer I/O
2026-06-22 16:05:55 +00:00
Yogthos
f0003151b0 jolt.ffi: read-bytes/write-bytes for fixed-length buffer I/O (UTF-8) 2026-06-22 12:04:19 -04:00
Dmitri Sotnikov
684c691f09
Merge pull request #164 from jolt-lang/spike/chez-bootstrap
jolt.ffi: :blocking (collect-safe) foreign calls
2026-06-22 16:02:14 +00:00
Yogthos
2a64e65a1c jolt.ffi: a :blocking option for collect-safe foreign calls
A library binding a blocking native call (accept/recv/connect/...) needs it
emitted __collect_safe so the thread deactivates for the call and doesn't pin
the stop-the-world collector. foreign-fn / defcfn take an optional trailing
:blocking; the backend emits (foreign-procedure __collect_safe ...). Needed for
the ring-janet-adapter socket-server port. ffi-binding-test asserts a thread
parked in a :blocking call doesn't block (collect).
2026-06-22 12:00:14 -04:00
Dmitri Sotnikov
f075a4004c
Merge pull request #163 from jolt-lang/spike/chez-bootstrap
Remove built-in jolt.sqlite/jdbc.core (db library owns it via jolt.ffi)
2026-06-22 15:25:21 +00:00
Yogthos
db9bed226f Remove the built-in jolt.sqlite / jdbc.core (libraries own native code)
The sqlite/jdbc functionality moves out of the host into the jolt-lang/db
library, which binds libsqlite3 (and libpq) itself via jolt.ffi. A baked
built-in jdbc.core would shadow the library's, so it's removed here. ring-app
gets jdbc.core from the db git dep instead.
2026-06-22 11:23:45 -04:00
Dmitri Sotnikov
e73e973ce7
Merge pull request #162 from jolt-lang/spike/chez-bootstrap
jolt.ffi: a Clojure FFI for libraries to bind native code
2026-06-22 15:01:48 +00:00
Yogthos
537cb360b4 Add a Clojure FFI so libraries can bind native code (jolt.ffi)
A jolt library can now bind its own native dependencies and expose a Clojure API
over them — no jolt built-in required. This is the foundation for moving the
http-client / db / adapter functionality out of the host and into real libraries.

- jolt.ffi/foreign-fn (sugar: defcfn) is a compiler special form: a compile-time
  -typed C signature lowers to a real Chez foreign-procedure (analyzer :ffi-fn ->
  backend foreign-procedure), so calls are typed and marshaled, not eval'd.
- host/chez/ffi.ss provides the rest under jolt.ffi: load-library, alloc/free,
  read/write/sizeof, ptr<->string, null/null?. Loaded after the loader snapshot
  so a library's (require '[jolt.ffi]) still loads the macro side.
- Types: int/uint/long/ulong/int64/uint64/size_t/ssize_t/iptr/uptr/double/float/
  pointer/string/void/uint8/char.

Validated end to end: a pure-Clojure file binds libc (getpid/strlen/abs) and
libsqlite3 (open/prepare/step/column/finalize over out-param pointers) and runs a
query. Gate test test/chez/ffi-binding-test.ss (make ffi); selfhost holds.
2026-06-22 10:59:51 -04:00
Dmitri Sotnikov
1276bfdd7e
Merge pull request #161 from jolt-lang/spike/chez-bootstrap
Namespace resolution fixes (:use/:only, refer-aware syntax-quote, alias-aware resolve)
2026-06-22 14:27:34 +00:00
Yogthos
ccf93c896a Resolve :use/:only refers, refer-aware syntax-quote, alias-aware resolve
Three related namespace-resolution fixes surfaced porting the clojure.tools.logging
library, all general:

- chez-register-spec! treats a :use :only vector like :require :refer, so a
  (:use [ns :only [names]]) clause actually brings those names in. Before, a bare
  reference to an :only'd name resolved to nil.
- syntax-quote (hc-sq-symbol) qualifies a referred name to its SOURCE namespace,
  not the compile namespace — so a macro that syntax-quotes a referred var (e.g.
  clojure.tools.logging/spy reaching clojure.pprint's pprint) expands correctly.
- resolve consults :as aliases and :refers like ns-resolve does; (resolve
  'alias/name) was returning nil because the alias wasn't expanded.
2026-06-22 10:25:35 -04:00
Dmitri Sotnikov
6b87f8dc73
Merge pull request #160 from jolt-lang/spike/chez-bootstrap
Rehost on Chez Scheme; remove the Janet host
2026-06-22 13:07:58 +00:00
Yogthos
45876998ad Docs: Chez-only, drop the Janet-era references and obsolete migration notes
Bring the docs in line with the actual implementation now that Chez is the sole
substrate.

Deleted the migration/spike/handoff artifacts that only documented the Janet
era or the port effort: the port plan, phase-0 and foundational-runtime spike
writeups (+ the stray root-level copy), the self-hosting design notes, the
architecture-refactor plan, and spike/chez/RESULTS.md.

Rewrote the current reference docs against the Chez facts: building-and-deps and
tools-deps (no jpm/build step — bin/joltc off the checked-in seed, deps via
jolt.deps into ~/.jolt/gitlibs), libraries (SQLite is built-in jdbc.core over
libsqlite3, not a Janet driver), the conformance/spec test-flow docs (the Chez
corpus runner + certify, no .janet harnesses), and the transient / type-hint /
seed-overlay design notes (Chez representations: mutable transients, flat
copy-on-write vectors, HAMT maps, the seed/overlay twin). Fixed the README
collections line (vectors aren't 32-way tries) and added the ffi/transient gate
targets. rfc 0001's numerics open-question is resolved (the Scheme tower).

Renamed the built-in HTTP adapter to jolt.http.server only (dropped the
ring-janet.adapter alias — a Janet-era name).
2026-06-22 09:05:35 -04:00
Yogthos
fe3fdf6b9c Transients: mutable backing instead of copy-on-write
The Chez port had landed transients as copy-on-write — each conj!/assoc!/etc.
rebuilt the whole persistent collection. Semantics were right but a transient
vector was O(n^2) to build (the persistent vector is a flat array, so every
conj! copied it); maps/sets were ~O(n log n) since the HAMT only path-copies.
This restores the Janet host's approach: true mutable backing, snapshot once on
persistent!.

  vec : a growable Scheme vector (capacity + fill count); conj!/pop! amortized
        O(1), persistent! hands off the buffer (exact fit) or trims once.
  map : a Chez hashtable keyed by key-hash/jolt= (value equality, nil-safe);
        persistent! folds it into a pmap.
  set : a Chez hashtable; persistent! folds into a pset.
  cow : fallback for anything else (e.g. a sorted coll) keeps the old
        copy-on-write path, preserving jolt's superset.

get/count/contains?/nth see through each representation. Building a 400k vector
went from minutes (quadratic) to ~50ms (linear). assoc! keeps the variadic
dangling-key nil-pad on both vectors and maps. test/chez/transient-test.ss pins
the invariants and the linear-time property; wired in as `make transient`.
2026-06-22 08:38:22 -04:00
Yogthos
6f433a1b3c Make blocking socket FFI collect-safe; fix http-client temp-file race
Two thread-safety bugs in the native FFI layer.

The HTTP server's accept/recv/send were plain foreign-procedures. A thread
inside a foreign call stays active for the stop-the-world collector, so the
accept loop sitting idle in accept() froze GC for the whole process whenever
another thread (a future, an async block) allocated. Mark the three blocking
calls __collect_safe so the thread deactivates for the call's duration —
collection proceeds while the accept thread waits. The args are an fd and
foreign-alloc'd buffers (outside the Scheme heap), so a collection mid-call has
nothing to move.

jolt.http-client built its -D header-file path from an unguarded (set! counter
(+ counter 1)) and counter mod 90000, with no per-process component. Concurrent
requests could compute the same path and clobber each other's headers. Use a
mutex-guarded monotonic counter plus the pid.

test/chez/ffi-server-test.ss exercises both (a (collect) while the server is
idle in accept(), temp-path uniqueness across threads, and a live request) and
is wired into the gate as `make ffi`.
2026-06-22 08:12:53 -04:00
Yogthos
4114766c71 Host gaps that blocked malli: map .iterator, two clojure.lang builders
malli loads and validates now. Three divergences surfaced building its registry
and :map schema:

dot-forms: (.iterator coll) on a map fell into the map-as-object branch and was
mis-read as a missing :iterator key (nil), so malli's -vmap got nil and crashed
on .hasNext. Route iterator through the collection-interop path — a jiterator
over the seq (the entry iterator for a map).

host-static: register clojure.lang.LazilyPersistentVector/createOwning (-vmap
fills an object-array then hands it over) and PersistentArrayMap/createWithCheck
(malli's eager entry parser relies on its duplicate-key throw; a missing class
was caught and mis-reported as ::duplicate-keys on every map schema).
2026-06-22 05:35:08 -04:00
Yogthos
6ab65a30e3 Fix arg evaluation order + host interop gaps so reitit/selmer/honeysql run
Shaking the ring-app example's real library stack out against jolt surfaced a
batch of divergences from JVM Clojure, the biggest being evaluation order.

backend_scheme: call and recur arguments were emitted as bare Scheme operands,
so Chez's unspecified (right-to-left) order won out. Clojure evaluates left to
right, which selmer's reader loop relies on: (recur (add-node ... rdr) (read-char
rdr)) consumed a char early and dropped the first chars of every {{tag}}. Bind
operands to fresh temps in a let* (only when two or more can have side effects,
so hot calls over locals/consts stay un-wrapped). emit-ordered already did this
for collection literals; generalize it.

host-contract: syntax-quote now resolves the alias part of a qualified symbol
(impl/foo -> clojure.tools.logging.impl/foo) instead of leaving it bare, which
limped along via short-name matching until two loaded namespaces (reitit.impl,
clojure.tools.logging.impl) shared the short name and it broke.

collections: key-hash masks with bitwise-and, not fxand — jolt-hash is set!-
decorated per type (records return their own hash) and Chez's equal-hash can be a
bignum, so a key's hash isn't always a fixnum.

seq: even?/odd? handle bignums (JVM accepts any integer; the fxand crashed).

records: Keyword/Symbol .sym/.getName/.toString (honeysql's :clj branch reads
(.sym k)); Throwable .getMessage/.toString over a Chez condition.

host-static: __register-class-ctor!/__register-class-statics! so a host shim
(reitit.trie-jolt) can mirror a Java class.

natives-str: String.intern returns the string.

sqlite: jdbc.core fetch/fetch-one kebab-case column keys (the jolt-lang/db
convention; created_at -> :created-at).

io: a relative io/file path resolves against JOLT_PWD (the user's cwd), not the
repo root the launcher cd'd to — matches JVM cwd semantics, so config.edn loads.

cli: render an uncaught jolt throw (ex-info message + ex-data, or a condition)
instead of Chez's opaque "non-condition value" dump.
2026-06-22 05:26:09 -04:00
Yogthos
10e3a00777 Host completeness for the ring stack (ring-app)
Gaps surfaced loading ring-core / hiccup / config / tools.logging on Chez:

- clojure.java.io/resource — resolve a named resource against the loader's
  source roots (no classpath), returning a slurp-able File.
- (Object.) constructor — a fresh distinct value (lock / unique sentinel).
- *out* / *err* as dynamic vars over a port-writer, so (binding [*out* *err*] …)
  and #'*out* compile and run (tools.logging, selmer).
- :refer :all now registers a refer-all relation, so an unqualified var from a
  (require '[ns :refer :all]) resolves at compile time — including #'var.
- java.lang.Character interop ((.toString \+) etc.).
- StringReader accepts a char[] ((StringReader. (char-array s))), not just a
  String.
- the \p{L} translation stops just below the UTF-16 surrogate gap (\x{D7FF})
  instead of \x{10FFFF} — a range across the surrogates made irregex's char-set
  construction call integer->char on a surrogate and crash.
2026-06-22 03:20:31 -04:00
Yogthos
5e916433b8 Native SQLite + an HTTP server over FFI (ring-app foundation)
sqlite.ss: jolt.sqlite + jdbc.core (jolt-lang/db's API) over the system
libsqlite3 — open/close, exec, a prepared query returning row maps, text/int/
double parameter binding, last_insert_rowid. The sqlite3 C API is non-variadic
so it binds directly.

http-server.ss: a minimal HTTP/1.1 server over BSD sockets (socket/bind/listen/
accept/recv/send via FFI), one connection at a time on a background accept
thread, synchronous Ring handlers. Parses the request line + headers + a
Content-Length body into a Ring request map (:body a StringReader), formats a
Ring response map back. Exposed as jolt.http.server and, for the example, as
ring-janet.adapter/run-server. macOS and Linux socket-option constants handled.
2026-06-22 02:50:53 -04:00
Yogthos
b251e9166e jolt.http-client (curl-backed) + format width/justify flags
A synchronous HTTP client def-var!'d into jolt.http-client (get/post/put/delete/
head/request -> {:status :headers :body}), with :headers, :body, :query-params,
:content-type and :insecure?. It shells out to the system `curl` rather than a
direct libcurl FFI: on Apple Silicon curl_easy_setopt is variadic and Chez's
fixed-signature foreign-procedure can't place the value arg on the stack where
the ABI expects it, so a direct bind silently drops the option. curl gives the
same native TLS/redirect/gzip with no per-platform C shim.

format now honours width and the -/0 flags (%-30s, %5d, %05d), not just %.Nf
precision — it was emitting the directive literally.
2026-06-22 02:44:29 -04:00
Yogthos
33664ed5e0 jolt.png: a built-in PNG writer (ray-tracer-multi)
A minimal truecolor PNG encoder over Chez bytevectors — CRC-32 / Adler-32
framing with DEFLATE "stored" blocks, so there's no compressor to carry. Restores
the jolt.png built-in (image/put!/write) the old host provided; def-var!'d into
the jolt.png namespace and loaded in the CLI before the loader's baked-namespace
snapshot, so (require '[jolt.png]) resolves with no source file. Output verified
to decode as a valid PNG (signature, IHDR, CRC-correct chunks, inflatable IDAT).
2026-06-22 02:33:27 -04:00
Yogthos
af163bad59 java.util.HashMap shim + a value in the "No method" error
A mutable Map keyed by jolt values (jolt-hash/=) with put/get/putAll/containsKey/
size/remove/keySet/values/entrySet — enough for libraries that build a fast
lookup table (malli's fast-registry doto's a HashMap then .get's it). The
"No method M for value" dot-dispatch error now includes the value, which makes
a wrong-type interop target far easier to pin down.
2026-06-22 02:28:17 -04:00
Yogthos
2de0543613 More library-compat fixes from porting the examples (markdown, malli)
Reader / loader:
- #?@ splicing reader conditionals now actually splice the matched collection's
  items into the enclosing sequence; the splice flag was read but ignored, so a
  binding vector like [a #?@(:clj [b (.foo b)])] lost its alignment.
- the file loader reads by position and skips a top-level form that reads as
  nothing (a :cljs-only #?, a #_ discard, a trailing comment) instead of
  treating it as EOF — which silently dropped the rest of a large .cljc file.
- jolt's reader feature set now includes :clj (was {:jolt :default}). jolt is a
  Clojure/JVM-compatible host that emulates clojure.lang.* and java.* interop,
  so it reads the :clj branch of a .cljc library, not :cljs. This also lets four
  more reader-conditional corpus cases pass (floor 2726 -> 2730).

Backend:
- munge-name escapes ' (prime) -> _PRIME_; a Clojure symbol like f' otherwise
  emitted a bare ' into Scheme, which is the quote reader macro and unbalanced
  the output.

Host shims:
- clojure.java.io/writer (pass through a StringWriter, file-back a path) and a
  readLine on the string reader, so line-seq over (io/reader …) works (markdown).

A better "unsupported destructuring pattern: <pat>" error message.
2026-06-22 02:25:11 -04:00
Yogthos
7e2704642b deps.edn resolution + a file loader + a project-aware CLI
joltc grew from a single -e expression into a real project runner. require now
loads a namespace's .clj/.cljc from the source roots transitively (load-once),
so a multi-file project works; the corpus/unit gates load compile-eval.ss but
not the loader, so their alias-only require is unchanged.

jolt.deps resolves a deps.edn into ordered source roots — git + local deps only
(no Maven), breadth-first so a top-level pin wins, with aliases (:extra-paths/
:extra-deps/:main-opts) and tasks. Git deps clone into a sha-immutable cache
($JOLT_GITLIBS, else ~/.jolt/gitlibs) by shelling out to git via a new
jolt.host/sh primitive. jolt.main dispatches run -m / -M:alias / -A / repl /
path / a deps.edn task. The launcher passes the user's cwd as JOLT_PWD (the
project dir) since it cd's to the repo root for the runtime's relative loads.
2026-06-22 02:06:05 -04:00
Yogthos
85cec65937 Compiler fixes surfaced porting the examples to Chez
Four runtime/reader gaps that blocked real libraries (hiccup, commonmark):

- reader: a type hint on a code form (^String (to-str x)) was lowered to a
  runtime (with-meta (to-str x) {:tag String}), mis-applying the hint to the
  call's RESULT and throwing when it's a string/number. A :tag hint on an
  evaluated form is compile-time only in Clojure — attach it to the form
  instead. Collection literals (^:foo [1 2 3]) still get runtime metadata.

- deftype: register the ctor globally by simple class name (like StringBuilder)
  so (Name. ...) interop resolves ns-agnostically. host-new resolved the ctor
  against the runtime current ns, which is the caller's, not the defining ns,
  once a deftype is used across files.

- protocol dispatch: canonical-host-tag now strips the clojure.lang. prefix too
  (clojure.lang.Keyword -> Keyword), and keywords/symbols carry the Named tag,
  numbers a Ratio tag. An (extend-protocol P clojure.lang.Keyword ...) was
  missing the dispatch, so e.g. hiccup rendered <:html> instead of <html>.

- regex: parse leading Java inline flags ((?s)/(?i)/(?m)) and pass the
  equivalent irregex options (single-line/case-insensitive/multi-line); irregex
  rejects the inline syntax. Adds a java.util.Iterator shim ((.iterator coll)/
  .hasNext/.next) for the run!-style loop hiccup compiles.
2026-06-22 02:06:05 -04:00
Yogthos
0d4f84e1f2 corpus gate: skip the racy future-cancel cases instead of banking the floor on them
The two future-cancel cases — (future-cancel (future 1)) and the
future-cancelled? variant — depend on whether future-cancel catches a
trivial future in-flight, which is pure thread-scheduling luck. They were
allowlisted but still counted toward `pass` whenever the race resolved
favorably, so the floor (2728) silently assumed both passed. On a fast dev
machine they always pass; on CI's loaded shared runner one races to a
divergence, dropping pass to 2727 and failing the gate.

Skip them like the undelivered-promise case (neither pass nor fail) so the
race can't perturb the count. Floor drops to the deterministic 2726.
2026-06-22 01:29:24 -04:00
Yogthos
dfc34e6e71 spec: set! now supports deftype mutable fields 2026-06-22 01:19:21 -04:00
Yogthos
424ce75cf6 mutable deftype fields: (set! field val) in a method
deftype fields tagged ^:unsynchronized-mutable / ^:volatile-mutable can now be
reassigned in place from a method, as on the JVM. A jrec stores fields as cons
cells, so a new jolt-set-field! mutates the pair with set-cdr!. The deftype macro
rewrites (set! mutable-field v) in a method body to (set! (.-field inst) v), and
the analyzer compiles a (set! (.-field obj) v) target to jolt-set-field! — so
both the rewritten symbol form and an explicit interop (set! (.-root this) v) go
through one path. Field reads remain a snapshot at method entry, which is correct
for the universal read-then-set pattern (a repeated set! of the same field in one
call would read the entry value).

Closes the set!-of-local SCI failures: SCI load 202 -> 205/218.
2026-06-22 01:19:03 -04:00
Yogthos
b9ab750983 spec/ebnf: macroexpand order, set!, letfn primitive, numeric tower
Bring the formal definition in line with this session's language work:
- grammar.ebnf: numbers are a real tower (exact integer / Ratio / double); the M
  suffix reads a real BigDecimal, N an exact integer (drop the stale Janet note).
- 02-reader S5: M is a real java.math.BigDecimal with scale-insensitive equality.
- 03-special-forms: document the read -> macroexpand -> analyze order (macros
  expand before special-form dispatch); special-form heads are not shadowable but
  macros are and value-position locals may be named like a special; set! on a var
  sets the innermost binding (else root); letfn is a primitive with letrec*
  semantics.
2026-06-22 01:03:48 -04:00
Yogthos
212cd0399a special-form heads are not shadowable
Found in a read/eval review: a local named like a special form wrongly took
over operator position. (let [if (fn ...)] (if true 1 2)) returned the fn, but
per spec section 3 (and the reference) special-form heads are not shadowable;
only macros are. Two fixes: drop the (not shadowed) guard on the special-form
branch of analyze-list (so an (if ...) head is always the special), and prefix
a local whose name is a Scheme keyword when emitting (so a value local legally
named if does not shadow the (if ...) the back end emits). Value-position
locals named if/or/case still work.
2026-06-22 01:01:53 -04:00
Yogthos
f18ae3bd46 macroexpand-first analyzer order; one macro path; defmacro/letfn fixes
The analyzer checked special forms before expanding macros, the reverse of the
canonical read -> macroexpand -> analyze order (Clojure/CLJS analyze-seq). Move
macroexpansion to the front of analyze-list. Knock-on fixes:

- letfn was both a (broken) macro expanding to let* AND a primitive special
  (analyze-letfn, proper letrec*). Macroexpand-first surfaced the macro, breaking
  mutual recursion; remove the macro, keep letfn a primitive.
- defmacro is now compiled by the analyzer (a :set-var-style :defmacro node that
  defs the expander fn via the fn macro — so destructuring arglists desugar — and
  marks the var a macro), so a non-top-level (when … (defmacro …)) works. The
  runtime spine's separate top-level defmacro interception is removed: one path.

SCI load 162 -> 202/218.
2026-06-22 00:54:16 -04:00
Yogthos
e6ee17b055 compile (set! *var* val)
The analyzer punted set! as uncompilable. Add it as a special form: (set! sym
val) on a var emits jolt-var-set, which updates the innermost thread binding (or
the root when unbound), returning val. A local target (deftype mutable field) or
an interop (.-field) target stays uncompilable for now. Also defines
*warn-on-reflection* (false) so set! on it resolves. SCI load 186 -> 196/218.
2026-06-22 00:40:46 -04:00
Yogthos
86aa89c832 uniquify duplicate fn params (macro _ _ expanders)
Chez rejects duplicate lambda formals, so any (fn [_ _] ...) failed to compile
— including every macro expander, whose &form/&env slots both expand to _. The
analyzer now renames each earlier duplicate param to a fresh name (Clojure binds
the last occurrence, so the earlier ones are shadowed and unreferenceable). SCI
load 162 -> 186/218.
2026-06-22 00:35:16 -04:00
Yogthos
3721423a64 real chunked seqs for vector seqs
A vector's seq is now a real chunked-seq (chunked-seq? true), matching Clojure/
CLJS. Each vector-seq cell carries its backing vector + element index as two
cseq fields (cvec/ci, no extra allocation vs the old lazy cell), so:
  - chunk-first hands out a 32-element block (a pvec slice), chunk-rest is the
    seq at the next block boundary — the ChunkedSeq contract (chunk-first ++
    chunk-rest == the seq);
  - reduce/transduce take a fast path that walks the backing vector by index in
    a tight loop with no per-element seq cells (reduce over a 1M-vector ~0.4s).
The seq cell stays a cseq, so first/rest/count/printing and the ~26 cseq?
dispatch sites are untouched. The eager chunk-buffer model (chunk-buffer/chunk/
chunk-cons) is preserved for the round-trip case. No seed change (runtime only).
2026-06-22 00:21:05 -04:00
Yogthos
0e4ccc97e0 (type record) returns its class-name string, not a symbol
(type r) returned a symbol user.TyR, so (= (symbol (str (type r))) (type r))
was true; the JVM's type is a Class (not a Symbol) so it's false. jolt models
classes as strings, so a record's type is now its ns-qualified class-name
string — equal to (class r), as on the JVM where type and class coincide for a
record. The symbol-keyed print-method defmethods already fall through to the
default record printing, so they're unaffected. Closes type-of-record.
2026-06-22 00:05:08 -04:00
Yogthos
ab96650fbb real BigDecimal type (bigdec, M literals)
bigdec / 1.5M / 0.0M silently produced doubles. Add a jbigdec value type
{unscaled, scale} over Chez exact integers (host/chez/bigdec.ss): value =
unscaled * 10^-scale. An M-suffix literal reads to a :bigdec form that the back
end lowers to jolt-bigdec-from-string (same IR-leaf path as #inst/#uuid); bigdec
coerces a number/string. Equality is by value (1.0M = 1.00M true, 3M = 3 false),
str drops the M and pr keeps it, class is java.math.BigDecimal, decimal? is true.
Arithmetic contagion isn't modelled (out of scope). The old corpus cases passed
spuriously as doubles; they now exercise a genuine BigDecimal.
2026-06-22 00:01:01 -04:00
Yogthos
7db5fabc8d resolve ^Type hint to canonical class name in var :tag
(def ^String tv ...) left (:tag (meta (var tv))) as the unresolved "String";
the JVM compiler resolves the hint to java.lang.String at def time. Add a
resolve-class-hint host seam (built from the existing class-token table) and
resolve a def's :tag through it in the analyzer. The reader path
(read-string "^String x") stays unresolved, matching the JVM (only the
compiler resolves). Closes ^Type-tag-on-var.
2026-06-21 23:52:47 -04:00
Yogthos
d1c2811d13 *in* is a Reader, not a map
(map? *in*) was true because *in* was a plain map of read-line-fn/read-fn
closures; the JVM *in* is a java.io.Reader so map? is false. A defrecord
doesn't help (records are maps). Make the reader a reify over a new IReader
protocol — a non-map value — and route read/read-line/read+string/line-seq
through its -read-line/-read-form/-read+string methods instead of keyword
access. with-in-str's __string-reader and the stdin *in* both reify it.
Closes *in*-bound + *in*-is-bound.
2026-06-21 23:45:24 -04:00
Yogthos
8ce00d29fd embed a namespace value spliced into a form (~*ns*)
A macro like (defmacro cur-ns [] `(str ~*ns*)) splices the live *ns* value
into its expansion, leaving an opaque jns object as a list element. The
analyzer had no way to carry a runtime value and threw uncompilable — the last
remaining corpus crash. Recognize a jns via the host contract (form-ns-value?)
and emit a :the-ns leaf that reconstructs it by name (intern-ns!) at the call
site, the same IR-leaf pattern as regex/inst/uuid. Closes unquote-*ns*-in-
template; corpus crash count -> 0.

A namespace fast path rather than a general constant pool: it's the only
embedded-value case in the corpus and the common real-world one (libs splice
~*ns*). A general pool can come later if other value types appear.
2026-06-21 23:40:59 -04:00
Yogthos
ccc76fd69f extenders excludes inline defrecord/deftype impls
deftype/defrecord inline protocol methods went through extend-type ->
register-method, so a record implementing a protocol inline showed up in
(extenders P) — the JVM only lists extend/extend-type/extend-protocol
registrations there (inline impls compile into the class). Add
register-inline-method: it registers for dispatch under the record tag but
skips the extender mark. The mark lives inside type-registry so the per-case
corpus prune restores it. Closes corpus lists-extended-type + seq-of-tags.
2026-06-21 23:35:11 -04:00
Yogthos
10d2b992f7 Prune stale corpus allowlist entries (now-passing print-method + misc)
The defmethod/print-method record cases, symbol-hint, and source-order entries no
longer diverge (closed by the defmethod-setup + earlier fixes); drop them. 0 new
divergences, corpus 2720/2741, 20 genuine gaps remain.
2026-06-21 22:49:34 -04:00
Yogthos
632a90cae2 definterface returns the name (not a var); ns-imports returns java.lang defaults
definterface now expands to (do (def name {}) 'name) so (var? (definterface ...))
is false, matching the JVM where it yields the interface Class. ns-imports returns
the 96 auto-imported java.lang classes (short symbol -> canonical name) so
(count (ns-imports 'user)) is 96. Re-minted for the macro change. Corpus 2718->2720.
2026-06-21 22:47:23 -04:00
Yogthos
9e0a930eb4 Typed-array identity + JVM flonum printing (Inc 3) 2026-06-21 22:36:14 -04:00
Yogthos
f2747679e9 Prune now-passing class/atom entries from the corpus allowlist
class number/string/keyword/name + atom?/instance? Atom pass after the class
refinements; drop their stale allowlist entries. Corpus 2705/2741 0 new div.
2026-06-21 20:06:21 -04:00
Yogthos
e3674d17a7 defmethod auto-create copies clojure.core dispatch (fix SCI regression)
The prior fix resolved an unqualified defmethod to clojure.core's multifn, which
broke SCI (it relies on per-ns shadow multimethods — hung loading core_protocols).
Keep the shadow, but when auto-creating it copy the dispatch fn + default from a
same-named clojure.core multifn (e.g. print-method's 2-arg dispatch) instead of
the 1-arg identity that crashed (print-method x w). Also trim the FQN class
tokens to value classes only (the collection interfaces shadowed names SCI uses).

Corpus 2705/2741 0 new div; SCI 162/218 restored; cross-ns + direct print-method
overrides work.
2026-06-21 20:04:13 -04:00
Yogthos
bb6c9eeb29 Class refinements: per-type class names, FQN tokens, instance? built-ins
- (class x) returns per-type JVM class names (Long/Double/Ratio/Character/Atom),
  not a blanket java.lang.Number.
- register fully-qualified class tokens (java.lang.Long, clojure.lang.Keyword,
  clojure.lang.Atom, ...) that self-evaluate to their name, so (= (class 1)
  java.lang.Long) and (instance? clojure.lang.Atom x) resolve.
- instance? recognizes Long/Double/Ratio/Character/Symbol/Atom/IFn built-ins.

Closes class number/string/keyword/name, instance? Atom, atom?. Corpus 2699->2705.
2026-06-21 19:12:40 -04:00
Yogthos
b03da19ba8 defmethod resolves the multifn like a var (find clojure.core's)
(defmethod print-method ...) from the user ns auto-created a stray user/print-method
with identity dispatch -> 'incorrect number of arguments 2' on (print-method x w).
Resolve an unqualified multifn name through current-ns -> :refer -> clojure.core
(like var resolution) before auto-creating. Fixes direct print-method/print-dup
override calls; pairs with the cross-ns defmethod fix.
2026-06-21 18:47:15 -04:00
Yogthos
4d1ec44676 JVM parity: unchecked-char -> char, pr of infinities -> ##Inf
Allowlist review found three addressable divergences:
- unchecked-char returned a number; the JVM returns a char.
- the readable printer (pr-str, coll elements, the -e/REPL printer) rendered
  infinities as Infinity/inf; Clojure's readable form is ##Inf/##-Inf/##NaN
  (str/print still gives Infinity). So (pr-str ##Inf) => ##Inf, (str [##Inf]) =>
  [##Inf], (str ##Inf) => Infinity.

Corpus 2695->2698; allowlist 43->40 (drop the 3 now-passing entries). Re-minted.
2026-06-21 17:55:05 -04:00
Yogthos
518683ccd6 syntax-quote leaves interop forms unqualified (jolt-z1zu)
A macro that syntax-quoted interop — `(.. (StringBuilder.) (.append x)) — had
its .method / Class. / .-field heads qualified to the compile ns (user/.append,
user/StringBuilder.), so they read as 'Unknown class user' at expansion. Like
Clojure, leave interop-head symbols bare in syntax-quote. Fixes any macro
templating interop, not just the one corpus case. Corpus 2694->2695.
2026-06-21 17:49:11 -04:00
Yogthos
f15a4e7747 reduce dispatches to a reify's IReduceInit reduce method (jolt-z1zu)
(reduce f init (reify clojure.lang.IReduceInit (reduce [_ f i] ...))) tried to
seq the reify and threw 'not seqable'. When the coll is a reify carrying a reduce
method, drive the reduction through it. Corpus 2693->2694.
2026-06-21 17:46:20 -04:00
Yogthos
31a453d492 (. obj :kw) is a keyword lookup (JVM parity)
The . special form rejected a non-symbol member; a keyword member now lowers to
an invoke of the keyword on the target ((. {:value 41} :value) => 41, as on the
JVM). Added a form-keyword? contract seam. Corpus 2692->2693. Re-minted.
2026-06-21 17:13:40 -04:00
Yogthos
45596d7239 import: bring a cross-ns deftype/record ctor into the current ns (jolt-c2l1)
(:import [other.ns Type]) was a no-op (import unbound), so (Type. ...) failed
with 'Unknown class'. Bind import to register each named type's ctor closure
under the current ns. Corpus 2691->2692.
2026-06-21 17:11:15 -04:00
Yogthos
547a8c6d17 The .. threading macro analyzes as a macro, not interop (jolt-c2l1 tail)
(.. x m ...) failed: the analyzer classified the .. head as a .method interop
call (method-head? matched any "."-prefixed name) and form-special?
(hc-interop-head?) also flagged it, so it never reached the macro check. Exclude
".." from both (the char after "." being "." means the threading macro, not
.method). Corpus 2690->2691. Re-minted.
2026-06-21 17:08:11 -04:00
Yogthos
e7f5bcb58d assoc! fills nil for a trailing lone key (JVM parity)
JVM assoc! is variadic: with a complete first pair present (>=3 kvs), a trailing
lone key fills nil ((assoc! t :a 1 :b) => {:a 1 :b nil}); a lone key alone (1 kv)
is still a wrong-arity throw. jolt delegated to the strict persistent assoc which
threw on any odd count. Pad a trailing nil for odd kvs >=3. Corpus 2688->2690.
2026-06-21 17:03:10 -04:00
Yogthos
c788e86f1a Cross-ns def/require/use/defmethod through the spine (jolt-c2l1)
The per-form eval passed a FIXED compile-ns to every subform of a top-level do,
so a runtime (ns ...)/(in-ns ...) didn't redirect later defs/refs — defs landed
in the wrong ns and qualified refs hit host-static ("Unknown class"). Thread
the current ns: each subform analyzes in (chez-current-ns), which ns/in-ns move.

That exposed two more gaps, now fixed:
- use refers ALL of a target's public vars (a refer-all table consulted by
  chez-resolve-refer) — was bound to plain require (explicit :refer only).
- defmethod on a QUALIFIED multifn (cf.mm/ext from another ns) resolves in the
  symbol's ns, not the current one (was auto-creating a stray multifn).

Corpus 2684->2688, 0 new divergences; floor raised. No re-mint (runtime shims).
2026-06-21 16:56:46 -04:00
Yogthos
76f8274603 sorted-map entries are real map-entries (jolt-jk23)
sorted-map seq/first/entries built plain [k v] vectors, so map-entry? was false
and key/val threw. Build them via a new jolt.host/map-entry seam (entry-flagged
pvec), matching a regular map's entries. Re-minted.
2026-06-21 16:43:05 -04:00
Yogthos
9ca30a236d CI runs behavior gates; self-host fixpoint is dev-only (jolt-8479)
The self-host byte-fixpoint (make selfhost) only holds on the Chez that minted
the seed — CI's Debian Chez emits byte-different output for some constructs
(isolated to the dedupe re-mint), so it failed there. The checked-in seed RUNS
correctly on any Chez, so CI now runs 'make ci' (corpus/unit/smoke/sci/certify);
'make test' keeps selfhost for local dev. Cross-version emit determinism tracked
in jolt-8479.
2026-06-21 15:46:40 -04:00
Yogthos
7d0e2f2b61 dedupe: add the 0-arg transducer arity (jolt-05i2)
(into [] (dedupe) coll) / (sequence (dedupe) coll) threw an arity error — dedupe
was [coll]-only. Add the 0-arg stateful transducer (tracks [seen? prev] in a
volatile, no sentinel). Re-minted.
2026-06-21 15:42:01 -04:00
Yogthos
f6937dd7df Bind clojure.core/float (= double on Chez)
Chez has no single-float type, so float coerces to a flonum like double.
Corpus 2683->2684; floor raised.
2026-06-21 15:39:21 -04:00
Yogthos
cf0b544baf Host interop fixes: ==/time/subvec/defonce + corpus cleanup
- == 1-arg returns true for any value (Clojure short-circuits before the number
  check), not 'requires numbers'.
- current-time-ms wired to now-millis so the time macro works.
- subvec truncates float/ratio indices via long (Scheme quotient rejects flonums).
- defonce checks bound? not var-get — in a top-level do the name is already an
  unbound interned cell, which var-get throws on.
- drop the line-seq corpus row (used janet/spit, N/A); allowlist char-array
  (needs Class/forName "[C").

Corpus 2678->2683, floor raised. Re-minted. Full gate green; CI green.

jolt-cf1q.7
2026-06-21 15:36:41 -04:00
Yogthos
dd0e0b55cc CI: invoke Chez via a scheme wrapper, not a symlink
Chez derives its boot-file name from argv0, so a symlink named chez looks for a
nonexistent chez.boot (CI failed at the first gate step). Replace the symlink
with a wrapper that exec's scheme, preserving argv0 so the boot files resolve.
2026-06-21 15:32:16 -04:00
Yogthos
39efb29134 Drop N/A janet.* cases from the corpus (jolt-0obq)
Remove the 17 rows that exercised the Janet FFI bridge (interop/janet bridge,
interop/jolt.interop) and the Janet build-time env scrub (host-interop/bake env
scrub, janet.os/setenv) — none exist on any non-Janet host, so they only added
crash noise. Portable interop is covered elsewhere. corpus 2920->2903 rows;
gate 2678/2742 0 new div, certify 0 new/0 stale.
2026-06-21 15:08:14 -04:00
Yogthos
017a1bc8c4 SCI conformance gate (pure Chez)
Re-port the SCI compatibility stress test to joltc: host/chez/run-sci.ss loads
borkdude/sci's own source (vendor/sci, re-vendored) through the spine and
requires its forms to compile+eval. Floor-gated at 160/218 forms (the tail is
genuine host gaps — set! on vars, some macro/def shapes); raise as they close.
Wired into 'make test' (skips if the submodule isn't checked out).

jolt-cf1q.6
2026-06-21 12:06:18 -04:00
Yogthos
48e2ef5910 Scrub dangling Janet references; drop dead Janet-coupled files
Rephrase comments that pointed at deleted Janet files (emit.janet, the seed
sources, 'the Janet back end punts ...') to present-tense descriptions of the
Chez behavior. Comment/docstring-only; the self-host fixpoint is unchanged
(comments don't affect the compiled seed).

Delete five files that were Janet-host shims with no Chez path: clojure.java.io
(provided natively by host/chez/io.ss), and jolt.{nrepl,png,interop,shell}
(the janet.* bridge, os/shell, janet.net — none exist on Chez).

jolt-cf1q.6
2026-06-21 12:01:04 -04:00
Yogthos
750ce05716 Docs + CI for the Chez-only substrate
Rewrite the README, CLAUDE.md build/architecture sections, test/chez/README,
and conformance SPEC for the Janet-free world: bin/joltc + make test, the
self-hosting bootstrap, the frozen JVM-sourced corpus. CI installs Chez + JDK/
Clojure and runs 'make test' (was Janet/jpm).

jolt-cf1q.6
2026-06-21 11:34:48 -04:00
Yogthos
58d03d67be Delete the Janet host — Chez is the sole substrate
Remove the Janet seed (src/jolt/*.janet: reader, value layer, vars/ns, the
tree-walking interpreter, the Janet backend, the optimizing compiler), the
Janet->Scheme cross-compiler (host/chez/{driver,emit,jolt-chez}.janet),
bin/jolt-chez, the jpm build (project.janet) and the Janet test runner
(run-tests.janet), plus the entire Janet test suite. jolt now builds and runs
on Chez alone: bin/joltc off the checked-in seed, bootstrap.ss to rebuild it.

The portable Clojure stays: jolt-core/**, host/chez/**.ss, and the stdlib +
tooling under src/jolt/clojure + src/jolt/jolt (read by the seed build, no
Janet). The gate is 'make test' (self-host, corpus, unit, cli smoke, certify).
Drop the sci and clojure-test-suite submodules (used only by deleted Janet
integration tests); irregex stays.

Filesystem corpus/unit cases that probed project.janet now probe README.md.

jolt-cf1q.6
2026-06-21 11:29:03 -04:00
Yogthos
5c1fdfc336 Pure-Chez test gates (no Janet)
Add a Janet-free gate so correctness can be judged with only Chez + Clojure:
- host/chez/run-corpus.ss: corpus.edn vs JVM expecteds, lifting the per-case ns
  isolation from the old Janet driver; reads corpus.edn via the jolt reader.
- host/chez/run-unit.ss + test/chez/unit.edn: the host-specific unit cases,
  evaluated in-process and compared to baked expecteds.
- host/chez/selfcheck.sh: self-host fixpoint (bootstrap.ss rebuild == checked-in seed).
- host/chez/smoke.sh: real bin/joltc CLI smoke.
- host/chez/remint.sh: re-mint the seed to a byte-fixpoint after a source change.
- Makefile: 'make test' runs the lot; 'make remint' rebuilds the seed.

Numbers match the Janet gate: corpus 2679/2757 0 new div, unit 450/450, certify
0 new/0 stale.

jolt-cf1q.6
2026-06-21 11:22:32 -04:00
Yogthos
9a273e71cd Move the Chez test oracle off Janet
The unit tests in test/chez/_*.janet now drive bin/joltc (the zero-Janet
spine) and judge against baked expected values instead of a live build/jolt
run. Ten of them captured the oracle from build/jolt per case; their values
are now literals (one env-dependent javastatic case became a predicate so it
stays portable). The rest already had literal expecteds with a redundant
build/jolt sanity check, now dropped.

Retire emit-test/emit-parity/reader-parity: they compared the Chez/Clojure
path against a live Janet evaluation, emitter, or reader. That migration check
is done, and run-corpus-zero-janet (Chez analyzer vs the JVM corpus) plus
certify.clj cover correctness now.

Rewrite the README for the current zero-Janet gate.

jolt-5oci
2026-06-21 09:48:49 -04:00
Yogthos
8d4f83e0f7 Retire the Janet-host corpus gates
run-corpus.janet drove a target binary (default build/jolt, the Janet host)
through the corpus and run-corpus-chez.janet was the all-flonum subset probe.
Both compare against corpus.edn, which is now JVM-sourced, so the all-flonum
hosts diverge on every numeric/host case. The zero-Janet spine gate
(run-corpus-zero-janet.janet) plus certify.clj are the oracles; drop these and
the stale test/chez/known-divergences.edn allowlist they used.
2026-06-21 01:59:15 -04:00
Yogthos
da775802d6 Source the conformance corpus from JVM Clojure; retire the prelude gate
corpus.edn :expected is now the value reference JVM Clojure produces, set by the
new test/conformance/regen-corpus.clj (one JVM process, per-row thread watchdog).
167 rows moved to the JVM value: ratios (/ 1 2)=>1/2, doubles (double 3)=>3.0,
shared-heap concurrency (the future/pmap/agent cases), clojure.math doubles. The
JVM is the spec; jolt is measured against it.

known-divergences.edn shrinks to the rows whose JVM value is an opaque host object
that can't round-trip to source (Java arrays, transients, atoms, beans, proxies,
chunks all print as #object[..@addr]) plus (fn* foo) and a few racy concurrency
cases (:flaky). The zero-janet gate's allowlist becomes the set of host gaps vs the
JVM spec (no Class/array/BigDecimal, :jolt reader, jolt's own printing).

Math/clojure.math sqrt/pow/floor/trig now return doubles (Chez returns exact for
exact args, e.g. (sqrt 9)=>3); JVM always returns a double.

extract-corpus.janet no longer writes corpus.edn unless asked (the test runner
imported it and was silently overwriting the JVM corpus with the spec sources'
placeholder answers). The prelude parity gate is deleted — the zero-janet spine +
certify.clj are the oracles.

zero-janet 2678 (0 new divergences), certify 0 new / 0 stale, emit-test 330/330.
2026-06-21 01:45:04 -04:00
Yogthos
467ad75ff7 Chez numeric tower: exact ints / Ratio / double for JVM parity (jolt-n6al)
jolt was all-flonum (one :number type, inherited from Janet whose only number
type is a double). The Chez runtime has a full numeric tower, so the zero-Janet
path now carries it = JVM Clojure semantics:

  (/ 1 2)      => 1/2      (exact Ratio, was 0.5)
  (integer? 3) => true   (integer? 3.0) => false   (float? 3.0) => true
  (ratio? (/ 1 2)) => true   (= 3 3.0) => false   (== 3 3.0) => true
  (+ 1 2) => 3 (exact)   (/ 1.0 2) => 0.5 (double)

jolt= was already exactness-aware (values.ss) and == is value-equality, so
=/== match the JVM split. The reader preserves exactness (integer literals exact,
a/b ratios exact rationals, decimals/exponents flonums); backend_scheme emit-const
renders exact ints/ratios and flonums faithfully; the value-position arithmetic,
count, int, compare, bit ops, parseLong, string .length/.indexOf, range,
timestamps, and array bytes return exact integers (= JVM int/long) instead of
coercing to flonum. double/parseDouble/clojure.math floor|ceil|signum stay double.

Only the zero-Janet path carries the tower (the Janet reader loses exactness into
a double before emit). The prelude/all-flonum path is unaffected for compiled code;
the runtime reader is shared, so a couple of all-flonum reader assertions become
value (==) assertions. ~16 numeric corpus cases now give the JVM tower value vs the
Janet-era :expected and are allowlisted as tower divergences (Chez == reference
JVM) pending the corpus flip to JVM (jolt-ecz0). No BigDecimal type (1M).

Re-minted. zero-janet 2682 (floor 2698->2682, the reclassified tower cases), 0 new
divergences; fixpoint 10/10, bootstrap 6/6, spine 35/35, cli 49/49; Janet gate 155
files 0 failed.
2026-06-20 23:09:27 -04:00
Yogthos
eb1c3298a4 Chez parity: trailing-apostrophe symbols + arglist return hints (jolt-vgrp, jolt-5540)
Two Chez reader bugs, both JVM-parity gaps:

inc'/+'/foo' (trailing apostrophe) were mis-read as a symbol followed by a
quote macro, because the reader treated ' as a terminator. In Clojure ' is a
NON-terminating macro char (constituent after the first char). Since the seed
is minted on Chez, (def inc' inc) became (def inc 'inc), clobbering inc's var
cell with its own symbol -- so (var-get (var inc)) returned the symbol, not the
fn. Drop ' from the token terminator set; a leading ' still quotes.

^bytes [b] / ^String [x y] return-type hints: the Chez reader lowered ^meta on
a collection to a (with-meta vec meta) form, but emitted a QUALIFIED
clojure.core/with-meta while the Janet reader emits a bare with-meta -- so the
fn/defn macros' unwrap logic (matching the bare head) slipped past it and choked
on a non-vector arglist. Emit bare with-meta to match Janet, and unwrap a
(with-meta <vec> _) arglist in analyze-fn as a backstop.

Re-minted the seed. zero-janet 2699, prelude 2652, Janet gate 155/0, fixpoint
10/10, bootstrap 6/6, all 0 new divergences.
2026-06-20 22:17:29 -04:00
Yogthos
53a189541c Chez parity: realized? on lazy-seq + conj! 1-arity identity
realized? threw 'not supported on' for a jolt-lazyseq record (the overlay
reads :jolt/type); add a jolt-lazyseq? arm to the post-prelude wrapper
reading the record's own realized? flag.

conj! 1-arity (conj! coll) is the transducer-completion arity and returns
coll as-is on the JVM, no transient check — we threw 'not a transient'.

Both gates: zero-janet 2696->2698, prelude 2649->2652, 0 new divergences.
2026-06-20 19:40:05 -04:00
Yogthos
32d2028559 Chez parity: STM stub + portable line-seq (janet.* audit, jolt-0obq)
Audit of the janet.*/jolt.interop/STM corpus cases vs Chez equivalents: the Janet
FFI-bridge cases (janet/string, janet/type, janet.math/sqrt, janet.string/ascii-
upper) test functionality already covered by PORTABLE corpus cases (str 29, type 9,
sqrt 3, upper-case 9), so they're safe to delete in Phase 5. Two needed a Chez
equivalent so they pass instead of being lost:

- clojure.lang.LockingTransaction/isRunning -> false (no STM on jolt).
- line-seq: the corpus case used janet/spit setup but was the SOLE line-seq
  coverage (0 other cases). Ported janet/spit->spit, and added a native Chez
  line-seq (drain a jhost io/reader + split on newline; no trailing empty line)
  that delegates a Janet map-reader to the overlay version.

zero-Janet 2694->2696, prelude floor 2648; self-host + Janet gate + JVM cert green.

jolt-cf1q.7 jolt-0obq
2026-06-20 18:49:28 -04:00
Yogthos
52a1ab5970 Chez parity: analyze any seq as a list form (macro/eval-built forms) (jolt-cf1q.7)
hc-list? required cseq-list? (a reader-built list), so a form built at runtime via
concat/map/cons — a lazy cseq with list?=#f — was rejected as "unsupported form".
In Clojure any seq is a valid call form, so accept any cseq. Lights up macros that
build their expansion with concat/list and (eval seq-form).

zero-Janet 2692->2694, 0 new divergences; self-host + Janet gate + JVM cert green.

jolt-cf1q.7
2026-06-20 18:15:30 -04:00
Yogthos
01f49f50c3 Chez parity: date/time + clojure.edn/read over a reader (jolt-cf1q.7/dcmm/7t3l/uicd)
Date/time (inst-time.ss): java.util.Date / java.sql.Timestamp ctors accept ms or
another date value (ms-of) -> a jinst; java.text.SimpleDateFormat (pattern + .format
via the existing format-ms UTC engine; .setTimeZone accepted); java.util.TimeZone/
getTimeZone. instance? answers Date true / Timestamp false for a jinst (a Date is
not a Timestamp on the JVM).

clojure.edn/read over a reader (io.ss + post-prelude): the overlay edn.clj's
drain-reader is janet/type-coupled, so Chez drains the jhost StringReader/
PushbackReader to a string and reads the first EDN form. Unblocks jolt-uicd.

Native Chez throughout (no vendoring): Chez date arithmetic + string ports. zero-Janet
2688->2692, 0 new divergences; self-host + Janet gate + JVM cert green.

jolt-cf1q.7 jolt-dcmm jolt-7t3l jolt-uicd
2026-06-20 18:03:58 -04:00
Yogthos
60b4bae105 Chez parity: deftype (P. args) constructor via host-new var fallback (jolt-cf1q.7)
deftype binds the type name as a VAR (the make-deftype-ctor closure), but (P. 5)
lowers to (host-new "P"), which only checked class-ctors-tbl -> "No constructor".
Fall back to resolving the class name as a var in the current ns / clojure.core
and invoking it — so (P. args) constructs the same jrec as the ->P factory, and
protocol method dispatch (.m / .-field) over it works.

zero-Janet 2685->2688 (no-constructor 5->2); prelude floor bumped to 2641 (the
delay batch's run). Self-host + Janet gates + JVM cert green.

jolt-cf1q.7
2026-06-20 17:36:54 -04:00
Yogthos
1d6e740668 Chez parity: delay / force / realized? (jolt-cf1q.7)
Add a thread-safe delay type (concurrency.ss): make-delay wraps a thunk; deref/
force run it once under a lock and cache (JVM delays are thread-safe + memoized).
delay? and realized?-on-a-delay are native; the overlay's `delay` macro
(-> make-delay) and `force` (-> deref) now work. realized? wrapper (post-prelude)
and the deref chain gain a delay arm. Removed the np-delay? stub from
natives-parity.ss (the real type lives in concurrency.ss).

Seed unchanged (no re-mint). zero-Janet 2673->2685, 0 new divergences; Janet gate
+ JVM cert green.

jolt-cf1q.7
2026-06-20 17:02:42 -04:00
Yogthos
4b4d677fe4 Chez corpus: eval ACTUAL top-level so runtime defmacro cases run (jolt-cf1q.7)
The zero-Janet runner wrapped each case as (= EXPECTED ACTUAL) and checked the
result was true. That nests ACTUAL's top-level (do ...), so a case like
(do (defmacro m ...) (m 1)) can't use the macro it just defined — the analyzer
punted defmacro -> "uncompilable" (35 cases).

Match certify.clj's eval-isolated instead: carry EXPECTED and ACTUAL as separate
sources and evaluate ACTUAL as its own top-level program (jolt-compile-eval unrolls
the top-level do, so a macro defined earlier is usable later), then compare to
EXPECTED with =. Evaluating ACTUAL from SOURCE (not (eval (quote A))) preserves the
reader's map-literal source order, so the eval-order cases still pass.

eval-corpus-zero-janet / program-corpus-zero-janet now use a 3-field TSV
(label/expected/actual); run-corpus-zero-janet's per-case debug path evals both
sides too. Only run-corpus-zero-janet uses these (the prelude gate is untouched).

zero-Janet 2642->2673 (analyzer "uncompilable" 35->4); 1 new allowlisted divergence
(`{:a ~x :b ~y} syntax-quote map construction doesn't preserve source eval order —
pmap is unordered). Janet gate + JVM cert green.

jolt-cf1q.7
2026-06-20 16:53:20 -04:00
Yogthos
ccab89e1d5 Chez parity: Java arrays + reader/macroexpand fns (jolt-cf1q.7)
host/chez/natives-array.ss: a jolt-array over a mutable Chez vector + object/typed
constructors (object-array/int-array/.../byte-array/make-array/into-array/to-array/
aclone), typed aset-*, byte/short coercions, bytes?/bytes/ints/..., and eager
chunk-buffer/chunk-* (Jolt doesn't chunk). count/nth/seq/get and jolt.host/ref-put!
are extended to see a jolt-array, so the overlay's aget/aset/alength work over it.
Numbers it produces are flonums (jolt's rep) so exactness-aware = holds. char-array
stays in io.ss (a char-SEQ that io/reader/str/slurp consume).

natives-parity.ss adds: __reader-features / -set!, reader-conditional, re-matcher,
delay? (stub — no delay type yet), macroexpand / macroexpand-1 (via the host-contract
macro seams).

A Chez array is a DISTINCT object (= the JVM), not a seq, so comparing a BARE array
to a list diverges from the Janet array-as-seq stub — those cases are allowlisted on
both gates (element ops aget/aset/alength/seq/vec pass). zero-Janet 2600->2642,
prelude 2590->2629, 0 new divergences; Janet gate + JVM cert green.

jolt-cf1q.7
2026-06-20 16:35:32 -04:00
Yogthos
1ba19759c8 Chez parity: missing native core fns + ns runtime fns (jolt-cf1q.7)
Native Chez shims for clojure.core fns that live in the Janet seed but had none
on the zero-Janet spine (they resolved to nil -> "not a fn"):

- host/chez/natives-parity.ss: hash / hash-combine / hash-ordered-coll /
  hash-unordered-coll (24-bit masked like core_extra), transient?, rseq (vectors +
  sorted colls), cat (transducer).
- jolt-invoke now dispatches a TRANSIENT vec/map/set as a fn (callable on the JVM):
  ((transient [10 20 30]) 1) -> 20.
- ns.ss: ns-resolve, ns-imports, remove-ns, intern, alias, ns-unalias, refer,
  ns-refers, refer-clojure, alter-meta!, reset-meta!, and a real ns-aliases (was a
  stub returning {}).
- runtime (require ...)/(use ...) now register :as/:refer into the Chez ns tables
  (was a no-op). The Chez analyzer already pre-registers at analyze time, but when
  the JANET analyzer compiled the form (prelude path) the Chez tables stayed empty,
  so ns-aliases/ns-resolve over an alias diverged — this fixes both paths.

Seed unchanged (overlay doesn't reference these at mint time). zero-Janet corpus
2567->2600, prelude 2557->2590, 0 new divergences on either; Janet gate + JVM cert
green. Filed jolt-vgrp for the pre-existing var-get-of-scalar-native-op quirk.

jolt-cf1q.7
2026-06-20 15:46:11 -04:00
Yogthos
d7cfafd3a9 Chez Phase 4: perf probe (test/chez/bench-chez.ss)
Mirrors test/bench/core-bench.janet's 8 compute programs + methodology (load the
runtime once, time compile+run of each, min of N) so the zero-Janet Chez path is
comparable to the Janet compile path.

Result: Chez is 3-33x faster than Janet across the set (~320ms vs ~5000ms total,
~15x), biggest on seq/map/reduce, smallest on fib (3.2x). So deleting Janet is a
perf win, which satisfies the "perf confirmed" precondition Phase 5 gates on. Chez
is ~2-28x slower than JVM Clojure, expected and not the bar (and the Janet-only
optimizing modes haven't been tried on Chez).

jolt-cf1q.5
2026-06-20 14:30:36 -04:00
Yogthos
c719e54543 Chez concurrency pt.3: async agents (per-agent serialized dispatch)
Replace the Janet synchronous agent shim (agent = atom, send applies inline) with
JVM-style async agents: send/send-off enqueue an action and a single worker thread
per agent applies them in order; deref reads the current (maybe not-yet-updated)
state without blocking; await blocks until the queue drains. A validator rejection
or a thrown action puts the agent in an error state (agent-error) and halts the
queue; restart-agent clears it. send and send-off share one serialized worker (a
superset of the JVM's fixed/cached pool split). Native versions re-asserted in
post-prelude over the overlay; await/restart-agent are new.

Corpus: the two "send/send-off applies" cases do (send a f) (deref a) with no
await, so they now read state before the action runs — diverging like the JVM
(the suite was literally "synchronous shim"). Allowlisted on both gates; floors
-2 (zero-Janet 2569->2567, prelude 2559->2557). cli-test covers async agents via
await (ordered 100-send dispatch, error capture) — 49/49. Janet gate + JVM cert
green; 0 new divergences on either corpus.

jolt-byjr
2026-06-20 14:22:37 -04:00
Yogthos
48ed72974f Chez concurrency pt.2: clojure.core.async on real-thread blocking channels
No mature Chez fibers library exists and this is a threaded Chez build, so a go
block is an OS thread and a channel is a mutex+condition blocking queue: <! / >!
are the blocking <!! / >!! and work anywhere (no CPS transform), like the Janet
stackful-fiber model but with real parallelism and a shared heap.

host/chez/async.ss provides chan (unbuffered rendezvous / fixed / dropping /
sliding), <! >! <!! >!! close! alts! timeout put! take! buffer ctors, channel
transducers, and go-spawn, all def-var!'d into clojure.core.async; go/go-loop/
thread are macros (mark-macro!) expanding to go-spawn, mirroring src/jolt/
async.janet. Binding conveyance rides the thread-parameter binding stack from
pt.1. alts! polls with a 1ms backoff (no cross-channel wait-set yet) and is
take-only, matching the Janet impl.

(require '[clojure.core.async ...]) resolves it with no file load — the vars are
resident and require just registers the :as/:refer.

cli-test covers go/buffered-drain/nested-<!/alts!/transducer/timeout/binding-
conveyance (43/43). core.async isn't in the conformance corpus (feature-gated
:async/core-async), so coverage is the Chez cli-test plus the existing Janet
core-async-spec. Seed unchanged (no .clj touched). Prelude corpus 2534->2559,
zero-Janet 2569, 0 new divergences on either; Janet gate + JVM cert green.

jolt-byjr
2026-06-20 13:48:10 -04:00
Yogthos
ec30c9e405 Chez concurrency pt.1: real OS-thread futures + blocking promises (shared heap)
future/future-call run the body on a native thread (fork-thread) over the SAME
heap — JVM semantics, not Janet's isolated-heap snapshot. deref blocks on a
mutex+condition latch; timed (deref f ms val) uses an absolute deadline.
promise is a real blocking promise (deref parks until deliver), replacing the
Janet non-blocking atom shim. future?/future-done?/future-cancelled?/future-cancel
/realized? are native (the overlay versions read Janet map keys); re-asserted in
post-prelude over the overlay. pmap/pcalls/pvalues (overlay, over future) light
up for free.

Thread-safety this forces:
- atoms get a per-atom mutex; swap!/swap-vals! are a JVM-style CAS loop (f runs
  outside the lock, so a watch/validator can deref the same atom); reset!/
  compare-and-set! are atomic.
- the dynamic binding stack becomes a Chez thread-parameter, so each future/thread
  has its own; Chez inherits it at fork, giving binding conveyance (the shim also
  installs an explicit snapshot).
- Thread/sleep really sleeps now (a worker sleeping doesn't block the parent).

Re-minted the seed: future-call now resolves at compile time, so pmap compiles to
a var-deref instead of the host-static-call fallback that crashed. image.ss
unchanged.

Corpus: the 2 snapshot cases now match the JVM (shared) not Janet (isolated) —
allowlisted on both Chez gates; the two racy future-cancel cases allowlisted;
"promise undelivered" (blocks on JVM/Chez, profile :bucket :timeout) skipped like
:throws. Zero-Janet corpus 2544 -> 2569, 0 new divergences, floor raised. Full
Janet gate + JVM cert green.

jolt-byjr
2026-06-20 13:11:31 -04:00
Yogthos
a23095b502 Chez: runtime eval / load-string / defmacro on the zero-Janet spine
The compiler image is already resident at runtime on the Chez spine, so eval
and load-string are just wiring: make them clojure.core functions instead of
analyzer special forms.

- eval / load-string are now functions, not special forms. Dropped "eval" from
  the host-contract special-symbol lists so it resolves as an ordinary var, and
  def-var! both in compile-eval.ss. eval takes an already-read form (e.g. from
  quote/list) and compiles+evals it in the current ns; load-string reads every
  form from a source string and evals each, returning the last.
- Runtime defmacro: jolt-compile-eval-form intercepts a (defmacro ...) form
  before analysis, defs the expander fn + mark-macro!s the var, exactly as
  emit-image.ss does at build time. The two helpers (macro-form? / defmacro->fn)
  move to compile-eval.ss and emit-image.ss reuses them.
- Top-level (do ...) is now unrolled form-by-form, like Clojure, so a defmacro
  or def in an earlier subform is visible (macro flag set / var interned) before
  a later subform is analyzed. This is what makes multi-form -e with a macro work.

Seed is byte-identical (no source references eval), so no re-mint; bootstrap-test
still passes. Zero-Janet corpus 2534 -> 2544 (eval/load-string cases now run),
0 new divergences; floor raised. Prelude corpus, JVM cert, full Janet gate green.

jolt-r8ku
2026-06-20 12:14:08 -04:00
Yogthos
a8b61283b3 Handoff: remaining blockers to removing Janet (Phase 4 -> Phase 5)
Next-session pickup doc. Self-host + conformance spec done; Janet removal gated on
runtime eval/macros (jolt-r8ku, do first), concurrency (jolt-byjr), host interop
(jolt-cf1q.7), deployment/perf (jolt-cf1q.5), then oracle-off-Janet + Phase 5
delete (jolt-cf1q.6). Includes constraints, verify commands, per-blocker approach
with file/line pointers, sequencing, and a file map.
2026-06-20 11:29:50 -04:00
Yogthos
6abbea3835 Fix 4 clojure.core bugs surfaced by JVM certification
The corpus certifier (test/conformance) flagged four cases where jolt's
hand-written :expected matched a real defect rather than Clojure. Fixed in the
jolt-core overlay, corrected the spec :expected, re-certified against JVM Clojure:

- ex-message: returns nil for a non-throwable (dropped the lenient string branch);
  still returns the message for ex-info. (jolt-l8e8)
- munge: preserves the argument's type — a symbol munges to a symbol, not a string.
  (jolt-hc35)
- print: (print nil) emits "nil", not "" (top-level nil guard; str yields "").
  (jolt-pqio)
- bounded-count: uses the counted? fast path (full count), else counts up to n via
  seq — was (min n (count coll)), wrong for counted colls. Added an uncounted-coll
  spec case. (jolt-2507)

Removed the 4 :bug entries from known-divergences.edn (now certified), regenerated
corpus + profile, re-minted the Chez bootstrap seed (clojure.core changed). Gates:
Janet 155/0, JVM certify clean, both Chez corpus gates 2534 (floors raised),
bootstrap 6/6, fixpoint intact.
2026-06-20 11:06:33 -04:00
Yogthos
f54e99cc08 Conformance inc3: promote the corpus to a documented portable spec
Makes the host-neutral corpus a first-class language specification with
conformance levels, not just a regression suite.

- [suite label] is now a unique, stable case id (extract-corpus disambiguates
  duplicate labels with ' (N)' — one collision existed).
- certify.clj --profile emits test/conformance/profile.edn: every non-portable
  case classified by the host feature it requires (numerics/double-only,
  concurrency/snapshot, host/jvm-interop, host/arrays, host/janet,
  async/core-async, runtime/eval, reader/jolt, printer/jolt, strictness/jolt,
  impl/representation, bug). 2670 of 2919 cases are portable (pass on any faithful
  Clojure); 249 are feature-gated.
- SPEC.md documents the contract: row schema, the JVM oracle, conformance levels,
  the feature vocabulary, and a worked new-runtime harness — so hosting jolt
  elsewhere and proving it correct is read-one-file mechanical.

Janet gate 155 files 0 failed; certify + zero-janet gates green.
2026-06-20 10:21:09 -04:00
Yogthos
635bbbcc27 Conformance inc2: fold the 355 inline conformance cases into the corpus
The corpus only ever saw test/spec/*-spec.janet; the 355 hand-written cases in
test/integration/conformance-test.janet (inline Janet, the lazy-seq / IFn /
destructuring / transducer essentials) were invisible to it and to any non-Janet
runtime. extract-corpus.janet now also pulls that (def cases ...) vector, deduped
by :actual, organized into 41 'conformance / <section>' suites recovered from the
file's ### headers. Corpus 2658 -> 2919 rows (+261 unique).

JVM certification: only 1 new divergence ((/ 2) => 0.5 vs 1/2, the all-double
numeric model) — classified. Chez gates: +1 known host gap (instance? Atom, atom
class identity, Phase 4) allowlisted in both runners; parity rose 2295 -> 2533 on
both, floors raised. Janet gate 155 files 0 failed; certifier green (0 new/stale).

Deferred: 41 non-literal core-async spec rows ((a "src") async-harness wrapper)
need harness context the corpus format doesn't carry — left for inc3.
2026-06-20 10:10:32 -04:00
Yogthos
b520eefa7a Conformance inc1: JVM-certify the corpus against reference Clojure
The corpus carried hand-written :expected values — a regression suite but a weak
spec (it checked jolt against its authors, not against Clojure). certify.clj runs
every corpus row's :actual and :expected through reference JVM Clojure 1.12.5 (fresh
user ns per case, output/stdin sunk, 5s per-case watchdog) and compares with =.

Result: of ~2487 vanilla-certifiable rows, 2416 match real Clojure exactly. The 71
divergences are all classified in known-divergences.edn — mostly deliberate
jolt-specific/host-model deltas (all-double numerics, snapshot concurrency, no-JVM
host model, jolt reader features, printer, strictness), plus 4 genuine bugs filed
as beads (jolt-l8e8 ex-message, jolt-hc35 munge, jolt-pqio print-nil,
jolt-2507 bounded-count).

certify-test.janet gates it: skips without clojure on PATH, else fails only on a
NEW (unclassified) divergence or stale allowlist entry; flaky timing-dependent
cases (future-cancel) tolerated either way. Full gate 155 files 0 failed.
2026-06-20 09:38:31 -04:00
Yogthos
d0fce540ed Chez Phase 3 inc9b: pure-Chez runtime CLI (bin/joltc, no Janet)
The runtime counterpart to bootstrap.ss. host/chez/cli.ss loads the checked-in
seed + the zero-Janet spine and compiles+evals a -e expression entirely on Chez;
bin/joltc execs it. With the seed checked in, a clone runs jolt with only Chez
installed — no Janet at build or run time. Multi-form -e wraps in (do ...) to
match Clojure. test/chez/cli-test.janet 9/9.
2026-06-20 06:53:05 -04:00
Yogthos
2c74476aed Chez Phase 3 inc9a: self-contained Chez bootstrap (no Janet in the loop)
Makes the inc8 fixpoint the actual build. host/chez/bootstrap.ss loads a seed
(prelude, image) pair and rebuilds the clojure.core prelude + compiler image from
source via the on-Chez compiler — read/analyze/emit all on Chez, zero Janet.

The seed (host/chez/seed/{prelude,image}.ss) is the checked-in bootstrap
compiler, minted once via the fixpoint (driver/mint-chez-seed* iterates
bootstrap.ss from the Janet-emitted pair to a joint byte-fixpoint). It's a joint
fixpoint: rebuilding from an up-to-date seed reproduces it exactly. So a fresh
checkout + Chez (no Janet) yields a working jolt.

test/chez/bootstrap-test.janet spawns only chez, asserts the rebuilt artifacts
match the seed byte-for-byte and compile+run real cases. Drift (seed sources
changed) fails the test with a re-mint pointer; host/chez/seed/README documents
re-minting.
2026-06-20 06:46:05 -04:00
Yogthos
81e587b2d7 Chez Phase 3 inc8: prelude fixpoint + fully-Chez-emitted system
Extends the fixpoint beyond the compiler image to the whole emitted system.
emit-image.ss now handles macros (defmacro -> bare expander fn + def-var! +
mark-macro!) and re-emits the clojure.core prelude (all tiers + stdlib) on Chez
via jolt-emit-prelude; driver's emit-image-on-chez takes an emit-fn arg.

The prelude converges at pstage3==pstage4 (one stage later than the compiler's
stage2==stage3) because macro expanders bake an auto-gensym id at emit time, so a
Janet-emitted macro carries a different id than a Chez-emitted one — only once
both stages load a Chez-emitted prelude does it stabilize.

fixpoint-test now proves: compiler stage2==stage3, prelude pstage3==pstage4, and
the fully Chez-emitted system (Chez prelude + Chez image, no Janet artifact in the
loop) compiles+runs real cases. 10/10.
2026-06-20 05:21:23 -04:00
Yogthos
509c23f06d Chez Phase 3 inc8: stage2==stage3 compiler-image fixpoint
The zero-Janet spine proves the on-Chez analyzer/emitter compile arbitrary
Clojure faithfully. This proves the stronger property: the on-Chez compiler
reproduces itself. emit-image.ss re-emits the compiler sources (jolt.ir +
jolt.analyzer + jolt.backend-scheme) ON CHEZ via the loaded image; feeding it
stage1 (the Janet cross-compile) yields stage2, feeding stage2 yields stage3.
stage2 and stage3 are byte-for-byte identical, and stage2 is a working compiler
(real cases compile+run through it). stage1 differs from stage2 only in gensym
numbering, so the fixpoint is stage2==stage3.

driver: emit-image-on-chez / program-emit-image spawn a fresh chez per stage
(clean gensym state). test/chez/fixpoint-test.janet gates it (skips without chez).
2026-06-20 05:08:30 -04:00
Yogthos
24ef2b8d4b Chez inc7: :refer support — Chez analyzer reaches parity with the oracle
(require '[clojure.string :refer [blank?]]) then an unqualified blank? now
resolves. chez-register-spec! registers :refer names (in addition to :as) into a
refer table; hc-resolve-cell's unqualified branch consults it before clojure.core.

Zero-Janet corpus parity 2293 -> 2295 = the Janet-hosted oracle's exact pass
count. The self-hosted Chez compiler (read -> analyze -> emit -> eval, no Janet)
now compiles every corpus case the Janet-hosted compiler does, with 0
divergences. Remaining failures are shared runtime breadth (host interop,
futures, runtime eval) deferred to Phase 4 / jolt-r8ku. Floor 2295.
2026-06-20 02:06:27 -04:00
Yogthos
28bb950efe Chez inc7: batched runner handles multi-line case sources
The batched zero-Janet runner wrote one case per TSV line, so a multi-line
source (e.g. a ;comment\n inside a map literal) split the line and the case was
truncated -> a false "apply non-procedure" crash. Escape \n/\t/\\ when writing
the cases file and unescape in the runner before eval.

Zero-Janet corpus parity 2288 -> 2293 (the 5 comment-in-map cases), 0
divergences — now within 2 of the Janet-hosted oracle (2295). Floor 2293.
2026-06-20 02:01:42 -04:00
Yogthos
7d0070d873 Chez inc7: reader features — ##Inf/##NaN, #(...), #?(...), ns :require aliases
Reader gaps the Chez-hosted analyzer hit where the Janet reader didn't:
- ##Inf / ##-Inf / ##NaN symbolic literals (## dispatch -> flonum).
- #(...) anonymous fn shorthand -> (fn* [p__N#] body), with % / %N / %& and the
  max-positional arity rule; scans + rewrites list/vector/set/map bodies.
- #?(...) reader conditional: feature set {:jolt :default}, first matching clause
  wins. #?@ splicing not yet supported (one niche case allowlisted).
- (ns name (:require [a :as x])) — the require pre-scan now also reads aliases
  from an ns form's :require/:use clauses, not just bare (require ...).

Zero-Janet corpus parity 2240 -> 2288, 0 divergences (2 now-reachable cases
allowlisted: str of Infinity inside a collection — same as the prelude gate —
and #?@ splice). spine-test 35/35; prelude parity 2295 unchanged, 0 new
divergences.
2026-06-20 01:57:36 -04:00
Yogthos
011e5d6337 Chez inc7: namespace :as aliases on the Chez-hosted analyzer
(require '[clojure.string :as s]) then s/foo crashed "Unknown class s": the
Chez analyzer resolved a qualified symbol's ns literally, and there was no
alias table (ns.ss jolt-ns-aliases was a stub, require a no-op). Add an alias
table + chez-register-spec! (parses [ns :as a]) in ns.ss; hc-resolve-cell now
resolves a qualified ns through it; compile-eval pre-registers a form's
require/use :as aliases before analysis (analysis precedes the runtime require,
so the runtime require staying a no-op is fine). The batched gate runner clears
the alias table between cases.

Zero-Janet corpus parity 2159 -> 2240, 0 divergences. spine-test 35/35.
2026-06-20 01:25:01 -04:00
Yogthos
8b86174b91 Chez Phase 3 inc7: corpus on the Chez-hosted analyzer + a 50x-faster gate
Point the Chez-HOSTED analyzer at the full parity corpus (read -> analyze ->
emit -> eval, all on Chez, no Janet) and close the divergences so the
self-hosted compiler is faithful: 0 divergences, 2159/2494 pass.

Keystone: the on-Chez emitter ran with prelude-mode off, so every call to a
non-native clojure.core fn tripped the "unsupported stdlib fn" out-of-subset
guard. The zero-Janet spine always has the full prelude loaded, so turn
prelude mode on in compile-eval.ss (22% -> 84% pass on a sample).

Faithfulness fixes (each was the Chez host/reader diverging from the Janet
analyzer; fixed in the keeper, not the seed):
- emit-const read a char's codepoint via (get v :ch) — the Janet rep; on Chez a
  char is native. Route through a new form-char-code host-contract fn (41 cases).
- next over a lazy seq returned the empty-list terminator (truthy), not nil, so
  butlast and other (if (next s) ...) loops ran one step too far — broke
  some->/some->>/cond->>.
- reader: radix literals (2r1010/16rFF/36rZ), #^ deprecated metadata, ^meta on
  collections (lowers to a runtime with-meta form like the Janet reader),
  map-literal source order (values eval left-to-right), and nested syntax-quote
  over a literal collapses at read time.
- keyword "a/b" splits into ns/name like the seed (destructure {:keys [x/y]}).
- form-syntax-quote-lower implemented on Chez (was a throwing stub).

7 divergences allowlisted: the same print-method-multimethod / host-class set
the prelude gate defers. 328 crashes remain = shared runtime breadth (host
interop, missing core fns, eval/load-string) deferred to Phase 4 / jolt-r8ku,
not compiler faithfulness.

Gate + speedup: test/chez/run-corpus-zero-janet.janet (floor 2159). Its batched
runner (driver/eval-corpus-zero-janet) runs every case in ONE chez process —
load the runtime once, guard + reset the user namespace per case — instead of a
fresh process per case: 1379s -> 1.6s.

spine-test 35/35; Janet gate 151/0; prelude parity 2295/2494 unchanged, 0 new
divergences.
2026-06-20 01:11:54 -04:00
Yogthos
d165198969 Chez Phase 3 inc6b: runtime macros on the zero-Janet spine
The on-Chez analyzer (inc6a) skipped macros, so let/when/->/defn didn't
expand from source. Each core/stdlib defmacro now emits into the prelude as
(def-var! ns name <expander fn>) + (mark-macro! ns name); form-macro?/
form-expand-1 on Chez look up the macro flag (rt.ss var-macro-table) and
apply the expander to the unevaluated arg forms, and the analyzer re-analyzes
the result. The expander's syntax-quote template was lowered to construction
code at cross-compile time, so it builds the expansion via __sqcat/__sqvec/
__sqmap/__sqset/__sq1 (new host/chez/syntax-quote.ss) as Chez reader forms.

Emit the bare (fn ...), not (def NAME (fn ...)): analyzing a def would
host-intern! NAME as a non-macro stub in the build ctx, and that stub makes a
later (require '[stdlib-ns]) skip loading the real macro — with-pprint-dispatch
then resolved as a fn and returned its unexpanded template. Wrapping the
lambda in def-var! manually never interns NAME. Fuller build-ctx isolation
(so stdlib cases pass instead of crash) tracked in jolt-lpvi.

__sqset builds a real set VALUE, not the reader's tagged-set form — a runtime
`#{~@xs} must be a set, not a map. form-set? additionally recognizes a pset so
a macro template's #{...} expansion still re-analyzes as a set literal.

spine-test 35/35 (20 macro cases: when/when-not/let/->/->>/and/or/cond/if-not/
defn run zero-Janet from source). Prelude parity 2280->2295, 0 new divergences.
Full Janet gate green.
2026-06-19 23:21:13 -04:00
Yogthos
c2d977cadf Chez Phase 3 inc6a: zero-Janet spine (analyzer + emitter on Chez), macro-free
Cross-compile jolt.ir + jolt.analyzer + jolt.backend-scheme to Scheme def-var!
forms via the existing Janet emit pipeline (driver/emit-compiler-image) and run
them ON CHEZ over a Scheme jolt.host impl. A macro-free Clojure expression now
compiles and runs with no Janet in the loop: read (reader.ss) -> analyze
(jolt.analyzer on Chez) -> IR -> emit (jolt.backend-scheme on Chez) -> eval.

host/chez/host-contract.ss is the jolt.host contract on Chez (the portable seam
the cross-compiled analyzer/emitter call): form-* over the Chez reader's forms,
resolve-global/compile-ns/host-intern!/late-bind? over the var-cell registry. ctx
is an opaque record carrying the compile ns. Native-op names are declare-var!'d
into clojure.core so +, *, <, ... classify as :var and the emitter's native-op
path lowers them. form-macro? is a #f stub and macro expansion / syntax-quote /
record hints are stubs for inc6b (runtime macros, jolt-r8ku).

host/chez/compile-eval.ss is the spine entry (read-string -> analyze -> emit ->
eval). driver gains emit-compiler-image / ensure-compiler-image (image caching)
and program-zero-janet / eval-zero-janet.

Two bugs fixed in the keeper, not reproduced:
- the Chez reader stores an unqualified symbol's ns as #f, but the analyzer tests
  (nil? ns); hc-sym-ns normalizes #f/'() -> jolt-nil. Without it every handled
  special (if/do/fn*) misanalyzed as a plain invoke.
- char (int->char) was missing from clojure.core on Chez; the emitter's
  chez-str-lit needs it for keyword/string consts. Added jolt-char to converters.ss.

Gate: test/chez/spine-test.janet 15/15 (Chez-hosted analyzer value == Janet-hosted
oracle through the same emitter/RT; only the analysis host differs). Full Janet
gate green (150 files). driver.janet is in the prelude fingerprint so the cache key
moved; prelude content is unchanged. jolt-chez fingerprint/ensure-prelude made
public for the test harness.
2026-06-19 21:16:24 -04:00
Yogthos
c47ae7fd22 Chez Phase 3 inc 5d: add jolt.reader/read-all; reader port logic complete
read-all reads every top-level form of a string (the parse-all the compile path
needs in inc 6). With this the portable reader covers everything reader.janet does
(atoms 5a, collections+quote/meta 5b, dispatch 5c, multi-form 5d).

Validated: reader-parity 149/149 (every construct); jolt.reader compiles and runs
on build/jolt ((require [jolt.reader]) (read-all …) works natively). The
interpreted reader overflows the eval stack on large/deeply-nested files, so real-
file/full-corpus validation happens when it's cross-compiled and run on Chez
(inc 7). Position tracking (checker) and the actual wire-in replacing reader.ss are
deferred — they ride inc 6 (jolt.reader on Chez). See new beads.
2026-06-19 20:26:27 -04:00
Yogthos
b12cb3c00b Chez Phase 3 inc 5c: reader dispatch (#) in jolt.reader
Ports the full # dispatch to the portable reader: #{} sets, #() anon-fns, #?/#?@
reader-conditionals, #_ discard, #' var-quote, #"" regex, #inst/#uuid/#tag tagged
literals, ## symbolic (Inf/-Inf/NaN), and #^ deprecated metadata. With this the
reader is feature-complete except position tracking + wire-in (inc 5d).

Reader-conditionals resolve clause-order against a portable feature set (atom
#{:jolt :default}); #? -> :skip / :form, #?@ -> :splice (the control protocol from
5b). #() uses the two-pass %-scan (collect indices, then rebuild replacing %N/%/%&
with gensym params) over the form tree via the jolt.host form-* contract. Three
host constructors added: form-make-set, form-make-tagged, form-gensym-name.

reader-parity 149/149. #() compares modulo gensym (canonicalize #-suffixed param
names by first-occurrence order — the two readers gensym different names but the
structure + %-mapping must match). ##NaN checked by the NaN!=NaN property. Full jpm
gate green (prelude pre-warmed). jolt-9ufe.
2026-06-19 20:00:57 -04:00
Yogthos
af64454f4e Chez Phase 3 inc 5b: reader collections + quote/deref/meta in jolt.reader
Ports list/vector/map literals and the quote family (' ` ~ ~@ @) + metadata (^)
to the portable Clojure reader. read-form now returns a [kind payload pos] control
triple (:form / :skip / :splice) instead of the Janet reader's :jolt/skip sentinel
FORMS — out-of-band control is collision-free and host-neutral (no tagged struct
to build or recognize). read-delimited dispatches the kinds; read-next-form skips
comments where a single datum is needed; read-map pairs k/v skipping trivia in
either slot. syntax-quote of a self-evaluating literal collapses at read time.

Four host constructors added to the contract (host_iface): form-make-list/vector/
map + form-sym-merge-meta (attach ^meta to a symbol). form-make-map reuses the
seed's reader-map (now public) for the source-order kv tracking. The portable
reader accumulates items in a jolt vector and the host builds its native form rep.

Gate: reader-parity 107/107 (lists/vectors/maps incl. nested + comments-in-coll,
quote/syntax-quote-collapse/unquote/deref, ^:dynamic/^Type/^{} meta). Full jpm gate
green (prelude cache pre-warmed — a cold cache races under the parallel gate when
the jolt-chez fingerprint changes; pre-existing, see new bead). jolt-sh1n.
2026-06-19 19:50:38 -04:00
Yogthos
afada6d4ff Chez Phase 3: fix +N reader literal in jolt.reader only (not the doomed Janet seed)
fix-bugs-dont-reproduce, scoped per the keeper rule: jolt-if19 (a leading + on a
numeric literal errored instead of reading as the positive number) is fixed in
jolt.reader (read-number* now strips a leading + like -, positive), the code we
keep. The Janet seed reader (reader.janet) is left untouched — it's deleted in
Phase 5, so fixing it is wasted work.

Since the seed reader stays buggy, reader-parity can't use it as the oracle for
these inputs: added check-correct to assert the portable reader against the hand-
verified value (+5 => 5, +42, +0xff => 255, +3.5). reader-parity 67/67. No Janet
binary/gate impact (jolt.reader is not yet in the binary path). jolt-if19.
2026-06-19 19:29:53 -04:00
Yogthos
4de586089c Chez Phase 3 inc 5a: port reader atom layer to jolt.reader (Clojure)
Starts taking the reader off Janet (src/jolt/reader.janet, 831 lines) into
portable jolt-core Clojure. jolt.reader holds the lexing/parsing LOGIC; form
construction + string->number parsing delegate to the jolt.host contract — a
Clojure source file can't write a {:jolt/type :symbol} literal (parses as a
tagged form) and the concrete representation is the host's to own. Same split the
analyzer/emitter already use. Once cross-compiled this runs on Chez so compile-
from-source needs no Janet reader.

inc 5a = the atom layer: whitespace/comments, symbols (+ nil/true/false),
keywords, strings (escapes), numbers (sign/hex/radix/ratio/fractional/exponent,
trailing N/M), characters. Collections, quote/deref/meta and dispatch (#) follow
in 5b/5c (throw not-yet-ported). Positions are char indices (Janet uses bytes);
identical for ASCII and the gate compares form VALUES, not positions.

host_iface.janet gains four reader primitives on the contract: form-make-symbol,
form-make-char, form-char-from-name, form-scan-number (the irreducible host bits
the portable reader rests on). Additive — new jolt.host interns, nothing else
changed.

Surfaced jolt-if19 (Janet seed reader: +N literals error instead of reading as N;
read-number strips only the - sign). The port reproduces it; both-throw counts as
faithful parity in the gate.

Gate: reader-parity 64/64 (symbols/keywords/strings/ints/hex/radix/ratio/floats/
exponent/N-M/chars). Full jpm gate green after clean rebuild, conformance 355x3.
jolt-50xx.
2026-06-19 19:17:17 -04:00
Yogthos
2a33da87d8 Chez Phase 3 inc 4: swap jolt-chez to the jolt.backend-scheme emitter; full corpus parity
driver.janet now compiles IR via the portable Clojure emitter (jolt.backend-
scheme) instead of emit.janet, at every entry point (compile-program, emit-core-
prelude, eval-e-with-prelude). The emitter is loaded into the ctx and called like
the analyzer. emit.janet stays only as the emit/program string-wrapper until
program assembly ports to Clojure with compile-from-source; its emit fn is no
longer called anywhere (emit-test's truthy-elision helper now uses the new
d/scheme-emit too). This takes the IR->Scheme emitter off Janet.

A form-by-form diff of the two emitters over the whole prelude found one gap:
emit-const missed char literals because a :jolt/type-tagged struct is not a plain
jolt map? — switched to the form-char? host contract. Diff then 0.

jolt-chez prelude fingerprint now includes backend_scheme.clj + host_iface.janet.

Gate: full prelude corpus 2280/2494, NEW divergence 0, same buckets as the Phase-2
emit.janet floor (36 emit-fail, 170 crash) — the Clojure emitter is byte-for-
behavior identical. emit-test 331/331 (now via the Clojure emitter), emit-parity
58/58. jolt-duot.
2026-06-19 18:52:33 -04:00
Yogthos
f9a665b3c4 Chez Phase 3 inc 3: try/throw + host-call + regex/inst/uuid + def-meta in jolt.backend-scheme
Completes the op coverage of the portable Clojure emitter — it now handles every
op emit.janet does (const/local/var/the-var/host/host-static/host-new/if/do/
invoke/vector/set/map/quote/throw/try/regex/inst/uuid/host-call/let/loop/recur/
fn/def). Adds emit-try (guard + dynamic-wind), :throw, :regex/:inst/:uuid, and
:host-call (jolt-host-call for rt-shimmed methods else record-method-dispatch).

def-meta + quoted-symbol-meta needed emit-quoted to reconstruct plain jolt VALUES
(metadata maps), not just reader forms. The blocker was that :meta arrived as a
raw Janet table embedded in the IR — jolt's count/map?/keys don't work on a table
(counter to jolt.ir's 'no host values embedded'). Fixed at the host seam:
h-sym-meta now returns the meta as an immutable struct, which is a portable jolt
map (jolt count/map?/keys work on a struct, and the Janet backend's merge/get
still do too). emit-quoted handles both reader forms (jolt.host form-* contract)
and jolt-value collections (native map?/vector?/set?/seq? branches, after the
form-* branches so reader forms win).

Gate: emit-parity 55/55 (incl try/catch/finally, ^:private def-var-with-meta!
structural check, inst/uuid eq, regex smoke, quoted-sym-meta). Full jpm gate
green after clean rebuild (seed change). jolt-me6m.
2026-06-19 18:16:36 -04:00
Yogthos
206fe94a95 Chez Phase 3 inc 2: quote + collection literals in jolt.backend-scheme
Adds :vector/:map/:set (emit-ordered, left-to-right element eval) and :quote to
the portable Clojure emitter. Collection-literal nodes carry already-analyzed IR
items so they just recurse; quote walks the RAW reader form.

emit-quoted walks the reader form via the jolt.host form-* contract (form-list?/
form-elements/form-vec-items/form-map-pairs/form-set-items/form-sym-*), the same
portable seam the analyzer uses — not host-native predicates, so it works
unchanged whether the form came from the Janet reader or the Chez reader. Reader
forms are raw host representations (Janet list=array, vec=tuple), so native
list?/vector? would not see them; the contract is the correct abstraction and
keeps the emitter host-neutral. Quoted-symbol metadata and def-meta still defer
to inc 3.

Surfaced a latent Janet-host bug (jolt-tg9s): (quote #{...}) evaluates to the raw
reader form instead of a reconstructed set, so (contains? (quote #{:p :q}) :p) is
false on build/jolt. The Chez emitter is correct (real Clojure: true); the parity
test asserts the verified value for those two cases.

Gate: emit-parity 42/42 (incl vector/map/set literals, coll/kw-as-fn, quoted
list/vec/map/set/symbol/nested). emit-test 331/331, conformance 355x3. jolt-7jvp.
2026-06-19 18:01:22 -04:00
Yogthos
d6847b3ba4 Chez Phase 3 inc 1: port emit.janet core ops to jolt.backend-scheme (Clojure)
First spine increment of self-hosting the compiler on Chez. The IR->Scheme
emitter is host/chez/emit.janet (Janet); to get the analyzer emitting its own
code on Chez with no Janet, the emitter logic has to be portable Clojure that
cross-compiles and runs on Chez itself.

jolt-core/jolt/backend-scheme.clj ports the core ops: const/local/var/the-var/
if/do/let/loop/recur/invoke (+ native-ops)/fn/def, plus the chez-str-lit/flonum/
munge/truthy-elision helpers and prelude-mode. Output is Scheme source text, op-
for-op with emit.janet. recur-target/known-procs are dynamic vars (auto-restore,
no throw-leak). Quote, collection literals, try/throw, host interop, regex/inst/
uuid and program assembly come in later increments (they throw not-yet-ported).

Gate: test/chez/emit-parity.janet loads the Clojure emitter interpreted on the
Janet host and runs each case through it -> Chez -> compares to the Janet CLI
oracle. 18/18 incl fib, factorial loop, multi-arity, variadic, higher-order,
#() shorthand, the mandelbrot kernel. emit-test 331/331 (emit.janet path
untouched), conformance 355x3. jolt-hg7z.
2026-06-19 17:34:25 -04:00
Yogthos
109bfcd09d Chez Phase 2: clojure.set + clojure.math + pprint (jolt-j5vg, jolt-22vo)
Close the remaining Phase-2 stdlib parity gaps.

clojure.set: pure Clojure over core, so just added to the prelude stdlib tier
(driver.janet stdlib-ns-files + jolt-chez fingerprint), same as clojure.edn.

clojure.math: not a .clj on the seed (native math/ bindings via jolt-h79), so
Chez gets its own host/chez/math.ss def-var!ing each fn over Chez native flonum
math. The analyzer already knows the ns (api.janet install-clojure-math!), so
refs lower to var-deref. Also adds the missing 'long' coercion to converters.ss
(int's sibling; several math cases wrap in long).

clojure.pprint: dropped the 2-arity's (binding [*out* writer] ...). *out* isn't
a bindable var in jolt — printing routes through the host (dyn :out) seam, so
the binding never redirected anything; it only made the defn uncompilable, which
the seed tolerated via interpreter fallback. Chez has no fallback, so the whole
pprint defn died. Dropping it is behavior-preserving (writer was always ignored)
and lets pprint compile cleanly. Both corpus cases pass.

Corpus parity 2259 -> 2280, crashes 191 -> 170, 0 new divergences. Floor raised.
New unit test test/chez/_stdlib.janet (27/27). Full Janet gate green (147 files).
2026-06-19 17:06:26 -04:00
Yogthos
6df60229b0 Chez Phase 2 (inc Y): runtime data reader + clojure.edn (jolt-r8ku)
Chez-side recursive-descent Clojure reader (host/chez/reader.ss) producing the
same jolt forms the Janet reader yields, behind the read-string / __parse-next /
__read-tagged seams the seed registers in eval_runtime.janet. That lights up the
whole *in* read family — read, read+string, with-in-str (read) — plus read-string
and read-string metadata, none of which needed an analyzer change (read-string is
a clojure.core seam, jolt-nil on the prelude until now).

Reader output is pinned to the Janet reader's shapes: numbers coerce to flonum
(the all-double model emit-const uses, else a read int isn't = a source int),
sets read as the {:jolt/type :jolt/set :value [...]} FORM, #tag/#inst/#uuid/#regex
as tagged forms (no data reader applied — read-string never evaluates), ^meta on
a symbol, and the ' ` ~ ~@ @ reader macros. clojure.edn is added to the prelude
tier; its edn->value builds the real set/tagged values and __read-tagged reuses
the inc X #inst/#uuid constructors. The reader-arity edn/read stays a lazy gap
(drain-reader is Janet-coupled) — read-string is the live path.

eval / load-string / runtime defmacro are still out: they need the compiler at
runtime, which is Phase 3 (self-host). Chez-only change, no Janet gate.

Parity 2238 -> 2259, 0 new divergences. _reader 47/47; all chez unit tests green;
emit-test 331/331.
2026-06-19 12:06:44 -04:00
Yogthos
c70f3bae34 Chez Phase 2 (inc X): #inst / #uuid literals + java.time (jolt-at0a)
The analyzer lowers a #inst/#uuid tagged form to a :inst/:uuid IR leaf, mirroring
the existing :regex node: the Janet back end punts to the interpreter (its
data-readers parse the literal, so seed behavior is unchanged), the Chez back end
emits jolt-inst-from-string / jolt-uuid-from-string.

host/chez/inst-time.ss is the Chez-native value layer: a jinst record holding
epoch ms (RFC3339 parsed via Hinnant civil/days math, with Clojure's partial
defaults and +/-hh:mm offsets), wired into jolt-get (so the overlay inst?/inst-ms
read it), jolt= / jolt-hash (instant identity as a map key), pr-str (#inst
"...-00:00"), str, type, and instance? java.util.Date. The java.time surface
(DateTimeFormatter ofPattern/ISO_LOCAL_DATE_TIME/ofLocalized*, the pattern engine,
Instant, ZoneId, LocalDateTime, FormatStyle, Locale, Date) ports java_base.janet
over host-static.ss's registries.

Corpus 2202->2238, 0 new divergences; clears the whole 'unsupported form'
emit-fail bucket. Full Janet gate green (analyzer/backend changes are
behaviour-preserving — #inst still parses through the interpreter's data-readers
on the seed).
2026-06-19 11:05:22 -04:00
Yogthos
62e636e52c Chez Phase 2 (inc W): reader-coupled io (jolt-at0a)
clojure.java.io/reader as an in-memory StringReader over slurp/string/char[]/
File; File .toURL/.toURI returning a url jhost (.toString/.getPath); slurp drains
a StringReader; char-array; with-open's __close seam over jhost readers and plain
:close maps. All in host/chez/io.ss (Chez-native, no analyzer change). Corpus
2191->2202, 0 new divergences. clojure.edn/read over a PushbackReader stays
jolt-r8ku (runtime read).
2026-06-19 10:40:19 -04:00
Yogthos
b38a7dd007 Chez Phase 2 (inc V): java.io.File + slurp/spit/flush/file-seq (jolt-yyud)
A File is a path-backed jfile record: (instance? java.io.File f) is true,
str/slurp coerce it to its path, and the File method surface (getName/
getPath/exists/isDirectory/isFile/listFiles/getParent) dispatches through
record-method-dispatch. slurp/spit/flush run over Chez's filesystem
primitives; file-seq's dir primitives (__file?/__dir?/__list-dir) and the
overlay's File branch (.isDirectory/.listFiles, which emit to jolt-host-call)
are jfile-aware. clojure.java.io/file + as-file are def-var!'d natively.

New host/chez/io.ss, a Chez-native implementation -- the seed's
clojure.java.io (io.clj) is a Janet-backed shim over janet.*/janet.file, so
it can't be reused. The analyzer resolves io/file because the seed ctx has
clojure.java.io loaded; only a runtime def-var! is needed. type/instance-
check/str-render/jolt-host-call are extended via the set!-wrap pattern (type
also re-def-var!'d since the var cell captured the old value).

Reader/StringReader-coupled io (io/reader, line-seq over a file, .toURL,
slurp over a reader) deferred to jolt-at0a.

Parity 2176 -> 2191, 0 new divergences. New test/chez/_io.janet 20/20.
2026-06-19 10:02:12 -04:00
Yogthos
77026fa9ec Chez Phase 2 (inc U): list? + map-entry-as-vector + clojure.walk (jolt-75sv)
list? was nil on Chez because one cseq record backs both lists and lazy/
realized seqs. Add a list? marker field (cseq v2) set only on the HEAD cell
of a list -- (list ...), quoted list literals, cons, reverse, conj onto a
list. rest/next/seq/map therefore yield unmarked seq cells, so they are
seqs and not list?, matching the seed (where rest-of-a-list is a non-list
seq). Empty () is treated as a list.

vector?: drop the map-entry exclusion. Clojure's MapEntry implements
IPersistentVector and the seed agrees -- (vector? (first {:a 1})) is true.
Only dot-forms' coll dispatch read jolt-vector?, where a 2-vector entry is
correct.

clojure.walk + clojure.template join the prelude stdlib tier. The driver
now evals each stdlib ns's requires -- and the ns form's (:require ...)
clause -- so an aliased ref (template's walk/postwalk-replace) resolves at
emit time instead of lowering to an Unknown class host-static. ns forms are
evaled for that side effect but not emitted, so the runtime *ns* doesn't
leak to the last stdlib ns.

Parity 2163 -> 2176, 0 new divergences. New test/chez/_walk.janet 39/39.
2026-06-19 09:30:18 -04:00
Yogthos
3319684a38 Chez Phase 2 (inc T): class native + bare class-token resolution (jolt-13zk)
Bare class names (String, Keyword, File...) evaluate to their JVM
canonical-name string, the same value (class x) returns, so
(= String (class "x")) holds and (defmethod m String ...) keys match a
(class ...) dispatch. New host/chez/host-class.ss ports
eval_resolve.janet's class-canonical-names + core_refs.janet's core-class
(scalar arms; collections/seqs are host-taxonomy-dependent and not class-
compared in the corpus).

The analyzer already resolves these names to clojure.core vars (the seed
ctx interns them via setup-class-ctors), so the back end emits
(var-deref "clojure.core" "String") and a runtime def-var! is all that's
needed -- no analyzer change, Janet path untouched. The class native MUST
land together with token resolution: alone it turns the bare-token corpus
cases (562/564) into divergences (this bit last session).

Parity 2154 -> 2163 (cases 560/562/563/564/2500-2503), 0 new divergences.
New test/chez/_class.janet 19/19.
2026-06-19 08:41:45 -04:00
Yogthos
b7864100cb Chez Phase 2 (inc S): atom watches + validators (jolt-mn9o)
The Chez atom record gains watches (an alist of key->fn) and validator
slots. swap!/reset! now validate the candidate value before storing and
notify watches after, in the seed's order (core_refs.janet) — the watch fn
is called (key ref old new). compare-and-set!/swap-vals!/reset-vals! route
through reset!/swap! so they validate + notify too.

add-watch/remove-watch/set-validator!/get-validator are native here and
re-asserted in post-prelude.ss: the clojure.core overlay implements them via
jolt.host/ref-put! on (get atom :watches), a Janet-table mutation a Chez atom
record can't answer, so its def-var! would otherwise clobber these. set-
validator! validates the current value immediately (Clojure throws if already
invalid).

Parity 2150 -> 2154, 0 new divergences. New test/chez/_atomwatch.janet 10/10.

The (class x) native and the clojure.walk port were explored this increment
but deferred: class alone makes the String-class-token corpus cases emit-and-
run with a wrong value (the bare token doesn't resolve yet) — filed jolt-13zk
to land the native together with token resolution; clojure.walk is blocked on
list? + map-entry-as-vector (jolt-75sv).
2026-06-19 04:33:07 -04:00
Yogthos
1f96351acb Chez Phase 2 (inc R): . / .-field dot-form desugar (jolt-kuic)
The analyzer lowers the `.` special form (. target member arg*) and the
.-field field-access head to a :host-call instead of leaving them
uncompilable. Janet behaviour is unchanged — its back end punts :host-call
to the interpreter, which re-runs the original `.` form via eval-dot.

The Chez back end routes a non-shimmed :host-call through
record-method-dispatch, extended by a new host/chez/dot-forms.ss with the
arms dispatch-member covers but the record/string base did not, mirroring
src/jolt/interop/collections.janet precedence:

  - collection interop first (count/seq/nth/get/valAt/containsKey on a
    vector/map/set), so (. {:count 9} count) is the entry count like the seed
  - field access for a "-name" member (records and maps)
  - the seed's universal object-methods (getMessage/getCause/toString/
    hashCode/equals) on a non-record map, winning over a field lookup
  - non-record map member: a stored fn is a method called with self, else
    the field value

Raw seqs are excluded from coll interop — the seed's behaviour there is
representation-dependent (plain (seq v) vs a lazy-seq) and a normalized cseq
can't mirror it. Also added getMessage/getLocalizedMessage/equals to the
string method surface so a thrown string / Exception. ctor (which keeps the
message string) answers .getMessage.

Parity 2134 -> 2150, 0 new divergences. New test/chez/_dotform.janet 26/26;
emit-test 331/331.
2026-06-19 00:32:30 -04:00
Yogthos
c90c4cb610 Chez Phase 2 (inc Q): Java class statics + constructors (jolt-avt6)
Lower host class interop on the Chez back end. The analyzer now turns a
non-var qualified ref `Class/member` into a :host-static node and a
`(Class. ...)` / `(new Class ...)` form into a :host-new node (ir.clj
gains both, with walker support). The Janet back end punts both to the
interpreter, so its behavior is unchanged (verified: dot-form, `..`
threading, shadowed `new`, and all interop still resolve via fallback).

The Chez emit lowers a value ref to host-static-ref, a call head to
host-static-call, and a constructor to host-new. host/chez/host-static.ss
is the runtime registry these resolve against — the Chez port of the
seed's class-statics / class-ctors / tagged-methods (java_base.janet +
host_io.janet), restricted to the java.lang/util/net/io surface portable
cljc code calls: Math, System (getenv/getProperty/exit/currentTimeMillis),
Long, Integer, Boolean, Character, String, Thread, Class, Pattern
(compile/quote/MULTILINE), URLEncoder/Decoder, Base64, the Number method
surface (byteValue/intValue/...), plus the StringBuilder, StringWriter,
StringReader, PushbackReader, HashMap, StringTokenizer, BigInteger,
String, MapEntry, and exception constructors. Constructed objects are
jhost records dispatched through record-method-dispatch.

Also: emit now evaluates collection-literal elements left-to-right
(emit-ordered) — Chez evaluates call args right-to-left, which had been
swapping side-effecting elements in [(read r) (read r)] and map literals.
This un-allowlisted the 6 eval-order corpus cases (the read-line trio +
the three map-construction cases). Removed `.write` from the
jolt-host-call fast-path so a StringWriter routes through dispatch.

java.time formatting, edn/read-over-readers, and slurp/with-open over
readers are deferred to a follow-up.

Corpus parity 2078 -> 2134 (floor raised), 0 new divergences; the
print-method builtin-override case is allowlisted (same multimethod gap,
newly reachable now that StringWriter constructs). emit-test 326/326,
_javastatic 51/51, conformance 355x3, full jpm test green.
2026-06-18 23:24:01 -04:00
Yogthos
a706a79b90 Chez Phase 2 (inc P): clojure.string namespace (jolt-nfca)
Bring the clojure.string namespace up on Chez so aliased refs like s/split,
s/upper-case, s/join, s/replace resolve and run.

Three pieces. (1) The Chez AOT driver analyzes the whole user form before any
require runs, so a (require '[clojure.string :as s]) never registered the
alias in time; eval-e-with-prelude now recursively pre-evals require/use forms
against the ctx, which loads the aliased ns and registers the alias so the
analyzer resolves s/X to a clojure.string var. (2) emit-core-prelude emits
stdlib namespaces (clojure.string) as their own def-var! tier through the same
analyze->emit pipeline, so the runtime var-deref resolves. (3) natives-str.ss
def-var!s the str-* primitives clojure.string.clj is written over (upper/lower/
trim/triml/trimr/find/reverse-b/join/split/replace/replace-all), plus no-op
require/use. Regex split keeps interior empties and honors the limit (ported
the seed re-split); regex replace does $N backref expansion and fn replacement
(ported replacement-for). new RT/clj files added to the prelude fingerprint.

Corpus prelude floor 2026 -> 2078 (+52), 0 new divergences. _strns 28/28 vs
build/jolt. Four previously-CRASHING cases now emit+run and surface pre-existing
gaps (read-line vector eval-order x3, instance? clojure.lang.Atom) — allowlisted
with notes. full jpm test + conformance x3 green.
2026-06-18 21:38:13 -04:00
Yogthos
3ab53ba938 Chez Phase 2 (inc O): host String methods (jolt-nfca)
Port the java.lang.String/CharSequence method surface to the Chez RT so
(.toUpperCase s), (.substring s a b), (.indexOf s x), the regex methods
(.matches/.replaceAll/.replaceFirst/.split), etc. run on a string target.

natives-str.ss holds jolt-string-method, ported from the seed's surface in
eval_resolve.janet: ASCII case mapping (byte-oriented like the seed), -1 on
indexOf miss, flonum numeric returns to match jolt's number model, Scheme
chars for charAt, and the regex methods over the irregex compiled via
jolt-re-pattern. record-method-dispatch gains a string? arm that falls
through to it (unsupported methods still throw).

Corpus prelude floor 2002 -> 2026 (+24), 0 new divergences. _str 27/27 vs
build/jolt; full jpm test + conformance x3 green.

The (. x m) dot-form (the . special form, distinct from .method sugar) and
the clojure.string namespace (needs prelude plumbing + Pattern) stay deferred.
2026-06-18 20:36:39 -04:00
Yogthos
b7158e0690 Chez Phase 2 (inc N): type (jolt-fmm4)
Implement (type x) on the Chez RT (host/chez/natives-meta.ss). Mirrors the
seed's core-type: the :type metadata wins when present, a record yields its
ns-qualified class-name SYMBOL (user.TyR — no-ns sentinel #f so it = the
overlay's (symbol (str t))), everything else a host-taxonomy keyword
(:number/:string/:vector/:map/:set/:seq/:fn/…). Total by construction — a
non-record value falling through to a crash would read as a divergence, so the
cond covers every value type incl. the host wrappers (atom/volatile/regex/var/
transient/uuid -> :jolt/*, a :jolt/type-tagged map like ex-info -> its tag,
sorted-set -> :jolt/sorted-set, sorted-map -> :map) and a final :object.

Also pin sequential?/seq? on lazy seqs (test/chez/_seqpred.janet): the inc M
seq? re-def-var! fix already covers sequential? transitively (sequential? is
overlay and delegates to seq?), so no code change — the earlier "still broken"
note was wrong, it assumed sequential? was native like seq?.

Prelude corpus parity 2000 -> 2002 (the two type cases), floor raised, 0 new
divergences. Gate: _type 37/37 + _seqpred 22/22 (both vs build/jolt oracle),
emit-test 321/321, full jpm test, conformance 355x3.
2026-06-18 19:40:17 -04:00
Yogthos
fc6551f877 Chez Phase 2 (inc M): dynamic var binding (jolt-2o7x)
Add the dynamic-binding cluster on the Chez RT: a per-thread binding stack
(host/chez/dyn-binding.ss) backing binding / with-bindings* / var-set /
thread-bound? / with-local-vars / with-redefs / bound-fn* /
get-thread-bindings / alter-var-root + the __local-var seam.

Frames are stored innermost-first as identity-keyed alists of mutable
(cell . value) pairs, so var-set updates the current binding in place. The
two var-read paths — var-deref (compiled code) and jolt-var-get (var-get /
deref on a cell) — are chained onto the stack so a `binding` frame is seen
by every read, with a fast path when the stack is empty. var-cells now hash
by ns/name so a var works as a map key (with-redefs builds (hash-map (var f)
v); get-thread-bindings returns a var-keyed map).

Fix a latent bug exposed once with-in-str could bind *in*: seq? didn't
recognize a lazy-seq. predicates.ss's jolt-seq? predates the lazyseq record,
and unlike the native-op dispatchers it's reached via var-deref, so the
patch must re-def-var! the var, not just set! the top-level binding.

Note: with-bindings* over a hash-map literal now returns the correct Clojure
value where the seed returns a stale one — the seed's PHM can't find a var
key (which is why its `binding` uses array-map); on Chez frames look up by
cell identity.

Prelude corpus parity 1972 -> 2000, floor raised. Gate: _dynbind 24/24,
prelude corpus 0 divergences, full jpm test, conformance 355x3.
2026-06-18 18:29:55 -04:00
Yogthos
737288bff5 Chez Phase 2 (inc L): var def-time metadata (jolt-zikh)
Capture a def's reader metadata on the Chez var. The :def emit now lowers a def
with non-empty metadata to def-var-with-meta!, which stores the user meta
(^:private / ^Type tag / docstring -> {:doc}) in an eq side-table keyed by the
var-cell. jolt-meta of a var-cell merges that onto {:ns :name} derived from the
cell, so every var reports {:ns :name} like Clojure with the def-time meta
layered on. (^{:map} metadata on a def name stays uncompilable for the compiler
generally — analyzer rejects it, the Janet back end punts to its interpreter,
which Chez lacks — so it's out of subset, not a meta-capture gap.)

Added natives-meta.ss to the prelude-cache fingerprint. Prelude parity
1969 -> 1972, 0 new divergences; the three var-metadata allowlist entries
(^:private / ^Type tag / docstring) dropped. New focused gate
test/chez/_var_meta.janet.
2026-06-18 17:13:46 -04:00
Yogthos
32e2d8bd58 Chez Phase 2 (inc K): namespace value model (jolt-yxqm)
Reimplement the ctx-coupled seed ns natives over the rt.ss var-table, since
Chez has no ctx. host/chez/ns.ss adds a jns namespace value + a registry and
binds find-ns/the-ns/create-ns/in-ns/all-ns/ns-publics/ns-map/ns-interns/
ns-aliases/resolve/find-var/ns-unmap/*ns* into clojure.core.

The resolve friction: native-ops (+, map, …) are inlined at emit so they have
no var-cell, and (resolve '+) was nil — diverging from Clojure where it's a
var. Added a defined? flag to the var-cell record (set by def-var!/declare-var!,
left false on a lazily-materialised forward ref) and def-var!'d every native-op
name to its value-position proc, so resolve returns the cell iff genuinely
defined. ns-unmap clears the flag. resolve never interns an empty cell
(var-cell-lookup is non-creating).

ns-name is overridden natively in post-prelude (the overlay reads
(get ns :name), nil on a jns record); the printers render a namespace as its
name. *ns* binds to the user ns; in-ns re-binds it. use/require cross-ns
switching stays deferred to Phase 3 (the analyzer bakes a def's target ns at
compile time).

Prelude parity 1951 -> 1969, 0 new divergences; four now-passing allowlist
entries dropped (ns *ns* cases + str-of-a-var). New focused gate test/chez/
_ns.janet (19 cases, expectations from the JVM-canonical build/jolt).
2026-06-18 16:49:35 -04:00
Yogthos
eb26ad0401 Chez Phase 2 (inc I+J): first-class vars + scalar natives
inc I (jolt-n7rz) — vars as first-class objects. (var x)/#'x lowered a
:the-var IR op the Chez emitter didn't handle (57 emit-fails, the biggest
bucket); emit it to the rt.ss var-cell and shim the static var ops in
vars.ss: var?/var-get/deref/var-as-IFn/var =/pr-str (#'ns/name) + a native
bound? (the overlay reads (get v :root), nil on a var-cell record). def now
RETURNS the var (#'ns/name) like Clojure — def-var!/declare-var! yield the
cell, not the value — so (var? (def x 1)) is true and pr-str-of-var/defn
pass (un-allowlisted). Dynamic binding (binding/with-bindings*/var-set/
thread-bound?) stays a follow-up; those crash on nil host prims (safe).
Var def-time metadata (^:private/^Type/doc) isn't captured yet — allowlisted.

inc J (jolt-snry) — scalar natives. natives-misc.ss: a juuid record
(random-uuid v4 / parse-uuid / uuid? / #uuid pr-str / str), a %-format
engine (%d/%s/%.Nf/%x/%c/%%; printf rides on it), a jtagged record
(tagged-literal + :tag/:form + #tag pr-str), bigint/biginteger as
near-identity over the all-flonum model. Overlay names (uuid?/random-uuid/
parse-uuid/tagged-literal?) re-asserted in post-prelude.ss.

Prelude parity 1898 -> 1951, 0 new divergences. Floor raised to 1951.
2026-06-18 15:44:10 -04:00
Yogthos
6581df2d17 Chez Phase 2 (inc H): volatiles + sequence/transduce
sequence and transduce were seed natives nil on the prelude; the stateful
transducer arities (take-nth/map-indexed/partition-by, overlay) drive a
volatile via volatile!/vswap!/vreset!/deref, also unshimmed — so the whole
(sequence xform coll) / (transduce xform f coll) surface crashed.

natives-xform.ss: native volatiles (a jvol record + volatile!/vreset!/
vswap!/volatile? + a deref arm); transduce/sequence built on the existing
into-xform / reduce-seq. The overlay vreset!/vswap!/volatile? drive a
volatile through ref-put!/:jolt/type (tagged-table only), so they're
overridden natively in post-prelude.ss.

driver.janet: drain each chez pipe to EOF instead of a single ev/read. A
program with a stdout side effect (newline) flushes in two writes and a
lone ev/read sometimes caught only the first chunk, dropping the trailing
value — an intermittent gate divergence. Now reads until the pipe closes.

Prelude parity 1886 -> 1898, 0 new divergences. Floor raised to 1898.
2026-06-18 14:47:03 -04:00
Yogthos
e434a7d352 Chez Phase 2 (inc G): lazy-seq bridge (make-lazy-seq / coll->cells)
The lazy-seq macro expands to (make-lazy-seq (fn* [] (coll->cells body)))
and lazy-cat to (concat (lazy-seq c) ...); both seed natives were nil on
the prelude, so every overlay fn built on lazy-seq — repeat/iterate/
cycle/dedupe/take-nth/keep/interpose/reductions/map-indexed/distinct/
interleave/tree-seq(->flatten)/partition-all/lazy-cat — hit apply-jolt-nil.

lazy-bridge.ss bridges to the cseq model: a jolt-lazyseq is a deferred
seq forced once by an extended jolt-seq; jolt-cons defers a lazyseq tail
so an infinite (repeat/iterate/cycle) stays lazy. A lazyseq is a new
value type, so the dispatchers that don't route through jolt-seq learn it
(sequential? for =/hash, plus count/empty?/nth/printers) or a raw
unrealized lazyseq escapes — the corpus compares (= [1 3 5] (take-nth …))
against it directly.

seq.ss: jolt-concat is now fully lazy (the rest isn't forced until the
first coll is exhausted), so a self-referential lazy-cat — fib =
(lazy-cat [0 1] (map + (rest fib) fib)) — no longer memoizes its tail as
empty by reading fib before its def binds.

Prelude parity 1837 -> 1886, 0 new divergences. Floor raised to 1886.
2026-06-18 13:53:02 -04:00
Yogthos
241095977f Chez Phase 2 (inc F): jolt.host ref primitives + sorted collections
jolt.host/tagged-table, ref-put!, ref-get resolved to jolt-nil on the
prelude, so the 25-sorted tier and every overlay fn that calls sorted?
(empty, ifn?, reversible?, map?, set?, coll?) hit apply-jolt-nil.

host-table.ss provides the three primitives over a Chez mutable htable
record and set!-extends the collection dispatchers (seq/count/get/
contains?/assoc1/dissoc/conj1/disj/empty?/keys/vals/coll?/map?/set? +
printers + equality + value-host-tags) with a sorted arm routing through
each value's :ops table — the seed dispatch pattern, same as records.ss.
first/rest/next fall out free since they seq first. Sorted colls print
in sorted order ({k v, k v} for maps) and = canonicalize like their
unordered counterparts.

emit.janet: a computed call operator (an :invoke/:if expr that yields an
IFn, e.g. ((sorted-map :a 1) :k)) now routes through jolt-invoke instead
of a raw Scheme application.

Prelude parity 1723 -> 1837, 0 new divergences. Floor raised to 1837.
2026-06-18 13:14:13 -04:00
Yogthos
b8e4e78372 Chez Phase 2 (inc E): meta / with-meta
meta/with-meta resolved to jolt-nil. Chez values don't carry metadata, so
collections use an identity-keyed side-table: with-meta returns a fresh copy of
the value (new identity) and records its meta there, leaving the original
unchanged (immutable-with-meta) and dropping meta on a later copying op. Symbols
carry meta in their existing field; meta on a non-metadatable value is nil.
vary-meta works over these. with-meta on a fn stays fn? (jolt is lenient).

emit.janet carries a quoted symbol's reader metadata (^:foo bar) onto the
emitted jolt-symbol so (meta 'x) sees it; symbol = still ignores meta.

Prelude parity 1701 -> 1723, 0 new divergences. jolt-rkbc.
2026-06-18 12:16:48 -04:00
Yogthos
f0419b560d Chez Phase 2 (inc D): records + protocols
The largest remaining crash bucket: defrecord/deftype/defprotocol/extend-type/
reify. make-deftype-ctor/make-protocol/protocol-dispatch/register-method/
satisfies?/extenders/instance-check/make-reified were ctx-capturing seed natives
resolving to jolt-nil.

records.ss adds a jrec type (tag + field alist), set!-extended into every
collection dispatcher (get/=/hash/count/keys/vals/seq/assoc/conj/contains?/map?/
pr-str/pr-readable/str) via the transients.ss capture-prev pattern, plus a
protocol registry (type-tag -> proto -> method -> fn) and dispatch over record
tags / host-type candidates. (get rec :jolt/deftype) returns the tag, so the
overlay record? works unchanged. A record equals another of the same type with
equal fields, is map?/coll? not vector?, prints #ns.Name{...}, and str uses a
custom Object toString impl when defined.

emit.janet :host-call now routes a non-shimmed method to record-method-dispatch
(was an emit-fail), so (.protoMethod record args) compiles; .-field access is
still analyzer-punted (deferred).

Prelude parity 1652 -> 1701, 0 new divergences. 4 print-method-multimethod cases
moved crash->allowlist (the printer doesn't consult a custom print-method yet).
jolt-jgoc.
2026-06-18 11:55:20 -04:00
Yogthos
5101ada8e0 Chez Phase 2 (inc C): bit ops + parse-long/parse-double
__bit-and/or/xor/and-not + bit-not/shift/set/clear/test/flip + unsigned-bit-
shift-right + parse-long/parse-double were unshimmed seed natives (jolt-nil).
Bit ops coerce the all-flonum operands to exact ints, operate, return flonums.
parse-* match the seed's strict shapes (nil on malformed, throw on non-string).

Prelude parity 1593 -> 1652 (inc B+C), 0 new divergences; 3 print-method/def-
returns-var cases moved crash->allowlist. jolt-cf1q.3.
2026-06-18 11:20:56 -04:00
Yogthos
d7c6290ec0 Chez Phase 2 (inc B): readable printer + output seams
__pr-str1/__write/__with-out-str/__eprint/__eprintf resolved to jolt-nil, so the
whole overlay print family (pr-str/pr/prn/print/println/print-str/prn-str/
println-str) hit the apply-jolt-nil crash bucket. jolt-pr-str is str-style
(strings raw); pr-str needs readable (pr) style — strings quoted+escaped at every
level. printing.ss adds the readable renderer (mirrors jolt-pr-str, recurses,
delegates scalars), plus the infinities long-form and a transient arm (those
were crash->divergence reveals once pr-str ran).

jolt-9zhh.
2026-06-18 11:20:56 -04:00
Yogthos
9b0d6acadc Chez Phase 2 (inc A): collection ctors + real map entries
set/hash-map/hash-set/array-map/rand resolved to jolt-nil on the prelude
(the apply-jolt-nil crash bucket) — the pmap/pset ctors already existed in
collections.ss, just bind the public clojure.core names to them.

Map entries are now a distinct type: a pvec carries an ent flag (default #f),
so an entry equals its [k v] vector and walks like one (nth/count/seq/=/hash/
print read only v) but is not vector? and is map-entry? — matching Clojure's
MapEntry. seqing a map produces flagged entries; vector? excludes them. This
unblocks key/val (overlay fns gated on map-entry?) and the every? map-entry?
cases.

Prelude parity 1534 -> 1593, 0 new divergences. jolt-agw6.
2026-06-18 10:44:33 -04:00
Yogthos
e98afcad13 Chez Phase 1 close-out: truthy? elision + end-to-end compute bench (jolt-nkcb)
Elide the redundant jolt-truthy? wrapper on an :if test that provably
yields a Scheme boolean (a native comparison/not, or a boolean const).
Sound because jolt-truthy? of #t/#f is identity. The hot fib/mandelbrot
tests are all comparisons, so this is a direct ceiling lever: fib(30)
end-to-end 24.0 -> 14.4 ms.

Add bench-pipeline.janet (JOLT_CHEZ_BENCH=1, opt-in) timing fib(30) +
mandelbrot(200) through the real pipeline vs the spike ceiling.
Mandelbrot 200 runs at 87 ms, at/below the 98 ms generic-flonum ceiling
- the substrate ceiling is reached end-to-end. fib sits at 2x its
hand-flonum ceiling; the residual is jolt's all-double number model
(typed fl*/fx* emission is Phase 4). Compile-only is total for the
compute subset (every form emits; Chez has no interpreter fallback).

Full parity unchanged at 1534/2494, 0 new divergences.
2026-06-18 09:05:54 -04:00
Yogthos
af680ed106 Chez plan: zero-Janet north star, self-host the compiler on Chez
Revise the epic's direction from a minimal Janet shim to ripping Janet
out entirely — Chez becomes the sole substrate. The missing spine: the
compiler pipeline itself only runs on Janet today (the analyzer executes
on the Janet host; the IR->Scheme emitter is host/chez/emit.janet). Phase
3 is re-scoped to self-host the compiler on Chez (emitter -> Clojure
jolt.backend-scheme, reader -> jolt-core, compile-from-source bootstrap
fixpoint). Phase 5 becomes a hard delete of both src/jolt/*.janet and
host/chez/*.janet. Sequencing: core parity first, then self-host, then
delete.
2026-06-18 08:27:22 -04:00
Yogthos
6af3e73595 Chez emit: codepoint-escape non-ASCII string literals (jolt-x0os)
Janet's %j renders a non-ASCII char as raw UTF-8 bytes (\xC3\xA9) and a
control char / DEL as \xHH with no terminating semicolon — both forms
Chez's reader rejects (invalid character \ in string hex escape). Replace
the %j string encoder with chez-str-lit: UTF-8-decode each char and emit a
Chez codepoint escape \x<cp>;, keeping \n/\t/\r/\"/\\ and staying
byte-identical to %j for printable ASCII. Applied to every string-content
site (string/keyword/symbol/var-name/regex-source).

Unblocks the p{L} utf-8 corpus case; prelude parity floor 1532 -> 1534.
2026-06-18 03:27:28 -04:00
Yogthos
6cc3dc2c7f Fix seed assoc! to throw on odd args (jolt-ea9k)
The transient assoc! accepted an odd key/val count and silently assigned
nil to the dangling key — non-Clojure, and inconsistent with the seed's
own plain assoc (which throws) and Clojure's assoc!. Make it throw.

Updates the 4 'assoc! odd args' transient spec rows to 3 :throws + 1
even-args positive, and regenerates corpus.edn. The Chez host already
threw on these, so this only realigns the corpus contract.
2026-06-18 03:27:21 -04:00
Yogthos
d7420deecb Chez Phase 1 (increment 3r): dynamic-var constants
host/chez/dynamic-vars.ss binds the two seed-native dynamic vars that aren't
emitted into the prelude: *clojure-version* (the {:major 1 :minor 11 ...} map) and
*unchecked-math* (false). Removes their two run-corpus-prelude allowlist entries
(now 11, all passing).

*ns* is deferred to jolt-b4kl: it needs a namespace value that is not a map yet
answers (get ns :name) for the overlay ns-name, plus str/find-ns support.

Parity 1530 -> 1532/2497, 0 new divergences. emit-test 305/305.
2026-06-18 01:43:16 -04:00
Yogthos
02139af0a1 Chez Phase 1 (increment 3q): multimethod dispatch + late-bind
host/chez/multimethods.ss implements the multimethod runtime: defmulti/defmethod
expand to defmulti-setup/defmethod-setup calls (+ get-method/methods/
remove-method/prefer-method/prefers). A jolt-multifn record carries its dispatch
fn and a jolt=-keyed method table; jolt-invoke dispatches it (exact match, then
isa?/hierarchy with prefer-method, then :default), reusing the overlay's
isa?/derive/make-hierarchy. The multifn's ns comes from a runtime chez-current-ns
(default user; the prelude load sets clojure.core for print-method/print-dup).

Two emit-side changes were needed:
- late-bind (:late-bind-unresolved? ctx flag, default OFF): defmulti expands to a
  bare-symbol setup call, so the analyzer doesn't intern the name and a forward
  reference '(area ...)' after '(defmulti area ...)' in one form was 'Unable to
  resolve symbol'. The strict compiler punts these to the interpreter; the Chez
  back end has none, so the flag lowers an unresolved symbol to a var-ref against
  the compile ns (open-world -e semantics). Set only by the Chez make-ctx /
  jolt-chez; the main compiler keeps strict resolution (host_iface late-bind?
  defaults nil).
- a :var call head now routes through jolt-invoke, since a late-bound var can hold
  a multifn (or keyword/coll IFn), not just a procedure. Transparent for
  procedures; the hot self-recursive call is a :local known-proc, stays direct.

Class-based dispatch ((class x)/String) deferred (needs deftype/class subsystem).

Parity 1506 -> 1530/2497, 0 new divergences. emit-test 302/302. Full janet gate
green (the analyzer flag is off there; suite flakiness under parallel load only).
2026-06-18 01:24:01 -04:00
Yogthos
e51cc2e47e Chez Phase 1 (increment 3p): misc seq/regex gaps + bug tracking
jolt-y1zq tail:
- 0-arg (conj) -> [] and 0-arg (conj!) -> a fresh transient vector
- nth sees through a transient (like get/count/contains?)
- irregex \p{...}/\P{...} property classes translate to the seed's ASCII char
  classes (regex.ss): \p{L} -> [a-zA-Z] + non-ASCII codepoints (the seed counts
  UTF-8 high bytes as letters), \p{N} -> [0-9], \p{Ps} -> [([{], etc. The
  translator tracks [...] nesting so a \p{} inside a class emits its content, not
  a nested class.

Two pre-existing bugs found and filed (tracked, not replicated):
- jolt-x0os: the Chez emitter mangles non-ASCII string literals into invalid Chez
  hex escapes (so p{L} utf-8 crashes on the input string, not the regex).
- jolt-ea9k: the seed's transient assoc! accepts odd args and assigns nil to the
  trailing key (non-Clojure; plain assoc throws). The Chez host throws
  (Clojure-correct); the 4 spec rows encoding the leniency are flagged in
  transients-spec.janet pending the seed fix.

Parity 1493 -> 1506/2497, 0 new divergences. emit-test 291/291.
2026-06-18 00:44:21 -04:00
Yogthos
cbb0f2ab4a Chez Phase 1 (increment 3o): transducer arities
The 1-arg map/filter/remove/take/drop/take-while/drop-while/mapcat now return a
transducer (fn [rf] rf'), and into gets a 3-arg (into to xform from). This was
the 'cdr () is not a pair' / 'incorrect number of arguments' crash bucket: the
emitter lowers (map f) and 3-arg into at an arity the native-op gate rejects, so
they fall to the value-position path and hit the bare jolt-map/jolt-into
procedure at the wrong arity. The fix is RT-side — case-lambda those procedures
plus jolt-into.

td-* factories ported from the seed (core_coll.janet); a reduced step stops the
fold via reduce-seq's existing short-circuit (inc 3n). transduce/comp/completing
are overlay and compose over these unchanged.

Parity 1467 -> 1493/2497, 0 new divergences. emit-test 278/278.
2026-06-17 23:51:06 -04:00
Yogthos
739b219d0e Chez Phase 1 (increment 3n): seq-native shims + reduced
The dominant prelude-parity crash bucket was 'apply non-procedure jolt-nil':
core fns calling seed-native seq fns (core_coll.janet) that have no Chez RT
shim, so var-deref returns jolt-nil. A static scan of the assembled prelude
turned up 52 referenced-but-undefined clojure.core names.

host/chez/natives-seq.ss shims the safe seq fns over the existing seq layer:
mapcat, take-while, drop-while, partition (collection arities only — the 1-arg
transducer forms are jolt-kxsr), and sort (compare default; a comparator may
return a 3-way number or a boolean less-than). reduced/reduced? is a jolt-reduced
record in seq.ss that reduce short-circuits on and deref unwraps, so unreduced
works. identical? = jolt= (the seed's definition).

Deferred list?: a Chez lazy seq and a list are both cseq, so it can't be told
apart without a distinct list type — a real divergence risk.

Parity 1407 -> 1467/2497, 0 new divergences. emit-test 263/263.
2026-06-17 23:16:26 -04:00
Yogthos
c28b5406ca Chez inc 3m: numeric-edge literal emit + variadic assoc!
##Inf/##-Inf/##NaN were emitted to bare inf/-inf/nan, which are unbound symbols in
Chez. emit-const now lowers them to +inf.0/-inf.0/+nan.0. The -e/element printer
renders them inf/-inf/nan (Chez's number->string gives +inf.0), and str renders the
long Clojure forms Infinity/-Infinity/NaN. assoc! is now variadic ((assoc! t k v
& kvs)) like Clojure.

Prelude parity 1382 -> 1407/2497, 0 new divergences. str of inf INSIDE a collection
still wants the long form (needs the Phase-2 recursive str renderer), so
[inf inside coll] is allowlisted. Transducer arities and the cdr-on-()/\p{} regex
gaps are split out to jolt-kxsr/jolt-y1zq.
2026-06-17 22:32:02 -04:00
Yogthos
1826c8b3e9 Chez inc 3l: transient collection RT shims
transient/persistent!/conj!/assoc!/dissoc!/disj!/pop! as copy-on-write over the
persistent collections (host/chez/transients.ss) — each op rebuilds the persistent
coll (no in-place perf) but the semantics match, so into/frequencies/group-by work.
Adds persistent disj over pset-disj. get/count/contains? are redefined to see
through a transient (frequencies and group-by both do (get tm k) on a transient
map); vector? on a transient vector is false, which group-by relies on.

Prelude parity 1326 -> 1382/2497, 0 new divergences. emit-test exercises the direct
transient ops via run-prelude and the overlay users (frequencies/group-by/into)
end-to-end through the bin/jolt-chez -e binary.
2026-06-17 21:58:35 -04:00
Yogthos
bb8b2d201c Chez inc 3k: converter + string-op RT shims
str/subs/vec/keyword/symbol/compare/int/double/gensym — host-coupled seed natives
the overlay assumes, now def-var!'d into clojure.core via host/chez/converters.ss
(loaded last, str reuses jolt-pr-str). Semantics match the seed: str-render-one
for str (nil->"", bare chars, regex source), the 3-way core-compare port, int
truncates. The symbol no-ns sentinel is #f to match emit's quoted-symbol lowering
(jolt-symbol #f "x"), so (= 'x (symbol "x")) holds — jolt= compares ns with strict
equal?, and jolt-nil vs #f would otherwise diverge.

Prelude parity 1220 -> 1326/2497, 0 new divergences. Floor raised to 1326; three
newly-reached *ns*/var-rendering cases added to the allowlist (Phase 2). run-prelude
and the per-case program file are now PID-unique so concurrent chez runs don't read
each other's half-written files.
2026-06-17 21:36:42 -04:00
Yogthos
0c9c7931fe Chez Phase 1 (increment 3j): assemble the clojure.core prelude, -e-capable jolt-chez
Emit every non-macro clojure.core form through the live analyzer -> Chez emit
pipeline as a def-var! prelude (prelude mode, tier dependency order), load it
before a user expression, and you get an -e-capable jolt-chez: analysis on Janet,
execution on Chez. driver/emit-core-prelude assembles it (each form behind a
silent load guard so the Phase-2 multimethod forms don't break the rest);
bin/jolt-chez is the -e CLI, caching the prelude on disk keyed by source hash.

run-corpus-prelude.janet is the full parity gate this opens, the prelude-backed
sibling of run-corpus-chez. First baseline: 1220/2497 evaluated cases pass, 0 new
divergences (10 allowlisted: dynamic vars, class names, eval-order — deferred
Phase 2). The rest is the punch-list: ~360 emit-fail (real host interop, out of
the analyzer subset) and ~900 runtime crashes, mostly core fns calling
host-coupled seed natives with no Chez shim yet (str/format/vec, transients).
Follow-ups jolt-t6cr/kl2l/q3w8/9ls5.

Two shims landed to get the prelude to load and run. atoms.ss: atom/deref/swap!/
reset! (+ the compare/vals kernel) — needed at load time for
global-hierarchy = (atom (make-hierarchy)). predicates.ss: the type predicates +
name/namespace/boolean the overlay assumes are seed natives, matching the seed's
strict semantics. post-prelude.ss re-asserts char?/atom? after the prelude: the
overlay implements those by reading :jolt/type, which is false for Chez-native
chars/atoms, so its def-var! would clobber the correct native versions.

Per-case Scheme files are PID-unique so a foreground -e never reads a half-written
file while the gate runs.
2026-06-17 20:50:42 -04:00
Yogthos
37c433bd4a Chez Phase 1 (increment 3i): regex via vendored irregex
Closes the last clojure.core prelude emit gap (parse-uuid): the whole
non-macro core now lowers to Scheme (prelude reach 355/355).

A #"..." literal analyzes to a :regex IR node. The Chez back end emits
a jolt-regex value over irregex (Alex Shinn, BSD), vendored as the
vendor/irregex submodule -- a portable Scheme regex with PCRE/Java-style
string patterns and first-class Chez support. host/chez/regex.ss wraps
jolt's re-* surface over it: irregex-match -> re-matches (anchored),
irregex-search -> re-find, groups as Clojure [whole g1 ...] vectors,
re-seq as a jolt seq. re-pattern/re-matches/re-find/re-seq/regex? are
def-var!'d into clojure.core so prelude / -e code resolves them.

They stay OUT of the subset native-ops on purpose: irregex's
Unicode/property-class semantics differ from the seed's byte-PEG
approximation, so keeping them prelude-only avoids dragging
engine-difference divergences into the subset-parity corpus. The Janet
back end punts :regex to the interpreter (the seed compiles #"..." to a
Janet PEG), so the main language is unchanged.

Only two adaptations for Chez's top level: a cond-expand shim (Chez's is
library-only) and a normalizing error wrapper (silences irregex's 1-arg
error warnings). rt.ss load is ~0.18s.

emit-test 131/131 (regex literal + re-* parity vs the CLI oracle);
prelude reach 355/355; Chez subset 672/672, 0 divergences; full gate
green.
2026-06-17 19:44:18 -04:00
Yogthos
b1cdfd1c9b Chez Phase 1 (increment 3h): host-interop method-call emit
(.method target arg*) now analyzes to a :host-call IR node instead of
punting at analyze. The Chez back end lowers it to a jolt-host-call
dispatch for the methods the RT shims (.write -> port display,
.isDirectory -> file-directory?, .listFiles -> directory-list); any
other method stays out of subset (clean emit-time reject, so it can't
read as a compiled-but-broken corpus divergence). The Janet back end
punts ALL :host-call to the interpreter, same shape as letfn: compiles
on Chez, interprets on Janet, zero change to the main language.

Closes the io tier's print-method defmethods and file-seq: prelude emit
reach 348 -> 354/355 (50-io 20/20). The one remaining gap is the regex
literal in parse-uuid (needs a regex engine on Chez; deferred).

emit-test 122/122; Chez subset 672/672, 0 divergences; full gate green.
2026-06-17 18:58:44 -04:00
Yogthos
0f7d2753a8 Chez Phase 1 (increment 3g): letfn + declare/def-no-init
Closes the last two non-host-interop prelude emit gaps.

letfn now analyzes to a :let node flagged :letrec — the binding fns are bound
into the env together before any spec is analyzed, so siblings and self resolve.
The Chez back end lowers it to letrec*; the Janet back end punts it at emit
(its sequential let* can't express the mutual recursion — same interpreter
fallback as before, just decided at emit-ir instead of analyze).

(def x) with no init (declare) analyzes to a :def with :no-init instead of
punting. Chez reserves the var cell via declare-var! (which doesn't clobber an
existing root — (do (def x 7) (def x) x) => 7); the Janet back end still punts
to the interpreter, which interns a genuinely-unbound var.

fallback-zero-test now checks emit-ir too, not just analyze-form, so the real
compile-vs-interpret decision is what it asserts (letfn/def-no-init analyze but
the Janet back end punts them). letfn stays in must-punt with an updated note.

Prelude emit reach 342 -> 348/355 (40-lazy now 13/13); Chez subset 664 -> 672,
0 divergences; emit-test 110 -> 117. Full gate green.
2026-06-17 18:27:34 -04:00
Yogthos
930800f9a6 Chez Phase 1 (increment 3f): quoted literals
Emit a :quote node by reconstructing the raw reader form as RT constructor
calls: symbol -> jolt-symbol, list (array) -> jolt-list, vector (tuple) ->
jolt-vector, map -> jolt-hash-map, set -> jolt-hash-set, scalars via emit-const.
The runtime value of a quote is just that literal data (the interpreter returns
the reader form verbatim).

Quote exposed a latent seq.ss bug: empty map/filter results were jolt-nil, but
Clojure's (map f []) is an empty seq, so (= () (map f [])) must be true. Return
jolt-empty-list (which seqs back to nil, so it's still a valid lazy-tail
terminator) instead — matching jolt-take/drop/rest/list.

Prelude emit reach 334 -> 342/355. Subset probe 632 -> 664/664 compiled, 0
divergences (quote + the seq fix pull 32 corpus cases into the subset). emit-test
110/110 (added 16 quote cases). corpus.edn regenerated (the 3 malformed-catch
spec rows). Full gate green.
2026-06-17 17:43:34 -04:00
Yogthos
d35783bf1b Reject a malformed catch clause with a clean error in both modes
jolt's catch is (catch class binding body*); the binding (3rd element) must be
a symbol. Neither the analyzer nor the interpreter validated it, so a non-symbol
binding crashed with an internal Janet error (expected integer key for array...)
and, in the interpreter, a malformed clause whose body never threw was silently
swallowed (returned the try value). Clojure rejects a non-class/non-symbol catch
clause; match that with an up-front error in analyze-try and eval-try.

Surfaced building the Chez try/throw emit. Regression rows in exceptions-spec
(runs x3 modes) plus a unit test asserting the clean message in interpret and
compile. jolt-kg6p.
2026-06-17 17:25:14 -04:00
Yogthos
ffa122440a Chez Phase 1 (increment 3e): throw/try/catch/finally + ex-info
Emit :throw as jolt-throw (Scheme raise of the raw jolt value, matching the
Janet compiled back end's (error v) — no envelope, so catch binds it directly).
Emit :try as guard (catch; the class is dropped in the IR, so it's catch-all)
plus dynamic-wind for finally. ex-info is a native-op building the tagged jolt
map {:jolt/type :jolt/ex-info :message :data :cause}, so the ex-data/ex-message/
ex-cause tier fns read it over jolt-get for free.

Prelude emit reach 303 -> 334/355 (:throw and :try gaps close). Subset probe
619 -> 632/632 compiled, 0 divergences (throw/try/ex-info pull 13 corpus cases
into the subset). emit-test 94/94 (added 11 throw/try/ex-info cases + uncaught
exits non-zero). Full gate green.
2026-06-17 17:10:38 -04:00
Yogthos
8f26433469 Chez Phase 1 (increment 3d): clojure.core prelude emit mode + gap catalog
Add a prelude emit mode to host/chez/emit.janet: when emitting clojure.core
itself (not user -e), a non-native clojure.core ref lowers to a runtime
var-deref instead of being rejected as out-of-subset, so core fns chain
through each other. Default (subset) mode is unchanged — the corpus probe
still rejects unimplemented core refs for a clean signal.

core-prelude-probe.janet walks the tiers through the live analyzer->emit
pipeline and catalogs reach + gaps (macros skipped; analyze-time only).
Baseline: 303/355 non-macro core forms emit. Remaining gaps are a tight
punch-list for the next increments: :throw (29), :quote (8), :try (2), Java
host interop (6), letfn (4), declare (2). Probe has a regression floor.

emit-test 83/83 (added prelude-mode lowering assertions); subset probe
619/619 unchanged; full gate green.
2026-06-17 16:29:29 -04:00
Yogthos
45208afff1 Chez Phase 1 (increment 3c): multi-arity + variadic fn emission
emit-fn lowered multi-arity fns to a Scheme case-lambda and variadic fns to
a rest-arg lambda; the Scheme rest list is coerced to a jolt seq (nil when
empty, via list->cseq), and the named-let wrapper runs that coercion only on
first entry so recur carries the seq directly. Single fixed arity keeps the
plain-lambda fast path (fib untouched).

Also fixes a latent leak in the module-global known-procs: a throw mid-emit
(uncompilable body) left the fn's name registered, so a later corpus case
binding the same name to a keyword emitted a direct call to a non-procedure.
The cleanup now runs on the error path too. Only surfaced once the new arity
support let +24 cases compile further before hitting an uncompilable fn.

Gate: emit-test 81/81, subset probe 619/619 compiled (was 595), 0 divergences,
2036/2655 out of subset; full run-tests green (125 files).
2026-06-17 16:08:27 -04:00
Yogthos
cb3cfaf0c2 Chez Phase 1 (increment 3b): seq tier + dynamic IFn dispatch on the Chez RT
Brings up the seq layer on the Chez runtime. host/chez/seq.ss adds one
lazy-capable node (cseq) that models Clojure's list, cons, and lazy seq -
all print as (...), all sequential-= to each other and to vectors. seq
coerces any seqable (vector/map/set/string/list/seq/nil) to a cseq or nil;
the empty seq is a distinct value printing () (rest of a 1-elem coll is ()
not nil, seq of empty is nil). Leaf ops: first/rest/next/seq/cons/list,
reverse/last, map/filter/remove/reduce/into, range/take/drop/concat/apply,
keys/vals, plus nth/peek/pop extended over seqs. map/filter/reduce apply
their fn arg through jolt-invoke, so a procedure, keyword, or collection all
work as the fn.

Dynamic IFn dispatch: a keyword/vector/coll held in a local (let binding or
fn param) and called as a fn now routes through the jolt-invoke fallback
(procedure? -> apply; keyword/coll -> lookup). The emitter only routes a
:local callee that isn't a known procedure - a named fn's self-recursion
name stays a direct call, so the fib hot path is untouched. Closes the 3
ex-known IFn divergences.

emit.janet: seq/pred ops added to native-ops with arity gates; value-position
clojure.core refs resolve to the RT procedure (native-ops names one for each),
with +/-/*// routed to flonum-coercing wrappers so higher-order arithmetic
((reduce + [])) keeps the all-double model. values.ss: cross-type sequential
=/hash so a vector and a list of the same elements are jolt= and hash alike.
rt.ss: printer learns seqs; top-level nil prints as the empty string (jolt -e
str-style). Fixed latent bug: (conj nil ...) now builds a list, not a vector.

Gates: emit-test 69/69 (fib/mandelbrot/collections/seq/IFn parity vs the jolt
oracle, fib(30) ~24ms unchanged). Subset probe 433/436 -> 595/595 compiled,
0 divergences (was 3 known), 2060/2655 out of subset. Full run-tests green
(125 files, conformance + suites included).
2026-06-17 15:19:18 -04:00
Yogthos
5c5d2cd1fc Chez Phase 1 (increment 3a): persistent collections on the Chez RT
Broaden the Scheme back end past the numeric/functional subset to vectors,
maps, and sets. host/chez/collections.ss adds a copy-on-write persistent
vector and a bitmap HAMT (the structure 0c measured self-hostable) backing
both maps and sets, keyed by jolt-hash and compared by jolt=. emit.janet
emits :vector/:map/:set literals to the rt constructors and lowers the leaf
ops (conj/get/nth/count/assoc/dissoc/contains?/empty?/peek/pop) via the
native-ops path, with a per-op arity gate.

Also: keyword/map literals in fn position lower to jolt-get ((:k m), ({:k v} k));
arity-1 comparisons emit the vacuous jolt truth (Scheme < rejects a non-number
even at arity 1); count returns a flonum and vector indices coerce from flonum,
both consequences of the all-double number model; values.ss = / hash and the
rt printer learn collections (maps/sets render in HAMT order, so the probe
compares unordered values via =, not printed form).

Subset parity 182 -> 433/436 compiled cases (2219/2655 out of subset), 0 new
divergences. The 3 known divergences are dynamic IFn dispatch (a keyword/vector
held in a local, called as a fn) — deferred to the IFn/protocol increment and
allowlisted in the probe. emit-test 31/31, full run-tests green (125 files).
2026-06-17 14:33:57 -04:00
Yogthos
9bbcc07c8f Chez Phase 1 (increment 2): live analyzer -> Chez, var cells, RT, mandelbrot
Wire the real pipeline end to end: host/chez/driver.janet boots a compile-mode
jolt ctx, runs the EXISTING Janet-hosted analyzer on actual Clojure source to
real IR, feeds it to the Scheme emitter, and runs the result on Chez. Analysis
stays on Janet (the analyzer ports to Chez in Phase 2); execution is on Chez.

emit.janet now consumes live IR (pv/phm-normalized like the Janet backend) and
covers what the analyzer actually emits, not the hand-built inc-1 shapes:
- core ops arrive as :var clojure.core/+ etc., not :rt — lowered to native
  Scheme via a native-ops table (mirrors backend.janet's), `=` to jolt=.
- var cells (host/chez/rt.ss): :def -> def-var!, :var -> var-deref. Late binding
  so cross-var calls (run -> count-point) and the entry crossing resolve at use.
- named fns (defn / fn self-name) bind via letrec so self-recursion resolves.
- unsupported stdlib/host refs (no core on Chez yet) are rejected at EMIT time
  (clean out-of-subset signal) instead of deref'ing to nil and failing at runtime.

Number model: jolt is all-doubles (no ratios; (/ 1 2) is 0.5), so literals emit
as flonums — matches the Janet host and keeps Chez out of exploding exact
rationals (mandelbrot). jolt-num->string prints integer-valued without ".0".

Two real bugs found via the corpus probe and fixed (regression rows added):
- loop bound in parallel (Scheme named-let) but Clojure loop is sequential — a
  later init must see earlier bindings; wrap a let* around the loop.
- #(...) shorthand gensyms params with a trailing `#`, invalid in Scheme — munge
  it to `_`.

Gate: test/chez/emit-test.janet runs the real analyzer -> Chez for (+ 1 2),
fib(30)=832040, mandelbrot run(40), and the two regressions, parity-checked
against the Janet oracle (6/6). First parity number via the new subset probe
(test/chez/run-corpus-chez.janet, JOLT_CHEZ_CORPUS=1): 182/182 compiled corpus
cases pass, 0 divergences; 2473/2655 out of subset pending core on Chez. Full
jpm/run-tests gate green (125 files). Chez tests skip cleanly without `chez`.

Perf note (unchanged plan): emitted fib(30) ~23ms vs hand-Scheme ~5ms — the
jolt-truthy? wrapper (~3x) plus flonum (not fixnum) arithmetic, both Phase-4
type-specialization levers.
2026-06-17 13:59:57 -04:00
Yogthos
874e3c7cf2 Chez Phase 1 (increment 1): IR -> Scheme emitter, real IR shapes
New back end half: host/chez/emit.janet consumes the host-neutral jolt IR
(ir.clj shapes) and emits Scheme, reusing the existing front-end (Option-2
backend swap). Covers the pure-functional subset: const/local/var/rt/if/do/let/
fn/invoke/def/loop/recur. Tested by hand-built IR run on Chez: (+ 1 2)=3,
fib(30)=832040, loop/recur sum=15 (4/4).

Finding: correct emit wraps every if-test in jolt-truthy?, costing ~3x on fib
(15.8ms vs hand-Scheme 5ms). Eliding the wrapper for known-boolean tests
recovers the ceiling (Phase-4 type-driven opt).

Remaining Phase 1: wire the live analyzer, var-cell late binding, RT module,
broader op coverage for mandelbrot.
2026-06-17 13:15:57 -04:00
Yogthos
b3d0a91e3e Chez Phase 0c + 0a hardening: collections decision + value-model fixes
0c: persistent HAMT on Chez is ~41x faster than Janet's HAMT on the collections
map-churn (258.6 -> 6.3 ms), ~15x off mutable-native (inherent persistence cost).
Decision: self-host the persistent collections in Clojure; substrate is not the
bottleneck. See docs/chez-phase0-results.md.

0a hardening: NUL-separated keyword intern key (no ns/name collision), non-finite
-safe jolt-hash. 37/37.
2026-06-17 13:10:19 -04:00
Yogthos
c9316dd372 Chez Phase 0a+0b: value model + host-neutral contract gate
0a (host/chez/values.ss): Jolt value model on Chez — nil sentinel distinct from
#f/'(), interned keywords, ns+meta symbols, exactness-aware = and consistent
hash. Chez numeric tower gives ratios/bignums free. 33/33 tests.

0b (test/chez/): extract test/spec/*.janet defspec tables into corpus.edn (2655
cases, valid as both EDN and Janet data), and a runner that drives ANY jolt
binary via the CLI boundary with per-case subprocess isolation. Pluggable target
(JOLT_BIN) so the same corpus gates every host. Baseline vs Janet build/jolt:
2641/2655, 14 known CLI divergences allowlisted; gate fails only on NEW ones.
2026-06-17 13:06:35 -04:00
Yogthos
b60177b03a Chez plan: lock host.* neutral bridge + :native deps.edn declaration
host.* replaces janet.* as the portable interop namespace (each host implements
it over its own FFI). Add a :native dep form so projects declare needed shared
libs (libcurl/openssl/zlib) — not git-fetchable, but surfaced to the user and
probed at load so a missing .so yields a precise error, not a raw dlopen fail.
2026-06-17 12:56:37 -04:00
Yogthos
9a5fb98f47 Chez plan: host interop + FFI shim libraries (examples acceptance corpus)
Account for jolt's layered interop surface on Chez — the janet.* bridge, the
FFI-backed java.* shim libs (http-client TLS/gzip, router, db), jpm-module Janet
deps (spork/http) — with ../examples as the end-to-end acceptance gate. New
epic child jolt-cf1q.7, gated behind Phase 2.
2026-06-17 12:40:06 -04:00
Yogthos
48d39ecd5a Chez port plan + beads epic (jolt-cf1q)
Phased plan for re-hosting jolt's substrate on Chez Scheme, organized around two
north stars: minimal host shim (push everything possible into self-hosted
jolt-core, drop the tree-walking interpreter) and the spec/conformance corpus as
the host-neutral correctness contract. Closes obsolete Janet-backend/cgen beads
superseded by the native substrate.
2026-06-17 12:31:21 -04:00
Yogthos
0087763dc9 Chez re-host spike: substrate ceiling vs Janet (fib/mandelbrot)
Hand-translate the two compute benches into the Scheme a jolt->Chez backend
would emit, to localize the execution-substrate ceiling without porting the RT.

fib 30: 246.6 -> 5.2 ms (~47x, fixnum). mandelbrot 200: 166.3 -> 13.4 ms
(~12.4x) ONLY with flonum-specialized ops; generic float ops box every flonum
and stay ~1.7x. 13.4 ms matches jolt's JOLT_CGEN C result, so Chez's native
compiler reaches the C ceiling with no cc step, REPL intact.

Size: Chez base 2.9 MB (AOT) / 4.0 MB (dynamic) vs Janet 2.21. Memory: Chez
~32-49 MB fixed baseline vs Janet ~12 MB (the one regression). RT-bound axes
(collections/binary-trees, where Chez's generational GC should help) not yet
measured. See spike/chez/RESULTS.md.
2026-06-17 12:07:35 -04:00
Dmitri Sotnikov
3c853429f9
Clojure-compat fixes enabling external HTTP-client support (#159)
Generic language fixes so a host-shim library (jolt-lang/http-client) can carry
clj-http-lite without baking any HTTP/native code into core:

- reader: accept #^ (deprecated metadata reader macro) as ^
- (str pattern) returns the raw regex source, not #"..." — pattern composition
  via (re-pattern (str p ...)) now works
- into/conj onto a map merges map items (was (into {} [{:a 1}]) -> {nil nil})
- a Var is callable as its current value (e.g. (wrap #'f) threading a var client)
- core-class reads a :class field off a thrown table, so libraries can throw
  typed exceptions whose (class e) is a JVM class name
- compiled catch unwraps the interpreter's :jolt/exception envelope (__unwrap-ex),
  so a typed throw keeps its class/message across the interpret/compile boundary
- slurp accepts a byte-array; io/copy is generic over :jolt/input-stream /
  :jolt/output-stream stream markers
- instance? gains a registry (__register-instance-check!) for library shim types
- new clojure.test namespace (deftest/is/are/testing/use-fixtures/run-tests with
  class-aware thrown?/thrown-with-msg?)

Spec: test/spec/clojure-interop-fixes-spec.janet.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-17 06:42:52 +00:00
Dmitri Sotnikov
951a3000e6
Fix five bugs surfaced by commonmark-app (jolt-3xur, dl4s, ik3a, w2wf, xtss) (#158)
* Fix four bugs surfaced by the commonmark-app example

- regex: bounded quantifiers {n,m} no longer expand exponentially. The
  desugaring inlined the continuation per optional level, doubling it each
  time, so {0,61} built a PEG the compiler expanded to ~2^61 nodes and hung.
  Each level is now its own grammar rule referenced by name (jolt-3xur).

- strings are a seqable of chars for vec/set/into, matching seq. They went
  through realize-for-iteration, which had no string case, so into/set got
  raw bytes (code points) and set threw; vec used string/from-bytes (1-char
  strings). realize-for-iteration now maps make-char over the bytes, and
  core-vec matches (jolt-dl4s).

- clojure.string/split takes Java Pattern.split limit semantics: negative
  keeps trailing empties, 0/omitted drops them (a no-match result stays
  [input]), positive caps the part count with the remainder unsplit. It used
  to delegate the limit to take, so a negative limit returned [] and the
  2-arg form never trimmed trailing empties (jolt-ik3a).

- System/exit is registered (maps to os/exit), so (System/exit n) works
  (jolt-w2wf).

* regex: single-digit backreferences \1..\9 (jolt-xtss)

\1..\9 now match the text captured by the corresponding group, so
patterns like ([-*_])\1\1 or (\w+) \1 work. The parser records which
groups are referenced; a referenced group additionally captures its text
under a tag and the backref compiles to a PEG (backmatch). Only referenced
groups change — they match possessively (the CPS-over-possessive-PEG engine
can't backtrack into a tagged capture), so backtracking back into a
backreferenced group isn't supported (rare). Unreferenced groups keep full
backtracking and position-based result capture unchanged.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-17 03:56:13 +00:00
Dmitri Sotnikov
8192fc9541
Keep a nil nested in a collection used as a map key (jolt-zcm9) (#157)
canon-key canonicalizes a collection key by re-keying a native Janet
table by the canonical form of each element/entry. canon-key returns nil
for nil, and a Janet struct can't hold a nil key or value, so a nil set
member / nil map key / nil map value was dropped during canonicalization
— #{nil 1} canonicalized like #{1} and collided as a map key. So
(count {#{nil 1} :a, #{1} :b}) was 1 and (get {#{nil 1} :a} #{1}) was :a.

Box a nested nil before it goes into the table. The marker has to be
value-hashable, not the identity-hashed mutable-table sentinel the
transients use: the canonical struct becomes a long-lived phm key, and
its hash must survive the marshal/snapshot/fork that init-cached relies
on — an unmarshalled table gets a fresh address, so its hash isn't
preserved and the map can't find its own key. An interned keyword hashes
by content. Collision risk is only a real value equal to that exact
keyword, the same negligible class as canon-key's existing set/map struct
aliasing.

The transient sentinel stays a mutable table (it's built and consumed
within one op, never crossing a marshal boundary, so identity hashing is
stable there).

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-17 02:11:38 +00:00
Dmitri Sotnikov
630eb2b9d3
Keep a nil element in sets and transient sets (jolt-bn2p) (#156)
Two paths dropped a nil set member. phs-seq read members via
phm-to-struct, whose Janet struct can't hold a nil key, so the nil was
lost on seq/print and on into-an-existing-set even though the backing phm
counted it (count/contains? then disagreed). Re-attach the nil from the
phm's has-nil slot, keeping struct-key order for the rest so set printing
stays stable.

The transient set keyed its native table by canon-key and stored the
member as the value; canon-key returns nil for nil and a nil value also
drops the entry, so conj!/disj!/contains?/persistent! lost a nil member
outright. Key by tbl-key (the same nil sentinel the transient map uses)
and box a nil value through tbl-nil-key, unboxing on persistent!.

A nil-containing set used as a map key still collides with the nil-free
one (canon-key drops nil during key canonicalization) — separate latent
bug, filed as jolt-zcm9.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-17 01:43:23 +00:00
Dmitri Sotnikov
2668a76837
group-by: transient-vector buckets + fix nil keys in transient maps (#155)
The map build already used a transient map, but each bucket was rebuilt with
a persistent (conj (get ret k []) x) per element — an O(log n) trie path
rebuild + alloc each. A coarse grouping (few large buckets) was bound on that
conj, not the map build. Buckets are now native arrays (transient vectors,
O(1) push) frozen once; distinct keys are tracked in a side vector so the
buckets freeze in place with no second map rebuild. A bucket's first element
stays a cheap persistent [x] and only promotes to a transient on the second,
so an all-singletons grouping pays no transient alloc.

  coarse (10/100 buckets, 50k): ~313ms -> ~125ms (~2.5x)
  2 buckets (50k):              ~322ms -> ~129ms (~2.5x)
  all-unique (50k):             ~949ms -> ~892ms (no regression)

Surfaced a latent bug: canon-key returns nil for a nil key and Janet tables
drop a nil key, so the canon-keyed transient map silently lost a nil-key
entry — group-by/frequencies/assoc!/into{} dropped the whole nil bucket
((group-by identity [nil nil 1]) gave {1 [1]}, not {nil [nil nil], 1 [1]}).
Route nil through a sentinel (tbl-key) at the transient-map keying sites;
persistent!/count/dissoc! work unchanged since the real [nil v] pair is kept
as the stored value, and phm already has its own has-nil slot. The transient
set has the analogous bug (needs phs nil support) — filed separately.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 23:10:08 +00:00
Dmitri Sotnikov
1873736045
phm/phs: bulk bottom-up HAMT build for the map/set builders (jolt-5vsp) (#154)
into {}, frequencies, group-by, set, into #{} and persistent! all built
their result by folding an immutable assoc/conj per element — each call
rebuilt the O(log32 n) trie path and allocated a fresh wrapper. Add a
one-pass bottom-up HAMT builder (phm-from-pairs) and route the builders
through it, the map/set analog of the pvec bulk build in #153.

phm-from-pairs partitions entries by hash and constructs the bin/array/
collision nodes directly, with the same bin<=16 / array-node>=17 promotion
the incremental path uses — so the trie is byte-identical to one built by
phm-assoc (validated across the size and branching boundaries, including
hash collisions, duplicate keys and the nil key). persistent! map/set and
the set constructor bulk-build; into {} keeps the small-scalar-map-stays-a
-struct rule via bulk-map-from-pairs; frequencies/group-by switch to the
canonical transient form and ride the fast persistent!.

50k A/B: into {} 704->270ms, frequencies 582->160, set 615->241,
into #{} 702->240, group-by 1358->919 (bound on persistent vector conj).

Gate: conformance x3, full suite (4718 >= baseline), new maps/sets bulk
boundary specs.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 22:24:38 +00:00
Dmitri Sotnikov
43e5426601
pv: build the vector trie bottom-up, not n incremental conjs (jolt-5vsp) (#153)
pv-from-indexed (behind vec/mapv/filterv/into-a-vector) built a pvec by calling
pv-conj once per element — each call allocated a fresh pvec wrapper and copied
the up-to-32 tail tuple, so building an n-vector was O(n) allocations + tail
copies. Replace it with a single bottom-up trie construction: chunk the elements
into 32-wide value leaves, group nodes 32-wide up to the root, split off the
tail. The structure is identical to the incremental one — tail-offset(n) =
((n-1)>>5)<<5 is exactly the trie/tail boundary, so nth/conj/assoc/seq read it
unchanged (validated against the old builder across the size boundaries).

into-a-vector likewise stops doing a persistent pv-conj per element: it
accumulates into a native array and bulk-builds once (the transient-style path).

Measured (50k): vec 211 -> 6 ms (~36x), into [] 197 -> 15 ms (~13x). mapv is
unchanged here — it's bottlenecked on lazy map realization, not the build.

The map/set builders (into {}, frequencies, group-by, set — all HAMT-backed)
need the same bulk treatment and are a separate follow-up. Gate: conformance x3,
full suite, new bulk-boundary rows in vectors-spec.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 21:31:26 +00:00
Dmitri Sotnikov
b771908e5b
jolt.png: host PNG encoder + overlay, render mandelbrot/raytracer demos (#152)
Add a PNG writer so the demos produce actual images. Two pieces:

- src/jolt/png.janet — the encoder (8-bit RGB, filter None, stored/uncompressed
  DEFLATE so no compressor is needed; correct CRC32 + Adler32). It lives in Janet
  because per-byte work in the overlay is far too slow (a byte-array aset loop is
  ~30s for 360k bytes, and CRC32 over even a tiny image would be worse). Janet's
  bit ops are 32-bit signed, so the 32-bit-unsigned arithmetic is done with plain
  number ops (doubles hold 2^32) plus byte-level bxor. Exposed to the overlay as
  janet.png/* by importing it into eval_base's module-load-env.

- src/jolt/jolt/png.clj — the jolt.png overlay wrapper: image / put! / write. The
  overlay only produces pixels; the host encodes them in one pass.

mandelbrot gets a `render` subcommand (jolt -m mandelbrot render out.png [size])
that colours count-point's escape counts; the numeric-arg bench path is untouched.

Verified end to end: macOS `sips` accepts the output (so CRC/zlib are valid).
png-test covers the encoder structure (signature/IHDR/IEND) and the overlay
round-trip.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 21:07:10 +00:00
Dmitri Sotnikov
e2b607f0f3
backend: lower tail loop/recur to while + state vars, not a per-iter closure (jolt-v28u) (#151)
emit-loop compiled every loop/recur to a self-recursive local closure called once
per iteration — relying on Janet TCO for stack safety but paying a fn frame + arg
bind each iteration. The jolt-5vsp spike localized the whole ~1.43x
jolt-over-hand-Janet gap on compute loops to exactly this.

Lower instead to a Janet `while` + state vars: the loop bindings become vars
carried across iterations, a recur writes them and raises a continue flag, and a
non-recur tail value falls out through a result var. recur-name routing in
emit-recur picks the while-set lowering for loops and leaves the fn-arity self-call
path untouched.

The one subtlety is closure capture: Janet closures capture vars BY REFERENCE, so
a closure built in the body over a shared mutable loop var would see the final
value ([3 3 3]) instead of its iteration's ([0 1 2]). Each iteration rebinds the
loop names into a fresh immutable `let` before running the body, which restores
per-iteration capture. recur reads those immutable bindings and writes the state
vars, so cross-referencing args (swap, fib) need no temps.

mandelbrot 218 -> 164 ms (~11.2x JVM, from 15x). fib is unaffected — it's
fn-arity recursion, not a loop. Regression spec in control-flow-spec covers
closure capture, no-clobber recur, nested loops, sequential init, recur through
let, and that fn-arity recur still works. Gate green (conformance x3, full suite).

Note: validating this after a rebuild needs JOLT_NO_DEPS_CACHE=1 — the deps-image
cache keys on the version string, not build identity, so it served stale codegen
(filed separately).

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 21:07:06 +00:00
Dmitri Sotnikov
6cace3db90
cgen: single-binary native build via jolt cgen-build (jolt-a7ds) (#150)
Fuse an app's native-compiled numeric-leaf fns plus its source into one
static executable: no sidecar .so, no toolchain on the target. The AOT path
(#148) already produced a prebuilt module + manifest; this links them into
the jpm-built exe so the app ships as a single file.

`jolt cgen-build -m NS -o OUT` stages a build dir (src/jolt-core symlinks
into the jolt tree, a generated cg.c of the hot fns, an uberscript bundle of
the app, and an entry that bakes the runtime, installs the native fns as var
roots, and runs -main), then runs `jpm build` there — declare-native builds
cg.a and declare-executable static-links it (jpm's create-executable marshals
the module cfns and calls its static entry at startup).

Build needs cc + jpm; the result needs neither. Mechanics that bit, codified
in cgen_build.janet: stdlib_embed slurps .clj cwd-relative so the build runs
in a repo-mirroring dir; jpm hardcodes ./project.janet and sets syspath=modpath;
the executable's dofile imports cg and static-links cg.a, neither ordered nor
release-built by default, so deps are wired explicitly; cleanup must lstat (the
tree symlinks must not be followed); the inner build runs --workers=1 so it
doesn't starve siblings in the parallel gate.

test/integration/cgen-build-test.janet builds the mandelbrot fixture, runs it
from a clean dir with no src/ and no cg.so, and checks the total at native
speed. Closes jolt-a7ds.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 20:34:10 +00:00
Dmitri Sotnikov
c22e6279fa
docs: update foundational-runtime handoff to current status (#149)
Lever 1 (native codegen) is built and merged (PRs #143-148): the floor is
localized, cgen translates numeric-leaf fns to C (JOLT_CGEN, 18x on mandelbrot,
cached), and the build-time AOT path deploys native code with no cc. Replaces the
stale START HERE (which still pointed at the now-done spike) with current status
and the open work: jolt-a7ds binary fusion, jolt-v28u while-lowering, jolt-l1l4
grammar widening, jolt-qx70 hot-fn detection.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 20:34:06 +00:00
Dmitri Sotnikov
bffb492c1c
cgen: build-time AOT — native fns without a toolchain on the target (jolt-a7ds) (#148)
Splits native codegen into a build phase (needs cc) and a deploy phase (none):

- gen-c-module/compile-module compile MANY numeric-leaf fns into ONE native
  module (the AOT shape), generalizing the one-fn-per-.so JIT path.
- Backend :cgen-collect? records each numeric-leaf defn's IR while the app loads
  as bytecode; cgen/aot-build compiles them into one module and write-manifest
  persists {sopath, [{ns name sym}]}.
- Backend :cgen-prebuilt + cgen/load-aot: the deploy run loads the prebuilt .so
  (via the native builtin, no cc) and installs each cfunction as the var root
  with the same timing as the JIT path, so callers direct-link to native code.
- toolchain-available? no longer crashes when cc is off PATH (os/execute raises
  on a missing exe) — a toolchain-less target now gets false.

Proven end-to-end in two processes (spike/native/aot-demo.janet): build with cc,
then deploy with cc removed from PATH -> count-point still native, mandelbrot
3288753 at 12.4ms (full 18x). Test: test/integration/cgen-aot-test.janet. Default
path unchanged; the modes are opt-in. Gate green (118 files).

Remaining for a literal single binary: fuse the .so + manifest into the jpm exe.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 18:09:22 +00:00
Dmitri Sotnikov
ce3c7df24b
cgen: content-address and cache the compiled .so (jolt-ihdp) (#147)
compile-fn now keys the .so on a hash of the generated C + the Janet ABI + the
platform, in a persistent cache dir (default jolt-cgen under TMPDIR, override
with JOLT_CGEN_CACHE_DIR; JOLT_CGEN_NO_CACHE=1 forces a rebuild). cc runs only on
the first build of a given fn; later runs with the same source reuse the cached
.so, so the per-startup compile cost is paid once.

mandelbrot 100 whole-process wall: cold ~0.71s -> warm ~0.21s (the ~0.5s cc
cost). These cache knobs don't shape output, so they stay out of
ctx-shaping-env-vars (same as the image-cache knobs). Test asserts the .so is
content-addressed and a second compile hits the cache without the source .c.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 17:37:35 +00:00
Dmitri Sotnikov
393656d8d9
cgen: install native fns into the compile path under JOLT_CGEN (jolt-ihdp) (#146)
Wires src/jolt/cgen.janet into the backend's :def emit. With JOLT_CGEN=1 (off by
default, needs direct-linking), a plain defn of a numeric-leaf fn is compiled to
C at def time and the cfunction installed as the var root, so direct-linked
callers embed native code. The fn is not inline-stashed when cgen fires —
callers must call the C fn, not inline the bytecode body. ^:redef/^:dynamic stay
bytecode.

The leaf-first rule falls out: run calls count-point (a user var), so run isn't a
numeric leaf and stays bytecode, calling the native count-point over the cheap
forward crossing. mandelbrot 200: 224ms -> 12.4ms (~18x), result unchanged.

Adds JOLT_CGEN to ctx-shaping-env-vars (rides the disk-cache key) and :cgen? to
resolve-run-mode. Default path (cgen off) is a no-op: cgen-root returns nil and
the normal bytecode emit runs. Gate green (117 files). Test:
test/integration/cgen-pipeline-test.janet.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 17:12:48 +00:00
Dmitri Sotnikov
19e8ee906a
cgen: jolt-IR -> C for numeric-leaf fns (jolt-ihdp) (#145)
First slice of the native-codegen tier. A new standalone module, src/jolt/
cgen.janet, that translates a numeric-leaf fn (numeric in/out, body uses only
native-op arithmetic + loop/recur/if/let/do) to a Janet native C module: params
unboxed to C doubles at entry, loop/recur lowered to a while loop, reboxed at
return. compile-fn runs cc and loads the .so via the native builtin, returning a
cfunction; it returns nil for non-candidates or when the toolchain is absent.

count-point compiles and matches the bytecode fn across the mandelbrot grid
(test/integration/cgen-test.janet, which skips the behavioral leg where cc/janet.h
are missing). Nothing wires this into the default compile path yet — detecting
hot fns and installing the C version onto the var cell is the next step.

See docs/foundational-runtime-lever1-native-codegen.md for the ceiling
(native-C ~18-22x faster than bytecode, edges out JVM) and the leaf-first rule.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 16:56:33 +00:00
Dmitri Sotnikov
a2ce6bb5f6
Spike: native codegen (lever 1) feasibility for jolt-5vsp (#144)
Probes the ceiling and incremental strategy for compiling hot fns to native C,
the only lever that moves the ~10.8x Janet-VM floor the localization spike found.

Native-C mandelbrot (Janet native module) runs ~10-12ms — faster than JVM
Clojure (14.2ms) and ~18-22x faster than jolt's 219ms. The boundary cost is
asymmetric: a bytecode loop calling a C hot-fn 40k times is nearly free (~11ms),
but a C fn calling back into bytecode via janet_call costs ~3.5us/call (~152ms,
no win). So the strategy is leaf-first / whole-hot-cluster compilation, crossing
only at cold edges. A plain cc-built .so (no jpm) loads at runtime via require at
full speed, so the native tier fits jolt's dynamic compile model.

Adds the spike artifacts under spike/native/ and the writeup. Next step is
jolt-ihdp (IR->C for the numeric subset). No source changes.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 16:30:17 +00:00
Dmitri Sotnikov
ae3f9f6e84
Spike: localize the mandelbrot 15x floor (jolt-5vsp) (#143)
The jolt-vs-hand-Janet-vs-JVM mandelbrot comparison splits the 15.4x floor
into two layers: a Janet-VM floor (~10.8x JVM, optimal while-loop Janet over
unboxed doubles — only native codegen moves it) plus a ~1.43x jolt loop-
lowering overhead on top. The overhead is entirely the loop/recur -> recursive-
closure-called-per-iteration lowering; hand-Janet written the same way matches
jolt, while a while+var/set version is 1.43x faster. So a cheap backend win
(jolt-v28u) sits above the structural native-codegen lever.

Adds the spike artifacts under bench/ and the results writeup; marks the spike
done in the handoff. No source changes.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 16:20:40 +00:00
Dmitri Sotnikov
6c3fec6065
Add foundational-runtime epic handoff (#142)
The targeted-specialization work (jolt-ffn) concluded that the constant-factor
gap vs JVM is structural, not per-form: three targeted passes (field-read,
inline cache, ctor descriptor-bake) all came back flat. mandelbrot (pure
compute) is ~15x off JVM and that's the floor — Janet bytecode VM + mark-sweep
GC + indirect calls.

This doc hands off the successor epic (jolt-5vsp): the foundational levers
(native codegen, GC-pressure reduction, deeper devirt+inline) and, importantly,
the spike to run first — localize the 15x floor by comparing jolt-compiled vs
hand-written-Janet vs JVM mandelbrot before committing to any big lever. Also
records what not to repeat.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 15:56:10 +00:00
Dmitri Sotnikov
6772e28eae
Specialize record field reads in method bodies; fix with-meta on symbols (#141)
A protocol method reads its fields through the generic guarded keyword lookup
because the method's `this` param is untyped. defrecord now hints `this` with
the record type, the per-form inference seeds ^Record-hinted params (the
:fn branch previously typed all params :any — only the whole-program path
seeded phints), and run-passes feeds the inference the record shapes. So a
hinted param's field reads bare-index instead of going through the :jolt/type
tag guard.

This needed a with-meta fix: (with-meta sym ..) returned a proto'd table, so
symbol? was false and the macro-attached hint broke fn destructuring. Symbols
now carry metadata in-place in their struct (matching how the reader attaches
^hint), keeping symbol? true, as in Clojure.

Modest on dispatch (~3-5%): the field read is a small fraction of a dispatch;
the machinery (record-tag + protocol lookup + wrapper) dominates, which is the
inline-cache target (jolt-ez5h). But it's a correctness fix and lets any
^Record-hinted code — not just methods — drop the field-read guard per-form,
not only under whole-program.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 15:14:59 +00:00
Dmitri Sotnikov
7d0b1d5695
Broaden the benchmark suite; add jolt-vs-JVM scorecard (#140)
The ray tracer is compute-bound and the three existing benches only cover
alloc / megamorphic-dispatch / collections. Add three axes the epic needs to
judge itself holistically:

- mono-dispatch: monomorphic protocol dispatch. Its jolt/JVM ratio (~110x) is
  *worse* than megamorphic (~76x) — the JVM inline-caches a runtime-monomorphic
  call site to near-free while jolt does a full registry dispatch (devirt only
  fires on statically-proven receivers). Points at the call-site inline cache.
- mandelbrot: pure float compute, no alloc/dispatch. The floor at ~15x — native
  arith already gets close to the JVM.
- fib: recursion, call + integer-arith overhead.

run.sh gains JVM=1, which runs each bench on JVM Clojure too and prints the
jolt/JVM ratio. collections sized up now that the map is a HAMT (jolt-684u).
README documents the axes and the current scorecard.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 14:50:38 +00:00
Dmitri Sotnikov
f0293fb4ee
Parallelize the test gate; cache cold-init tests, drop the benchmark from it (#139)
run-tests.janet runs the same file set as `jpm test` across a pool of worker
processes (one `janet FILE` each, ev-based). The full gate goes from ~790s
serial to ~98s here (8x), and more on CI where the heavy files don't thrash on
swap. CI and the docs point at it; `jpm test` still works serially.

Three things dominated the wall:

- Nine integration tests cold-built a compile ctx (~8s each); switch them to
  api/init-cached so they share the prebuilt image. The cache key already
  fingerprints the ctx-shaping env vars, so the direct-link ones share one DL
  image and the rest share the plain one.
- core-bench's main ran on every gate (~35s of benchmark loops that assert
  nothing); gate it behind JOLT_BENCH=1.
- cli-test spawned `janet src/jolt/main.janet` ~20 times at ~8s cold each
  (340s under parallel load, and it was the whole wall); prefer build/jolt
  (~20ms baked ctx) when present, fall back to from-source for an unbuilt tree.

type-check-test stays on cold init: a snapshot-loaded ctx loses the success
checker's op/msg detail (jolt-vley). jolt-pria tracks caching from-source
startup generally, which would let cli-test drop the build/jolt preference.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 14:23:02 +00:00
Dmitri Sotnikov
307b65b45b
Fix -m arg drop under whole-program cache (jolt-4mui) + RFC 0003 sync (#138)
* Bind *command-line-args* after the deps-image cache swap (jolt-4mui)

Under whole-program (deps-image cache active), `jolt -m NS ARG` dropped ARG:
run-main set *command-line-args* on the current ctx, but a cache HIT then
replaced ctx with the saved image (via `set ctx cached`), whose *command-line-
args* was whatever got baked when the image was saved. The stale binding won at
`(apply NS/-main *command-line-args*)`, so -main ran with the wrong (usually
default) args — silently, for any optimized -m program.

Move set-command-line-args to AFTER the cache swap so it binds on the final ctx.
Repro/regression in deps-cache-args-test.janet: first run builds the image
(arg "first"), second run (cache hit) must echo "second", not the baked "first".

* docs: RFC 0003 — phm is a HAMT, sorted colls a red-black tree

The transients RFC described phm as "bucket-based copy-on-write" and mused about
"if it ever becomes a HAMT" — it is one now (jolt-684u), and sorted maps/sets are
a red-black tree (jolt-0hbr). Update the deviation/future-work notes accordingly.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 13:34:08 +00:00
Dmitri Sotnikov
82525b6a81
sorted-map/set: red-black tree instead of O(n) sorted vector (jolt-0hbr) (#137)
Sorted collections were a sorted VECTOR — insert-at = (into (conj (subvec es
0 i) x) (subvec es i)) is O(n) per assoc with a large constant, so building was
O(n^2): 2000 entries took 55.6s.

Replace the rep with a red-black tree (assoc/dissoc/get/contains O(log n)),
ported from the ClojureScript PersistentTreeMap (cljs.core: tree-map-add /
balance-left / balance-right / tree-map-append / balance-*-del). This tier (25)
loads before 30-macros so deftype isn't available; a node is a plain vector
[color k v left right] and cljs's BlackNode/RedNode methods become functions —
the algorithm is unchanged. A sorted-set stores elements as keys with a nil
value; its ops project the key.

The seed read the old :entries vector directly for equality/printing; route
those through a new :entries op that materializes ascending from the tree
(core_types/sorted-entries-arr + main.janet's printer).

2000 sorted-map assocs: 55.6s -> 0.98s (57x); now O(log n) (per-op cost flat
from n=2000 to 10000). Correctness in test/integration/sorted-rbtree-test.janet
(shuffled insert ordering, delete rebalancing, custom comparator, comparator
lookup, subseq, count); sorted specs + full gate green. (key/val on sorted
entries stays a pre-existing gap — entries are pvecs not host tuples; jolt-jk23.)

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 06:04:46 +00:00
Dmitri Sotnikov
91e246c682
Persistent hash map: HAMT instead of O(n) copy-on-write (jolt-684u) (#136)
* Add benchmark suite for alloc/dispatch/collection workloads (jolt-1r86)

The ray tracer is float-compute-bound (devirt, alloc removal, type-proving all
measured flat on it), so it can't validate the optimization passes. Add a small
cross-language suite (AWFY + CLBG style, portable Clojure) isolating the axes it
misses:

  binary-trees  allocation / GC pressure (escaping short-lived records)
  dispatch      megamorphic protocol dispatch (~1M dispatches/s; WP can't devirt)
  collections   persistent map/vector churn

bench/run.sh runs them; bench/README.md maps each to the pass it exercises.

collections immediately surfaced jolt-684u: the persistent hash map is O(n) per
assoc (flat copy-on-write bucket array, not a HAMT) — n=4000 assocs take 50s.
Invisible to the ray tracer (no maps).

* Persistent hash map: HAMT instead of O(n) copy-on-write (jolt-684u)

The map was a flat bucket array whose assoc copied the whole array every insert
(O(n)/assoc, O(n^2) to build). Compounding it, small maps are Janet structs that
only promoted to phm for collection keys — never for size — so a scalar-key map
stayed an O(n)-copy struct forever. Building a 4000-entry map took ~50s.

Two fixes, following ClojureScript's design:

- phm.janet is now a HAMT (hash array mapped trie): BitmapIndexedNode /
  ArrayNode / HashCollisionNode, 32-way, 5 hash bits per level, structural
  sharing — assoc/dissoc/get are O(log32 n). Translated from cljs.core, adapted
  to Janet's 32-bit bit-ops (the hash is carried unsigned, the level index is
  extracted with arithmetic, and bits are tested with band against 1<<i since
  brushift rejects negative bitmaps). The public phm-* API and the value shape
  (:jolt/type :jolt/phm, :cnt) are unchanged; transients are a separate rep and
  untouched.

- core_coll promotes a struct map to a phm past 8 entries (not only for
  collection keys), mirroring cljs PersistentArrayMap -> PersistentHashMap, so
  incremental building isn't O(n^2).

20000 raw assocs: 7.1s -> 0.105s. The collections benchmark: 16.7s -> 0.2s.
Correctness covered by test/unit/phm-hamt-test.janet (oracle vs a Janet table,
nil keys, dissoc, a real hash-collision pair, and a sub-linear-assoc guard);
full gate green.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 05:01:22 +00:00
Dmitri Sotnikov
c13a8ee402
Add benchmark suite for alloc/dispatch/collection workloads (jolt-1r86) (#135)
The ray tracer is float-compute-bound (devirt, alloc removal, type-proving all
measured flat on it), so it can't validate the optimization passes. Add a small
cross-language suite (AWFY + CLBG style, portable Clojure) isolating the axes it
misses:

  binary-trees  allocation / GC pressure (escaping short-lived records)
  dispatch      megamorphic protocol dispatch (~1M dispatches/s; WP can't devirt)
  collections   persistent map/vector churn

bench/run.sh runs them; bench/README.md maps each to the pass it exercises.

collections immediately surfaced jolt-684u: the persistent hash map is O(n) per
assoc (flat copy-on-write bucket array, not a HAMT) — n=4000 assocs take 50s.
Invisible to the ray tracer (no maps).

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 04:41:49 +00:00
Dmitri Sotnikov
bd9e542c78
Propagate declared record hints through the whole-program fixpoint (jolt-3ko) (#134)
A ^Record param hint was applied only at the final re-emit (reinfer-def), not
during the inter-procedural fixpoint. So a hinted param with no callers stayed
:any while inference ran, and a field read off it (e.g. (:origin ^Ray r)) never
told a non-inlined callee that its arg is a Vec3 — the callee's params stayed
unproven and its field reads kept the dynamic guard.

Seed declared hints as a param-type floor in the fixpoint: phint-seed (passes/
types) resolves an arity's :phints to positional record types via the
record-shapes registry, and infer-unit! initializes each fn's fresh param slots
from them instead of nil. A fixed declared type can't poison the least-fixpoint
the way an early-iteration :any would, and a hinted param now propagates its
(and its field reads') types to its callees during inference.

Scope: this closes the hinted-propagation gap. It does NOT help the ray tracer,
which uses zero ^-hinted params (only hinted fields) — its remaining type gap is
unhinted record-param inference on recursive/non-inlined hot fns, and per the
jolt-15jq A/B it's allocation-bound regardless (jolt-8flj). Tracked on the bead.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 03:38:03 +00:00
Dmitri Sotnikov
30a12f39ff
Fold jolt-deps into the jolt binary (#133)
Dependency resolution now lives in the `jolt` CLI itself instead of a separate
jolt-deps executable. `jolt` resolves a deps.edn into JOLT_PATH/JOLT_APP_PATHS
in-process and dispatches the deps subcommands:

  jolt -M:alias [args]   run the alias :main-opts
  jolt -A:alias CMD      run CMD with the alias paths
  jolt run FILE          resolve, then run FILE
  jolt path | tasks | task NAME

A deps.edn in the working dir is auto-resolved for the runnable commands
(repl/-m/-e/nrepl-server/FILE), so e.g. `jolt -M:nrepl` (or plain
`jolt nrepl-server`) starts an nREPL with the project and its deps loaded.

The runtime core stays deps-agnostic — it only reads JOLT_PATH. The resolver
(deps.janet) is reached only from the CLI entry and loads jpm lazily, so a run
with no deps.edn never touches it and an app baked from its own jolt/api entry
never links it. resolve-deps-argv only resolves on an explicit deps command or
when a deps.edn is present; help/version never do.

jolt-deps stays as a thin deprecation shim that forwards to `jolt`, so existing
scripts keep working. Docs (README, CLAUDE.md, building-and-deps, tools-deps)
and the help text updated.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 10:30:28 +08:00
Dmitri Sotnikov
048de0200d
Scalar-replace short-lived record allocations (jolt-15jq) (#132)
scalar-replace already folds non-escaping const-key map literals
((:k {:k a ..}) -> a, and drops a let-bound map that doesn't escape).
Extend the same fold to record constructors: a (->Rec a b c) is a
positional struct whose declared field order lives in the record-shapes
registry, so a field read on a non-escaping ctor folds to the matching
positional arg and the allocation disappears.

Direct form (:field (->Rec ..)) and the let-bound form both handled,
threaded through run-passes via a per-unit shape registry (new
jolt.host/record-shapes accessor). Soundness: ctor args must be pure
(duplicated/discarded like map vals), arg count must equal the field
count, and only declared-field reads fold — a record answers the virtual
:jolt/deftype key with its type tag and any other key with nil, neither
of which is a positional arg, so those keep the allocation. pure? now
treats a record ctor of pure args as pure, so nested records (a Ray
holding a Vec3) fold bottom-up.

Allocation-bound microbench (non-escaping record built + field-read in a
hot loop): 69.6s -> 2.4s, landing on the no-record arithmetic baseline.
The ray tracer is unchanged — its vec3 results escape (returned/stored
each op), so they genuinely allocate; that's a separate problem.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 01:21:36 +00:00
Dmitri Sotnikov
ae593b0f66
Keep record :type through depth-capping (jolt-3ko) (#131)
cap truncates a deep type's field VALUES to :any so the inter-procedural
fixpoint stays finite, but it rebuilt the struct via mk-struct and dropped the
record :type tag along the way. The tag is identity — independent of field
depth — so a record stored in a deep container (a Sphere in a world vector, a
material on a hit) degraded to a plain struct, and devirtualization (jolt-41m)
and record? folding silently stopped firing on it.

Preserve :type alongside :shape when capping. Verified: a protocol call on a
record read out of a vector now devirtualizes (the call node gets :devirt-type,
which needs the receiver's record type). Sound — the tag stays accurate; only
field values below the depth cap are truncated.

No measurable wall-clock change on its own (jolt's protocol dispatch is already
cheap), but it restores the record fast path / devirt / record?-folding on
records-in-containers, and unblocks downstream work that keys off record types.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-16 08:21:40 +08:00
Dmitri Sotnikov
04d3e192ed
Dead-code elimination in jolt uberscript (jolt-atg) (#130)
A bundle is closed-world — everything it needs is inlined and nothing is
required afterward — so a user defn unreachable from the entry's reference
graph can be dropped. The bundler now computes reachability from main-ns/-main
plus every non-prunable form and drops dead defn/defn- by exact source span
(formatting and reader macros in the surviving code are untouched).

Conservative and sound: only plain defn/defn- are prunable; a defn is kept if
its bare or ns-qualified name appears in any kept form, the closure runs to a
fixpoint, and any use of resolve/ns-resolve/requiring-resolve/find-var/intern/
eval/load-string disables pruning entirely. A parse failure on any file also
falls back to verbatim bundling, so the command stays as robust as a plain
concatenation. defmethod/defrecord/extend bodies are non-prunable and scanned,
so a fn reached only via dynamic dispatch stays live.

New reader/parse-all-spans returns [form start end] byte offsets so the drop
is a verbatim slice, not a re-print.

30-fn library used by a 3-fn entry: bundle 1114 -> 437 bytes (61% smaller, 27
dead fns dropped), output byte-identical.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 22:25:44 +00:00
Dmitri Sotnikov
a029afc127
Fold type predicates on proven types (jolt-wcw) (#129)
When the collection-type inference proves an argument's type, number?/
string?/keyword?/record?/nil?/some? fold to a compile-time boolean. A
const-fold now runs after inference so a folded predicate propagates and
collapses any if it gates to the taken branch.

Sound by construction: only a provable answer folds, and only when the
argument is side-effect-free (a const or local) so dropping its evaluation
is a no-op. Unknown types (:any/:truthy) and impure args keep the call.
vector?/set?/map? are left out — the :vec tag conflates a real vector with
a range/seq, so vector? could be wrong.

50M-iter loop, same shape isolated with a carry-only control: number? call+
branch 5080ms, predicate folded 1365ms — matching the 1417ms control floor,
so the 3.7x is entirely the eliminated call+branch.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 21:43:15 +00:00
Dmitri Sotnikov
b6fd49f5a8
docs: note malli, markdown-clj, hiccup in libraries.md (#128)
Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 20:21:25 +00:00
Dmitri Sotnikov
61d621521b
Library ports: get hiccup running, verify malli (reader + interop fixes) (#127)
* Reader: #() params survive syntax-quote (auto-gensym names)

#(...) named its synthesized params with bare gensyms, so a #() written inside a
syntax-quote had its params qualified to the current ns by sq-symbol — and a
qualified symbol isn't a valid fn param. hiccup's compiler emits
`(let [sb# ..] (iterate! #(.append sb# %) ..)), which broke with "Unable to
resolve symbol: ns/_NNNN".

Name the params with a trailing # (auto-gensym suffix, like Clojure's p1__N#) so
syntax-quote maps them consistently and leaves them unqualified. Harmless outside
a backtick (just a regular symbol name).

* interop: String/valueOf static + String is a CharSequence

Two interop gaps surfaced bringing up hiccup and malli:
- String/valueOf(Object): hiccup's compiler stringifies attribute values with
  (String/valueOf (or arg "")). Added the static — "null" for nil, else core-str.
- (instance? CharSequence s) returned false for a string; String implements
  CharSequence, and malli's :re validator gates on it before matching, so :re
  schemas always failed. instance-check now answers true for strings.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 19:36:13 +00:00
Dmitri Sotnikov
d223f40c91
Scope whole-program optimization to app namespaces (jolt-87e) (#126)
Under JOLT_OPTIMIZE a -m program run inferred + specialized EVERY loaded
namespace, including every transitive dependency. On a dep-heavy app that's
prohibitive: malli-app cold-started in ~2m10s (hundreds of dep namespaces, each
run through the per-form inline + inference passes).

The closed world a whole-program pass reasons over is the APP, not its
libraries. jolt-deps now passes the project's own source roots (its deps.edn
:paths) to the runtime as JOLT_APP_PATHS. A namespace loaded from an app root
gets full optimization (and joins the one whole-program fixpoint); a dependency
namespace compiles at default cost — :inline? off for its load, so the per-form
optimize passes don't run over library code — staying direct-linked but
generically typed (the open-world default). With no app roots declared (a bare
program run, or jolt without jolt-deps) everything counts as app, so behavior is
unchanged.

malli-app JOLT_OPTIMIZE cold start: 2m10s -> 4.5s. Compute-heavy programs whose
hot code is their own namespaces (the typed ray tracer) are unaffected — their
code is app code and still fully optimized (9s/frame render). Applied at runtime
in main for the same baked-at-build-time reason as JOLT_PATH; added to the
ctx-image cache key. Help text corrected: optimization is opt-in, not default.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 18:02:54 +00:00
Dmitri Sotnikov
d200725811
Support mutable deftype fields under shapes (jolt-c3q) (#125)
A deftype field tagged ^:unsynchronized-mutable / ^:volatile-mutable is set!-able,
but under direct-link immutable records are shape-rec tuples, so set! errored
("Can't set! field on non-deftype: tuple").

A deftype with any mutable field now opts out of the shape-rec layout and uses
the existing :jolt/deftype table form regardless of :shapes? — set! already
mutates that form and field reads route through the tagged-table path. Such a
type is also not registered as a shape, so the inference never emits a bare-index
read against the table. Immutable deftypes/records keep the fast shape-rec.

deftype extracts per-field mutability from the field metadata and passes it to
make-deftype-ctor, which picks the representation at ctor-build time.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 17:45:29 +00:00
Dmitri Sotnikov
8ee04eed8e
Fix record field-reorder redefine (jolt-wf4) and builtin-shadowing local names (jolt-fjb1) (#124)
* Don't direct-link a var redefined earlier in the same unit (jolt-wf4)

defrecord/deftype expands to (do (def R (make-deftype-ctor ...)) (def ->R R) ...),
so the ->R alias references R within one compiled unit. Under direct-link a var
ref embeds the cell's root as a compile-time constant, but on a redefine R's old
root is still in place when that unit compiles — the (def R new) sibling hasn't
run yet — so ->R sealed to the stale pre-redef ctor. (defrecord R [x a])
(defrecord R [a x]) (:a (->R 10 20)) read the old [x a] layout and returned 20.

Track the vars a unit (re)defines and force their later in-unit references to the
live indirect deref. The cell is registered only after its own init is emitted,
so a recursive self-reference inside the init still direct-links (it runs after
the def completes); only sibling references after the def go indirect.

* Emit Janet's `in` as a value so a user local can't shadow it (jolt-fjb1)

The back end emits `in` to deref var cells ((in cell :root)) and index
shape-recs. It emitted the bare symbol, so a user local named `in` shadowed
Janet's builtin in the surrounding scope and the generated cell-deref called the
user's value as a function — "<table> called with 2 arguments, possibly expected
1". malli's explainer binds [value in acc], so m/explain hit this on every
schema (m/validate was unaffected — its path doesn't bind `in`).

Embed `in`'s function value at the emit sites (as jolt-call/core-get already
are); a value in head position can't be shadowed. Fixes m/explain on malli
(loaded with JOLT_FEATURES=clj so its .cljc reader-conditionals resolve).

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 16:46:00 +00:00
Dmitri Sotnikov
747ac87e03
Merge pull request #123 from jolt-lang/fix-outstanding-bugs
Fix outstanding bugs: clojure.walk lists, deftype toString, ISeq call forms
2026-06-15 15:01:45 +00:00
Yogthos
80aae55977 Evaluate non-array ISeq forms (cons/concat/lazy-seq) as calls (jolt-2rx)
eval-form treated only a reader LIST (a Janet array) as a call; a runtime-built
list — a plist or lazy-seq from cons/concat/list or ~@ (list?/seq? true but
array? false) — fell through to self-eval. So (eval (cons '+ '(1 2))) returned
the list as data instead of 3, and a macro whose output contained such a subform
left it unevaluated. Add a plist?/lazy-seq? branch that coerces to the element
array via d-realize and dispatches through eval-list; an empty list self-evals.

The analyzer already punts these forms to the interpreter (analyze's :else ->
uncompilable -> interpreter fallback), so this one interpreter branch fixes the
correctness bug across the eval and macro-expansion paths; compiling them
directly (vs punting) would be a separate perf change. Verified: conformance
355/355, syntax-quote ~@ splice, list values unchanged.
2026-06-15 10:27:34 -04:00
Yogthos
a2c4fe317b Honor a deftype's custom Object/toString in .toString and str (jolt-rt6n)
A deftype with (Object (toString [_] s)) had its toString ignored: the generic
object-methods "toString" fired in dispatch-member before the record's own
method (the record isn't a tagged shim, so that guard passed), and str rendered
the #Type{...} data repr instead of routing through toString.

- dispatch-member: a record's own method (instance/reified/protocol) now wins
  over the generic object-methods table — so .toString/.equals/.hashCode on a
  record use the record's definitions; plain records still reach object-methods.
- str: add a late-bound record-tostring-cb (wired per-ctx by
  install-print-method-cb!, mirroring print-method-cb) that str-render-one
  consults for records — a deftype with a custom toString renders via it, plain
  records keep the data repr. pr-str is unchanged.

Needed by hiccup's RawString. Adds deftype-tostring-spec (.toString + str +
concatenation + a regression guard that record-less-toString keeps its repr).
2026-06-15 10:22:09 -04:00
Yogthos
fb1ec25c69 Fix clojure.walk to descend lists and seqs (jolt-khk)
walk only handled vector?/map? and fell through :else for everything else, so
postwalk over a quoted list (a plist) never touched its elements —
postwalk-replace with symbol keys silently no-op'd, which broke
clojure.template/apply-template (found during reitit work). Add list? (rebuild as
a list) and seq? (map over it) branches after the vector/map ones so concrete
collections stay authoritative. Adds walk-spec covering list/seq walking plus a
vector/keywordize regression guard and the apply-template trigger.
2026-06-15 10:16:47 -04:00
Dmitri Sotnikov
1ffb348c5e
Merge pull request #122 from jolt-lang/refactor-phase1-interop-subdir
Refactor phase 1: split host-interop into a src/jolt/interop/ subdir (jolt-jx5l)
2026-06-15 13:55:41 +00:00
Yogthos
7cb86e8f7b Refactor phase 1: split host-interop into a src/jolt/interop/ subdir (jolt-jx5l)
The 813-line host_interop.janet (java.lang statics, java.time, the 458-line
java.io/util/net/sql/text install-io!, and collection interop all in one file)
becomes a 19-line aggregator over src/jolt/interop/:
- java_base.janet  — java.lang statics (Math/Thread/System/Long) + the java.time
                     shim + the shared chr/pad2/formatter coercion helpers
- host_io.janet    — java.io/util/net/sql/text shims; imports java_base for the
                     shared helpers it reuses (it constructs java.time values and
                     formats dates)
- collections.janet— the late-bound .iterator/.nth/.count/.seq interop hooks

host_interop.janet just loads the three and runs install!/install-io!/
install-collections! in order. chr/pad2/formatter become public (they cross the
java_base->host_io boundary now). The registry machinery (class-statics/
tagged-methods/register-*!) stays in the evaluator, which loads first — moving it
out touches the hot dot-dispatch and is left as a separate step.

Pure move, no behavior change. Also fixes a cache footgun the subdir exposed:
source-fingerprint walked src/jolt/ with a non-recursive (os/dir), so an edit to
an interop/ file would not have busted the ctx image cache (stale-image bug) —
made it recurse, keyed by repo-relative path.

Full gate green: host-interop-spec (130+ rows across every JDK area), ctx-image
cold/warm, conformance x3, suite >=4695/88, fixpoint.
2026-06-15 09:37:28 -04:00
Dmitri Sotnikov
10482dae35
Merge pull request #121 from jolt-lang/refactor-phase4b-ctx-image
Refactor phase 4b: share the ctx-image load/save dance (jolt-q5ql)
2026-06-15 13:26:38 +00:00
Dmitri Sotnikov
bf30323772
Merge pull request #120 from jolt-lang/refactor-phase5c-coll-dispatch
Refactor phase 5c: dispatch core collection ops on :jolt/type via one case (jolt-bvek)
2026-06-15 13:09:15 +00:00
Yogthos
21721473a5 Refactor phase 4b: share the ctx-image load/save dance (jolt-q5ql)
init-cached (core image) and the deps-image (main.janet) each hand-wrote the same
fork->slurp->validate-ctx?->rewire load and the snapshot->tmp->atomic-rename save.
Extract load-ctx-image / save-ctx-image (in api.janet, beside snapshot/fork): the
two callers now differ only in the validity predicate they pass — none for the
core image (its source fingerprint is already in the path), a deps-manifest mtime
check for the deps image. The per-process print-method-cb rewiring an image
restore must replay is threaded as a callback so the helpers don't depend on core.

Kept in api.janet rather than a new ctx_image.janet module: Janet's `use` doesn't
transitively re-export, and snapshot/fork already live in api and are consumed via
(use ./api) by main and the test harnesses — a separate module would force every
caller to import it directly. (load-image/save-image collide with Janet builtins,
hence the ctx- prefix.)

Full gate green; ctx-image-test cold/warm + deps tests pass.
2026-06-15 09:08:56 -04:00
Yogthos
dd4100db73 Refactor phase 5c: dispatch core collection ops on :jolt/type via one case (jolt-bvek)
core-count/core-seq/core-conj each walked a chain of (and (table? x) (= :jolt/X
(get x :jolt/type))) predicates — re-fetching the type tag per predicate and, for
conj, a 14-deep nested if. Replace with a single (case (get coll :jolt/type) ...)
per op: the type is read once and the arm calls the concrete op directly. Host
values (tuple/array/nil) and tuple-based shape-recs carry no :jolt/type and stay
in a per-op fallback cond (shape-rec? kept before tuple? — shape-recs ARE tuples).
Factor the two map-conj paths (phm vs struct) into one conj-into-map parameterized
by the assoc fn.

Behavior-preserving: the :jolt/type tags are disjoint, so the case is an exact
re-expression of the predicate chain; the fallbacks reproduce the originals
(incl. count erroring on a raw host table, which only seq ever handled).

This is the perf-neutral realization of the planned collection vtable. A shared
:jolt/type->{ops} table was tried first and REGRESSED core-bench ~2-4% (table
lookup + indirect call on the hot path); the inline case instead runs ~1.7%
FASTER than main (4864ms vs ~4950ms baseline) since one type-get + a jump beats
the sequential predicate chain. Full gate green (suite >=4695/88, fixpoint).
2026-06-15 08:51:34 -04:00
Dmitri Sotnikov
dd64fd4e74
Merge pull request #119 from jolt-lang/refactor-phase3b-try-shape
Refactor phase 3b: keep :try IR nodes structs, not phms (jolt-26dm)
2026-06-15 09:26:03 +00:00
Dmitri Sotnikov
53967ce221
Merge pull request #118 from jolt-lang/refactor-phase3a-ir-walk
Refactor phase 3a: one map-ir-children combinator for IR rewrite walks (jolt-26dm)
2026-06-15 09:15:46 +00:00
Yogthos
130a6c5c4d Refactor phase 3b: keep :try IR nodes structs, not phms (jolt-26dm)
analyze-try assoc'd :catch-sym/:catch-body/:finally nil-when-absent, so a try
with no catch (or no finally) carried a nil-valued key — which makes the node a
phm in jolt's map representation and forces the back end to densify it
(norm-node) before reading :op. That's the map-nil-representation trap Phase 2
already cleaned up for def/fn/arity nodes. Add those keys only when the clause is
present, matching the arity :rest discipline; a try node stays a fast struct.

Behavior-invisible: emit-try reads each key with a nil-safe (node :k) and gates
on it, so an absent key and a present-nil key are indistinguishable to every
consumer. Adds ir-try-shape-test asserting the node shape across all four
try/catch/finally combinations plus end-to-end eval.

Note on scope: the plan's "delete the defensive norm-node calls" is NOT done — it
can't be. {:op :const :val nil} (e.g. (def x nil)) and nil map keys are
inherently phm, so the emit-dispatch norm-node guards a real case, not a
present-or-absent artifact. This PR removes a source of gratuitous phm nodes
rather than the densification itself. Full gate green.
2026-06-15 05:09:02 -04:00
Dmitri Sotnikov
61e4bbf2de
Merge pull request #117 from jolt-lang/dedup-eval-dot-member
Dedup the two .method dot-dispatch arms into one dispatch-member (jolt-eos3)
2026-06-15 08:58:01 +00:00
Yogthos
8789777323 Refactor phase 3a: one map-ir-children combinator for the IR rewrite walks (jolt-26dm)
Six bottom-up IR rewrites (const-fold, inline-node, subst, flatten-lets,
subst-lookup, scalar-replace) each hand-listed every op's child positions —
~250 lines of identical "recurse children, rebuild" arms that had to be kept in
sync whenever an op was added. Extract one map-ir-children into ir.clj that
knows each op's child layout; each walk keeps only its genuine specials
(const-fold's invoke/if, inline-node's invoke, subst's local/let alpha-rename,
scalar-replace's invoke/let folds) and delegates the rest.

The combinator is total over the op set, so the walks are now total too: a
couple soundly gain coverage they previously skipped (const-fold now folds
inside :try; subst-lookup now recurses :def inits, which fixes a latent dangling
ref where a dropped const-key-map binding was referenced inside a def). These
are sound — all six are result-preserving optimizations — and 3-mode conformance
+ fixpoint confirm identical program behavior.

map-ir-children is shape-preserving for :try (recurses :catch-body/:finally only
when present, never assoc's nil) so it can't turn a struct node into a phm.
Written with cond/get only, matching the passes' tier, so no new load-order dep.

Predicates (body-closed?/pure?/local-escapes?), the type-threading infer, and the
Janet backend emit stay as-is: their conservative :else defaults / [type node]
threading / host language don't fit a node-rebuilding combinator.

Adds ir-passes-test coverage for folding reaching fn/loop/try bodies. Full gate
green (conformance x3, suite >=4695/88, fixpoint stage1==2==3, inline-sra + devirt).
2026-06-15 04:57:42 -04:00
Dmitri Sotnikov
b54bd6c856
Merge pull request #116 from jolt-lang/refactor-phase5de-seed-overlay-docs
Refactor phase 5d/5e: seed↔overlay registry + boundary docs (jolt-bvek)
2026-06-15 08:36:07 +00:00
Yogthos
0406b315a9 Dedup the two .method dot-dispatch arms into one dispatch-member (jolt-eos3)
eval-dot copy-pasted its entire dispatch chain across the (. obj method args...)
and (. obj member) forms — string/number/object/tagged-shim lookup duplicated,
hand-synced on every interop change. Extract one dispatch-member that takes the
evaluated args plus a has-args flag. The shared head (string/number/object/
tagged) is single-sourced; the genuinely divergent tails (call form: record →
native field → coll-interop(args); bare form: zero-arg coll-interop → field /
zero-arg method) stay branched on has-args. The guards that differed between the
arms (object-methods checks table? only; tagged dispatch checks table-or-struct;
bare-form tagged dispatch requires the member present) are preserved verbatim,
keyed off has-args, so behavior is identical.

Adds a "dot dispatch arms" spec locking the divergent cases: zero-arg vs
with-arg coll-interop, record/deftype zero-arg vs with-args methods, -field
access. Full gate green.
2026-06-15 04:35:52 -04:00
Yogthos
793b55f1f3 Refactor phase 5d/5e: seed↔overlay registry + rep↔API boundary docs (jolt-bvek)
5d: document the seed↔overlay boundary and add a drift check. core fns split
across a Janet seed (core-X, registered in core-bindings) and a Clojure overlay;
five names (char?/sorted?/sorted-map?/sorted-set?/transduce) carry a defn in
both, with the overlay copy authoritative and the seed copy internal-only. The
into-vs-transduce home asymmetry was undocumented. Adds docs/seed-overlay-registry.md,
SEED-TWIN: comments at the five seed sites, and a build-time drift check
(test/unit/seed-overlay-registry-test.janet) that recomputes the twin set from
source and fails if it diverges or a twin leaks into core-bindings.

5e: rep↔API pointer comments in pv/plist/phm/phs/lazyseq (representation lives
here; Clojure-facing ops dispatch in core_coll/core_types) and back-pointers in
core_coll. No behavior change — comments, docs, one source-analysis test.

Full gate green (suite ≥4695 pass / ≥88 clean files), drift check passes.
2026-06-15 04:22:05 -04:00
Dmitri Sotnikov
6c0142e625
Merge pull request #115 from jolt-lang/refactor-phase4-config-cache
Refactor phase 4: lift run-mode into config + fix cache-key footgun (jolt-q5ql)
2026-06-15 07:32:11 +00:00
Dmitri Sotnikov
7632be811b
Merge pull request #114 from jolt-lang/refactor-phase3c-dedups
Refactor phase 3c: dedup collection readers + phm bucket scans (jolt-26dm)
2026-06-15 07:15:44 +00:00
Yogthos
fbca0890f5 Refactor phase 4: lift run-mode into config.janet + fix the cache-key footgun
main.janet held ~45 lines of env-knob policy (open-mode / direct-link / optimize
/ shapes / whole-program gates) that couldn't be unit-tested without the CLI, and
two disk-image caches (api/init-cached, main/deps-image) each hand-built a
POSITIONAL "%q|%q|..." key that silently misaligned if a ctx-shaping knob was
added in only one place.

config.janet now owns:
  ctx-shaping-env-vars  the canonical list of env vars that shape the built ctx
  ctx-cache-key         a labeled key (name=value) over a prefix + every shaping
                        var, so adding a knob updates BOTH cache keys at once and
                        can't positionally alias two different builds
  resolve-run-mode      [open-mode? main-entry?] -> the ctx env knob map

main shrinks to: compute open-mode?/main-entry? from argv, call resolve-run-mode,
install the knobs. Both deps-image-path (main) and image-cache-path (api) build
their keys via ctx-cache-key. New test/unit/config-test.janet locks in the
run-mode cases and asserts every ctx-shaping env var participates in the key.

Scope: this is 4a + the 4b cache-key footgun fix. The optional 4b cleanup
(folding the load/save image dance + aot marshal helpers into one ctx_image
module) is left for a follow-up — it's lower value and higher blast radius.

No behavior change (cache keys now key on a superset of env vars, so at worst a
one-time cold rebuild). Gate green: conformance 355x3, clojure-test-suite 4718
pass (>= 4695 baseline), config-test, full jpm test exit 0.
2026-06-15 03:15:24 -04:00
Dmitri Sotnikov
5ad8e8281a
Merge pull request #113 from jolt-lang/refactor-types-split
Refactor phase 5a: split types.janet into value-layer modules (jolt-bvek)
2026-06-15 06:59:50 +00:00
Yogthos
33c39e8741 Refactor phase 3c: dedup the collection readers and phm bucket scans
Two small structural dedups from the plan's Phase 3c.

reader.janet: read-list/read-vector/read-set each had a copy of the same
read-loop (skip whitespace, stop at close char, drop #_ discards, splice #?@).
The skip/splice logic had drifted between them once. Hoisted into one
read-delimited [s pos close err] -> [items end]; the three readers now just wrap
its result. read-map keeps its own loop (its key/value pairing needs a different
value-slot scan).

phm.janet: phm-bucket-find/contains?/assoc/dissoc and phm-get each open-coded the
same stride-2 key scan. Extracted bucket-index-of [bucket k] -> index|nil; all
five now share it.

No behavior change. Gate green: conformance 355x3, clojure-test-suite 4718 pass
(>= 4695 baseline), full jpm test exit 0.
2026-06-15 02:58:16 -04:00
Dmitri Sotnikov
aab7c6396c
Merge pull request #112 from jolt-lang/refactor-evaluator-split
Refactor phase 2a: split evaluator.janet + explode eval-list (jolt-oudv)
2026-06-15 06:46:42 +00:00
Yogthos
24b3d59b13 Refactor phase 5a: split types.janet into value-layer cluster modules
types.janet held five concerns under one generic name. Split into sibling
modules along its existing section boundaries:

  types_symbols    characters + symbol helpers
  types_var        Var
  types_ns         Namespace
  types_ctx        Context (+ inst/uuid values)
  types_protocols  protocol/type registry + shape-records

types.janet is now a pure aggregator: it loads the clusters in dependency order
and re-exports their defs (import :prefix "" :export true), so every consumer
keeps its single (use ./types) unchanged.

Order-preserving (statically verified: zero backward references, zero
cross-cluster private helpers — the cleanest of the seed splits). No behavior
change.

Gate green: conformance 355x3, clojure-test-suite 4718 pass (>= 4695 baseline),
full jpm test exit 0.
2026-06-15 02:42:42 -04:00
Yogthos
0a531dd1e8 Refactor phase 2a: split evaluator.janet + explode eval-list into named handlers
evaluator.janet was a 2597-line file with a 680-line eval-list. Split into
cluster modules behind a re-export aggregator (same pattern as core):

  eval_base     forward vars, syntax-quote, ns-loading, registries, jolt-invoke
  eval_resolve  symbol/var resolution, params, destructuring, class lookup
  eval_runtime  protocols, multimethods, deftype/reify, install-stateful-fns!
  eval_special  the special forms (eval-list dispatch)

evaluator.janet stays the module every consumer imports: it loads the clusters
in dependency order and re-exports their defs (import :prefix "" :export true),
so the five (use ./evaluator) consumers are unchanged. It still owns the
eval-form entry that ties resolution + special forms + map/coll evaluation.

In eval_special, the giant eval-list match is exploded: each multi-line arm is
now a named (defn eval-<form> [ctx bindings form] ...) — eval-def, eval-fn*,
eval-let*, eval-loop*, eval-try, eval-set!, eval-dot, etc. — and eval-list is a
thin dispatch table over them. "where is try handled" is now `grep eval-try`.

Order-preserving (statically verified: no symbol used before its cluster loads;
zero backward refs). 27 helpers shared across clusters are now public so `use`
shares them. The two near-duplicate .method dot blocks are NOT merged here — that
is a behavior-sensitive dedup tracked separately (jolt-eos3); this PR is pure
moves + the mechanical eval-list explosion, no behavior change.

Gate green: conformance 355x3, clojure-test-suite 4718 pass (>= 4695 baseline),
full jpm test exit 0.
2026-06-15 02:29:23 -04:00
Dmitri Sotnikov
ea5c3452de
Merge pull request #111 from jolt-lang/refactor-core-split
Refactor phase 2b: split core.janet into cluster modules (jolt-nma8)
2026-06-15 06:21:16 +00:00
Yogthos
0c324178d4 Refactor phase 2b: split core.janet into cluster modules behind a re-export aggregator
core.janet was a 3013-line grab-bag. Split into six order-preserving cluster
modules:

  core_types  vector helpers, predicates, math, comparison, equality
  core_coll   collections, transducers, seqs, HOFs, constructors
  core_print  string + pr-str/str rendering
  core_io     I/O, files, JDBC, compare, type
  core_refs   arrays, bit ops, coercions, hash, atoms/refs
  core_extra  additional clojure.core fns, transients, hashing

core.janet stays the module everyone imports: it loads the clusters in
dependency order and re-exports each one's defs (import :prefix "" :export true),
so every consumer keeps its single (use ./core) with no change. The aggregator
also owns core-bindings and init-core!, which reference fns from every cluster.

The split preserves definition order exactly (verified: no symbol is used in a
cluster that loads before its definition), so seed load-order semantics are
unchanged. Three private helpers used across clusters (map-entries-of,
map-assoc1, str-render-one) are now public so `use` shares them; the five
forward vars (canon-key/jolt-equal?/pr-render/core-compare/print-method-cb) each
stay within their owning cluster.

No behavior change. Gate green: conformance 355x3, clojure-test-suite 4718 pass
(>= 4695 baseline), full jpm test exit 0.
2026-06-15 02:04:17 -04:00
Dmitri Sotnikov
c9f60d70c4
Merge pull request #110 from jolt-lang/refactor-passes-split
Refactor phase 2c: split passes.clj into fold/inline/types (jolt-8qrw)
2026-06-15 05:58:04 +00:00
Yogthos
8e634e7106 Refactor phase 2c: split passes.clj into fold/inline/types behind a façade
passes.clj was a 1486-line grab-bag mixing three weakly-coupled concerns. Split
along the clusters the review mapped (only run-passes + the dirty flag were
shared):

  jolt.passes.fold    const-fold + the shared scalar-const? predicate (base)
  jolt.passes.inline  inline + flatten-lets + scalar-replace
  jolt.passes.types   collection-type inference + success checker + driver API
  jolt.passes         façade: run-passes + :refer re-exports of the driver fns
                      the back end looks up by name

scalar-const? was used by both the inline pass and the inference walk, so it
moves to fold (the base layer) and both refer it. The check-mode state stays
private to jolt.passes.types behind a new run-inference fn; run-passes calls it.

build-compiler! loads the three in dependency order before the façade, mirroring
the existing jolt.ir -> jolt.analyzer bootstrap. No behavior change. Also fixed
the stale ns docstring that listed four passes and omitted the type system.

Gate green: conformance 355x3, clojure-test-suite 4718 pass (>= 4695 baseline),
full jpm test exit 0.
2026-06-15 01:41:28 -04:00
Dmitri Sotnikov
b06855db1f
Merge pull request #109 from jolt-lang/refactor-phs-split
Refactor: extract PersistentHashSet from phm.janet (jolt-bvek)
2026-06-15 05:19:05 +00:00
Yogthos
bad418e917 Refactor: extract PersistentHashSet from phm.janet into phs.janet
Completes the phm.janet decomposition (jolt-bvek): after lazyseq left, the set
follows. phm.janet is now purely the PersistentHashMap; phs.janet is the thin
set layer over it (members are keys -> true), and (use ./phm) for the builders.

Importers using set?/phs-* via (use ./phm) add (use ./phs); backend's emitted
set literal head changes phm/make-phs -> phs/make-phs. Behaviour unchanged
(sets verified interpreted + compiled; full gate green).
2026-06-15 00:54:05 -04:00
Dmitri Sotnikov
7cffe85298
Refactor: extract LazySeq from phm.janet into lazyseq.janet (#108)
phm.janet held the PersistentHashMap, the PersistentHashSet, AND the LazySeq
primitives — a lazy sequence has nothing to do with hash maps; both were just
tagged tables, which is why they shared a file (jolt-bvek). An agent looking for
lazy-seq realization would never grep phm.janet.

Move the LazySeq section (lazy-seq?/make-lazy-seq/realize-ls/ls-first/ls-rest/
ls-rest-cached/ls-seq/ls-count/lazy-cons) to a new self-contained lazyseq.janet
(janet builtins only, no jolt deps). Importers that used the fns through
(use ./phm) add (use ./lazyseq); host_interop's one phm/lazy-seq? becomes
lazy-seq?. Behaviour unchanged (covered by test/unit/lazy-seq-test.janet + the
full gate). phs split is a follow-up.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 04:41:39 +00:00
Dmitri Sotnikov
62fe11fa0d
Refactor phase 1: consolidate host interop into one module (#107)
Cheap-version consolidation of the host-interop sprawl (jolt-jx5l). The JVM
class/method shims were scattered across javatime.janet, evaluator.janet,
api.janet and core.janet; the recent hiccup/markdown/malli fixes landed ad hoc.

- Rename javatime.janet -> host_interop.janet. The name described ~20% of its
  contents (java.io/util/net/sql/lang all lived there); it is now the one home
  for host shims, and greppable.
- Move the four hardcoded static tables (Math/Thread/System/Long) out of the
  evaluator and register them through the generic class-statics registry. The
  special-case dispatch in resolve-sym is deleted — one mechanism, not two.
- Move the collection-interop wiring (set-coll-realizer!, set-coll-interop!,
  malli's LazilyPersistentVector/PersistentArrayMap statics) from api.janet into
  host_interop's install-collections!. api's 40 lines of wiring become a single
  (import ./host_interop).

No behavior change. Full per-package src/jolt/interop/ split is the follow-up;
string/number/object method tables and core's File/JDBC ctors stay put for now
(they're coupled to the dot-dispatch / collection layer). Full gate green.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 03:56:25 +00:00
Dmitri Sotnikov
910c4b6c99
Protocol/interop fixes to run metosin/malli (jolt-ltwk) (#105)
* Protocol/interop fixes to run metosin/malli

Bringing up malli (schema validation) surfaced a batch of protocol and host-interop
gaps. m/validate now works across the schema vocabulary (predicates, :map incl.
nested/optional, :vector, :tuple, :enum, :maybe, :and, bounded int/string).

- extend-type and reify now accept MULTIPLE protocols in one form (each bare
  symbol switches the current protocol). reify records every protocol it
  implements, so instance?/satisfies? recognise all of them.
- Protocol method params support destructuring: reify/extend-type/deftype/
  defrecord emit (fn ...) (which desugars patterns) instead of raw fn*.
- instance? of a PROTOCOL works like satisfies? for reify/record instances,
  matching short names across qualified/bare protocol references.
- @x reads as the qualified clojure.core/deref, so it still derefs where a ns
  excludes and rebinds deref (malli does). Updated reader-test + the reader
  spec/grammar (S11, deref rule).
- Java collection interop on jolt collections: .nth/.count/.valAt/.get/.seq/
  .containsKey route to the clojure.core equivalent (1-arg and 0-arg paths).
- java.util.HashMap capacity/load-factor constructors + .putAll.
- A class used as a value resolves to its instances' type, so Pattern -> the
  regex type (malli keys class-schemas by it).
- Shims for malli's load path: LazilyPersistentVector/createOwning and
  PersistentArrayMap/createWithCheck statics.

m/explain not yet working (jolt-fjb1). Full gate green.

* satisfies? recognizes reify, consistent with instance?

A reify's protocol methods are instance-local, so they aren't in the global type
registry that type-satisfies? consults — satisfies? returned false for a reify
even when it implemented the protocol. Check the protocols the reify records on
itself (the same :jolt/protocols list instance? uses), matching short names like
instance? does. Covers single- and multi-protocol reify.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 03:20:33 +00:00
Dmitri Sotnikov
d1f73f1740
Architecture refactor: plan + phase 0 (dead code + bugs) (#106)
* Add architecture refactor plan

Synthesizes a six-part architectural review into phased, gate-validated cleanup
work. Targets LLM-maintainability: one home per feature, no god-files, explicit
checked contracts, no copy-paste dispatch. No code changes yet — the plan only.

* Refactor phase 0: dead code + isolated bugs

Pure cleanup ahead of the structural phases (docs/architecture-refactor-plan.md).
No behavior change except the two bug fixes, which are covered by a regression row.

Dead code (all verified zero-reference or overridden):
- core-resolve / core-satisfies? / core-type->str seed stubs + bindings —
  resolve and satisfies? are interned by install-stateful-fns! (the seed copies
  were shadowed); type->str was an inert SCI stub with no callers.
- find defined twice in 20-coll.clj; the dead copy returned a plain vector
  (wrong — the live def at :787 returns a real map-entry) with a comment that
  contradicted it.
- mark-hint (passes.clj), phs-to-struct (phm), shape-vals / ns-imports-fn
  (types) — unreferenced.
- redundant local pad2 in javatime (module-level one already in scope).

Bugs:
- File.toURL stored :url but every :jolt/url method reads :spec, so a URL from
  (.toURL file) returned nil from all its methods. Now stores :spec (+ spec row).
- pl-rest had a no-op (if (plist? r) r r); collapsed to r.
- :map-shapes? was missing from the deps-image cache key — two runs differing
  only in map-shapes could reuse each other's image.

Also dropped read-quote's unused pos param. Full gate green.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 03:01:55 +00:00
Dmitri Sotnikov
88b7d13401
Stdlib/reader fixes to run markdown-clj (#104)
Bringing up yogthos/markdown-clj surfaced a batch of Clojure-conformance gaps:

- clojure.java.io/writer returned nil for a Writer/StringWriter (it only
  handled paths); now passes a Writer or file handle through, like reader does.
- StringWriter had no :close field, so with-open errored closing it.
- java.io.Reader had no .readLine method (only the :read-line-fn used by
  line-seq); markdown's main loop calls .readLine directly.
- Writer.write(int) wrote the int's digits instead of the char for that code.
  StringBuilder.append(int) keeps Java semantics (the digits) — the two differ,
  so the char-code path is local to the writer, not shared render-piece.
- drop-while over a string errored in array/slice; it now char-seqs the string
  like take-while/remove already do.
- re-seq returned an empty seq instead of nil on no match, so
  (if-let [m (re-seq ...)] ...) always took the truthy branch — an infinite loop
  in markdown's thaw-string.
- The #() reader didn't scan % inside map {} or set #{} literals, so
  #(identity {:text %}) compiled as a 0-arg fn.

re-seq-nil and the #() map/set scan are general bugs, not markdown-specific.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 01:48:15 +00:00
Dmitri Sotnikov
288b20956c
Shim java.util.Iterator: (.iterator coll)/.hasNext/.next over jolt collections (#103)
Some Clojure libraries loop with the Java Iterator protocol — e.g. hiccup's
iterate! does (let [it (.iterator coll)] (while (.hasNext it) (f (.next it)))).
jolt had no Iterator, so (.iterator coll) returned nil and the loop did nothing
(silently dropping content). Add an (.iterator coll) object-method that returns a
:jolt/iterator over any seqable, with hasNext/next; the collection is materialized
via core's realize-for-iteration (late-bound through the evaluator since core
loads after it, wired in api). Found while bringing up weavejester/hiccup.

Co-authored-by: Yogthos <yogthos@gmail.com>
2026-06-15 00:28:45 +00:00
Dmitri Sotnikov
874b6140ee
Merge pull request #102 from jolt-lang/startup-perf
Fast startup: inference opt-in, dependencies compile once (jolt-87e)
2026-06-14 22:47:52 +00:00
Yogthos
9f1cebc097 Fast startup: inference is opt-in, and dependencies compile once (jolt-87e)
Two things made a program run catastrophically slow to start (ring-app: 111s),
which is backwards — direct-linking should make startup FASTER (Clojure: smaller
classes, faster startup; Stalin: whole-program analysis is strictly ahead-of-
time, zero startup cost). We were doing the opposite: paying compile-time work at
every startup.

1. Decouple inference from direct-linking. Direct-linking (cheap compile-time
   call resolution) was bundled with the whole inference/specialization pipeline
   (inlining + scalar replacement + structural types + the per-ns re-emit
   fixpoint) — the expensive part. Now a program run direct-links + uses records
   by default (fast, faster calls) but the inference is opt-in via JOLT_OPTIMIZE
   (or an explicit JOLT_DIRECT_LINK / the native build). ring-app: 111s -> 6s.

2. Compile dependencies once. The baked core loads in ~10ms, but a program still
   re-compiled every dependency namespace (reitit, ring, selmer, honeysql, …)
   from source on every run — the whole 6s. Snapshot the ctx after the require
   chain to a per-project image (the same marshal/fork the core image uses),
   keyed on version + entry + source roots + flags, with a per-file mtime
   manifest so any edit invalidates it. First run compiles + caches; later runs
   fork the image (~10ms) and skip compilation. ring-app: 6s -> ~1s warm,
   competitive with the JVM. JOLT_NO_DEPS_CACHE disables.

Full gate green (all -m integration tests now exercise the cache); deps-image
invalidation verified (touching a source recompiles).
2026-06-14 18:21:48 -04:00
Dmitri Sotnikov
cf271e218a
Merge pull request #101 from jolt-lang/fix-record-vector-pred
Records are not vectors: fix vector?/sequential? for shape-recs (jolt-14k)
2026-06-14 22:21:17 +00:00
Yogthos
8a3019a64e Records are not vectors: vector?/sequential? false for shape-recs (jolt-14k)
Under direct-linking a record is a Janet tuple (its shape-rec), and core-vector?
just delegated to jvec? which is true for any tuple — so (vector? a-record) and
(sequential? a-record) returned true. That broke map-destructuring of a record:
the destructure coerce treats a sequential source as & {:keys} kwargs and does
(apply hash-map x), so destructuring a record fed its entries to make-phm as a
flat kv-list and corrupted. Surfaced as reitit's router crashing on a wildcard
route ('expected integer key for tuple in range [0,5), got 5') whenever the new
direct-link default was on; minimal repro is (let [{:keys [a]} (->R ...)] ...).

Fix: core-vector? excludes shape-recs, matching Clojure (a record is not a
vector or sequential). jvec? is unchanged for internal representation dispatch.
Regression cases added to record-declared-shape-test.
2026-06-14 17:45:30 -04:00
Dmitri Sotnikov
8c29168de2
Merge pull request #100 from jolt-lang/default-direct-link
Direct-link + whole-program by default for program runs
2026-06-14 20:01:28 +00:00
Yogthos
6abfd660ce Direct-link + whole-program by default for program runs; open for the REPL
Running a program is a closed world — every namespace is required, then it runs
to completion — so make it direct-link by default (inlining, record shapes, the
inference's specialization), and for a -m/-M entry auto-enable the whole-program
cross-namespace inference pass. A decomposed multi-namespace program was ~3.7x
slower than the same code in one namespace purely because per-namespace
inference can't see a caller in a not-yet-loaded namespace; this closes that for
the common case with no flags and no hints.

Interactive modes (repl, -e, nrepl-server) stay indirect/open — they have to let
you redefine vars, which direct-linking seals against. Opt-outs:
JOLT_NO_DIRECT_LINK forces the open path even for a program run (hot-reload,
runtime redefinition); JOLT_NO_WHOLE_PROGRAM keeps direct-linking but per-ns;
JOLT_DIRECT_LINK / JOLT_WHOLE_PROGRAM still force-on. Namespaces required inside
-main (after the batch pass) fall back to per-ns inference.

The success checker (RFC 0006) rides on the inference for free, but a casual
program run shouldn't spam type warnings just because it now direct-links, so its
default-on is suppressed when direct-linking was auto-enabled (:direct-link-auto?);
an explicit JOLT_DIRECT_LINK or JOLT_TYPE_CHECK still turns it on. whole-program-
test and devirt-test opt their per-ns baseline out of the new auto-default.

Docs: RFC 0005 gains 'Compilation modes and defaults' + 'Cross-namespace
inference'; RFC 0004 documents cross-ns/param hints; self-hosting-compiler and
--help updated. Full gate green.
2026-06-14 15:44:01 -04:00
Dmitri Sotnikov
230a0c2160
Merge pull request #99 from jolt-lang/field-types
Records across namespaces: field type hints, cross-ns hints, whole-program var const-linking
2026-06-14 16:58:30 +00:00
Yogthos
1525c7adfb Record type hints resolve across namespace boundaries (fields and params)
A ^RecordType hint only resolved against the current namespace's ctor key, so a
hint naming a record defined in another namespace degraded to :any. That made a
decomposed multi-namespace program much slower than the monolith: per-namespace
inference can't see a record param's callers in other namespaces, and the
declared hint that could have typed it was dropped.

Resolution now works cross-namespace, for both record FIELD hints (defrecord)
and fn PARAM hints, in both spellings — ^Vec3 where the type is referred and
^v/Vec3 where the namespace is aliased:

- reader keeps a tag's namespace qualifier (^t/Ray -> "t/Ray", was "Ray").
- make-deftype-ctor-impl indexes each ctor closure by value; record-hint-ctor-key
  resolves a hint name against the COMPILE ns (referred names live there; aliases
  resolve through it) and maps the type var's root back to its home ctor key.
  Using the ctor value, not the var's :ns, is what makes :refer work — :refer
  re-interns a fresh var whose :ns is the referring ns.
- the analyzer captures record param hints as arity :phints [name ctor-key];
  reinfer-def seeds those param types, so a record param is typed even with no
  inferred caller — the open-world / cross-ns case.

Effect on the multi-namespace ray tracer: per-ns compile 30.4s -> 7.9s with
param hints, matching whole-program (8.1s) and the single-ns monolith (8.3s).
cross-ns-hints-test covers field + param hints, refer + as, and the reader tag.
2026-06-14 12:40:14 -04:00
Yogthos
4018eb87ed Const-link stable vars under whole-program; direct-link cfunction roots
direct-var? now treats a cfunction root the same as a function root, so a
call/ref to a native fn (clojure.math/sqrt et al.) embeds the value instead of
a per-call cell deref. This was the hot indirection in the ray tracer — sqrt
runs every bounce — and it applies in every direct-link build, not just
whole-program.

const-link? is new and whole-program-only: in a closed world every non-dynamic
var has a stable root, so embed it as a constant (quoted unless it's already
callable) rather than reading the cell each reference. Covers what direct-var?
can't — ^:redef vars (reloading is off under the flag), data defs, and record
type/ctor roots. Dynamic vars stay indirect; a nil (not-yet-defined) root stays
indirect and the whole-program re-emit picks it up once the root is in place.

Measured on the records ray tracer: hot-path indirect refs (sqrt + data vars)
gone; the only indirect refs left are cold defrecord self-references. whole-
program-test now also checks a ^:redef fn and a data def so the per-ns vs
whole-program comparison guards const-link soundness.
2026-06-14 11:03:28 -04:00
Yogthos
840f699f54 record field type hints -> per-field value types in inference (jolt-3ko)
A record field can carry a type hint — ^Vec3 (a defined record type) or ^:num —
and the inference now resolves it so reading the field back yields that exact type
instead of :any. A Vec3 stored in a Ray field reads out as Vec3, so the vec ops on
field-read values prove their reads (bare-index). This is Stalin's per-slot type
sets, but DECLARED rather than inferred: the exact shape is known up front.

- deftype captures each field's :tag / :num metadata (was stripped) and passes it
  to make-deftype-ctor; the ctor registers per-field tags, resolving a record-type
  hint to its ctor-key (same-ns) so the inference can look it up directly.
- call-ret-type builds a record's struct type with field types resolved from the
  hints, recursing into nested record types (depth-bounded for self/cyclic types).

Measured: a nested-record read loop (:r (:origin ray)) runs 1.3s with ^Vec3 hints
vs 7.1s without — 5.5x. This is the lever the ray tracer needed (vecs flow through
container fields); records without it read back as :any and stay unproven.
2026-06-14 07:00:29 -04:00
Dmitri Sotnikov
872cea1c37
Merge pull request #98 from jolt-lang/stalin-techniques
Devirtualize protocol dispatch on known record receivers (jolt-41m)
2026-06-14 10:44:04 +00:00
Yogthos
fa02c8f93d devirtualize protocol dispatch on known record receivers (jolt-41m)
A protocol method call compiles to (protocol-dispatch proto method this rest) — a
runtime registry walk (type-tag -> proto -> method) on every call, ~19x a direct
call. When the inference proves the receiver (arg 0) is a known record type, the
call now resolves to a DIRECT method call at compile time, skipping the registry.

- defprotocol registers each method's var-key 'ns/method' -> [proto method] (a
  ctx-capturing register-protocol-methods! emitted into the do-block); infer-unit!
  feeds it to the inference via a box (like record-shapes).
- the record-ctor return type carries :type (the record tag) so the inference
  knows the receiver type; the :else invoke case annotates a protocol call whose
  arg0 has a known :type with :devirt-{type,proto,method}.
- emit-invoke resolves the impl via find-protocol-method at emit time and emits a
  direct call to the embedded impl fn value. Unknown/polymorphic receivers (no
  proven :type) fall back to the dispatch path unchanged.

Measured: removes the dispatch overhead (14.7s -> 9.3s on a 10M-call loop); the
remaining cost is the method body itself (non-inlined, unproven reads) — inlining
the resolved method is the follow-up (jolt-t6r) toward direct-call speed.

Sound under the closed-world assumption direct-linking already makes (the impl is
resolved + embedded at compile time). Adds devirt-test (subprocess: dispatched ==
devirtualized across polymorphic dispatch, unknown-receiver fallback, and
heterogeneous collections). Stalin's compile-call/callee-environment is the model.
2026-06-14 03:33:48 -04:00
Dmitri Sotnikov
a83792cc51
Merge pull request #97 from jolt-lang/perf-alloc
Records with declared shapes for fast field access (jolt-t34)
2026-06-14 06:33:03 +00:00
Yogthos
75a1352d22 whole-program closed-world inference, opt-in (jolt-t34)
JOLT_WHOLE_PROGRAM (requires direct-linking) defers the per-namespace inference
and runs ONE fixpoint over every user unit at once, so param types propagate
across namespace boundaries — a non-inlined fn's record params get proven from
its callers in another unit, which the per-ns pass can't see. Sound only under
the closed-world assumption (no later eval/redefinition) the flag asserts; slow,
memory-heavy builds are the documented trade-off (the reason it's opt-in).

infer-unit! now takes one ns-name OR a list; infer-program! gathers all recorded
user namespaces and runs the existing fixpoint over the union (re-emit was already
ns-agnostic — keyed by var-key, callee-first). The evaluator defers + records each
unit under the flag; run-main triggers infer-program! after all requires, before
-main. Off by default — per-ns behaviour unchanged.

Measured: a recursive (non-inlined) cross-ns record reader runs 1.66x faster
(8.9s -> 5.3s) — params proven -> bare-index reads. NOTE: small accessor fns are
INLINED cross-ns and records carry GLOBAL declared shapes, so most record reads
are already proven without this pass; the win is for non-inlined hot fns, and it's
the foundation for future whole-program work (devirtualization, unboxing).

Adds whole-program-test (subprocess soundness: per-ns and whole-program produce
identical results on a cross-ns record program).
2026-06-14 00:25:56 -04:00
Yogthos
f5a5b25d59 records use declared-shape layout with fast field access by default (jolt-t34)
Records (defrecord/deftype) are now shape-recs in a direct-linking unit by
default — no JOLT_SHAPE flag. A record's shape is DECLARED, so the inference
proves field reads by a lookup, not fragile shape inference, and they bare-index.
Result: ~1.4x faster than the :jolt/deftype table form on a record-heavy loop
(3.9s vs 5.5s), driven by cheaper construction + proven bare-index reads.

Two gates now:
- :shapes?     — shape-recs active; records use declared-shape layout + bare
                 index reads. On with direct-linking (where the inference runs).
- :map-shapes? — also shape generic const-key maps. Opt-in (JOLT_SHAPE), because
                 shaping maps net-loses on unproven reads (measured). Records win.

- call-ret-type types a record ctor (->Name) as a struct of its declared shape,
  fed from a ctx-env registry populated at deftype; field reads on the result
  bare-index. (set-record-shapes!/set-map-shapes! wired through infer-unit!.)
- sidx reads the field's position from the :shape vector AS-IS (declared order
  for records, str-sorted for map literals) — no re-sort — so any field order
  bare-indexes correctly. The map :map case only sets :shape under :map-shapes?.
- record-shape-for interns the descriptor per (type, fields), not per type: a
  record redefined with different fields now gets a fresh descriptor instead of a
  stale one (fixes redef descriptor staleness; old instances stay valid).

Adds record-declared-shape-test (declared-order reads, incl. non-alphabetical
fields, through fn boundaries + protocol method bodies). Known pre-existing edge
case filed as jolt-wf4 (direct (:f (->R …)) read returns nil after a record is
redefined with different fields; let-bound read works; repros without shapes).
2026-06-14 00:04:15 -04:00
Yogthos
4e7b8f792f shapes: revert default-on to opt-in; faster get-or-shape inline read (jolt-t34)
Measurement (vec-heavy benchmark + read-mechanism micro-benchmarks) showed that
shape-rec'ing generic const-key maps is a net loss in a bytecode VM: an unproven
field read can't beat a native Janet struct-get (one opcode), and an inline cache
doesn't transfer to a VM jolt doesn't control. The win is real only for PROVEN
reads (bare-index beats struct) — i.e. construction-heavy or inference-friendly
code, not general map-through-fn code.

So shapes are opt-in again (JOLT_SHAPE), not defaulted on in direct-link. Kept the
get-or-shape fix: the non-proven shape read now indexes inline off the descriptor
instead of calling core-get (which was ~2.5x slower per read). :shapes? is the
single opt-in gate; image cache keys on JOLT_SHAPE + JOLT_NO_SHAPE.

Next: records with declared shapes get fast field access by default (the general
case), which is where the proven-read win actually lands.
2026-06-13 23:41:26 -04:00
Yogthos
7900173d45 type-aware record equality + assoc extension + record-shape test (jolt-t34 R3)
Record shape-recs now compare type-aware like the table form: eq-map-pairs
returns nil for a record so equality falls to deep=, which is type-aware via the
per-type interned descriptor. A record equals only a same-type record, never a
plain map (cross-type and record-vs-map were wrongly equal under JOLT_SHAPE).

assoc of a new (undeclared) key keeps the record type and grows a slot, matching
Clojure's record-extension semantics.

Adds test/integration/record-shape-test.janet locking in the runtime record
mechanism (tag, field access, virtual :jolt/deftype, type-preserving assoc,
dissoc demotion, type-aware equality, #ns.Type{...} printing).
2026-06-13 20:40:50 -04:00
Yogthos
2900d36176 records as shape-recs carrying a type tag (jolt-t34 R3)
Records (defrecord/deftype) become shape-recs whose descriptor also carries
:type, under JOLT_SHAPE (flag off keeps the :jolt/deftype table form). A record
is a Janet tuple [descriptor field0 ...] in declared field order, so the
positional ->Name ctor maps args straight to slots.

- record-tag unifies the type accessor over both reps; instance?/satisfies?/
  protocol dispatch and the . interop path go through it.
- virtual :jolt/deftype key on record shape-recs (via shape-get) keeps every
  existing (get obj :jolt/deftype) dispatch site working.
- emit-kw-lookup's non-proven fallback routes any tuple to core-get (shape
  aware), not gated on compile-time JOLT_SHAPE — core is baked without the flag
  but still receives user shape-recs.
- assoc keeps the type (same-shape in place, new key grows a slot Clojure-style);
  dissoc of a declared field demotes to a plain map; pr prints #ns.Type{...}.
- JOLT_SHAPE added to the image-cache key (it shapes core's own compiled paths).
2026-06-13 20:40:50 -04:00
Yogthos
e61a175f91 feat: completeness preservation — shapes survive cap and same-shape joins (jolt-t34 R2)
The inference dropped the complete :shape whenever it rebuilt a struct type
(cap) or joined two (join-t/merge-fields), so a vec3 retrieved from a container
or a fn param typed across call sites lost its layout and every field read fell
to the slow descriptor path. Two fixes:

- cap preserves :shape: capping truncates field VALUES below the depth limit but
  never the key SET, so the layout is still complete. It also recurses into
  fields, so a shaped value nested in a container (a vec3 inside a hit-info)
  keeps its own :shape — which is what lets (:r (:normal hit-info)) bare-index.
- join-t preserves :shape when both sides are the SAME complete shape (the
  merged struct has the same keys); different shapes still drop it. This carries
  the shape through if-joins and the inter-procedural fixpoint's call-site joins.

Result: the ray tracer goes from 22s (R1, correct-but-descriptor-path) to 4.36s
— 2.7x FASTER than the 11.7s no-shape baseline, and ~3x the JVM (was 8.5x), with
byte-identical output. The compounding of cheaper tuple construction plus
bare-index reads across the whole render far exceeds the per-op estimate.

Gate green flag-off, suite 4718, default-path bench even, transparency intact.
2026-06-13 20:40:50 -04:00
Yogthos
c6b964c1b1 feat: general shape-record mechanism — consistent representation + transparency (jolt-t34 R1)
Removes ALL hardcoding. Every constant-key map literal in user code becomes a
shape-rec (a Janet tuple [descriptor v0 v1 ...]); the descriptor is interned
per key SET with a single canonical key order owned by the runtime
(types/shape-sort), so every site that builds or reads a shape agrees.

Consistency is the foundation: shape-recs are made UNCONDITIONALLY for
const-key user maps (emit-map), not gated on the inference — so a value's
representation always matches what the type system claims, which was the bug
that broke the earlier generalization (a ray built as a struct but bare-indexed
as a shape). Gated on :inline? so it applies to user data only, never to core
or the compiler's own IR-node maps.

Transparency layer (the shape-rec block now lives in types.janet, reachable by
core + evaluator + backend): get, assoc, dissoc, count, contains?, map?, first,
seq, the central realize-for-iteration normalizer (keys/vals/reduce-kv), eq-map-
pairs + eq-seqable (equality), pr-render (print), jolt-call (compiled IFn), and
the interpreter's coll-lookup all handle shape-recs. A new spec
(shape-transparency-test) asserts each op matches the equivalent struct map,
including nil/false values (which shape-recs store positionally — unlike
structs).

The ray tracer renders byte-identically (mean 122.04). Gate green with the flag
off, suite 4718. NOT yet faster — every field read currently takes the
descriptor path because the inference drops the complete :shape through joins
and containers; that's Round 2 (completeness preservation). All behind
JOLT_SHAPE (off by default).
2026-06-13 20:40:50 -04:00
Yogthos
e377c223b6 wip: generalize shape mechanism off the hardcoded vec3 shape (jolt-t34)
Removes the {:r :g :b} hardcoding. ANY constant key set is now a shape:
- inference: a struct type from a map LITERAL carries :shape (its canonical
  str-sorted key vector — completeness); joins/access-inferred structs lack
  it, so they never get a bare index. The literal node and lookup subjects
  carry the shape; the back end derives the index from it.
- backend: emit-map turns any shape-tagged const-key map into a shape tuple;
  emit-kw-lookup reads the field by bare index when the complete shape is
  proven, else by the value's own descriptor (so a shape-rec whose :shape was
  dropped by a join still reads correctly).
- runtime: core-get and core-assoc handle shape-recs.

Status: CORRECT for direct field access, container round-trips, and assoc
(minimal repros pass). NOT yet complete — the full ray tracer still hits an
uncovered path (a shape-rec reaching a map op without coverage: keys/vals/
count/seq/equality/print/jolt-call/dissoc/contains?/the interpreter's
coll-lookup all still need shape-rec branches). And the perf win needs
COMPLETENESS PRESERVATION through joins/containers (merge-fields/cap drop
:shape today, so nested vec3 access falls to the descriptor path, slower than
a struct get) — without it the general version is slower than the vec3
prototype.

All behind JOLT_SHAPE (off by default). Gate green with the flag off, suite
4718. This preserves the general design; the transparency layer + completeness
preservation are the remaining multi-session work.
2026-06-13 20:40:50 -04:00
Yogthos
d33fb85041 prototype: shape-record representation for vec3 maps (jolt-t34, JOLT_SHAPE)
Validated prototype of the hidden-class object-model change. A vec3-shaped
{:r :g :b} map literal is represented as a cheap Janet tuple [shape vb vg vr]
instead of a struct (~2x cheaper to construct); a lookup on a value the
inference PROVES is the shape reads by bare index with no runtime check.

Result on the ray tracer (direct-link): 12.3s -> 10.7s (~13% faster), with
byte-identical pixel output. The shape value flows transparently through
hit-info/ray/material containers and the colors vector; core-get handles it
(inline check, no fn call) so an unspecialized access is still correct.

Key lessons baked in: the lookup MUST compile to a bare index (a runtime
shape check, even inlined, taxed every field read and made it 2.5-3.4x
SLOWER) — so the inference gained a :shape hint (struct type with keys
exactly {:r :g :b}) that the back end turns into (in m idx). The descriptor
is quoted when embedded (its keys are a parens tuple Janet would otherwise
try to CALL).

All behind JOLT_SHAPE (off by default). Gate green, suite 4718, default-path
bench even. Scoped to the one shape; NOT yet sound in general (assumes every
vec3-shaped value is a shape-rec, true under the flag for the ray tracer) nor
fully transparent (only core-get + the inlined lookup; jolt-call/equality/
print/keys not yet covered). Those are the next steps toward a real feature.
2026-06-13 20:40:50 -04:00
Dmitri Sotnikov
50d8b13ca3
Merge pull request #96 from jolt-lang/migratus-fixes
remove logging from core
2026-06-14 00:39:48 +00:00
Yogthos
92df2cebf7 remove logging from core 2026-06-13 20:23:18 -04:00
Dmitri Sotnikov
52bea0b620
Merge pull request #95 from jolt-lang/migratus-fixes
Migratus fixes
2026-06-13 23:13:43 +00:00
Yogthos
cf1fdfdb24 feat: run the real clojure.tools.logging (defmacro/syntax-quote/ns + host shims)
Pivot from a jolt reimplementation to running the upstream library verbatim.
Vendors the real clojure/tools/logging.clj; jolt provides the backend and the
host primitives it needs. Language features (broadly useful for real Clojure
libs), all covered in 3-mode conformance + spec suites:

- defmacro: multi-arity dispatch (jolt-q8l) and a docstring + attr-map + params
  head (jolt-qnr) — the 4-arity log macro and every level macro need these.
- syntax-quote resolves an alias-qualified symbol to its target ns (jolt-9av),
  so a macro template (impl/get-logger) resolves at the use site.
- the ns macro unwraps ^{:map} metadata on the ns name (jolt-8w2 workaround,
  matching def/defn/defmacro).
- a namespace object self-evaluates, so ~*ns* can be spliced into a template.

Host shims (ported from / modeled on clojure where applicable):
- clojure.string/trim-newline (ported, CharSequence interop -> count/subs)
- agent/send-off/send (minimal synchronous stubs; jolt has no thread pool/STM)
- clojure.lang.LockingTransaction/isRunning -> false
- a minimal clojure.pprint (pprint/with-pprint-dispatch/code-dispatch, for spy)
- clojure.tools.logging.impl: a jolt stderr LoggerFactory backend (the library's
  designed pluggable extension point)

docs/libraries.md lists tools.logging; grammar.ebnf metadata note clarified.
Conformance 355/355 x3 modes; full jpm test gate green.
2026-06-13 18:50:53 -04:00
Yogthos
72e36f46de test+docs: spec coverage for migratus-enablement fixes; list migratus
Adds spec/conformance coverage for everything landed enabling migratus:
- conformance corpus (runs interpret/compile/self-host): def 3-arg docstring,
  def/defn ^{:map} name meta, defmacro arity-clause + docstring, defmulti
  docstring, assoc-nil/assoc-in real maps, try multi-body + finally-on-success,
  current-ns restore after a caught throw, cross-ns methods visibility.
- spec suites: host-interop (exception ctors, Character/Thread/Long, Timestamp/
  SimpleDateFormat, java.io.File model + File-aware file-seq, clojure.tools.logging),
  regex (Pattern statics + MULTILINE + quote, String .matches/.replaceAll/
  .replaceFirst), maps (assoc on nil), multimethods (defmulti docstring +
  value-based methods/get-method), macros (defmacro arity-clause + name meta).

Rewrote clojure.tools.logging/spy as a single variadic arity (jolt defmacro
takes only the first clause of a multi-arity macro — jolt-q8l).

docs/libraries.md: add migratus and the next.jdbc compatibility layer, with the
janet-lang/sqlite3 int64 caveat for 14-digit timestamp ids.

Full gate green; conformance 350/350 x3 modes.
2026-06-13 17:45:05 -04:00
Yogthos
4f59958d92 fix: try/catch/finally correctness + current-ns restore (jolt-96m + try bug)
Two try bugs that blocked migratus's migrate (run/table-exists? are multi-body
try/catch/finally with janet-interop jdbc calls, which punt to the interpreter):
- The interpreter's try took only form 1 as the body, dropping later body forms
  before the clauses, and did not run finally on the success path when a catch
  was present. Rewritten to collect every body form before the first
  catch/finally and to always run finally (via defer, so it fires even if a
  catch body throws).
- current-ns leaked after a caught throw from an INTERPRETED fn (it sets ns to
  its defining ns and can't restore on unwind). The interpreter's try now
  restores; the compiled try (emit-try) snapshots the ns at entry and resets it
  in the catch via new __current-ns/__set-current-ns! core fns. Statement
  values also get a :close key so with-open can close them.

With these, migratus migrate/rollback round-trips on jolt (verified end to end
against sqlite). Conformance 335/335 x3, full gate green.
2026-06-13 17:16:40 -04:00
Yogthos
eec3edf632 test: fallback-zero uses a resolvable multifn for get-method/methods
methods/get-method now take the multimethod VALUE (Clojure semantics), so the
arg must resolve to compile. The isolated analyzer never defines mf, so point
these two at print-method (a real defmulti) instead.
2026-06-13 16:11:27 -04:00
Yogthos
b304c43333 feat: java.io.File model + multimethod/assoc/defmulti fixes for migratus (jolt-hjw)
File API (jolt-hjw): io/file and (File. …) build a tagged :jolt/file value
(instance? File true) with a full method surface (isFile/isDirectory/exists/
getName/getPath/getAbsolutePath/listFiles/toPath/delete/createNewFile/…) backed
by os/ and file/. file-seq is File-aware (leaves are File values). str/slurp/spit
coerce :jolt/file to its path. ClassLoader/getSystemClassLoader + a classloader
stub whose getResource returns nil degrade migratus's classpath lookup to the
filesystem. java.nio.file Path/FileSystem/PathMatcher are shimmed just enough for
script-excluded?'s glob (recursive * / ? matcher).

Three bugs found getting migratus's migration discovery to work:
- (assoc nil k v) returned a raw janet table, not a map, so assoc-in built tables
  that count/seq rejected. Now returns an immutable map.
- methods/get-method resolved the multimethod symbol at runtime in the current
  ns, so a bare multifn ref in its defining ns saw an empty table once defmethods
  lived elsewhere. Now they take the multimethod VALUE and recover the var via a
  registry (Clojure semantics).
- defmulti now drops a leading docstring/attr-map (migratus's multimethods carry
  docstrings) instead of treating the docstring as the dispatch fn.

Conformance 335/335 x3, clojure-test-suite at baseline.
2026-06-13 16:04:30 -04:00
Yogthos
9813186ef9 fix: def supports the 3-arg docstring form (def name doc value) in compile mode (jolt-6ym)
The analyzer always took (nth items 2) as the value, so (def x "doc" 42)
bound x to the docstring and dropped 42. Now it mirrors the interpreter:
when there are 4+ items and item 2 is a string, item 2 is the docstring
(attached as :doc meta) and item 3 is the value. Conformance 335/335 x3.
2026-06-13 15:41:59 -04:00
Yogthos
19505a5944 feat: jolt-side java.sql interop + defn/defmacro meta & arity-clause fixes
For the migratus next.jdbc shim (jolt-0z5):
- core.janet: __jdbc-wrap-conn / __jdbc-conn-raw / __jdbc-make-stmt builtins.
  A connection is a tagged wrapper over a jdbc.core conn carrying a clj :exec
  callback so the host Statement.executeBatch runs SQL without a janet->clj call.
- javatime.janet: tagged-methods for :jolt/jdbc-conn (setAutoCommit/isClosed/
  close/getMetaData), :jolt/jdbc-meta (getDatabaseProductName), :jolt/jdbc-stmt
  (addBatch/executeBatch/close); java.sql.Timestamp ctor -> millis.
- evaluator.janet: instance? case for Connection/java.sql.Connection so
  migratus's do-commands runs SQL through its Connection branch.

Two defn/defmacro fixes found loading migratus.core (both rooted in the reader
representing ^{:map} name metadata as a with-meta form, jolt-8w2):
- defmacro special form: unwrap a with-meta name (mirrors def), and handle the
  arity-clause form (defmacro name ([params] body...)) like fn/defn — a params
  vector reads as a tuple, an arity clause as a list (array).
- defn overlay: pass the bare (unwrapped) name to fn while def keeps the meta.

Conformance 335/335 x3 modes.
2026-06-13 15:28:54 -04:00
Dmitri Sotnikov
30267fe67a
Merge pull request #94 from jolt-lang/perf-type-inference
Perf type inference
2026-06-13 19:12:39 +00:00
Yogthos
e2d33df484 feat: clojure.tools.logging shim for migratus (jolt-nzg)
Faithful shim of the real clojure/tools.logging public API rather than a
bespoke logger. clojure.tools.logging.impl provides the Logger/LoggerFactory
protocols (matching upstream signatures) plus a jolt stderr-backed factory;
find-factory returns it instead of probing slf4j/log4j/jul. The macro surface
(logp/logf, level macros + f-variants, log, enabled?, spy) dispatches through
those protocols. Departures forced by no-JVM: log* writes directly (no agent/
LockingTransaction), and the throwable-first-arg branch is dropped since
(instance? Throwable x) is always false for jolt exception values.

Workarounds for two jolt gaps found en route (filed as jolt-9av, jolt-6ym):
macro bodies fully-qualify impl refs because syntax-quote does not resolve ns
aliases, and def docstrings are kept in comments because (def name doc value)
is mishandled.

stderr via __eprint. trace/debug suppressed by default (impl/*level* :info).
Conformance 335/335 x3 modes.
2026-06-13 14:58:47 -04:00
Yogthos
f9a1849ec8 docs: RFC 0006 — mark negative/never types resolved (jolt-wwy) 2026-06-13 14:48:39 -04:00
Yogthos
328f88636e feat: negative/never types — calling a non-function, wrong-arity (jolt-wwy)
Two provably-wrong cases the inference already has the facts for, closing the
last RFC 0006 open question:

- Calling a non-function. At an :invoke whose callee is provably :num or :str
  (the only non-callable types — keywords/maps/vectors/sets are IFn), report
  "cannot call a number as a function". Default level (no closed-world: the
  callee type is inferred at the call site). Covers (5 1), ("hi" 0),
  ((+ 1 2) :k), a let-bound number, and a var holding a number (via vtype-box
  in direct-link). A union is non-callable only when every member is, so
  ((if c 1 :k) x) is accepted (:kw is callable). Verified zero false positives
  on the ray tracer, which calls maps/keywords/vectors as fns throughout.

- Wrong arity to a user fn. The registered single-fixed-arity sig (jolt-zo1)
  makes a mismatched arg count provably throw; reported under the
  JOLT_TYPE_CHECK_USER opt-in (same closed-world boundary; ^:redef/variadic
  skipped). Caught at compile time before the runtime arity error.

Both fold into the existing infer walk, carry :pos for file:line:col, and keep
no-false-positives. Gate green, suite 4718, conformance 335/335, runtime bench
even (compile-time only).
2026-06-13 14:48:14 -04:00
Yogthos
6abcb11e92 feat: misc host-interop shims for migratus (jolt-3v0)
Character/isUpperCase + isLowerCase (ASCII, on :jolt/char structs);
Thread/interrupted (false, no real threads) + Thread/currentThread stub
with a .getContextClassLoader classloader stub; Long/valueOf; java.net.URI
constructor (string round-trip); and a minimal java.util.Date / TimeZone /
SimpleDateFormat supporting the y M d H m s tokens migratus's timestamp uses,
backed by os/date (UTC default). Full gate + 3-mode conformance green.
2026-06-13 14:00:50 -04:00
Yogthos
4e3984e9c0 feat: host-interop shims for migratus (exceptions + regex Pattern)
jolt-6xk: resolve bare exception class symbols (Exception,
IllegalArgumentException, InterruptedException, Throwable) by consulting
class-ctors/class-canonical-names in the unqualified symbol-resolution
fallthrough, mirroring the qualified path. Constructors already existed
in javatime; throw/catch/.getMessage already handled the string payload.

jolt-47b: java.util.regex.Pattern statics (compile/quote/MULTILINE) that
return jolt's native :jolt/regex value so str/replace, re-matches, and
.split accept them transparently. MULTILINE maps to a (?m) prefix routed
through the regex engine's inline-flag path; the engine gains (?m) anchor
support with the non-multiline branches left byte-identical. String
methods .matches/.replaceAll/.replaceFirst added. .split dispatches on
compiled-regex via a :jolt/regex tagged-method.

Full jpm test gate green.
2026-06-13 13:49:42 -04:00
Yogthos
e071d09170 docs: RFC 0006 — mark unions/user-fns/positions/default-on resolved
Update the status, strictness levels, and open questions to reflect what
landed: bounded unions (jolt-pz5), user-fn domains behind
JOLT_TYPE_CHECK_USER (jolt-zo1), precise file:line:col (jolt-fqy), and the
checker folded into one inference walk that piggybacks on direct-link
specialization (on by default there, opt-in in plain builds). Align the
error-reporting example with the actual output format.
2026-06-13 13:47:36 -04:00
Yogthos
088232b778 feat: success checker on by default in direct-link builds, free (jolt audit)
Checking inherently needs an inference pass (~2.6x compile as a standalone
pass). But direct-link builds ALREADY run one inference pass for
specialization (run-passes' infer-top), so checking can ride along: set a
check-mode flag, turn checking? on during that existing pass, and collect
the diagnostics after — ~2% overhead measured on the ray tracer, vs 2.6x
for the separate pass.

So the checker now defaults to `warn` in direct-link builds (where it's
nearly free) and stays OFF in plain REPL/dev builds (no inference to ride,
no forced cost — opt in with JOLT_TYPE_CHECK there). JOLT_TYPE_CHECK still
overrides in both directions (off to disable, error to escalate).

It checks the POST-optimization IR, which matches what the optimized
program actually evaluates — scalar-replace only drops provably-pure code,
an accepted opt-mode divergence, so no real error is hidden. The loaders
enable position tracking whenever checking will run (env-selected or
direct-link). type-check! (the standalone pass) stays for plain builds;
both paths share report-diags!.

cli-test pins: plain build silent, direct-link warns by default,
JOLT_TYPE_CHECK=off disables. Gate green, suite 4718, runtime bench even.
2026-06-13 13:46:16 -04:00
Yogthos
69af83da89 refactor: fold success checking into the inference walk
The checker ran a separate check-walk that re-inferred each argument's
subtree AND recursed into it — quadratic in expression nesting. Fold the
diagnostic emission into `infer` itself (gated by a checking? flag, off
during the optimization fixpoint): one O(n) walk that both types and
checks. Removes check-walk entirely; check-form now drives infer.

This is a cleanup and removes the deep-nesting blowup, but it does NOT make
warn-by-default cheap: on a real 360-line file the checker still adds ~2.6x
compile time (277ms -> 720ms). That cost is the structural inference pass
itself, which checking inherently requires — not redundancy. A cheap
default-on path would need either piggybacking on the inference direct-link
already runs, or a lighter scalar-only checker inference. Gate green,
type-check tests pass.
2026-06-13 13:02:17 -04:00
Yogthos
37949ac602 fix: unary arithmetic type-error message no longer crashes the reporter
rewrite-message assumed janet's BINARY arithmetic dispatch error shape
("could not find method :+ for 1 or :r+ for "a""). Unary inc/dec/- on a
non-number produce "could not find method :+ for "x"" — no "or :r" clause
— so orpos was nil and the reporter itself threw "could not find method :+
for nil", burying the real error. Handle the unary form. Found auditing
the RFC 0006 checker's default (checker-off) path. Regression row in
cli-test.
2026-06-13 12:26:56 -04:00
Yogthos
f2d65addc8 feat: precise file:line:col source locations for type errors (jolt-fqy)
RFC 0006 error reporting wanted file:line:col but IR nodes carried no
position, so diagnostics read only "type error in <ns>: <msg>". Now:

    type error /tmp/scene.clj:5:5: `inc` requires a number, but argument 1 is a string

The reader records each LIST form's absolute start offset in a table keyed
by form identity (lists are fresh arrays, never interned), gated behind a
flag the loaders enable only when JOLT_TYPE_CHECK is on — zero cost off.
Keying by identity makes positions survive macroexpansion exactly when the
user's own sub-form is spliced through, and absent for macro-synthesized
structure: a `(inc :k)` written inside `(when c ...)` reports at its own
line, never at the expansion's generated if/do.

The analyzer stamps the offset onto :invoke nodes (form-position host
contract fn); the checker carries it into each diagnostic as :pos; the
loaders stash the file's source + path on the env (save/restored across
nested requires); backend/type-check! converts offset -> line:col via the
reader's line-col and renders the RFC format. Falls back to the ns when no
position is available (synthetic forms), so it is never worse than before.

Gate green, conformance 335/335, suite 4718, runtime bench even (positions
are compile-time only; off by default).
2026-06-13 12:18:53 -04:00
Yogthos
824b30defd feat: report provably-wrong calls to user functions, opt-in (jolt-zo1)
The success checker fired only against core-fn error domains (stable, not
redefinable). This adds reporting of a call that passes a provably-wrong
type to a USER fn whose body requires otherwise — e.g. a fn that only does
arithmetic on a param, called with a string.

As check-walk sees defs it registers each non-redefinable single-fixed-arity
user fn's {:params :body} in module state (user-sig-box, accumulating across
forms like rtenv-box — a def must precede its call). At a call site (strict
mode only) the body is re-checked with ONE parameter bound to its concrete
argument type, others :any; if that produces a diagnostic the all-:any body
did not, the argument alone is provably wrong and the call is reported.
Monotonic — binding a concrete type can only add error-domain hits — so still
no false positives. A cycle guard (checking-box) terminates mutual recursion.

Gated behind JOLT_TYPE_CHECK_USER (orthogonal to the warn/error level)
because it rests on the closed-world assumption, weaker than the core-fn
case. check-form gains a strict? arity; the default path is unchanged and
user-fn code runs only when the checker is enabled. ^:redef/^:dynamic and
multi/variadic fns are not registered (their body is no stable requirement).

Gate green, suite 4718, conformance 335/335.
2026-06-13 11:56:21 -04:00
Yogthos
9f076937af feat: bounded union types in the RFC 0005 lattice (jolt-pz5)
The success checker (RFC 0006) used to lose differing if-branches to :any
and accept the use. (inc (if c "a" :k)) typed the if as :any — sound but
imprecise, since the value is provably {:str | :kw}, every member of which
is in inc's error domain.

Adds {:union #{T...}} to the lattice: join-t forms a scalar union of
differing branches instead of collapsing to :any, capped at 4 distinct
scalars (the member space is the five scalar tags, so the lattice stays
finite and the inter-procedural fixpoint still terminates). The checker's
not-number?/not-seqable? report a union only when EVERY member is in the
error domain — any valid member accepts the call, so still no false
positives. type-name renders "a string or a keyword".

Unions are scalar-only and carry no :struct/:vec/:set key, so every
structural predicate already treats them as opaque — specialization sees
them exactly as :any and codegen is unchanged. Gate green, suite 4718,
conformance 335/335, bench even.
2026-06-13 11:39:55 -04:00
Yogthos
1c0b3fe9bd docs: mark RFC 0005/0006 implemented, note follow-up work 2026-06-13 11:01:42 -04:00
Yogthos
9867c33079 feat: success-type checker (RFC 0006) — flag provably-wrong core calls
Reuse the structural inference from RFC 0005 as a loose type checker. It reports
a core-fn call only when an argument's inferred type is concrete and lies in
that op's throwing error domain, and accepts everything ambiguous (:any, a
union that joined to :any, :truthy). By construction it never produces a false
positive: a correct program has nothing to report even in error mode.

The curated error-domain table starts with the clearest throwing cases:
arithmetic on a provable non-number, and count/first/rest/next/seq/nth on a
provable non-seqable scalar. Lenient operations like (get 5 :k) and (:k 5),
which return nil rather than throw, are deliberately not listed.

Checking is decoupled from specialization: it runs whenever JOLT_TYPE_CHECK is
warn or error, regardless of :inline?, reading the knob at compile time so no
rebuild is needed. warn prints to stderr, error fails the form's compilation,
off (the default) skips it entirely. Core init stays clean under the flag.

jolt-y3b
2026-06-13 11:00:58 -04:00
Yogthos
9bc7b27245 perf: structural type inference (RFC 0005) — nested access typed, hint-free
Replace the ad-hoc inference lattice (a flat :struct-map tag plus {:vec ELEM})
with one recursive structural type: {:struct {field -> T}}, {:vec T}, {:set T},
scalar tags, and :any. A keyword lookup now returns its field's type, so nested
access like (:r (:direction ray)) is typed end to end and drops its guard. join
is field-wise and element-wise with a depth cap of 4 so the inter-procedural
fixpoint still terminates.

The back end honors a struct hint on any subject node, not just locals, so an
inferred field type on a nested lookup specializes. The orchestrator's fixpoint
joins through the portable join-types so compound types no longer collapse to
:any.

Ray tracer goes 12.8s to 11.0s with no hints, matching the explicit ^:struct
version (10.9s). Render checksum unchanged (1915337), full gate green,
conformance x3 modes pass.

jolt-5uj
2026-06-13 10:44:40 -04:00
Yogthos
e7473f38cf docs: RFC 0005 structural type inference + RFC 0006 success type checking
0005 proposes replacing the ad-hoc inference lattice with one recursive
structural type (a struct carries its field types, a vector its element type,
recursively), so a lookup returns its field's type and nested access is typed
end to end. It unifies :struct tracking with field tracking, subsumes the
current inference phases, and is the soft-typing (HM + a dynamic top) design:
structural types + core-fn type schemes, solved by lattice join with :any as
top instead of unify-or-fail. Includes the depth cap for termination and an
explicit design-problems section.

0006 (follow-up, depends on 0005) reuses the inference as a loose type checker
in the success-typing discipline (Dialyzer): report only PROVABLY-wrong code
(a concrete type in an operation's throwing error-domain), accept everything
ambiguous, never a false positive. Curated error-domain table, strictness
levels (off/warn/error), clear located messages, and the soundness boundaries
(closed-world, macros, unions).
2026-06-13 10:17:21 -04:00
Yogthos
5f05a99010 feat: Phase 2 vector-op specialization — count/nth on inferred vectors (jolt-d6u)
The inference now tags a :local it proved to be a vector with :hint :vector, and
the back end specializes (count v) -> pv-count (skipping core-count's dispatch
chain) and the 3-arg (nth v i default) -> pv-nth. The 2-arg nth is deliberately
NOT specialized: pv-nth returns nil out-of-bounds where Clojure nth throws.

Sound, conformance 335/335 x3 and full jpm test pass; type-infer-phase2-test
pins the specialization and the 2-arg exclusion.
2026-06-13 09:28:16 -04:00
Yogthos
09e5af02c9 feat: Phase 3 collection-element types + HOF awareness + ordered re-emit (jolt-d6u)
Extends the inference lattice with a parametric vector type {:vec ELEM} and
threads element types through the program:
- vector literals, conj/into, and range produce element-typed vectors;
- reduce/map/mapv/filter/filterv seed their closure's element (and reduce's
  accumulator) param, so a lookup inside the closure over a vector-of-structs
  specializes (the HOF-element-awareness piece);
- a var reference carries a VALUE type — a fn var is :truthy (non-nil, sealed
  root), a def var carries its inferred init type (e.g. a color table is
  {:vec :struct-map}); element-returning fns (rand-nth/first/nth/...) yield the
  collection's element type. These let the dynamically-built scene's sphere
  maps type as structs.
The inter-procedural fixpoint now also infers non-fn def value types, and the
recompile re-emits the WHOLE unit callee-first (reverse-topological) so a
caller re-embeds its recompiled, now-specialized callees and a call site
compiled after the pass links the whole chain.

Result on the ray tracer (no hints): the chain closes — hittables infers to
{:vec :struct-map}, hit-sphere's hittable param to :struct-map — and the render
goes 13.1s -> 12.8s. That is only ~3%, far short of the explicit hint's 1.22x.
The remaining gap is nested field access: a lookup RESULT like (:direction ray)
is :any, so (:r (:direction ray)) stays guarded, and the vec3 fns (called with
such values) can't be typed struct. The hint asserts the vec3 params directly
and propagates through inlining; matching it needs field-shape types
(ray.direction : vec3, vec3.r : number) — a structural extension (Phase 4).

Sound: a seeded full render produces an identical checksum (1915337);
conformance 335/335 x3 and the full jpm test pass; type-infer-phase3-test pins
the element-typing + HOF mechanism. Phase 2 (vector nth/count specialization)
was deprioritized — it is orthogonal to this benchmark.
2026-06-13 06:53:21 -04:00
Yogthos
ea1d9a23e1 feat: Phase 1 inter-procedural collection-type inference (jolt-767)
Closed-world (optimization mode): after a unit loads, infer-unit! runs a
whole-unit fixpoint over the call graph and recompiles. A fn's param types are
the lub of its in-unit call-site arg types; its return type is the lub of its
tail positions; iterated to a least fixpoint. Param types are RECOMPUTED FRESH
each iteration (not accumulated) because :any is the lattice top — joining an
early-iteration :any would poison the result permanently. Closures inherit the
enclosing tenv so captured locals keep their types (their own params shadow to
:any). A fn whose var escapes as a VALUE keeps :any params (its callers aren't
all visible). Each fn is then re-inferred with its param types seeded and
re-emitted; recompiled bodies are semantically identical, so correctness holds
regardless of order. Sound under source distribution + whole-program compile
(the consumer compiles all call sites together).

Plumbing: the portable pass (jolt.passes) gained inter-procedural primitives —
set-rtenv!, infer-body (types a body, collects its call sites), reinfer-def
(seeds param types), and escape tracking. The back end stashes each
single-fixed-arity defn's :def IR (:infer-ir); the evaluator triggers
infer-unit! after a unit loads (via an env hook, opt mode only).

Result and honest finding: the fixpoint correctly types scalar-flowing params
(ray-cast/hit-all/hit-sphere all get the ray param as :struct-map, no hint),
but the ray tracer does NOT speed up — its dominant lookups are on `hittable`,
the element of the `hittables` vector threaded through `reduce`, which stays
:any. Typing it needs collection-element types (vector<struct>) plus HOF-element
awareness (knowing reduce applies the closure to elements), which is beyond
inter-procedural param inference. The explicit ^:struct hint reaches it (it
types the reduce closure param directly), which is why the hinted run is 1.22x.

Verified: conformance 335/335 x3, full jpm test; new type-infer-phase1-test
pins the fixpoint, the escape gate, the seeded re-inference, and correctness.
2026-06-13 04:45:13 -04:00
Yogthos
3c20383851 feat: Phase 0 intra-procedural collection-type inference (jolt-6sr)
A forward, soft-typing-style pass (simplified HM: monovariant, never-fails,
lattice top = :any) in jolt.passes, run after the inline/scalar-replace
fixpoint when the optimization mode is on. It types expressions from literals
and arithmetic, flows the type through let bindings, and joins at if-branches.
Where a keyword-lookup subject is PROVEN to be a plain struct map it sets
:hint :struct (the same channel a manual hint uses, so the back end drops the
:jolt/type guard); where the type is :any it leaves the dynamic guard in place.

Sound by construction: a concrete type is assigned only when proven (scalar
keys with non-nil/non-false values for a struct-map), so a wrong bare get can't
happen. This is the foundation; on its own it mostly overlaps Route 1
scalar-replacement (which already eliminates non-escaping let-bound maps), so
its standalone win is small. Phase 1 (inter-procedural) is where escaping
params get typed.

Verified: conformance 335/335 x3, full jpm test; new type-infer-test pins the
flow rules and the sound :any fallback (cases force the map to escape so the
test isolates inference from scalar-replacement).
2026-06-13 01:46:34 -04:00
Yogthos
4b44bcd5fd test: cover ^:struct on let bindings (jolt-94n) 2026-06-12 20:25:15 -04:00
Yogthos
5f59c02b69 feat: expand type-hint lookup specialization (^Record, get-form, checked mode, docs)
Builds on the ^:struct keyword-lookup hint:

- ^TypeName for records. A tag naming a defrecord/deftype now resolves to the
  struct fast path: record instances are tables tagged :jolt/deftype (not
  :jolt/type), so a raw keyword get is correct for them. A new host contract fn
  record-type? detects a record by its ->Name constructor; a non-record tag
  (^String, ^long, ...) is ignored, as before.

- (get m :k) and (get m :k default) now get the same inlined keyword lookup as
  (:k m): the representation guard fast path when unhinted, and the bare get
  when the subject is ^:struct/^Record. A variable/number/string key still
  falls through to core-get. The two call shapes share one emitter
  (emit-kw-lookup).

- JOLT_CHECK_HINTS=1 turns a violated hint into a clear runtime error (naming
  the local and key) by keeping the guard and throwing on the tagged arm. It is
  off by default with zero cost to normal builds (a hinted lookup still emits a
  bare get), and is part of the image-cache fingerprint. This is the answer to
  "a lying hint is silent": opt into checking during development.

- Docs: RFC 0004 records the design, soundness contract, and measurements; the
  reader spec gains S12b (hints are semantically transparent; jolt recognizes
  ^:struct and ^Record as lookup-optimization assertions).

There is no Clojure keyword equivalent for "plain map / fast keyword access"
(Clojure hints are class names), so ^:struct stays a jolt-specific flag,
analogous to ^:dynamic.

Verified: conformance 335/335 in all three modes and the full jpm test pass; a
seeded ray-tracer render is byte-identical hinted vs unhinted; the struct-hint
test covers record hints, the get-form, inline propagation, and the checked-mode
error. Full render with hints holds at 13.3s -> 10.9s (1.22x).
2026-06-12 20:20:25 -04:00
Yogthos
c4be5d8a0e perf: hint-driven keyword-lookup guard elimination (^:struct)
A constant-keyword lookup (:k m) currently emits a guarded form,
(if (get m :jolt/type) (core-get m k) (get m k)), to tell a plain struct
(raw get is correct) from a phm/sorted/transient (needs core-get). On a
struct that guard is a second get, so the lookup costs ~36ns where a bare
get is ~20ns. Profiling the ray tracer (jolt-dad) showed keyword lookups are
~50% of a render and the guard is the only avoidable part, but dropping it
needs to know statically that the subject is a plain struct.

Type hints are exactly that information, and jolt already parses them and
otherwise ignores them. This wires one through: a local hinted ^:struct
asserts a plain struct/record map, so a (:k local) lookup on it skips the
guard and emits a bare get. The hint rides on the binding symbol into the
analyzer, which records it per-local and attaches it to :local IR nodes; the
back end reads it on the lookup subject. It also propagates through inlining:
when the inliner let-binds a non-trivial arg to a fresh local, it carries the
called fn's param hint onto that local, so lookups inside the spliced body
keep the bare path. This is a programmer assertion, like a Clojure type hint
(an inaccurate hint just makes the raw get return the wrong value, the same
contract as a wrong ^String), so it stays opt-in and off by default.

On the ray tracer (with inlining on) this is 13.3s to 10.9s, 1.22x, taking it
to 7.8x JVM from 9.4x after the inline pass. The unhinted path emits identical
code (the fast arm is just factored out), so nothing changes without hints.

Verified: a seeded full render produces an identical checksum hinted vs
unhinted; conformance 335/335 in all three modes and the full jpm test pass;
new test/integration/struct-hint-test.janet pins the guard removal, the
inline propagation, and that an accurate hint is correctness-preserving.
2026-06-12 17:45:18 -04:00
Dmitri Sotnikov
e41832c05d
Merge pull request #93 from jolt-lang/perf-ir-inline-sra
perf: AOT escape analysis (IR inlining + scalar replacement)
2026-06-12 20:42:58 +00:00
Yogthos
b5075b73be perf: AOT escape analysis (IR inlining + scalar replacement)
Adds two IR passes to jolt.passes that run when a unit opts into
direct-linking (JOLT_DIRECT_LINK=1, off by default). The inline pass splices
small direct-linked fns at their call sites, copy-propagating trivial args so
that scalar replacement can then see map literals across the call boundary.
Scalar replacement is AOT escape analysis: a map allocation whose only use is
constant-keyword lookup is dropped and each (:k m) is replaced with the value
at :k, both for a literal lookup subject and for a non-escaping let-bound map.
Inlining and scalar replacement iterate to a capped fixpoint, since inlining
exposes literals that scalar replacement then collapses.

The back end stashes the body IR of each single-fixed-arity defn on its var
cell (inline-stash!), and the portable pass reads it through two new jolt.host
contract fns (inline-enabled?, inline-ir). Inlining is gated on :inline?, which
is off for all of init so core and the self-hosted compiler compile exactly as
before (const-fold only); api/init and main re-read JOLT_DIRECT_LINK so the
flag works both for a freshly built context and for the build-time-baked one in
the shipped binary.

Only inline-safe targets are spliced: a single fixed arity, no recur/loop/fn/
try crossing the boundary, within a size budget, a closed body (no free locals
beyond the params, so a self-recursive fn's name reference can't dangle), and
not ^:redef / ^:dynamic. Bodies are fully alpha-renamed so no spliced name can
collide with a caller local.

On the ray tracer this is 15.3s -> 13.0s (1.18x). The ceiling is honest: that
workload's cost is dominated by lookups on maps that genuinely escape (rays,
hits, materials) and by dynamic dispatch (the reduce closure, the :scatter fn),
which escape analysis cannot remove. On allocation-bound code where the
temporaries are local it is far larger: a vec3 reflect+dot loop goes 9.3s ->
0.38s (25x), with the loop body reduced to pure arithmetic.

Verified: full jpm test passes (inline off, no regression); conformance 335/335
in all three modes and the clojure-test-suite both pass with inline on; new
inline-sra-test pins the transform and its semantics.
2026-06-12 15:58:50 -04:00
Dmitri Sotnikov
0cfb5a982e
Merge pull request #92 from jolt-lang/fix-507-p3c
fix: map literal evaluation order; land the local-callee call inline
2026-06-12 18:22:01 +00:00
Yogthos
15d599c0f3 fix: map literals evaluate in source order; land the local-callee inline
jolt-p3c: Clojure evaluates map-literal entries left to right, but the
reader represented map forms as bare janet structs, so entries ran in
hash order. The reader now carries [k v ...] source order out-of-band —
on a struct PROTOTYPE (keys/kvs/length ignore protos, so macros that
get/keys literal map forms see no change; jolt-equal? was already
structural) and as a plain field on the phm rep (nil key/value). The
analyzer (form-map-pairs), the interpreter's map eval, both
syntax-quote walks, and core-sqmap (the lowered `{...} builder — the
array-map case, where Clojure also preserves insertion order) all honor
it, so the order survives macroexpansion in both modes.

jolt-507 root-caused: the parked inline put a LOCAL in janet call-head
position for the first time, and janet resolves head symbols against
the macro table before lexical upvalues — clojure.core/repeat's
self-name local expanded as janet's (repeat n & body) macro, compiling
the self-call into a countdown loop returning nil. Everything in the
issue (interpose, interleave) traced to that one name collision. The
emitter now rebinds local callees to reserved _fp$ symbols (argument
positions never consult the macro table), and the inline — direct
calls for function locals, jolt-call only for IFn-collection
leftovers — lands. Spec rows pin locals named repeat/seq/with called
in head position.

Gate green, suite 4718 steady, bench even with main.
2026-06-12 14:09:50 -04:00
Dmitri Sotnikov
34d32ea3bf
Merge pull request #91 from jolt-lang/perf-maps-math
perf: inline keyword lookup + map literals, clojure.math, indexed reduce
2026-06-12 17:26:11 +00:00
Yogthos
8b2be06b68 perf: inline keyword lookup + map literals, clojure.math, indexed reduce
jank's ray tracer benchmark (examples/ray-tracer) drops from 165.6s to
15.8s per render (10.5x) — from 118x JVM Clojure to 11x. The changes
mirror the optimizations in jank's June 2026 post, adapted to the
janet backend (jolt-4vr, jolt-h79):

- (:kw m) emits an inline lookup instead of variadic jolt-call ->
  core-get's predicate chain. The guard is (get m :jolt/type): janet
  compiles get to an opcode (~17ns) where a struct? cfunction call
  costs ~85ns/lookup. :jolt/type is reserved (the reader rejects it in
  map literals) and every table rep that must not be raw-indexed
  carries it — phm tables now tagged too — so tagged values route to
  core-get and everything else gets janet get, which matches core-get
  for keyword keys on structs/records/nil/arrays. 929ns -> 90ns.
- {:k v ...} literals with scalar const keys emit let-bound values, an
  `and` truthiness test (pure branch opcodes), and a native (struct ...)
  call instead of variadic build-map-literal + runtime kv re-scan. nil
  or false values fall back to build-map-literal, which keeps Clojure's
  nil-entry semantics via the phm rep. 890ns -> 246ns.
- native-op additions: min/max (janet's are variadic with the same
  numeric semantics), nil?/some? lowered to janet's fastfun = / not=
  against nil, and not.
- clojure.math (Clojure 1.11) installed as a namespace whose vars hold
  janet's math natives directly, so calls direct-link. Math/sqrt-style
  interop stays in the frozen interpret-only punt set (~5us/call); this
  is the compiled route (~30ns).
- reduce over pvec/tuple/array iterates indexed in place — it was
  copying the whole pvec into a fresh array on every reduce call — and
  stops at `reduced` instead of scanning the tail.
- interpreter coll-lookup gains the sorted-coll arm: (:k (sorted-map ..))
  was nil in interpret mode (compiled mode had it right).

map-fastpath-spec pins keyword-invoke/map-literal semantics (16+15
rows incl. nil-value maps, records, sorted, vectors-of-internals) and
clojure.math (10 rows). Two pre-existing bugs found and filed while
writing it: map literals evaluate entries in reader-hash order
(jolt-p3c), and an attempted local-callee call inline that breaks
overlay lazy self-recursion is parked with a repro (jolt-507).

Gate green, conformance 335/335 x3, suite 4718 steady, core bench even
with main back-to-back.
2026-06-12 13:15:03 -04:00
Dmitri Sotnikov
ca83b4a1de
Merge pull request #90 from jolt-lang/error-round-5
errors: reader errors carry file:line:col (round 5)
2026-06-12 15:15:39 +00:00
Yogthos
026ad888cd errors: reader errors carry file:line:col (jolt-2o7.5)
Syntax errors were positionless ('Unterminated list', no idea where).
Now they use Clojure's shape:

    Error: Syntax error reading source at (src/app/syn.clj:3:8): Unmatched delimiter: ]
      at src/app/top.clj:1

The reader's 15 error sites raise a {:jolt/reader-error :msg :pos}
struct carrying the byte offset (every site already had pos in scope).
The parse entry points convert offset -> line:col on demand and
re-raise the formatted message: parse-string against its own string
(no file), parse-all-positioned against the full source with the
file threaded in from the loaders — rebasing slice-relative offsets
onto the original source so positions stay absolute. No per-token
cost; nothing is tracked until an error actually happens.

Unmatched-delimiter messages match Clojure's 'Unmatched delimiter: )'
wording. cli-test rows assert positions for unterminated string/list,
unmatched delimiter through a require (composing with round 4's
'while loading' chain), and bad ## tokens. Gate green, suite 4718
steady, bench within noise.
2026-06-12 10:55:42 -04:00
Dmitri Sotnikov
2a3dfd223d
Merge pull request #89 from jolt-lang/error-round-4
errors: load errors carry file:line and the require chain (round 4)
2026-06-12 14:51:45 +00:00
Yogthos
6d082e9b1d errors: load errors carry file:line and the require chain (jolt-2o7.4)
A failing top-level form now reports where it lives:

    Error: Cannot add 1 and "boom" — + expects numbers
      at /path/src/app/broken.clj:3
      while loading /path/src/app/mid.clj
      while loading /path/src/app/top.clj

The reader has no per-form positions (round 5), but the loaders know
exactly which slice of source each form came from: parse-all-positioned
(reader) counts newlines around parse-next and returns [form line]
pairs; load-ns, load-string and load-ns-source evaluate through a
positioned loop that on error stashes the innermost form's {:file
:line} on the env and appends each file unwound through to a loading
chain. report-error prints both, suppressing the synthetic <eval>
strings the CLI feeds itself (the require/apply one-liners).

load-string takes an optional file arg; run-file passes the script
path so script errors name the script. cli-test rows cover the
3-requires-deep case, script files, and that one-line -e output stays
clean. Gate green, suite 4718 steady, bench within noise.
2026-06-12 10:40:42 -04:00
Dmitri Sotnikov
32f733b6f6
Merge pull request #88 from jolt-lang/error-round-3
errors: unresolved symbols error with Clojure's message (round 3)
2026-06-12 14:18:28 +00:00
Yogthos
c230e70ed7 errors: unresolved symbols error with Clojure's message (jolt-2o7.3)
A typo'd symbol used to auto-intern an unbound var and die later as
'Cannot call nil as a function' with no hint which symbol. Now:

    $ jolt -e '(undefined-fn 1)'
    Error: Unable to resolve symbol: undefined-fn in this context

The analyzer's :unresolved fallthrough now punts to the interpreter
(whose resolver raises the message above when the form runs) instead of
emitting a var-ref that interned the var. A punt rather than a hard
throw because runtime-interning forms (defmulti's setup) legitimately
reference the var they're about to create from a nested do.

Pulling that thread surfaced three real bugs the leniency was masking:

- h-resolve-global resolved unqualified symbols against ctx-current-ns,
  which during analysis is jolt.analyzer — so user-ns vars NEVER
  resolved through it; the lenient arm happened to emit the right ns.
  Now resolves against the compile ns like the qualified branch.
- Top-level (do ...) wasn't split: Clojure compiles and EVALS each
  child in sequence so earlier children's runtime effects (defmulti's
  intern) are visible while later children compile. eval-toplevel now
  splits.
- The stdlib itself had forward references the auto-intern hid:
  10-seq's transducers used vreset!/vswap! from 20-coll (moved to
  10-seq); in 20-coll qualified-ident?/realized?/list*/underive
  referenced defs declared later in the file (reordered); sorted? and
  partition-all are genuinely later-tier and got (declare ...).

Test rows updated where they encoded the old leniency: ir-passes'
dead-branch row (unresolved in a dead branch is an error, as in
Clojure), compile-mode's ctx-isolation row (other ctx now errors
instead of reading nil), cli rows assert the new message. Gate green,
conformance 335/335 x3, suite 4718 steady, bench within noise.
2026-06-12 10:07:48 -04:00
Dmitri Sotnikov
3fd0af33b6
Merge pull request #87 from jolt-lang/error-rounds-1-2
errors: user-readable messages and stack traces (rounds 1+2)
2026-06-12 13:07:46 +00:00
Yogthos
e0442f84e4 errors: user-readable messages and stack traces (jolt-2o7 rounds 1+2)
Before: (+ 1 "a") printed 'could not find method :+ for 1 or :r+ for "a"'
followed by three janet frames pointing at jolt internals. After:

    Error: Cannot add 1 and "a" — + expects numbers
      at app.deep/level3

Round 1 — compiled fns carry their Clojure identity:
- The analyzer's recur target (which doubles as the compiled janet fn's
  name) is now ns/fn-name (_r$app.deep/level3--N), so janet stack traces
  name the user's fns; defn passes the self-name through to fn.
- eval-toplevel re-raises with propagate instead of protect+error — the
  failing fiber's stack was being discarded, which is why every trace began
  at eval-toplevel.
- require/maybe-require-ns route loaded namespaces through the loader's
  compile-or-interpret eval-toplevel via a ctx hook (the evaluator can't
  import the loader). Previously REQUIRED namespaces always ran interpreted:
  slower, and their fns were anonymous in traces.

Round 2 — report-error presents for users (rephrase-inspired):
- The full trace text is stashed at the innermost eval-toplevel boundary
  (janet's debug/stacktrace walks the fiber propagation chain; debug/stack
  cannot), then filtered: _r$ frames demangled to ns/fn-name, jolt-internal
  and [eval] frames dropped. JOLT_DEBUG=1 restores the raw janet trace.
- Message rewrites: janet arithmetic dispatch -> 'Cannot add X and Y — +
  expects numbers'; compiled arity -> Clojure's 'Wrong number of args (N)
  passed to: ns/fn'; nil-call gets an undefined-symbol hint (round 3 will
  fix resolution properly).

6 cli-test rows assert the exact user-visible output. Gate green, suite
4718 steady, bench within noise.
2026-06-12 08:58:08 -04:00
Dmitri Sotnikov
a40c715ade
Merge pull request #86 from jolt-lang/reitit-enablement
host: enable reitit routing — runtime JOLT_FEATURES, class-shim hooks, get-on-strings
2026-06-12 05:31:03 +00:00
Yogthos
f3a9b9ab1f host: scoped reader-feature toggle, clojure.template, java.util.Locale alias
Follow-ups for running reitit alongside default-feature libraries in one app:

- __reader-features / __reader-features-set! exposed to Clojure so a namespace
  can load a clj-targeted lib (reitit, under :clj) WITHOUT forcing the whole
  process to :clj — set features, require, restore. Honeysql/selmer/ring stay
  on the default set they were validated under. (jolt vector args coerced to a
  janet array — janet map over a pvec iterates keys otherwise.)

- clojure.template added to the stdlib (verbatim, pure Clojure over
  clojure.walk) — honeysql's :clj branch requires it.

- java.util.Locale registered as a qualified alias (+ ROOT) for selmer's :clj
  Locale/US use.
2026-06-12 01:22:28 -04:00
Yogthos
6ab76efd19 host: enable reitit — runtime JOLT_FEATURES, class-shim hooks, get-on-strings, java shims
Everything reitit-core needs to load unmodified from git under :clj features:

- The baked binary now re-reads JOLT_FEATURES at startup (like JOLT_PATH).
  reader-features-set! runs at module load = BUILD time for a binary, so a
  process opting into :clj (to read a lib's :clj branches) was ignored, and
  unmatched #?(...) forms silently spliced to nothing — defn of a fn with an
  empty arglist, hence the cryptic index errors.

- (get s i) indexes a string and returns the char, as in Clojure (nth did;
  get returned nil). reitit's path parser is (get path i)-based — without
  this every route read as static.

- Class-shim registration exposed to Clojure: __register-class-statics! /
  __register-class-methods! / __register-class-ctor!, so a library can mirror
  a Java class jolt doesn't ship (the reitit.Trie mirror lives in jolt-lang/
  router on top of these).

- Java surface reitit's :clj branches call: .getMessage (on exceptions and
  strings) and a small universal object-method set, .intern, java.util.HashMap
  (a mutable map wrapper). Plus defprotocol already took keyword options.

Gate green; clojure-test-suite 4715 -> 4718 (the get-on-strings fix).
2026-06-12 01:09:00 -04:00
Dmitri Sotnikov
0ad6529d44
Merge pull request #85 from jolt-lang/honeysql-enablement
core: defprotocol accepts docstring + keyword options (honeysql)
2026-06-12 03:04:42 +00:00
Yogthos
e1a6d77b0c core: defprotocol accepts docstring + keyword options (honeysql)
Clojure's defprotocol takes an optional docstring and leading keyword
options (:extend-via-metadata true) before the signatures; jolt's macro
fed the option keyword to (first sig). honeysql declares its InlineValue
protocol exactly that way — with the fix, all four honeysql namespaces
load unmodified from git and the formatter produces correct sqlvecs for
selects/inserts/updates/deletes/joins/:inline. Listed in libraries.md.
2026-06-11 22:56:12 -04:00
Yogthos
b2c9970583 docs: jdbc.core (jolt-lang/db) in libraries.md 2026-06-11 22:38:09 -04:00
Dmitri Sotnikov
df909d0914
Merge pull request #84 from jolt-lang/bake-env-scrub
host: JOLT_BAKE_ENV_ALLOWLIST scrubs the env during image bakes (jolt-s3j)
2026-06-12 02:00:28 +00:00
Yogthos
59853492dd host: JOLT_BAKE_ENV_ALLOWLIST scrubs the env during image bakes (jolt-s3j)
A native-executable build bakes the jolt ctx, and env-reading libraries
(config.core/load-env) snapshot the ENTIRE build environment into it — jpm
marshals that into the binary. GitHub push protection caught real API
tokens inside an example's build output this way.

With JOLT_BAKE_ENV_ALLOWLIST set (comma-separated names — a project's
build.sh exports it for the bake), System/getenv serves only the listed
variables: single-name reads of unlisted vars return nil and the full
snapshot is filtered. Unset — every normal runtime — reads stay live and
unfiltered, so a baked binary that re-reads env at startup (config.core/
reload-env) sees the real runtime environment.

Verified A/B on ring-app: a planted token appears in the unscrubbed binary
(strings | grep: 1 hit) and not in the scrubbed one (0), which still serves.
Direct janet.os/environ bridge calls remain unfiltered host access, as
documented.
2026-06-11 21:50:45 -04:00
Yogthos
d8260fb587 docs: greeter retired — ring-app is the example 2026-06-11 21:22:02 -04:00
Dmitri Sotnikov
2009de2a76
Merge pull request #83 from jolt-lang/ring-core-host-shims
host: ring-core enablement — shims, class tokens, vendored spork/http
2026-06-12 01:19:26 +00:00
Yogthos
9aa1479d81 deps: jolt-deps finds the jolt binary next to itself
Running a checkout's build/jolt-deps by path failed with ENOENT unless
build/ was also on PATH: exec-jolt spawned a bare "jolt". Resolution is
now $JOLT_BIN, then the jolt sitting beside this binary (argv[0]'s
directory — the pair is built together), then PATH.
2026-06-11 21:10:10 -04:00
Yogthos
06e0899578 deps: :jpm/module coordinates; the janet.* bridge autoloads jpm modules
The vendored spork/http is gone — jpm owns janet packages. In its place:

- The janet.* bridge autoloads jpm-installed modules on first reference:
  janet.spork.http/server requires spork/http from the module path and
  caches its bindings (failures are negatively cached). Works for any
  module, in every mode, including inside net/server connection fibers.

- deps.edn grows a :jpm/module coordinate: jolt-deps verifies the module is
  importable at resolve time, optionally running `jpm install` on the
  :jpm/install package once when it isn't, and otherwise fails with the
  install hint. Contributes no source roots. ring-app declares spork/http
  this way.

Docs: README's interop section, docs/tools-deps.md (:jpm/module reference),
and the ring-app README (including the jpm-version caveat for spork HEAD's
.janet native sources, which older jpm rejects).
2026-06-11 20:58:43 -04:00
Yogthos
9aadbf42fd docs: vendored spork/http note in the interop section; ring-core/ring-codec in libraries.md 2026-06-11 20:42:41 -04:00
Yogthos
9ba8e0870c core: the fn-macro hint unwrap that belonged in the previous commit
(^bytes [b])-style return hints reach fn as a (with-meta [b] {:tag ...})
form; unhint sheds the wrapper through the rebuild path so the clause
representation never changes. The host-interop hint rows exercise it.
2026-06-11 20:39:58 -04:00
Yogthos
8212bc76f7 host: ring-core enablement part 2 — class tokens, vendored spork/http, reader-draining slurp
Class names evaluate to their canonical class-name STRINGS (the same values
class returns), so a (defmulti m (comp class :body)) matches (defmethod m
String ...) — ring.util.request dispatches exactly this way. Constructor
sugar and the new special form resolve the actual ctor from the registry
when given a token; dispatch-only names (InputStream, File, ISeq, ...) are
interned for defmethod position. nil is a legal multimethod dispatch value
now (sentinel-keyed: janet tables drop nil keys) — ring keys body-string's
no-body case on it.

spork/http is VENDORED (vendor/spork/http.janet, MIT) and baked into the
image, reaching the jolt layer as janet.spork.http/* through the janet.*
bridge — whose lookup is now a chain (runtime fiber env, module env, the
vendored registry), which also fixes janet/* resolution inside net/server
connection fibers (they carry a foreign env). jolt.http is rewritten over
the spork client (its old net/request never existed; also fixes its own
get shadowing clojure.core/get).

Also: slurp accepts opts and DRAINS reader shims (ring middleware slurps
request bodies), clojure.string/replace takes fn replacements with Clojure's
match-or-groups argument, .indexOf int needles are char codes, .getBytes on
the String surface, and ^bytes-style return hints on param vectors parse
(the fn macro unwraps the with-meta form).

Suite steady at 4715/5348; conformance x3 green; deps-conformance medley +
cuerdas green (the stuartsierra/dependency failure predates this change).
2026-06-11 19:50:52 -04:00
Yogthos
7c26a182e8 host: ring-core enablement — java.net/util shims, protocol dispatch gaps, IReduceInit, spork/http bridge
The interop surface ring.util.codec needs (registered through the javatime
shim registries): URLEncoder/URLDecoder (www-form-urlencoded in pure janet),
Charset/forName, Base64 encoder/decoder, Integer/valueOf with radix +
parseInt, StringTokenizer, clojure.lang.MapEntry (a 2-tuple), a String ctor
from bytes, .getBytes on the String surface, and a java.lang.Number method
surface (byteValue and friends).

Protocol fixes: extend-protocol on java.util.Map/Set/List now dispatches
(maps — phm/struct/sorted/records — never produced host tags and fell to
Object), lazy seqs gained their ISeq tags, and a nil extension arm works
(group-by-head and extend-type both choked on the nil head). reduce
dispatches to a reified clojure.lang.IReduceInit's own reduce method, which
is how ring-codec tokenizes.

jolt-deps learns :deps/root (tools.deps monorepo subdirectory checkouts —
ring-core lives inside ring-clojure/ring). spork/http, when jpm-installed,
reaches the jolt layer as janet.spork.http/* through the janet.* bridge
(soft: nothing requires it unless used).

The protocol fixes alone let 29 more clojure-test-suite assertions execute:
5319 -> 5348 run, 4706 -> 4715 pass.
2026-06-11 19:05:50 -04:00
Dmitri Sotnikov
58f9a5e596
Merge pull request #82 from jolt-lang/printer-cold-types
core: cold tagged-type printing migrates to print-method defmethods
2026-06-11 22:44:06 +00:00
Yogthos
af3e49a89c core: cold tagged-type printing migrates to print-method defmethods
The first per-type migration print-method unlocked: uuid, regex, transient,
and channel rendering move from host pr-render branches to io-tier
defmethods (exact same output). The renderer's tagged fallthrough now
dispatches ANY remaining :jolt/* value through the print-method hook before
the raw pairs view — so every tagged type is user-overridable, atoms
included ((defmethod print-method :jolt/atom ...) fires nested), and future
per-type migrations are pure overlay additions.

Hot types (numbers, strings, symbols, collections) stay native, and inst/
namespace/var stay host for now — their formatters (rfc3339, display names)
live there anyway. A transient's :kind is read with jolt.host/ref-get: get
on a transient is the dispatched collection lookup (same trap as sorted
colls).

Before the hook is wired (init-time error messages) tagged values fall
through to the pairs view — bootstrap rendering only.
2026-06-11 18:35:21 -04:00
Dmitri Sotnikov
8efdd9956d
Merge pull request #81 from jolt-lang/print-method-multimethod
core: print-method is a real multimethod (jolt-g1r); records print canonically
2026-06-11 22:19:26 +00:00
Yogthos
1e4a0a6d53 core: print-method is a real multimethod (jolt-g1r); records print canonically
print-method/print-dup are now multimethods in the io tier with Clojure's
exact dispatch ((:type meta) keyword, else type — core.clj 3693). On jolt the
dispatch value for a record is its quoted full-name symbol, since class names
aren't values here.

Records used to pr-str as the raw janet table; the renderer's record branch
now prints Clojure's #ns.Type{:k v} syntax, and first consults a callback the
api wires up after the overlay loads — so a user defmethod on a record type
fires everywhere: top level, nested in collections, through pr/prn/pr-str.
Builtin overrides (a :number method) fire only on direct print-method calls;
pr keeps the native fast path (documented divergence).

java.io.Writer arrives as a shim beside the StringReader/StringBuilder ones:
a :jolt/writer tagged value with write/append/flush/toString, a StringWriter
ctor, and a sink variant the renderer callback uses.

Two latent host bugs fixed on the way: the interpreted syntax-quote splice
blew up on ~@nil (an interpreted macro's empty & rest binds nil — first tier
user of defmulti found it; d-realize now treats nil as the empty seq), and
(print-method x nil) now throws like the JVM instead of returning nil.

10 spec rows; bench dead even (sandwich run); greeter green on a fresh
binary.
2026-06-11 18:10:11 -04:00
Yogthos
eff8cb99a5 docs: track libraries confirmed to work on jolt
config and Selmer from the greeter example; medley and cuerdas from the
deps-conformance battery.
2026-06-11 17:44:57 -04:00
Dmitri Sotnikov
174b923d30
Merge pull request #80 from jolt-lang/shrink-batch-2
core: 16 more bindings to the overlay — promise/deliver, the proxy surface, JVM-shape stubs
2026-06-11 21:40:07 +00:00
Yogthos
89e67fbc47 core: 16 more bindings to the overlay — promise/deliver, the proxy surface, JVM-shape stubs
Batch 2 of the post-shrink sweep, all pure compositions or documented stubs:
enumeration-seq/iterator-seq (seq), promise/deliver (an atom — deref of an
undelivered promise stays nil, single-threaded host), bean, uri?,
special-symbol? (an evaluated set of quoted symbols — a QUOTED set literal
stays an unevaluated reader form on jolt, which the first version tripped
over), print-method/print-dup (inert until jolt-g1r), and the whole proxy
surface (mappings/call-with-super/init/update pass through, the constructive
half throws). The seed loses 16 defns and bindings; nothing kept.
2026-06-11 17:33:29 -04:00
Dmitri Sotnikov
001bb0c4c6
Merge pull request #79 from jolt-lang/shrink-batch-bitops-set
core: variadic bit ops, set? covers sorted sets, if rejects extra forms
2026-06-11 21:13:09 +00:00
Yogthos
c6f6b7deb7 core: variadic bit ops, set? covers sorted sets, if rejects extra forms
Three canonical-conformance fixes from the post-shrink batch:

- bit-and/bit-or/bit-xor/bit-and-not get Clojure's variadic arities as
  20-coll shells folding the binary host ops (now __bit-* seams). 2-arg call
  sites still compile to the native janet op via the backend's native-ops
  table. The passes.clj constant-fold table now names the seams — the public
  fns are overlay and don't exist when the compiler loads (this briefly broke
  every compile-mode init).

- core-set? recognizes the :jolt/sorted-set representation (jolt-dpn):
  (set? (sorted-set 1)) was false, and ifn? on sorted sets inherited the bug.

- (if) / (if test) / (if test then else extra) throw in both the analyzer
  and the interpreter — spec 03-special-forms X1, now marked verified.

Suite 4704 -> 4706; bench and the greeter example benchmark are flat.
2026-06-11 17:04:26 -04:00
Dmitri Sotnikov
621dc8e310
Merge pull request #78 from jolt-lang/jolt-6xn-arity
core: enforce fn arity in both modes (jolt-6xn); canonical seq-to-map-for-destructuring
2026-06-11 20:22:02 +00:00
Yogthos
9e9fd19450 core: enforce fn arity in both modes (jolt-6xn); canonical seq-to-map-for-destructuring
Fixed arities now throw Clojure's ArityException shape — 'Wrong number of
args (N) passed to: name' — on any count mismatch; variadic arities on fewer
than their fixed params. The compiled path already enforced fixed arities via
janet's native fn check and multi-arity dispatch; this adds the check to the
interpreter's single-arity closures (the oracle was silently dropping extra
args and giving a raw tuple-index error for missing ones) and guards the
compiled single-variadic wrapper's minimum. Messages carry the fn name when
there is one. 16 spec rows; the update.cljc suite row flipped green (4703 ->
4704).

Enforcement exposed that seq-to-map-for-destructuring had drifted: the spec
row called the 1-arity fn with two args, and the body silently dropped a
trailing unpaired element. Replaced with the canonical Clojure 1.11 version
(even pairs build the map, a single trailing element passes through — so
(f {:b 2}) kwargs calls work — and an unpaired key throws).

Also: transients RFC notes tuple support from the seed-shrink rounds.
2026-06-11 16:20:14 -04:00
Dmitri Sotnikov
501c6bf9c9
Merge pull request #77 from jolt-lang/seed-shrink-r6-printer
core: pr/prn/pr-str/print/println to the overlay; pr-str escapes strings now
2026-06-11 18:20:28 +00:00
Yogthos
1de5ede246 core: pr/prn/pr-str/print/println to the overlay; pr-str escapes strings now
Round 6 of the seed shrink (the printer round, scoped by the perf wall). The
five wrappers move to 20-coll over two new host seams: __write (push a string
to *out*) and __pr-str1 (render one value readably). The renderer itself
stays in the seed — it's representation-coupled (pvec/phm/phs/sorted
internals) and shared with the hot str, and rendering through overlay calls
would pay the per-element call cost everywhere big values get printed.
print-method as a real multimethod is follow-up work.

The new spec rows caught a renderer bug: string bodies were never escaped, so
(pr-str "a\"b") didn't round-trip through the reader. pr-render now
escapes quote/backslash/control chars per Clojure.
2026-06-11 14:12:12 -04:00
Dmitri Sotnikov
5cac9efd4e
Merge pull request #76 from jolt-lang/seed-shrink-r5-into-transduce
core: transduce/eduction/->Eduction to the overlay; into stays seed (perf wall)
2026-06-11 18:00:59 +00:00
Yogthos
2557e3295f core: transduce/eduction/->Eduction to the overlay; into stays seed (perf wall)
Round 5 of the seed shrink. transduce is the canonical 5-liner over reduce
(which already honors reduced and steps lazy seqs); eduction composes with
comp and stays eager into a vector (documented divergence, as before);
td-comp — eduction's last caller — is deleted from the seed. transient
accepts tuples now (reader vectors / map entries), so (into [] (first {:a 1}))
keeps working everywhere a vector does.

into was moved, benched, and moved back: the overlay call layers cost the
into-vec suite ~11% back-to-back (536 vs 480ms), the same per-call wall that
sent even?/odd? home in round 4. A transient conj! fast path didn't pay for
itself either (jolt call overhead dominates, not the per-element conj). The
seed keeps core-into + its private transduce machinery; the binding count
still drops by three.
2026-06-11 13:52:35 -04:00
Dmitri Sotnikov
b9c7a623bb
Merge pull request #75 from jolt-lang/seed-shrink-r4-syntax-tier
core: zero?/pos?/every? to 00-syntax, char? to the overlay; fix rest/next over sets and maps
2026-06-11 17:33:31 +00:00
Yogthos
a1a9fd9949 core: zero?/pos?/every? to 00-syntax, char? to the overlay; fix rest/next over sets and maps
Round 4 of the seed shrink. zero?, pos?, every? move to the syntax tier
(empty? and the analyzer use them — raw def+fn* per the file constraint);
char? joins the tagged-value predicates in 20-coll. coll? stays seed: host
set? doesn't cover sorted sets (filed jolt-dpn) and the tag check from the
overlay would hit the sorted-coll get trap. pos? guards number? explicitly —
the staged recompile emits bare > as the native janet op, which orders
strings (zero? gets the same guard; spec rows lock both plus neg?).

The canonical every? seq-walks its coll, which exposed that rest/next over
sets, phms, struct maps and sorted colls fell into core-rest's indexed
fall-through and walked the wrapper table's INTERNAL fields — (next #{1 2})
was (nil nil), (clojure.set/subset?) broke. core-rest now seqs those
representations (branches placed AFTER the hot vector/lazy paths; the first
ordering cost seq-pipe 4x). Suite rises 4700 -> 4703; baseline 4660 -> 4695.

even?/odd? are back in the seed after the bench A/B: (filter even? ...) pays
an extra call layer per element through the overlay (seq-pipe 262 -> 1100ms).
They join the perf-wall list with the lazy hot fns.
2026-06-11 13:24:51 -04:00
Dmitri Sotnikov
407d656240
Merge pull request #74 from jolt-lang/seed-shrink-r3-pure-leaves
core: move the pure-leaf fns to the overlay; memfn is a real macro now
2026-06-11 16:48:40 +00:00
Yogthos
7ca88ab2b5 core: move the pure-leaf fns to the overlay; memfn is a real macro now
Round 3 of the seed shrink. To the overlay: identity, constantly, neg?,
even?, odd? (20-coll, ahead of their first in-tier uses), not= and unreduced
(00-syntax — the kernel and seq tiers use them), ==, ensure-reduced,
halt-when, parse-boolean, parse-uuid, newline, seque, array-seq, to-array-2d,
and the masking unchecked-byte/short/char/float/double coercions. parse-uuid
validates via re-matches over a new __make-uuid host binding (overlay source
can't write :jolt/type map literals). memfn moves to 30-macros as a working
macro over the .method call sugar instead of a fn that throws.

Behavior fixes toward Clojure, each with spec rows: == now throws on
non-numbers instead of comparing them, and halt-when is the canonical
::halt-map version (the halt value replaces the whole reduction result, no
double completion). list? and map-entry? stay in the seed — both are
representation-coupled (plist/tuple checks).

clojure-test-suite goes 4701 -> 4700: update.cljc expects
(update {:k 1} :k identity 1 2 3 4) to throw an arity error, and jolt fns
don't enforce fixed arity anywhere (pre-existing, language-wide — the seed's
Janet identity threw natively). Filed as jolt-6xn; fixing it should flip
several suite rows at once.
2026-06-11 12:38:12 -04:00
Dmitri Sotnikov
65687c69ae
Merge pull request #73 from jolt-lang/seed-shrink-r2-aliases
core: move promoting/unchecked arithmetic aliases, int?, num to the overlay
2026-06-11 10:55:56 -04:00
Yogthos
9ac0f54e72 core: move promoting/unchecked arithmetic aliases, int?, num to the overlay
Jolt numbers don't overflow, so +'/-'/*'/inc'/dec' and the whole unchecked-*
family are just the checked ops — now one-line defs in core/20-coll.clj
instead of ~25 seed bindings. int? and num move the same way.

unchecked-divide-int now goes through quot, so dividing by zero throws like
the JVM instead of silently truncating infinity. unchecked-int/long gain
char handling via int, matching Clojure ((unchecked-int \a) => 97). The
masking byte/short/char coercions are not aliases and stay in the seed for
a later round.

Also drops a second duplicate set of unchecked defns that was shadowing the
first at module load.
2026-06-11 10:47:14 -04:00
Dmitri Sotnikov
826bee29d4
Merge pull request #72 from jolt-lang/seed-shrink-r1-dead-code
core: delete seed fns shadowed by the overlay; fix methods/remove-all-methods
2026-06-11 10:35:29 -04:00
Yogthos
a06c39266a core: delete seed fns shadowed by the overlay; fix methods/remove-all-methods
The seed copies of inst?/inst-ms and the multimethod table ops
(get-method/methods/remove-method/remove-all-methods/prefer-method) are dead
code — the overlay redefines all of them (20-coll.clj, 30-macros.clj) — so
they're deleted along with duplicate bindings-table entries (unreduced,
eduction, unchecked-inc/dec/add/subtract).

New spec rows for methods/remove-all-methods caught two real bugs in the
evaluator's setup fns: methods-setup returned the live host table (count
rejected it, and callers could mutate dispatch state), and
remove-all-methods-setup swapped in a fresh table the dispatch closure never
saw. methods now returns a phm snapshot; remove-all-methods clears in place.
2026-06-11 09:21:25 -04:00
Dmitri Sotnikov
1a2841b02f
Merge pull request #71 from jolt-lang/greeter-example-fixes
deps + compile fixes surfaced by the greeter example
2026-06-11 01:50:06 -04:00
Yogthos
703a59d40b core: close the compile-path gaps that broke uberscripting Selmer + config
Loading these libs via require worked (load-ns-source interprets, macros
expand lazily) but the same code inlined by uberscript routes through
eval-toplevel and compiled, surfacing four gaps:

- a ^{:map} metadata def name reads as (def (with-meta name m) v); the
  analyzer died extracting the name (config.core's defonce env). It now
  throws uncompilable so the interpreter, which handles it, takes over.
- declare was a no-op, so a compiled forward reference to a declared
  name that collides with a janet root binding bound to the host fn
  (selmer.parser's (declare parse) compiled to janet's 1-arg parse).
  declare now expands to no-init defs, the interpreter interns them,
  and the analyzer routes no-init def to the interpreter.
- class? was missing (selmer.util's exception macro calls it at
  expansion time). Always false, like ratio? — no Class objects here.
- require of an unlocatable namespace silently left an empty ns behind,
  deferring the failure to an unresolved symbol far from the cause. It
  now throws like Clojure's FileNotFoundException. Namespaces entered
  in-session count as loaded (Clojure puts them in *loaded-libs*), and
  the SCI bootstrap opts out via :lenient-require? since its
  clj-targeted requires can't all exist on this host.
2026-06-11 01:31:50 -04:00
Yogthos
c2511fee7e deps: make the .cpcache key stable and keep git fetch off stdout
The cache key was built with janet's (hash ...), which is seeded per
process, so it never matched across invocations and every jolt-deps run
re-resolved and re-fetched. Store the raw key material instead.

Run the jpm git calls silenced (:silent) with a one-line progress note
on stderr — the checkout chatter ("HEAD is now at ...") was landing on
stdout and corrupting the documented JOLT_PATH=$(jolt-deps path) capture.

Covered by two new deps-resolve checks: a cross-process cache-hit probe
(sentinel-tampered cache file) and a stdout-cleanliness check against a
local file:// git dep.
2026-06-11 01:31:37 -04:00
Dmitri Sotnikov
9cb3369f50
Merge pull request #70 from jolt-lang/config-shims
java shims for yogthos/config + three conformance fixes
2026-06-11 00:19:33 -04:00
Yogthos
d161e16df6 core: java shims for yogthos/config + three conformance fixes
yogthos/config now loads and runs end to end: config.edn/.lein-env
deep merge, env vars keywordized and type-converted, PushbackReader
over io/reader, reload-env via alter-var-root. New shims:
java.io.PushbackReader (read/unread), Long/parseLong, BigInteger.,
Boolean/parseBoolean, System/getProperties; clojure.edn/read now
drains an actual reader (it used to read one LINE from a raw janet
file handle, so multi-line config files and shim readers both broke).

Three real bugs shaken out, each with regression specs:

- An empty rest arg bound () instead of nil. ((fn [& r] (if r 1 2)))
  returned 1; the truthy () sent config's (or (.exists f) required)
  down the wrong branch. Fixed in the interpreter and the compiled-fn
  emission. Internal apply boundaries (protocol dispatch, core-apply)
  now accept a nil seq like Clojure's (apply f x nil).

- seq/map over a raw janet table (System/getenv, os/environ) yielded
  nothing, so config's read-system-env came back empty. Raw host
  tables now seq as kv entries like any map, in core-seq,
  realize-for-iteration, and coll->cells. The old spec row hid this
  behind a vacuous (every? pred empty) — replaced with one that
  asserts non-emptiness.

- edn/read single-line limitation, as above.

test/integration/config-lib-test.janet runs the real library from
~/src/config (skips when absent).
2026-06-11 00:11:04 -04:00
Dmitri Sotnikov
1673a0f024
Merge pull request #69 from jolt-lang/deps-tasks
deps: global gitlibs-style clone cache + :tasks runner
2026-06-10 23:30:57 -04:00
Yogthos
9ab36eb97f deps: global gitlibs-style clone cache + :tasks runner
Git clones now default to a shared, sha-immutable cache —
$JOLT_GITLIBS, else <config-dir>/gitlibs — instead of a per-project
./jpm_tree, the tools.gitlibs ~/.gitlibs model. Passing tree
explicitly still works (tests do). The resolved-roots cache moves
out of the clone tree to the project-local .cpcache/jolt-deps.jdn,
since roots depend on the project while clones don't.

deps.edn grows :tasks, the honest subset of babashka's: a string
task is a shell command, a map task is {:main-opts [...] :doc}.
jolt-deps tasks lists them (merged user+project), jolt-deps task
NAME runs one. Bare-expression tasks are out of scope: the reader
hands back parsed data and round-tripping to source is fragile.

Also fixes load-config skipping the symbol-key normalization when
only one config file existed — :tasks/:deps keys stayed raw reader
symbols (which embed positions and never compare equal), so lookups
missed. Regression rows in deps-tasks-test; docs updated for the
whole tools.deps surface (aliases, -A/-M, user config, conflicts,
gitlibs cache, tasks).
2026-06-10 23:22:36 -04:00
Dmitri Sotnikov
0a01afe595
Merge pull request #68 from jolt-lang/deps-aliases
deps: aliases, user config merge, tools.deps conflict semantics
2026-06-10 23:21:52 -04:00
Yogthos
add80c3018 deps: aliases, user config merge, tools.deps conflict semantics
deps.edn :aliases now work the tools.deps way, scoped to what jolt
supports (git/:local, no maven): :extra-paths and :extra-deps
accumulate across selected aliases, :main-opts is last-wins. The CLI
grows -A:dev:test (selects aliases for path/run/repl/-e) and
-M:alias (runs the alias :main-opts through jolt). A user-level
deps.edn ($JOLT_CONFIG, else $XDG_CONFIG_HOME/jolt, else ~/.jolt)
merges under the project file: :deps and :aliases merge per key with
the project winning, everything else replaces.

Resolution is now breadth-first so a top-level coordinate always
beats a transitive one for the same lib (it was DFS first-wins —
a dep's pin could shadow the project's own). Conflicting coordinates
for one lib warn on stderr with both coords and which won. Also
fixes dedup keying: it hashed the symbol struct's string repr, which
carries reader position metadata, so the same lib from two files
never deduped.

resolve-deps-cached keys on project edn + user edn + aliases.
Tests: deps-aliases-test and deps-conflicts-test, local deps only.
2026-06-10 23:13:46 -04:00
Dmitri Sotnikov
f50fdad0ee
Merge pull request #67 from jolt-lang/java-time-shims
java.time + java.io shims: Selmer renders end to end
2026-06-10 22:38:08 -04:00
Yogthos
d584369dda core: java.time + java.io shims — Selmer renders end to end (jolt-ea7)
Selmer now loads and renders templates on jolt: variables, filters
(upper, date with JVM patterns), if/for tags, nested lookups, HTML
escaping, and render-file with its last-modified template cache.

New src/jolt/javatime.janet provides the java.time surface Selmer's
date filters use (DateTimeFormatter/Instant/ZoneId/LocalDateTime/
FormatStyle/Locale, epoch-ms backed, host-local timezone) plus the
java.io/java.lang/java.net shims its template reader needs
(StringReader, StringBuilder, URL, File/separator, Class/forName).
Everything registers through three new evaluator registries
(class-statics, tagged-methods, class-ctors), so the module is data
plus an install call.

Fixes shaken out along the way, each load-bearing for Selmer and
correct on their own:
- :refer :all silently referred nothing (it iterated the :all keyword)
- ns :import ignored vector specs and didn't share deftype ctor vars
- dot calls on deftype/reify instances never consulted the protocol
  registry, so (.render-node node ctx) failed where (render-node ...)
  worked
- instance? rejected expression type args like (Class/forName "[C")
- char-array didn't accept a string
- io/resource now searches the loader's source roots (the classpath
  analog); io/reader handles char arrays, URLs, readers, and returns
  an in-memory reader with :read-line-fn for file paths
- String .split (regex, JVM trailing-empty semantics), file-path
  methods (.toURI/.toURL/.getPath/.lastModified/.exists)
- System/getProperty (os.name & co), the janet/* bridge now works
  inside env-less fibers, and qualified class names that syntax-quote
  mangles (selmer.util/StringBuilder) fall back to the ctor registry

Spec rows cover the shim surface; test/integration/selmer-test.janet
runs the real Selmer from ~/src/selmer (skips cleanly when absent).
2026-06-10 22:29:53 -04:00
Dmitri Sotnikov
123c58560a
Merge pull request #66 from jolt-lang/unicode-regex
regex: \p{...} property classes; interop: the String method surface (cuerdas green)
2026-06-10 21:38:08 -04:00
Yogthos
1898df99bc regex: \p{...} property classes; interop: the String method surface (cuerdas is green)
\p{L}/\p{Lu}/\p{Ll}/\p{N}/\p{Z}/\p{Ps}/\p{Pe} (+\P negation) land in
both escape positions of the regex compiler, mapped onto the byte PEGs:
ASCII exact, any high byte (inside a UTF-8 sequence) counts as a LETTER —
so ^\p{L}+$ accepts UTF-8 words while \p{N}/\p{Z} stay ASCII. (?u) was
already a tolerated no-op flag. Unknown property names error at compile.

Chasing the acceptance target (cuerdas via deps-conformance) pulled in the
rest of its clj-compat chain, each a real gap:
- the deps-conformance harness reads libraries under clj-compat reader
  features (deps are clj/cljc by definition — without :clj, cuerdas's
  #?(:clj (instance? Pattern x)) branches resolved to NIL bodies)
- instance? knows Pattern/java.util.regex.Pattern (regex values) and
  Character (cuerdas's rx/regexp? gate on split)
- the java.lang.String method surface: .toLowerCase/.toUpperCase/.trim/
  .indexOf(-1 on miss)/.lastIndexOf/.substring/.charAt/.startsWith/
  .endsWith/.contains/.replace/.equalsIgnoreCase/... — ASCII case mapping,
  unknown methods error (the old path silently returned nil)
- the (.method obj args) SUGAR now desugars to (. obj method args) in the
  interpreter — it was never implemented (bare .method heads resolved as
  vars, hence 'Cannot call nil')
- Long/MAX_VALUE / MIN_VALUE statics (f64 approximations)

deps-conformance: medley ok, cuerdas ok (was check-error); dependency now
loads its clj branches and fails only on its single-segment ns resolution.
30 new spec rows (11 regex, 19 interop). Gate exit 0.
2026-06-10 21:29:22 -04:00
Dmitri Sotnikov
621ca5cf75
Merge pull request #65 from jolt-lang/bug-batch
core: three bug fixes — ifn?, prefer-method dispatch, reader comments in map values
2026-06-10 21:11:10 -04:00
Yogthos
c06af7c9f4 core: three bug fixes — ifn?, prefer-method dispatch, reader comments in map values
ifn? (jolt-1vx) is the canonical IFn set in the overlay: fns, keywords,
symbols, maps (sorted included), sets, vectors, and vars — NOT lists. The
seed version said true for lists and false for struct maps and vars.
Mutable-mode caveat documented (vectors and lists share the array repr
there). 13 predicate rows.

Multimethod dispatch (jolt-heo) now collects EVERY isa-matching method key
and picks the dominant one — x dominates y when prefer-method'd over it or
(isa? x y) — and two matches with no dominant is an ambiguity ERROR, as in
Clojure. It used to take whichever key the table yielded first, silently
ignoring prefer-method. The prefers store upgrades to Clojure's
{x -> set-of-dominated} shape, shared between the dispatch closure and
prefer-method-setup via the var; prefers becomes a macro over a setup fn
(the store lives on the VAR — the multifn value can't carry it, so the old
fn read {} forever). 6 multimethod rows + the conformance row updated to
the canonical shape (335x3).

The reader (jolt-ou8) kept the pending KEY when a comment or #_ sits in a
map's VALUE slot: the old code dropped both, desyncing kv pairing — the
real value became the next key and the closing brace landed in value
position ('Unmatched closing brace'). Selmer's deps.edn (a '; for
development (REPL, etc)' comment between key and value) now parses; 6
reader rows incl. nested commented maps.

Gate: jpm exit 0, conformance 335x3, all tests passed.
2026-06-10 21:03:14 -04:00
Dmitri Sotnikov
19606730a3
Merge pull request #64 from jolt-lang/ir-passes
compiler: IR pass pipeline + constant folding (jolt-2om)
2026-06-10 19:37:22 -04:00
Yogthos
35e8821a92 compiler: IR pass pipeline + constant folding (jolt-2om, nanopass-lite)
jolt.passes is the new portable pipeline stage between the analyzer and the
back end: pure IR -> IR rewrites, total over node :ops (unknown ops pass
through with folded children), loaded with the compiler namespaces and
resolved lazily by analyze-form (JOLT_NO_IR_PASSES=1 disables — the same
escape-hatch pattern as the macro oracle). The shape is flatiron's opt.clj
applied to the jolt IR, which is what jolt-2om asked for.

The first pass is constant folding: a call of a foldable numeric SEED fn
(the later tiers don't exist when the compiler loads) whose args are all
constant numbers becomes a constant, and an if with a constant test becomes
the taken branch (dead-branch elimination — the untaken side never even
resolves). Folding computes with the ACTUAL jolt fns, so results match
runtime semantics by construction; a fold that would throw (mod 5 0) is
left for runtime.

Two walk lessons paid for in debugging: let/loop bindings are
[name init-ir] PAIRS, not maps (assoc'ing :init into a pair corrupts it);
and a throw inside the interpreted pass unwinds past the interpreter's ns
restores, so analyze-form restores the compile ns after the (protected)
pass call — without that, one pass error left current-ns in jolt.passes and
the rest of the tier compile resolved against the wrong namespace (sort-by
landed on the 2-arg JANET builtin).

ir-passes-test pins folds, conservatism (free vars, throwing folds), and
end-to-end eval. Gate exit 0.
2026-06-10 19:29:36 -04:00
Dmitri Sotnikov
40da75cee4
Merge pull request #63 from jolt-lang/pmap
core: lazy realization shared across walks (once-only effects); pmap family
2026-06-10 19:20:28 -04:00
Yogthos
4a1a9e3aec core: lazy realization is shared across walks (once-only effects); pmap family
Every walk over a lazy seq created FRESH wrapper tables around the shared
rest-thunks (ls-rest, ls-seq/ls-count, realize-for-iteration, the printers,
reduce — each had its own make-lazy-seq loop), so independent walks re-ran
the thunks: side effects duplicated, and a doall'd seq of futures was
re-spawned serially by the deref walk. Every walker now goes through
ls-rest-cached, which memoizes the rest wrapper on its node — thunks run
exactly once, as in Clojure. Costs ~10% on walk-heavy benches (the per-node
cache get/put — Clojure's LazySeq pays the same); net still -9% vs the
pre-linear-walks baseline. Three regression rows pin once-only effects and
value stability across walks.

On top of that: pmap/pcalls/pvalues (jolt-oeu) over the real-thread futures
— spawn-all-then-deref (the once-only fix is what makes the doall actually
mean that), snapshot semantics documented, multi-coll arity via the
canonical vector-zip. System/currentTimeMillis + nanoTime land as System
statics (the realtime clock — os/time is whole seconds, which quantized
every elapsed measurement to 1000ms). Seven pmap rows incl. a generous-
margin parallelism check (4 x 200ms sleeps under 700ms after warmup).
2026-06-10 19:14:49 -04:00
Dmitri Sotnikov
599c0468f7
Merge pull request #62 from jolt-lang/native-ops-batch2
backend: extend native-ops — mod/rem///bit ops emit janet opcodes
2026-06-10 19:08:51 -04:00
Yogthos
e64c1f1b36 backend: extend native-ops — mod/rem///bit ops emit janet opcodes (jolt-5lm)
The native-ops table grows from 9 to 16, each verified for semantic parity
with the jolt fn before inclusion (incl. negative operands): mod is floored
on both sides, rem (janet %) truncates, / is variadic with (/ x) -> 1/x.
quot is deliberately absent — janet div floors where Clojure truncates.
jolt's bit fns are 2-arg (unlike Clojure's variadic), so the bit ops emit
native only at exactly that arity; bit-not is unary. Eight new conformance
rows pin compile=interpret on the new ops at their guarded arities (334x3).

map-read 10.8 -> 9.2 ms (the (mod i 100) in its loop inlines). From the
flatiron review's unchecked-primitive-loops idea.
2026-06-10 19:02:19 -04:00
Dmitri Sotnikov
377fe706e9
Merge pull request #61 from jolt-lang/linear-seq-walks
core: seq walks over concrete collections are linear (bench TOTAL -18%)
2026-06-10 19:02:14 -04:00
Yogthos
4f78f6be33 core: seq walks over concrete collections are linear (bench TOTAL -18%)
Reviewing flatiron's morsel batching for applicability turned up something
better hiding under the lazy machinery: every cell over a concrete
collection was produced by slicing the REMAINDER ((tuple/slice c 1) per
element in coll->cells, and rest/next sliced the same way), so any full walk
was O(n^2). mapv over 40k elements took 10.4 SECONDS; a 20k-element
first/rest loop took two.

Cells over indexed collections now walk by INDEX (one shared indexed-cells
helper, O(1) per step), and rest/next of a vector/tuple/list return an O(1)
lazy view from index 1 — which also makes (rest [1 2 3]) a SEQ, as in
Clojure (it was a vector-typed slice; seq?/vector? rows pin the change).

mapv 40k: 10405 -> 182 ms. rest-loop 20k: 2040 -> 31 ms. Whole bench:
seq-pipe -28%, into-vec -24%, str-join -18%, hof -26%, TOTAL 4565 -> 3753
(-18% vs main, back to back). Chunked seqs (jolt-yqc) drop in priority:
the quadratic walks were the actual cost; chunking now only amortizes
per-element closure allocation.

Nine regression rows incl. 20k/50k linear-scaling smoke tests. Gate exit 0.
2026-06-10 18:56:19 -04:00
Dmitri Sotnikov
c24c7617b8
Merge pull request #60 from jolt-lang/ns-alias-unify
core: one alias store (jolt-ark); '.' classified as a special form
2026-06-10 18:53:08 -04:00
Yogthos
e311018d55 core: one alias store (jolt-ark); '.' classified as the special form it is
require-:as wrote the string-keyed :imports table (which resolution reads)
while ns-aliases read the symbol-keyed :aliases table (which nothing wrote)
— so (ns-aliases) was always empty and the alias fn had to write both as a
bridge. :aliases (alias-name string -> ns-name string) is THE store now:
require :as and the alias fn write it, both resolution paths read it first
(falling back to :imports for class imports, which is all that table holds
now), ns-unalias removes one entry, and ns-aliases presents Clojure's
{alias-symbol -> namespace object} shape built from it. ns-resolve's
qualified path goes through the same lookup.

Also: the coverage dashboard's last 'resolvable-not-interned' entry was '.'
— which (resolve '.) returns nil for on the JVM too; the tool now classifies
it as the special form it is, and that category reads ZERO.

7 new unified-alias spec rows (require/alias/ns-unalias round-trips through
both the resolution and introspection views); the white-box namespace test
tracks the accessor rename. Gate exit 0.
2026-06-10 18:37:19 -04:00
Dmitri Sotnikov
7c2d556dc5
Merge pull request #59 from jolt-lang/variadic-recur
evaluator: recur into a variadic arity binds the rest seq directly (fixes the hang)
2026-06-10 18:29:08 -04:00
Yogthos
502eabdb2d evaluator: recur into a variadic arity binds the rest seq directly (fixes the hang)
(recur (inc acc) (rest xs)) re-entered the fn through its varargs collector,
so the rest seq came back wrapped in a fresh 1-element rest list — xs never
emptied and the interpreter hung (jolt-4df; the compile path was already
correct). recur now re-enters through a dedicated entry that binds the LAST
arg directly as the rest param (n-fixed + 1 args, Clojure's contract), in
both the single-arity and multi-arity fn* paths; the shared body runner keeps
the ns-swap/restore in one place, and fixed arities still re-dispatch through
the arity dispatcher exactly as before.

Six spec rows: the original repro, zero-fixed variadic, rest-empties-to-nil,
multi-arity variadic recur, nil rest, and a fixed-arity control.
2026-06-10 18:23:13 -04:00
Dmitri Sotnikov
4a4f658f97
Merge pull request #58 from jolt-lang/edn-ns-fixes
core: edn :readers/:default opts; uncaught throws no longer leak the callee's ns
2026-06-10 18:22:43 -04:00
Yogthos
6260230231 core: edn :readers/:default opts; uncaught throws no longer leak the callee's ns
clojure.edn was nearly complete (sets, #uuid/#inst, :eof all landed earlier);
the :readers opt was ignored and :default missing. Both work now — the
reader stores a tag as a :#name keyword, so the lookup normalizes it to the
symbol Clojure keys :readers with; :default gets (tag value); built-in data
readers stay the fallback. 8 new edn spec rows. This was the last open item
of jolt-0mb (the vendored walk/zip/data/edn battery has been green for a
while: 34/33/61/50, all clean).

Chasing the probe cascade ('Unable to resolve symbol: edn/...' after one
error) found a real evaluator bug: an interpreted fn body runs with
current-ns rebound to its DEFINING ns and restores it with a plain trailing
call — an UNCAUGHT throw skips every restore on the way out, leaving the ctx
stuck in the deepest callee's ns, where alias-qualified lookups then fail
(the same cascade previously seen via sci). The repair lives at the
TOP-LEVEL boundary (loader/eval-toplevel saves the entry ns and restores it
on error before re-raising) — NOT per-call defer/try, which builds a fiber
per frame and blew the C stack on deep interpreted recursion (file-seq)
when tried first. Regression tests cover the cross-eval leak and that
aliases keep resolving.
2026-06-10 18:16:06 -04:00
Dmitri Sotnikov
fef99db6d8
Merge pull request #57 from jolt-lang/spec-untested-vars
spec: rows for every untested var (131 -> 0); five real bugs found by the probes
2026-06-10 17:58:28 -04:00
Yogthos
d06b3fe636 spec: rows for every untested var (131 -> 0); the probes found five real bugs
test/spec/untested-vars-spec.janet adds 143 rows asserting jolt's documented
behavior for the whole implemented-untested category — primed arithmetic,
the array/aset/coercion stubs, unchecked-*, the chunk family, JVM-shape
stubs (class/bean/proxy/memfn as resolve-only or :throws), ns/REPL
machinery, and the misc seqs. tools/spec_coverage.py now checks each var as
a whole TOKEN in the test sources (call-position-only matching missed *1,
+', ., .., /, and bare transducer refs like cat).

Writing rows from probed truth surfaced five real bugs, all fixed:
- comp with a jolt-IFn stage silently returned nil ((comp seq :content)) —
  raw Janet keyword application is not jolt invoke. comp is the canonical
  overlay defn now (fixed-arity composed fn, so the hot 1-arg path is two
  direct calls); the seed keeps a private td-comp only for the transducer
  machinery. hof bench +9% vs native, the price of correct IFn dispatch.
- extend (the fn) was a nil-expanding stub MACRO shadowing any definition;
  it's a real fn over register-method now, and extends? (a constant-false
  stub) is real over extenders
- (.. x f g) hit the 'ClassName.' constructor branch (a name ending in a
  dot) and died; .. is the canonical threading macro now
- aclone errored on pvecs; ns-interns/ns-imports returned live host tables
  that count/seq reject (now structs)

Thread/sleep + Thread/yield land as Thread statics beside Math/: sleep parks
the WORKER's own event loop (each future thread has one), which makes timed
deref provably fire — futures-spec gains the timeout-fires, sleep-in-body,
and timed-out-future-still-completes rows. The futures impl itself already
ran on real OS threads (ev/spawn-thread + marshalled results); jolt-ejx was
stale.

Dashboard: implemented+tested 433 -> 564 of 694; implemented-untested and
missing-portable are both EMPTY. Gate: jpm exit 0, all tests passed.
2026-06-10 17:52:30 -04:00
Dmitri Sotnikov
daab2ab4cc
Merge pull request #56 from jolt-lang/missing-core-vars
core: the last seven missing-portable vars — coverage gap closed (jolt-brh)
2026-06-10 17:28:11 -04:00
Yogthos
6445f461bb core: the last seven missing-portable vars — coverage gap closed (jolt-brh)
The dashboard's missing-portable category is now EMPTY (was 35 when the issue
was filed; this session's io/leaf work had already landed most of them).
The final seven:

- extenders — ctx-capturing clojure.core fn over the protocol type-registry:
  the type-tags implementing a protocol, as symbols; nil when none
- find-keyword — keyword: jolt keywords have no intern table, so it always
  finds (babashka makes the same call)
- inst-ms* — the raw Inst method; one inst representation, so = inst-ms
- read+string — over the 50-io readers, which now expose :buf and :fill-fn;
  returns [form exact-text-consumed], EOF throws or yields [eof-value ""]
  with the 3-arity, works for string AND stdin readers
- with-local-vars — fresh free-standing var cells (__local-var seam) bound as
  locals; var-get/var-set work on any cell
- with-open — canonical recursive expansion closing through the __close seam:
  a map-like value's :close fn or a host file (no .close interop here);
  nested closes run inner-first, finally runs on throw
- with-precision — body evaluates with precision/:rounding accepted and
  ignored (doubles, no BigDecimal context) — documented divergence

30 new spec rows (test/spec/missing-vars-spec.janet); coverage.md
regenerated: implemented+tested 426 -> 433, missing-portable 7 -> 0.
Gate: jpm exit 0, all tests passed.
2026-06-10 17:22:28 -04:00
Dmitri Sotnikov
3051f4264a
Merge pull request #55 from jolt-lang/phm-resize
phm: grow the bucket array past load factor 2 (map-read 4.5x recovery)
2026-06-10 17:12:04 -04:00
Yogthos
24a8522b51 phm: grow the bucket array past load factor 2 (map-read 4.5x recovery)
The phm had a FIXED 8 buckets, so a 100-entry map was a ~12-entry linear
scan per lookup — and phm-get walked the bucket twice (contains? then find).
This went mostly unnoticed until the canonical zipmap (batch 2) started
returning phms where kvs->map had built structs for scalar keys, regressing
the map-read bench ~7x (jolt-s3y).

phm-assoc now rehashes into a doubled bucket array when the count passes 2
entries/bucket (done on the fresh copy, so persistence is untouched);
phm-get is single-pass with a presence flag (nil values still distinguish
from missing); key= tries identity/scalar equality before paying for
canonicalization; the bucket count is derived from (length (m :buckets)),
not a constant, so any already-marshaled map keeps working. core-contains?'s
phm branch goes through phm-contains? instead of poking buckets directly.

map-read 48.5 -> 10.9 ms (the residual vs the pre-batch-2 6.7 is the
canonicalizing-representation constant); map-build steady; bench TOTAL 4457
vs 4565 on main back-to-back. New unit case crosses the resize boundary at
500 entries: every key found, nil values present, collection keys canonical,
dissoc + persistence intact. Gate: jpm exit 0, conformance 326x3.
2026-06-10 17:06:03 -04:00
Dmitri Sotnikov
ce4f387a2f
Merge pull request #54 from jolt-lang/ci-green
ci: fix the one red test on CI runners; document the gate + porting gotchas
2026-06-10 17:04:25 -04:00
Yogthos
b1486d58a0 ci: fix the one red test on CI runners; document the gate + porting gotchas
nrepl-test was CI's only failure: the server subprocess ran main.janet from
source, paying the full compile-mode init, which outran the 5s connect poll
on slow runners (locally it always won the race). The test now prefers
build/jolt — its ctx is baked at build time, so it accepts in ~20ms and CI
builds it anyway — falls back to source, polls up to 60s under a 90s
watchdog, and dumps the server's stderr when startup fails so the next CI
failure is diagnosable.

CLAUDE.md's placeholder sections become real: build/test commands with the
run-the-gate-with-a-real-exit-code protocol (a piped gate once shipped masked
spec failures), the seed/overlay/tier architecture sketch, and the porting
gotchas that have each bitten at least once (leaf verification, stub-breaks-
self-recursion, tier macro ordering, ref-get vs get on attached-ops wrappers,
:jolt/type map keys, expander-called fns, canonical-port policy) — previously
only in local bd memories.
2026-06-10 16:59:12 -04:00
Dmitri Sotnikov
1d373299a4
Merge pull request #53 from jolt-lang/inline-var-deref
backend: inline the var deref at indirect call sites (fib 1.75x)
2026-06-10 16:42:56 -04:00
Yogthos
639effd58c backend: inline the var deref at indirect call sites (fib 1.75x)
An indirect global reference emitted ((var-get 'cell) ...) — a function call
per deref, whose body is a binding-stack check plus a root read. The emitter
now inlines that: (if (in 'cell :dynamic) (var-get 'cell) (in 'cell :root)).
Non-dynamic vars — the vast majority of references — pay two native table
ops and a branch instead of a call; dynamic vars take the full var-get
(thread-binding walk). Redefinition stays live (:root is read per call) and
binding semantics are exact: the :dynamic check is PER CALL, not at emit,
because a (def ^:dynamic x) in the same compiled unit marks the cell dynamic
only when the def runs — the same reason JVM Clojure's Var.deref() checks
the thread-bound bit every call (an emit-time variant was 1.7x faster still
on fib but failed conformance exactly there).

fib 130 -> 74 ms (1.75x); bench TOTAL 4564 -> 4437 back-to-back. This
displaces the gen-counter inline-cache design from jolt-8sq: with var-get's
existing fast path, resolution was never the cost — the call was. A
gen-guarded cache would add state per site to save nothing further, and
couldn't skip the dynamic check anyway.

Found while benching: map-read regressed ~7x back in batch 2 (canonical
zipmap builds a phm where kvs->map built a struct) — filed as jolt-s3y.

Gate: jpm test exit 0, conformance 326x3, suite >= baseline.
2026-06-10 16:42:46 -04:00
Dmitri Sotnikov
19d59cba85
Merge pull request #52 from jolt-lang/leaf-shrink-batch4
core: Stage 3 — leaf batch 4: sort-by + rand family + char tables
2026-06-10 16:30:23 -04:00
Yogthos
3d7de8ff90 core: Stage 3 — leaf batch 4: sort-by + the rand family + char tables to the overlay
sort-by, rand-int, shuffle, random-uuid, char-escape-string, and
char-name-string move to 20-coll over the two host seams that stay (rand and
sort — they ARE the randomness/ordering primitives). Canonical upgrades ride
along: sort-by defaults its comparator to compare, so nil sorts FIRST (the
kernel fn used host ordering and put nil last); rand-int truncates toward
zero via int (the kernel fn floored, wrong for negative n); shuffle is a
pure-functional Fisher-Yates over vector assoc and rejects non-collections
(a string is seqable but not shuffleable, as on the JVM — the honest gate
caught that one); random-uuid builds over rand-int and validates through
parse-uuid; the char tables are char-keyed Clojure maps (Clojure's shape —
the seed keeps its private code-keyed copies for pr-render).

22 new spec rows. Gate: jpm test exit 0 verified, suite 4698 >= 4660, bench
parity with main back-to-back (4733 vs 4817).
2026-06-10 16:30:17 -04:00
Dmitri Sotnikov
17f2474be2
Merge pull request #51 from jolt-lang/early-defn-recompile
core: staged recompile for early defns; keys/vals/empty? leave the seed (jolt-4j3)
2026-06-10 16:16:31 -04:00
Yogthos
63eb6eca6e core: staged recompile for early defns; keys/vals/empty? leave the seed (jolt-4j3)
recompile-defns! is the defn analog of recompile-macros!: pre/at-kernel
overlay defns (00-syntax's destructure and friends; the kernel tier too in
interpret mode) load as interpreted closures, the evaluator stashes their fn
source on the var (:defn-src, scoped by a flag only api/load-core-overlay!
sets), and the end-of-init pass compiles them and swaps the var root. With
that in place, keys/vals/empty? — the fns the 00-syntax expanders call at
expansion time — move to the top of 00-syntax as raw fn* defs (canonical:
keys/vals project (seq m), so sorted maps come back in comparator order and
(keys {}) is nil; empty? keeps O(1) count dispatch with seq's cell check only
for the lazy/list fallback). The sorted tier drops its now-dead :keys/:vals
ops.

Correctness fixes that surfaced once the gate was run with a REAL exit code
(the previous 'jpm test | grep' gates reported grep's exit and masked spec
failures across #48-#50):
- map conj is strict again: a non-nil/non-map arg must be a 2-element vector
  ('Vector arg to map conj must be a pair'), and merge inherits it — the
  batch-2 canonical merge had silently dropped the validation
- conj onto a lazy seq prepends (it fell into the MAP fallback); upstream
  clojure.data/diff relies on (conj seq x) via set/union over keys, so diff
  now matches Clojure exactly
- (seq {}) / (seq #{}) / empty phm are nil, not ()
- key/val are strict (a plain vector is not an entry); find mints a REAL
  entry as the first entry of a one-entry map, nil values intact
- the sci avoid-method-too-large stub passes its registry map through
  instead of returning a raw host table (strict conj rejected it; sci's
  clojure-core registry is also no longer discarded)

Test updates: lazy-infinite pins take-nth realization at 5 (was 7 — the
canonical lazy impl realizes fewer); self-host asserts the analyzer IS loaded
in interpret mode (compiled expanders, PR #50) and is NOT in the
:compile-macros? false oracle. 18 new maps-spec rows.

Gate: jpm test exit 0 (verified directly, not through a pipe), conformance
326x3, suite 4698 >= 4660.
2026-06-10 16:16:23 -04:00
Dmitri Sotnikov
e6f562c175
Merge pull request #50 from jolt-lang/macro-expansion-path
core: compiled macro expansion in every mode (+123 suite passes)
2026-06-10 15:37:39 -04:00
Yogthos
dc1f6d9755 core: compiled macro expansion in every mode (+123 suite passes)
Macros are ordinary compiled fns in Clojure's model; compile mode has had
that since the staged bootstrap, but interpret mode — the conformance
battery's default — kept interpreted expanders, so every distinct (and ...)/
(cond ...) call form, and every fresh form produced by a recursive expansion,
ran an interpreted closure. ensure-macros-compiled! now runs in every mode:
interpret-mode init loads the tiers fast-interpreted, then one pass at the
end builds the analyzer (which itself stays interpreted there) and compiles
all stashed expanders; user defmacros after init compile too. The new
:compile-macros? opt (JOLT_INTERPRET_MACROS=1) preserves the fully-
interpreted oracle, and joins the ctx-image cache key.

Battery: 4700 pass / 90 clean files / 7 timeouts, from 4577 / 87 / 9 — two
macro-heavy files stopped timing out and 149 more assertions execute. The
compiled-expander delta proper is +67 passes (oracle mode on the same tree
measures 4633). Baselines raised 4540->4660, clean 86->88. Interpret init
grows 0.12s -> 1.12s for the analyzer build; init-cached amortizes it to ~5ms
per process.

New macro-expansion-test pins: expanders compiled in interpret mode (core +
post-init user defmacros), uncompilable bodies fall back interpreted and
still work, compile mode unchanged, oracle opt-out honored.

Follow-up filed (jolt-4j3): the same staged-recompile treatment for early
overlay DEFNS, which is what still pins keys/vals/empty? to the seed.
2026-06-10 15:37:32 -04:00
Dmitri Sotnikov
1ec2fa4adf
Merge pull request #49 from jolt-lang/leaf-shrink-batch3
core: Stage 3 — leaf batch 3: empty/assoc-in/update-in + interpose/take-nth
2026-06-10 15:26:47 -04:00
Yogthos
780b6474ff core: Stage 3 — leaf batch 3: empty/assoc-in/update-in + interpose/take-nth to the overlay
empty, assoc-in, and update-in move to 20-coll.clj as the canonical recursive
ports; interpose and take-nth move to the lazy tier WITH their canonical
transducer arities (volatile-based), so the seed's td-interpose/td-take-nth
helpers go too. (empty lazy-seq) is () now — the kernel fn returned a bare
host table for it.

keys/vals/empty? stay put for now: they're expander-coupled — 00-syntax's
when/and/or/cond/destructure expanders call them at expansion time, which
happens during the kernel-tier compile, before any later tier exists. They
move when early defns get the staged-recompile treatment macros already have.

26 new spec rows (incl. transducer arities through sequence/into and laziness
checks against (range)). Gate green: conformance 326x3, suite >= baseline,
full jpm test.
2026-06-10 15:26:41 -04:00
Dmitri Sotnikov
1340b7c52f
Merge pull request #48 from jolt-lang/leaf-shrink-batch2
core: Stage 3 — leaf batch 2: sixteen more seed fns to the overlay; retire MIGRATION.md
2026-06-10 15:17:07 -04:00
Yogthos
0e71b193e5 core: Stage 3 — leaf batch 2: sixteen more seed fns to the overlay; retire MIGRATION.md
key/val/select-keys/zipmap/merge/merge-with/get-in/memoize/partial/
trampoline/some?/true?/false?/max/min/reverse move to 20-coll.clj as the
canonical Clojure definitions, plus find — which was previously missing from
jolt entirely (select-keys/merge-with/memoize build on it). Two behavior
fixes ride along: memoize now caches nil results (the kernel fn re-computed
them — canonical find-based impl), and conj of nil onto a map is a no-op as
in Clojure (it errored; the canonical merge relies on it). max/min keep the
JVM NaN behavior by construction (pairwise >/<). not= stays: the kernel tier
(subvec) uses it.

One new tier-ordering rule, learned the hard way: a tier may only use macros
from tiers that load BEFORE it — memoize's if-let (30-macros) broke compiled
init while interpret mode passed, because compile expands macros at tier
load and the interpreter expands lazily. Now documented in the migration
workflow note.

MIGRATION.md is gone — task tracking lives in beads (jolt-ded; the per-batch
workflow, tier-order rules, perf wall, and remaining candidates are in bd
memory core-migration-workflow). The doc's candidate lists had gone stale
against the actual seed anyway.

43 new spec rows. Gate green: conformance 326x3, suite >= baseline, full
jpm test, bench at parity with main back-to-back (4851 vs 4831 TOTAL).
2026-06-10 15:16:47 -04:00
Dmitri Sotnikov
af34d94870
Merge pull request #47 from jolt-lang/leaf-shrink-batch
core: Stage 3 — leaf batch: nine more seed fns to the overlay
2026-06-10 14:57:10 -04:00
Yogthos
3faee14271 core: Stage 3 — leaf batch: complement/fnil/clojure-version/bigdec/numerator/denominator/supers/munge/test to the overlay
Nine more seed leaves move to 20-coll.clj (verified leaf-by-leaf: defn +
core-bindings entry only, no internal callers). fnil is upgraded to Clojure's
canonical 2/3/4-arity — it patches only the first 1-3 arguments; the old
kernel fn patched every position it had a default for, which Clojure does
not. The rest carry their kernel semantics over unchanged (bigdec is a
double, numerator/denominator throw, supers is #{}, munge rewrites dashes).

16 new spec rows incl. the fnil arity-contract cases. Gate green:
conformance 326x3, suite 4577, full jpm test (2:18 — first full run with the
ctx image cache on main).
2026-06-10 14:56:50 -04:00
Dmitri Sotnikov
ee6447adae
Merge pull request #46 from jolt-lang/in-reader-family
core: Stage 3 — the *in* reader family is Clojure (50-io tier)
2026-06-10 14:53:17 -04:00
Yogthos
c665b8eb9f core: Stage 3 — the *in* reader family is Clojure (50-io tier)
*in*, read-line, read, with-in-str, and line-seq land as a new overlay IO
tier (core/50-io.clj). *in* is a dynamic var holding a reader — a plain map
of two closures, :read-line-fn (next line, nil at EOF) and :read-fn (next
form, advancing past exactly that form). The default *in* reads real stdin
with a shared leftover buffer, so read and read-line interleave correctly;
with-in-str rebinds *in* to a string reader over one atom-held buffer —
(read) consumes its form, a following (read-line) returns the rest of that
line, as in Clojure. read has the 0/1/3 arities (EOF throws, or returns
eof-value when eof-error? is false).

The Janet seed grows two seams next to read-string: __stdin-read-line (one
line off stdin, newline stripped) and __parse-next (one form off a string ->
[form rest], nil at end of input) — and loses the line-seq stub.

Two traps hit and documented for future tiers: a map LITERAL with :jolt/type
as a key is read as a tagged form (don't tag overlay value maps), and a
leftover seed stub holding the same name breaks direct-linked self-recursion
— the overlay line-seq's recursive call bound to the stub's root, truncating
after one line. The stub's string-splitting behavior is kept as a documented
extension.

20 new io-spec rows (read-line EOF/interleave, read arities + eval round-trip,
line-seq incl. real-stdin paths). Gate green: conformance 326x3, suite 4577,
full jpm test.
2026-06-10 14:52:40 -04:00
Dmitri Sotnikov
101027a0cd
Merge pull request #44 from jolt-lang/sorted-colls-clojure
core: Stage 3 — sorted collections are pure Clojure (canonical port)
2026-06-10 14:51:50 -04:00
Dmitri Sotnikov
8cec486588
Merge pull request #43 from jolt-lang/transients-rfc
docs: RFC 0003 — transients design note
2026-06-10 14:51:23 -04:00
Dmitri Sotnikov
11691ddb2b
Merge pull request #42 from jolt-lang/ctx-image-cache
core: AOT context image — init-cached recovers the bootstrap cost across processes
2026-06-10 14:51:19 -04:00
Yogthos
55a3ebf93f core: Stage 3 — sorted collections are pure Clojure (canonical port)
sorted-map/sorted-map-by/sorted-set/sorted-set-by/sorted?/sorted-map?/
sorted-set?/subseq/rsubseq now live in their own overlay tier (25-sorted.clj).
A sorted coll is a tagged host table with a comparator-ordered :entries
vector, a 3-way :cmp, and the tier's op implementations ATTACHED to the value
(:ops map): the seed's conj/assoc/get/seq/count/... branches are each a
one-line call through (coll :ops), so the ops travel with the value — correct
across contexts, forks, and AOT images, no module-level hooks to re-wire.
The host surface grows by three minimal value primitives: jolt.host/
tagged-table, ref-put! (already there), and ref-get — a raw field read,
because plain get on a sorted coll IS the comparator lookup and reading
:entries with it recurses.

This fixes a pile of Clojure-correctness gaps the Janet kernel had:
- lookup/membership now go through the COMPARATOR: (contains? (sorted-set 1)
  1.0) was a deep= scan, (conj (sorted-set 1) 1.0) and assoc of a
  comparator-equal key now no-op/replace as in Clojure
- equality is representation-agnostic: (= (sorted-map :a 1) {:a 1}) and
  (= (sorted-set 1 2) #{1 2}) were false
- iteration was broken: (map inc (sorted-set 3 1 2)) errored
  (realize-for-iteration and coll->cells had no sorted branches)
- empty?/empty saw the host wrapper, not the collection: (empty? (sorted-map))
  was false, (empty sc) returned a bare table; it now keeps the comparator
- sorted colls canonicalize as map keys; comparator fns may be boolean
  predicates or 3-way (Clojure's fn->comparator)
- sorted-map throws on odd kv count; conj nil is a no-op

Also fixes jolt-h86 en passant: into-conj had no branch for sets (or sorted
colls) and silently returned the target unchanged — (into #{} [:a :b]) was
#{}. The fallback now folds conj. Regression rows in sets-spec.

sorted-spec grows to 77 rows (comparator-based membership, equality,
empty/rseq/printing, seq-fn interop, subseq/rsubseq on maps). Gate green:
conformance 326x3, suite 4577 (vs 4566 prior — the battery gained rows),
sorted+sets specs, full jpm test, bench at parity with main back-to-back
(4521ms vs 4619ms TOTAL under identical load).
2026-06-10 14:39:02 -04:00
Yogthos
d6c5552fda docs: RFC 0003 — transients semantics and why they stay in the Janet seed
Pins down what a transient is in Jolt (tagged table over a native Janet
array/table, canonical-keyed for maps/sets), where behavior deviates from the
JVM (O(n) transient/persistent! edges with O(1) native ops between, no
owner-thread check — same as Clojure 1.7+, transient-of-list leniency), and
the three reasons the machinery is seed-resident rather than a migration
candidate: it IS the mutation kernel, it sits under the seed's own dispatch,
and the value layer is declared irreducible. Exists so the kernel-shrink
ladder (jolt-tzo) doesn't revisit transients every round.
2026-06-10 13:58:47 -04:00
Yogthos
0e3584884f core: AOT context image — init-cached recovers the bootstrap cost across processes
init in compile mode is ~2.4 s (tier loading, analyzer self-compile, macro
recompilation), paid by every process that builds a ctx from source — each
jpm-test file, embedders, workers. init-cached marshals the built ctx to a
disk image (same root-env dicts as snapshot/fork) and later processes
unmarshal it in ~5 ms, any process: nothing from the baking process is
needed at load.

The cache key fingerprints the embedded .clj stdlib (which covers jolt-core:
analyzer, IR, core tiers), the .janet seed sources next to the module, the
janet version, the init opts, and the env knobs that shape a ctx (JOLT_PATH/
MUTABLE/AOT_CORE/FEATURES) — any change rebuilds. Corrupt or non-ctx images
fall back to a rebuild (unmarshal of garbage can 'succeed' with a scalar, so
the shape is checked, not just the throw). Writes are atomic (tmp + rename)
so racing cold starts never publish a torn image. JOLT_NO_IMAGE_CACHE=1
opts out; JOLT_IMAGE_CACHE_DIR overrides the location (default TMPDIR).

Test consumers switch to init-cached (harness, suite-worker, conformance,
the behavioral unit/integration tests); tests that validate the bootstrap
itself (bootstrap-fixpoint, staged-bootstrap, aot round-trip, direct-linking)
and the deps tests (tmp-dir :paths would fragment the key) keep real init.
Full jpm test: 2:46 -> 1:58 (~29%). New ctx-image-test covers cold/warm,
cross-process load (subprocess runs defn/redef/macros/protocols/multimethods
off the baked image), per-opts keying, and corrupt-image fallback.
2026-06-10 13:57:37 -04:00
Dmitri Sotnikov
bf885078f9
Merge pull request #41 from jolt-lang/stage3-retire-bootstrap
Stage 3: retire the bootstrap compiler
2026-06-10 13:32:21 -04:00
Yogthos
cbab7f66df core: Stage 3 — retire the bootstrap compiler (compiler.janet deleted)
The 1104-line Janet bootstrap compiler existed to build jolt.ir/jolt.analyzer
and the kernel tier before the self-hosted analyzer could exist. It is
replaced by the interpreter + one fixpoint turn:

1. bootstrap-load-source loads the compiler sources INTERPRETED (the
   evaluator can run the analyzer — it always could).
2. After the overlay is up, self-compile-compiler! re-runs the kernel tier,
   jolt.ir, and jolt.analyzer through the SELF-HOSTED pipeline — the
   interpreted analyzer compiles itself, and steady state runs compiled with
   no bootstrap compiler involved.

Measured: init {:compile? true} 1093 -> ~2400 ms (the one-time interpreted
pass + self-compilation), but steady-state compilation is 2.8x FASTER
(100 forms: 134 -> 48 ms) — the self-hosted pipeline emits better code than
the bootstrap did. An AOT image for init cost is future work (aot.janet's
machinery is the natural vehicle).

The bootstrap's runtime kernel moves to backend.janet (jolt-runtime-env,
ctx-janet-env, build-map-literal); aot imports it from there. The
uncompilable-error? punt check unwraps the interpreter's exception struct
(the interpreted analyzer's throw arrives wrapped). compile-string/
compile-file (the bootstrap's source-text emitter API, no callers outside
the bootstrap's own unit test) are removed with it, as is compiler-test.

Gate green across everything incl. fixpoint stage1==2==3, AOT round-trip,
uberscript, CLI; conformance 326x3; suite 4566 >= 4540; bench in band.
2026-06-10 13:32:14 -04:00
Dmitri Sotnikov
90a35a5dc0
Merge pull request #40 from jolt-lang/stage3-shrink2-hierarchy
Stage 3: the hierarchy system is pure Clojure (canonical port)
2026-06-10 13:19:12 -04:00
Yogthos
c1a54fa2de core: Stage 3 — the hierarchy system is pure Clojure (canonical port)
make-hierarchy/derive/underive/isa?/parents/ancestors/descendants are now
Clojure in the overlay — Clojure's own pure-map implementation: a hierarchy
is {:parents {tag #{..}} :ancestors {..} :descendants {..}}, the 3-arity
forms are PURE (derive returns a new hierarchy), and the 1/2-arity forms swap
a private global-hierarchy atom.

This fixes three correctness gaps the Janet kernel had: multi-parent derive
(the kernel's :parents held a single parent per tag), TRANSITIVE descendants
(the kernel tracked direct children only), and vector-pair isa?
((isa? [child1 child2] [parent1 parent2])). Cyclic and duplicate derives now
behave like Clojure (throw / no-op).

Multimethod dispatch was the kernel's only internal caller: defmulti-setup's
dispatch closure now calls the overlay's isa? through a lazily-resolved var
(cached per multimethod); a :hierarchy option is an atom (deref per dispatch,
matching Clojure's var semantics) or a plain hierarchy map. The Janet kernel
(types.janet) and the core-* wrappers are deleted.

20 new hierarchy spec rows (pure 3-arity incl. cycle/duplicate edges, global
+ dispatch incl. custom :hierarchy atoms). Gate green: conformance 326x3,
suite 4566 >= 4540, all batteries, bench in the session band.
2026-06-10 13:19:05 -04:00
Dmitri Sotnikov
232e4b137c
Merge pull request #39 from jolt-lang/stage3-tier-shrink
Stage 3 tier shrink: 26 pure-over-core leaves move to the overlay
2026-06-10 13:10:37 -04:00
Yogthos
d4c065edfe core: Stage 3 tier shrink — 26 pure-over-core leaves move to the overlay
The representation predicates (sequential?/associative?/counted?/indexed?/
reversible?/seqable?, boolean?/double?/float?/infinite?, the qualified/simple
ident predicates), the realization boundaries doall/dorun, list*, find,
realized?/force, pop, the print-str family, and rand-nth/random-sample are
now Clojure in 20-coll, expressed over the overlay's own predicates and
primitives — no Janet representations referenced.

Several got MORE correct in the move: dorun honors its bounded-n arity (the
seed ignored n); find works on vectors by index (contains? gives it free);
indexed? no longer claims seq results; seqable? no longer claims arbitrary
tagged tables (atoms were "seqable"); realized? reads the tagged slots via
the established get pattern.

Seed: 3427 -> ~3200 lines, 371 -> ~345 core-* fns. Gate green (conformance
326x3, suite 4569 >= 4540, stdlib battery, all specs+unit); bench in noise.
2026-06-10 13:10:30 -04:00
Dmitri Sotnikov
6699ef47d3
Merge pull request #38 from jolt-lang/stage3-ns-var
*ns*: the current-namespace dynamic var
2026-06-10 13:05:22 -04:00
Yogthos
817495dd51 core: *ns* — the current-namespace dynamic var
*ns* is interned in clojure.core holding the current NAMESPACE OBJECT, kept
in sync by ctx-set-current-ns through a var table cached on the env (one
table put on the hot path; core-bench A/B neutral). A thread binding
(binding [*ns* ...]) shadows the root through var-get as usual. in-ns now
returns the namespace object (Clojure); str renders a namespace as its name
and pr-str as #namespace[name]. The ns-designator helper accepts namespace
objects (they are tagged STRUCTS — the old table?-based check never matched
them), so (ns-aliases *ns*) / (ns-unalias *ns* 'a) work — SCI and the jank
syntax-quote corpus use exactly that shape.

Known divergence (documented): inside an interpreted fn, *ns* reflects the
fn's defining ns (jolt's resolution model rebinds current-ns per call);
top-level and load-time reads match Clojure.

Gate green (conformance 326x3, suite 4572 >= 4540, all batteries,
specs+unit +8 *ns* rows); bench neutral.
2026-06-10 13:05:15 -04:00
Dmitri Sotnikov
4f872fce94
Merge pull request #37 from jolt-lang/stage3-turn2b
Stage 3 turn 2b: host IO, ns introspection, thread-binding family
2026-06-10 12:54:12 -04:00
Yogthos
d61c86a068 core: Stage 3 turn 2b — host IO, ns introspection, thread-binding family
The names turn 2a's leak removal exposed as honestly missing, now proper:

- slurp/spit/flush (host-classified): path-based IO over Janet files; spit
  takes :append; flush flushes *out*. printf prints formatted (no newline)
  over the existing format. file-seq walks paths via two host dir primitives
  through the overlay's tree-seq.
- ns-map / ns-unmap / ns-refers (ctx fns). ns-refers required fixing the
  refer MODEL: refer/use/:refer now map the SOURCE VAR into the target ns
  (the Clojure model) instead of copying its value into a new var — so
  source-ns redefinitions propagate, the :macro flag travels for free, and
  refers are identifiable by the var's home :ns.
- Thread-binding family: with-bindings*/with-bindings, bound-fn*/bound-fn,
  bound?, thread-bound?, get-thread-bindings. The captured binding map is a
  Janet struct keyed by the var tables — the exact frame representation
  var-get reads — so it re-pushes correctly (a phm frame is invisible to
  var lookup).
- load-string and eval interned as VALUES at the api layer (they need the
  loader's compile-or-interpret routing); the eval special form still
  handles direct calls.

Suite 4532 -> 4572 (baseline floor 4540 across timeout variance, clean 86),
conformance 326x3, stdlib battery, all specs+unit (+21 turn-2b rows).
Coverage: missing-portable 27 -> 10 (left: the *in*-model readers, the
with-local-vars/with-precision/extenders tail).
2026-06-10 12:53:47 -04:00
Dmitri Sotnikov
a75a26860b
Merge pull request #36 from jolt-lang/stage3-turn2
Stage 3 turn 2a: close the implicit Janet root-env leak
2026-06-10 12:43:36 -04:00
Dmitri Sotnikov
c6ccdcf4b1
Merge pull request #35 from jolt-lang/ci-node24-bump
CI: bump checkout/cache actions to v5 (Node 24)
2026-06-10 12:43:33 -04:00
Yogthos
c7b0ad9d84 core: Stage 3 turn 2a — close the implicit Janet root-env leak
resolve-sym's last resort silently resolved any unknown Clojure symbol
against Janet's root environment — leaking Janet builtins with JANET
semantics into Clojure code: (type 1) was Janet's :number, (gensym) returned
Janet symbols (the long-documented (symbol (str (gensym))) macro landmine
existed BECAUSE of this), compare/slurp/int?/any? likewise. The explicit
janet/ prefix is the deliberate interop channel; the implicit fallback is
gone — an unresolved symbol is an error.

What the leak was masking, now proper interned vars:
- gensym: jolt's own (already existed, never interned) — returns real jolt
  symbols; the macro landmine is dead
- compare: full Clojure total order (nil-first, numbers, strings, keywords,
  symbols by ns/name, booleans, chars, uuid/inst, vectors by length then
  elementwise; cross-type throws)
- type: :type metadata override, deftype/record tag as symbol, else a
  taxonomy keyword (host-classified)
- int?: core-integer? — which had a latent bug the leak hid: (integer?
  ##Inf) was true (floor of inf is inf); NaN/infinities now excluded
- any?: constantly true (Clojure 1.9; SCI's namespaces.cljc needs it)
- jolt.interop/janet-type now uses the explicit (janet/type x) channel
- evaluator-test uses init (a bare make-ctx resolved EVERYTHING via the leak)

Suite 4470 -> 4532+ pass / 86-87 clean (proper compare unlocks the sort
files); baselines raised. Conformance 326x3 (+5 rows), +22 predicate spec
rows, stdlib battery green, all specs+unit. Coverage dashboard now counts
previously-leak-resolvable names honestly (missing-portable 19 -> 27).
2026-06-10 12:43:08 -04:00
Yogthos
da69d00b2c ci: bump checkout/cache actions to v5 (Node 24)
GitHub forces Node 24 for JavaScript actions starting June 16 2026; the v4
actions run on deprecated Node 20. v5 of both run on Node 24.
2026-06-10 12:31:44 -04:00
Dmitri Sotnikov
d0c605ac9d
Merge pull request #34 from jolt-lang/fix-edn-regression
Fix strict-map? regression in clojure.edn; apply EDN built-in tags
2026-06-10 12:26:05 -04:00
Yogthos
d8ffe386e6 edn: fix strict-map? regression in edn->value; apply EDN built-in tags
CI caught what the local per-change gate missed (clojure-stdlib-suite-test
wasn't in the habitual gate list): strict map? (PR #28) correctly stopped
treating tagged structs as maps, which silently disabled edn->value's
(and (map? x) (= :jolt/set ...)) set-form branch — edn/read-string returned
raw reader forms for #{...}. Reader FORMS are now detected by :jolt/type
directly, never via map?.

While here, EDN's built-in tagged elements are applied properly: a
:jolt/tagged form routes through the registered data reader (__read-tagged,
a ctx-capturing core fn over :data-readers), so (edn/read-string "#uuid ...")
yields a real UUID value and #inst an instant — per the EDN spec. The
read_string battery goes 47 -> 50 (above the old 49 baseline: the uuid
asserts pass now); baseline raised to 50.
2026-06-10 12:25:45 -04:00
Dmitri Sotnikov
517f7ba762
Merge pull request #33 from jolt-lang/preflight-inst-sq
Pre-flight: #inst instant values + syntax-quote literal collapse
2026-06-10 12:19:54 -04:00
Yogthos
e58be2fbd2 core: #inst instant values + syntax-quote literal collapse (spec 2.3/2.4)
#inst (jolt-rnh): an instant is an immutable tagged struct
{:jolt/type :jolt/inst :ms <epoch-millis>} — equality and map-key hashing by
INSTANT, so different offsets denoting the same moment are =. The reader
parses RFC3339 with Clojure's partial-timestamp defaults (#inst "2020" is
2020-01-01T00:00:00.000Z) and errors on malformed input; inst?/inst-ms in
the overlay; pr-str prints the canonical
#inst "yyyy-MM-ddThh:mm:ss.fff-00:00" and round-trips; str gives the bare
RFC3339 string. Self-evaluating in the evaluator (like uuid/chars).

Syntax-quote (jolt-l2a): a syntax-quoted self-evaluating literal (string,
number, boolean, nil, keyword, char) collapses to the literal at READ time,
matching Clojure's reader — so nested/adjacent backticks over literals are
inert: (= "meow" ```"meow") is true (jank pass-adjacent). Symbols still
qualify and collections still template. General nested syntax-quote over
non-literals remains UNVERIFIED in spec S25.

Tests: inst-spec (18 cases incl. partial defaults, offsets, round-trip),
+10 literal-collapse reader rows, +5 conformance rows (321x3). Spec
02-reader S20/S25 updated to normative. Suite stable 4470/86.
2026-06-10 12:19:23 -04:00
Dmitri Sotnikov
094152a8c3
Merge pull request #32 from jolt-lang/fix-doc-refs
Fix doc/ -> docs/ references missed in the consolidation
2026-06-10 12:12:12 -04:00
Yogthos
b3f2b19bf7 docs: fix doc/ -> docs/ references missed in the consolidation merge 2026-06-10 12:11:53 -04:00
Dmitri Sotnikov
99f822b91a
Merge pull request #31 from jolt-lang/reader-feature-keys
Spec §2 (reader) + RFC 0002 feature-key decision + docs consolidation
2026-06-10 12:10:47 -04:00
Yogthos
808ce6a725 docs: consolidate doc/ into docs/
One documentation root: the prose docs (building-and-deps, self-hosting
architecture/compiler, tools-deps, grammar.ebnf) join the spec and RFCs under
docs/. References in README and deps-conformance-test updated.
2026-06-10 12:10:28 -04:00
Yogthos
fdfd086df6 reader: feature set #{:jolt :default}, clause-order matching (RFC 0002)
jolt no longer satisfies :clj in reader conditionals. The shortcut was a
measured net liability: :clj branches carry JVM interop and JVM-specific
test expectations jolt fails, and they shadowed :default branches jolt
passes. A/B over the suite: clj,default = 4967 assertions / 4324 pass / 119
errors; jolt,default = 5069 / 4470 / 81 (+146 pass, -38 errors, +8 clean
files). Baselines raised to 4470/86.

Matching is now by CLAUSE order like Clojure — the first clause whose key is
in the feature set wins (#?(:default 5 :clj 6) is 5 everywhere); the old code
scanned for :clj first, then :default, regardless of position.

Foreign clj-targeted libraries are a property of the LOADING CONTEXT, not the
platform: reader-features-set! opts a load into a compatibility set, and the
SCI bootstrap/runtime tests load SCI under ["jolt" "clj" "default"] (its
.cljc selects implementations via :clj with no :jolt branches).
JOLT_FEATURES remains the process-wide override.

RFC 0002 records the decision with the measured data; spec 02-reader S18 is
now normative (clause order, documented feature set, per-context override).
Reader tests updated to the portable set + an opt-in round-trip.
2026-06-10 11:40:06 -04:00
Yogthos
2224e40afc docs: spec §2 (reader) — grammar, reader-macro catalog, syntax-quote contract
The lexical-syntax chapter, granularity modeled on jank's 62-file
per-construct reader corpus: token grammar (whitespace/comments, collections
with read-time duplicate checks, numbers incl. the N/M tower question,
symbols/keywords incl. ::auto-resolution, strings/chars), the quote-family
sugars, the full #-dispatch catalog with normative entries (anonymous fn
%-derivation, discard composition, reader conditionals, symbolic floats,
tagged literals), and the syntax-quote contract (core/alias/current-ns
qualification, template-stable gensyms, ~' idiom, distribution through
collections).

Adapting the corpus surfaced and filed three findings, recorded as labeled
divergences/UNVERIFIED in the chapter: nested syntax-quote doesn't collapse
(S25, (= "meow" ```"meow") is false), #inst reads as a bare string (identity
data reader, no instant type), and jolt satisfies :clj in reader
conditionals (feature-key policy under review).

reader-forms-spec gains 11 chapter-cited rows (discard stacking, ##Inf/
##-Inf/##NaN, :default conditionals, qualified var-quote identity, gensym
stability within vs across templates) — all passing.
2026-06-10 11:23:38 -04:00
Dmitri Sotnikov
4d4402b75a
Merge pull request #30 from jolt-lang/spec-35-vars-batch-a
Spec 35-var batch A: 1.11 parsers, map/partition variants, with-redefs, ns fns (+3 bug fixes)
2026-06-10 23:17:26 +08:00
Yogthos
eb7a9f1b20 core: spec 35-var batch A — 1.11 parsers, map/partition variants, with-redefs, ns fns
Fifteen vars from the spec coverage gap (docs/spec/coverage.md):
parse-long/parse-double/parse-boolean (strict validation; scan-number alone
accepts 0x10), newline, current-time-ms (host clock for time), update-keys/
update-vals (PHM base, collisions last-wins), partitionv/partitionv-all/
splitv-at (lazy seqs of vectors; splitv-at's tail stays a seq, matching the
reference), with-redefs/with-redefs-fn (roots restored on throw), time,
macroexpand (expand-1 to fixpoint), alias/ns-unalias (write BOTH alias stores
— require :as uses string-keyed :imports while ns-aliases reads :aliases;
split filed), ns-publics (symbol-keyed map; publics == interns, no privacy).

Three pre-existing bugs fixed along the way:
- (partition n step pad coll) misparsed pad as the coll and returned ()
- (require 'bare.symbol) rejected — only vector specs were accepted
- analyze-form leaked the interpreted analyzer's ns on a punt: a throw out of
  the analyzer left current-ns=jolt.analyzer and :compile-ns set, so the
  fallback interpretation resolved user vars against the wrong namespace
  (bit (var user-sym) under compile mode)

And one overlay-authoring landmine documented in-code: a 20-coll fn must not
use 30-macros macros (with-redefs-fn's dotimes compiled as a forward ref that
resolved to the macro fn at runtime) — loop/recur instead.

Gate: conformance 316x3 (+14 rows), suite 4324 pass / 78 clean (was 4081/72;
parse_*/update_* files now contribute), baselines raised, all specs+unit,
fixpoint, self-host, sci, staged. Coverage: missing-portable 35 -> 20.
2026-06-10 11:16:54 -04:00
Dmitri Sotnikov
894af34b4c
Merge pull request #29 from jolt-lang/language-spec-rfc
RFC 0001: a specification for the Clojure language (+ spec skeleton, coverage dashboard)
2026-06-10 22:57:09 +08:00
Yogthos
7003926eda docs: language specification RFC + spec skeleton with normative exemplars
RFC 0001 proposes a normative, implementation-independent Clojure language
spec (the reader, evaluation model, special forms, data types, seq/laziness
contracts, namespaces/vars, and the portable clojure.core surface) to the
standard of R7RS/Racket — Clojure has none, and every alternative
implementation re-derives semantics from the reference and folklore. The
spec is executable-first: every numbered normative statement cites its
conformance test or is marked UNVERIFIED.

docs/spec/ carries the front matter (conformance terms, entry format, host
classification), the special-form catalog with worked normative entries for
if and let*, the core-library entry format with worked entries for first,
reduce, and parse-uuid, and a generated coverage dashboard over the 694-var
ClojureDocs inventory (tools/spec_coverage.py cross-references the surface
against jolt's interned+resolvable vars and the test suites).

Measured baseline: 380 implemented+tested, 154 implemented-untested, 35
portable-but-missing (filed), 22 resolvable-but-not-interned (filed — seed
fns invisible to resolve/ns-publics), rest classified host/JVM/concurrency.
2026-06-10 10:53:44 -04:00
Dmitri Sotnikov
bd5142abac
Merge pull request #28 from jolt-lang/strict-map-coll-predicates
Strict map?/coll?: tagged structs are values, sorted colls are colls
2026-06-10 22:35:17 +08:00
Yogthos
526de12ad1 core: strict map?/coll? — tagged structs are values, sorted colls are colls
map? treated ANY struct as a map, so (map? 'sym), (map? \a), and
(map? (random-uuid)) were all true; coll? had the same wart. Meanwhile both
were FALSE for sorted maps (and coll? for sorted sets and records), which are
collections in Clojure. Now: a map is a plain struct literal, a phm, a sorted
map, or a record; coll? additionally includes sorted sets. Tagged structs
(anything with :jolt/type) are values.

The loose-map? dependents (destructure's clause ordering, defn's attr-map
guard) already guarded with symbol? first, so nothing relied on the wart —
full gate green, and the suite gained: 4074 -> 4081 pass / 72 clean files
(map_qmark/coll_qmark assertions now correct); baselines raised. 22 new
strictness spec cases.
2026-06-10 10:34:59 -04:00
Dmitri Sotnikov
b836a5d499
Merge pull request #27 from jolt-lang/uuid-support
Proper uuid support: fix random-uuid, add parse-uuid, real UUID values (jolt-6s2)
2026-06-10 22:27:56 +08:00
Yogthos
e44a7a9820 core: proper uuid support — fix random-uuid, add parse-uuid, real uuid values
uuid support was broken end to end (jolt-6s2): random-uuid built malformed
strings ((string/format "%x") with no zero-padding, so hex groups came out
short, and no variant bits), uuid? was hardcoded false, the #uuid data reader
was identity (bare string), and parse-uuid didn't exist. Nothing tested it.

A UUID is now an immutable tagged struct {:jolt/type :jolt/uuid :str <lower>}
(make-uuid, types.janet) — struct value equality gives case-insensitive = and
map-key/set hashing for free, and the evaluator treats it as self-evaluating
(like chars). random-uuid emits a correct RFC 4122 v4 (zero-padded 8-4-4-4-12,
version nibble 4, variant 8-b). parse-uuid validates the canonical shape,
returns nil on a bad string, throws on a non-string (Clojure 1.11). uuid? is
an overlay tag predicate. str renders the bare string; pr-str renders
#uuid "..." and round-trips.

Tests: uuid-spec (30 cases: format, parse edge cases from the suite, value
semantics, reader literal), 6 conformance cases x3 modes. The suite's
parse_uuid/random_uuid/uuid_qmark files now contribute: 4049 -> 4074 pass,
71 clean files; baselines raised.
2026-06-10 10:27:24 -04:00
Dmitri Sotnikov
c14c129627
Merge pull request #26 from jolt-lang/stage2-freeze-punt-set
Freeze the punt set: fallback-zero asserts the exhaustive fallback surface
2026-06-10 21:54:47 +08:00
313 changed files with 46600 additions and 18755 deletions

236
.github/workflows/release.yml vendored Normal file
View file

@ -0,0 +1,236 @@
name: release
# Build the self-contained joltc binary for each platform and attach it to the
# GitHub Release when a v* tag is pushed. The binary bundles the runtime,
# compiler, jolt-core + stdlib source, the Chez boots, and a launcher stub, so it
# runs AND compiles jolt apps with no Chez or cc on the user's machine (jolt-eaj).
#
# No Apple notarization, mirroring dirge: macOS users who download the tarball
# clear Gatekeeper quarantine once (`xattr -d com.apple.quarantine joltc`), or
# install via a Homebrew tap that de-quarantines on install.
on:
push:
tags:
- 'v*'
workflow_dispatch: {} # dry-run the build matrix without tagging
permissions:
contents: write # create/update the GitHub Release and upload assets
jobs:
build:
name: build ${{ matrix.target }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
target: x86_64-linux
shell: bash
# No x86_64-macos: GitHub is retiring the macos-13 Intel runner (jobs
# queue forever). Intel Macs build from source. macos-14 is arm64.
- os: macos-14
target: aarch64-macos
shell: bash
- os: windows-latest
target: x86_64-windows
shell: msys2 {0}
defaults:
run:
shell: ${{ matrix.shell }}
steps:
- uses: actions/checkout@v5
with:
submodules: recursive # vendor/irregex, used by the Chez regex shim
# --- Linux: build Chez from source. The apt chezscheme ships petite+scheme
# only, with no kernel dev files (libkernel.a, scheme.h), which build-joltc
# needs to cc-link. Same setup as .github/workflows/tests.yml. ---
- name: Install build dependencies (Linux)
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y build-essential git liblz4-dev zlib1g-dev libncurses-dev uuid-dev
- name: Cache Chez Scheme (Linux)
if: runner.os == 'Linux'
id: cache-chez
uses: actions/cache@v4
with:
path: /opt/chez
key: chez-${{ runner.os }}-v10.4.1-x11off
- name: Build Chez Scheme from source (Linux)
if: runner.os == 'Linux' && steps.cache-chez.outputs.cache-hit != 'true'
run: |
git clone --depth 1 --branch v10.4.1 https://github.com/cisco/ChezScheme.git /tmp/chez-src
cd /tmp/chez-src
./configure --installprefix=/opt/chez --threads --disable-x11
make -j"$(nproc)"
sudo make install
sudo chown -R "$USER" /opt/chez
- name: Put chez on PATH (Linux)
if: runner.os == 'Linux'
run: |
# Installed as `scheme`; the build invokes `chez`. A wrapper that execs
# scheme keeps argv0 so Chez finds its boot files, and sits next to
# scheme so build.ss derives the csv dir (libkernel.a/scheme.h) from it.
printf '#!/bin/sh\nexec /opt/chez/bin/scheme "$@"\n' > /opt/chez/bin/chez
chmod +x /opt/chez/bin/chez
echo '/opt/chez/bin' >> "$GITHUB_PATH"
# --- macOS: Homebrew chezscheme ships `chez` plus the csv kernel dev files
# (libkernel.a, scheme.h, *.boot), which is all build-joltc needs. ---
- name: Install Chez Scheme (macOS)
if: runner.os == 'macOS'
run: brew install chezscheme lz4
# --- Windows: MSYS2/MinGW-w64 toolchain + Chez built from source (ta6nt).
# The whole job runs in the msys2 shell so cc/xxd/paths behave; the
# produced joltc.exe is a plain Windows binary (no MSYS runtime dep). ---
- name: Set up MSYS2 (Windows)
if: runner.os == 'Windows'
uses: msys2/setup-msys2@v2
with:
msystem: MINGW64
update: false
# inherit the runner PATH so GITHUB_PATH additions (the chez wrapper
# dir) are visible inside the msys2 shell
path-type: inherit
install: >-
git make vim unzip zip
mingw-w64-x86_64-gcc
mingw-w64-x86_64-lz4
mingw-w64-x86_64-zlib
mingw-w64-x86_64-ntldd
- name: Cache Chez Scheme (Windows)
if: runner.os == 'Windows'
id: cache-chez-win
uses: actions/cache@v4
with:
path: chez-install
key: chez-${{ runner.os }}-v10.4.1-mingw64
- name: Build Chez Scheme from source (Windows)
if: runner.os == 'Windows' && steps.cache-chez-win.outputs.cache-hit != 'true'
run: |
git clone --depth 1 --branch v10.4.1 https://github.com/cisco/ChezScheme.git /tmp/chez-src
cd /tmp/chez-src
./configure --threads
make -j"$(nproc)"
# `make install` drives the unix installsh through cmd and dies; the
# build tree has everything — assemble the layout by hand. Boot files
# sit next to scheme.exe (that's where the Windows kernel looks).
inst="$GITHUB_WORKSPACE/chez-install"
mkdir -p "$inst/bin" "$inst/csv"
cp ta6nt/bin/ta6nt/*.exe "$inst/bin/"
cp ta6nt/bin/ta6nt/*.dll "$inst/bin/" 2>/dev/null || true
cp ta6nt/boot/ta6nt/petite.boot ta6nt/boot/ta6nt/scheme.boot "$inst/bin/"
cp ta6nt/boot/ta6nt/petite.boot ta6nt/boot/ta6nt/scheme.boot "$inst/csv/"
cp ta6nt/boot/ta6nt/scheme.h "$inst/csv/"
cp ta6nt/boot/ta6nt/equates.h "$inst/csv/" 2>/dev/null || true
cp ta6nt/boot/ta6nt/libkernel.a "$inst/csv/" || { echo "libkernel.a not found:"; find ta6nt -name "*.a" -o -name "kernel*"; exit 1; }
- name: Put chez on PATH (Windows)
if: runner.os == 'Windows'
run: |
bindir="$GITHUB_WORKSPACE/chez-install/bin"
{ echo '#!/bin/sh'; echo "exec \"$bindir/scheme.exe\" \"\$@\""; } > "$bindir/chez"
chmod +x "$bindir/chez"
echo "$bindir" >> "$GITHUB_PATH"
echo "JOLT_CHEZ_CSV=$GITHUB_WORKSPACE/chez-install/csv" >> "$GITHUB_ENV"
# cc is the build's compiler name; alias it to mingw gcc
{ echo '#!/bin/sh'; echo 'exec gcc "$@"'; } > "$bindir/cc"
chmod +x "$bindir/cc"
- name: Show Chez version
run: chez --version
# build-joltc compiles in a fresh Chez and cc-links; the checked-in seed is
# the compiler image, so no selfhost re-mint (that byte-fixpoint is a
# dev-machine check — see jolt-8479). `make joltc-release`, not `make joltc`.
- name: Build joltc (release)
run: make joltc-release
env:
# Bake the release tag into the binary (build-joltc falls back to
# `git describe` when this is empty, e.g. a workflow_dispatch dry run).
JOLT_VERSION: ${{ startsWith(github.ref, 'refs/tags/') && github.ref_name || '' }}
- name: Inspect the binary (Windows)
if: runner.os == 'Windows'
run: |
set +e
ls -la target/release/
ntldd target/release/joltc.exe 2>&1 | head -20
./target/release/joltc.exe -e '(+ 1 2)'
echo "exit=$?"
# Sanity: the built binary runs (no Chez needed) and self-reports a value.
- name: Smoke the binary
run: |
out="$(./target/release/joltc -e '(reduce + (range 10))')"
test "$out" = "45" || { echo "joltc -e gave '$out', want 45"; exit 1; }
# The binary is a self-contained COMPILER: it must `build` an app with no
# jolt source on disk. Run from an isolated dir (nothing but the tiny app)
# so a build that reaches for host/chez/*.ss on the filesystem fails here,
# not on a user's machine.
- name: Smoke a self-contained build
run: |
joltc="$(pwd)/target/release/joltc"
work="$(mktemp -d)"
mkdir -p "$work/app/src/app"
printf '{:paths ["src"]}\n' > "$work/app/deps.edn"
printf '(ns app.core)\n(defn -main [& _] (println "built:" (reduce + (range 10))))\n' \
> "$work/app/src/app/core.clj"
( cd "$work/app" && "$joltc" build -m app.core -o app )
out="$("$work/app/app")"
test "$out" = "built: 45" || { echo "self-contained build ran '$out', want 'built: 45'"; exit 1; }
# A built binary must also run the DYNAMIC require path: a namespace not
# in the static ns graph compiles from the source roots at runtime, so the
# boot's top-level defines must be visible to the runtime compiler's eval
# (issue #290: this died with "variable var-deref is not bound").
- name: Smoke a runtime require in a built binary
run: |
joltc="$(pwd)/target/release/joltc"
work="$(mktemp -d)"
mkdir -p "$work/app/src/app"
printf '{:paths ["src"]}\n' > "$work/app/deps.edn"
printf '(ns app.extra)\n(defn greet [s] (str "Hello, " s "!"))\n' \
> "$work/app/src/app/extra.clj"
printf '(ns app.core)\n(defn -main [& _]\n (println ((requiring-resolve (quote app.extra/greet)) "runtime")))\n' \
> "$work/app/src/app/core.clj"
( cd "$work/app" && "$joltc" build -m app.core -o app )
out="$(cd "$work/app" && ./app)"
test "$out" = "Hello, runtime!" || { echo "runtime require ran '$out', want 'Hello, runtime!'"; exit 1; }
- name: Package
run: |
ver="${GITHUB_REF_NAME}"
name="joltc-${ver}-${{ matrix.target }}"
mkdir -p "dist/${name}"
cp README.md LICENSE "dist/${name}/"
if [ "${{ runner.os }}" = "Windows" ]; then
cp target/release/joltc.exe "dist/${name}/joltc.exe"
( cd dist && zip -r "${name}.zip" "${name}" && sha256sum "${name}.zip" > "${name}.zip.sha256" )
else
cp target/release/joltc "dist/${name}/joltc"
tar -C dist -czf "dist/${name}.tar.gz" "${name}"
( cd dist && shasum -a 256 "${name}.tar.gz" > "${name}.tar.gz.sha256" )
fi
ls -la dist
- name: Upload to the GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v2
with:
files: |
dist/*.tar.gz
dist/*.tar.gz.sha256
dist/*.zip
dist/*.zip.sha256
fail_on_unmatched_files: false

View file

@ -1,6 +1,7 @@
name: tests
# Run the full test suite (jpm test) on every push and pull request.
# Run the gate (make test) on every push and pull request. Chez is the sole
# substrate; the JVM is used only as the conformance oracle (certify).
on:
push:
pull_request:
@ -8,50 +9,65 @@ on:
jobs:
test:
runs-on: ubuntu-latest
env:
JANET_VERSION: v1.41.2 # bump to match the version Jolt is developed against
# Per-file deadline for the clojure-test-suite battery. Finite files finish
# in well under 1s; the genuinely-infinite ones get killed at any deadline.
# A generous value gives slow CI runners headroom so a sub-second file
# spiking doesn't time out and drop total-pass below the baseline.
JOLT_SUITE_TIMEOUT: "20"
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
with:
# Submodules: vendor/sci (SCI bootstrap/runtime tests) and
# vendor/clojure-test-suite (the cross-dialect conformance battery,
# asserted against a baseline by clojure-test-suite-test.janet).
submodules: recursive
submodules: recursive # vendor/irregex, used by the Chez regex shim
- name: Cache Janet build
id: cache-janet
# Build Chez from source rather than the distro package: the apt
# chezscheme ships petite+scheme only, with no kernel dev files
# (libkernel.a, scheme.h), so `jolt build` (the buildsmoke gate) can't link
# a binary and would skip. The source build provides them, plus the libs the
# kernel links against.
- name: Install build dependencies
run: |
sudo apt-get update
sudo apt-get install -y build-essential git liblz4-dev zlib1g-dev libncurses-dev uuid-dev
- name: Cache Chez Scheme
id: cache-chez
uses: actions/cache@v4
with:
path: /tmp/janet
key: janet-${{ env.JANET_VERSION }}-${{ runner.os }}
path: /opt/chez
key: chez-${{ runner.os }}-v10.4.1-x11off
- name: Build Janet
if: steps.cache-janet.outputs.cache-hit != 'true'
- name: Build Chez Scheme from source
if: steps.cache-chez.outputs.cache-hit != 'true'
run: |
git clone --depth 1 --branch "$JANET_VERSION" https://github.com/janet-lang/janet.git /tmp/janet
make -C /tmp/janet
git clone --depth 1 --branch v10.4.1 https://github.com/cisco/ChezScheme.git /tmp/chez-src
cd /tmp/chez-src
# --disable-x11: the expression editor's X11 clipboard isn't needed and
# would pull an X11 build/link dep.
./configure --installprefix=/opt/chez --threads --disable-x11
make -j"$(nproc)"
sudo make install
sudo chown -R "$USER" /opt/chez
- name: Install Janet
run: sudo make -C /tmp/janet install
- name: Install jpm
- name: Put chez on PATH
run: |
git clone --depth 1 https://github.com/janet-lang/jpm.git /tmp/jpm
# bootstrap.janet resolves jpm/cli.janet relative to the cwd, so it
# must run from inside the jpm checkout.
cd /tmp/jpm
sudo janet bootstrap.janet
# Installed as `scheme`; the gate invokes `chez`. A wrapper that execs
# scheme keeps argv0 so Chez finds its boot files. Placed next to scheme
# so build.ss derives the csv dir (libkernel.a/scheme.h) from it.
printf '#!/bin/sh\nexec /opt/chez/bin/scheme "$@"\n' > /opt/chez/bin/chez
chmod +x /opt/chez/bin/chez
echo '/opt/chez/bin' >> "$GITHUB_PATH"
/opt/chez/bin/chez --version
- name: Janet version
run: janet -v
- name: Install JDK + Clojure (certify oracle)
run: |
sudo apt-get install -y default-jdk rlwrap
# --retry + --fail so a transient CDN error retries instead of handing
# bash an HTML error page (a 2min timeout page flaked a run)
curl --fail --retry 5 --retry-delay 10 --retry-all-errors -L -O \
https://github.com/clojure/brew-install/releases/latest/download/linux-install.sh
head -1 linux-install.sh | grep -q '^#!' || { echo "installer download corrupt"; cat linux-install.sh | head -5; exit 1; }
sudo bash linux-install.sh
clojure --version
- name: Build executable
run: jpm build
- name: Run tests
run: jpm test
- name: Gate
# `make ci` runs the behavior gates (corpus/unit/smoke/buildsmoke/sci/
# certify). buildsmoke now links a real binary (the source-built Chez has
# the kernel dev files). The self-host byte-fixpoint (make selfhost) is a
# dev-machine check — it only holds on the Chez that minted the seed. See
# jolt-8479.
run: make ci

4
.gitignore vendored
View file

@ -1,4 +1,8 @@
AGENTS.md
.DS_Store
CLAUDE.md
build/
target/
.clj-kondo/
.dirge/
.claude/

3
.gitmodules vendored
View file

@ -1,3 +1,6 @@
[submodule "vendor/irregex"]
path = vendor/irregex
url = https://github.com/ashinn/irregex.git
[submodule "vendor/sci"]
path = vendor/sci
url = https://github.com/borkdude/sci.git

View file

@ -1,96 +0,0 @@
# Agent Instructions
This project uses **bd** (beads) for issue tracking. Run `bd prime` for full workflow context.
> **Architecture in one line:** Issues live in a local Dolt database
> (`.beads/dolt/`); cross-machine sync uses `bd dolt push/pull` (a
> git-compatible protocol), stored under `refs/dolt/data` on your git
> remote — separate from `refs/heads/*` where your code lives.
> `.beads/issues.jsonl` is a passive export, not the wire protocol.
>
> See [SYNC_CONCEPTS.md](https://github.com/gastownhall/beads/blob/main/docs/SYNC_CONCEPTS.md)
> for the one-screen overview and anti-patterns (don't treat JSONL as the
> source of truth; don't `bd import` during normal operation; don't
> reach for third-party Dolt hosting before trying the default).
## Quick Reference
```bash
bd ready # Find available work
bd show <id> # View issue details
bd update <id> --claim # Claim work atomically
bd close <id> # Complete work
bd dolt push # Push beads data to remote
```
## Non-Interactive Shell Commands
**ALWAYS use non-interactive flags** with file operations to avoid hanging on confirmation prompts.
Shell commands like `cp`, `mv`, and `rm` may be aliased to include `-i` (interactive) mode on some systems, causing the agent to hang indefinitely waiting for y/n input.
**Use these forms instead:**
```bash
# Force overwrite without prompting
cp -f source dest # NOT: cp source dest
mv -f source dest # NOT: mv source dest
rm -f file # NOT: rm file
# For recursive operations
rm -rf directory # NOT: rm -r directory
cp -rf source dest # NOT: cp -r source dest
```
**Other commands that may prompt:**
- `scp` - use `-o BatchMode=yes` for non-interactive
- `ssh` - use `-o BatchMode=yes` to fail instead of prompting
- `apt-get` - use `-y` flag
- `brew` - use `HOMEBREW_NO_AUTO_UPDATE=1` env var
<!-- BEGIN BEADS INTEGRATION v:1 profile:minimal hash:7510c1e2 -->
## Beads Issue Tracker
This project uses **bd (beads)** for issue tracking. Run `bd prime` to see full workflow context and commands.
### Quick Reference
```bash
bd ready # Find available work
bd show <id> # View issue details
bd update <id> --claim # Claim work
bd close <id> # Complete work
```
### Rules
- Use `bd` for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown TODO lists
- Run `bd prime` for detailed command reference and session close protocol
- Use `bd remember` for persistent knowledge — do NOT use MEMORY.md files
**Architecture in one line:** issues live in a local Dolt DB; sync uses `refs/dolt/data` on your git remote; `.beads/issues.jsonl` is a passive export. See https://github.com/gastownhall/beads/blob/main/docs/SYNC_CONCEPTS.md for details and anti-patterns.
## Session Completion
**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds.
**MANDATORY WORKFLOW:**
1. **File issues for remaining work** - Create issues for anything that needs follow-up
2. **Run quality gates** (if code changed) - Tests, linters, builds
3. **Update issue status** - Close finished work, update in-progress items
4. **PUSH TO REMOTE** - This is MANDATORY:
```bash
git pull --rebase
git push
git status # MUST show "up to date with origin"
```
5. **Clean up** - Clear stashes, prune remote branches
6. **Verify** - All changes committed AND pushed
7. **Hand off** - Provide context for next session
**CRITICAL RULES:**
- Work is NOT complete until `git push` succeeds
- NEVER stop before pushing - that leaves work stranded locally
- NEVER say "ready to push when you are" - YOU must push
- If push fails, resolve and retry until it succeeds
<!-- END BEADS INTEGRATION -->

View file

@ -1,70 +0,0 @@
# Project Instructions for AI Agents
This file provides instructions and context for AI coding agents working on this project.
<!-- BEGIN BEADS INTEGRATION v:1 profile:minimal hash:7510c1e2 -->
## Beads Issue Tracker
This project uses **bd (beads)** for issue tracking. Run `bd prime` to see full workflow context and commands.
### Quick Reference
```bash
bd ready # Find available work
bd show <id> # View issue details
bd update <id> --claim # Claim work
bd close <id> # Complete work
```
### Rules
- Use `bd` for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown TODO lists
- Run `bd prime` for detailed command reference and session close protocol
- Use `bd remember` for persistent knowledge — do NOT use MEMORY.md files
**Architecture in one line:** issues live in a local Dolt DB; sync uses `refs/dolt/data` on your git remote; `.beads/issues.jsonl` is a passive export. See https://github.com/gastownhall/beads/blob/main/docs/SYNC_CONCEPTS.md for details and anti-patterns.
## Session Completion
**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds.
**MANDATORY WORKFLOW:**
1. **File issues for remaining work** - Create issues for anything that needs follow-up
2. **Run quality gates** (if code changed) - Tests, linters, builds
3. **Update issue status** - Close finished work, update in-progress items
4. **PUSH TO REMOTE** - This is MANDATORY:
```bash
git pull --rebase
git push
git status # MUST show "up to date with origin"
```
5. **Clean up** - Clear stashes, prune remote branches
6. **Verify** - All changes committed AND pushed
7. **Hand off** - Provide context for next session
**CRITICAL RULES:**
- Work is NOT complete until `git push` succeeds
- NEVER stop before pushing - that leaves work stranded locally
- NEVER say "ready to push when you are" - YOU must push
- If push fails, resolve and retry until it succeeds
<!-- END BEADS INTEGRATION -->
## Build & Test
_Add your build and test commands here_
```bash
# Example:
# npm install
# npm test
```
## Architecture Overview
_Add a brief overview of your project architecture_
## Conventions & Patterns
_Add your project-specific conventions here_

367
LICENSE
View file

@ -1,143 +1,179 @@
Eclipse Public License - v 1.0
Eclipse Public License - v 2.0
THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE
PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF
THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE
PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION
OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
1. DEFINITIONS
"Contribution" means:
a) in the case of the initial Contributor, the initial code and
documentation distributed under this Agreement, and
a) in the case of the initial Contributor, the initial content
Distributed under this Agreement, and
b) in the case of each subsequent Contributor:
b) in the case of each subsequent Contributor:
i) changes to the Program, and
ii) additions to the Program;
where such changes and/or additions to the Program originate from
and are Distributed by that particular Contributor. A Contribution
"originates" from a Contributor if it was added to the Program by
such Contributor itself or anyone acting on such Contributor's behalf.
Contributions do not include changes or additions to the Program that
are not Modified Works.
i) changes to the Program, and
ii) additions to the Program;
where such changes and/or additions to the Program originate from and
are distributed by that particular Contributor. A Contribution
'originates' from a Contributor if it was added to the Program by such
Contributor itself or anyone acting on such Contributor's behalf.
Contributions do not include additions to the Program which: (i) are
separate modules of software distributed in conjunction with the Program
under their own license agreement, and (ii) are not derivative works of
the Program.
"Contributor" means any person or entity that distributes the Program.
"Contributor" means any person or entity that Distributes the Program.
"Licensed Patents" mean patent claims licensable by a Contributor which
are necessarily infringed by the use or sale of its Contribution alone
or when combined with the Program.
"Program" means the Contributions distributed in accordance with this
"Program" means the Contributions Distributed in accordance with this
Agreement.
"Recipient" means anyone who receives the Program under this Agreement,
including all Contributors.
"Recipient" means anyone who receives the Program under this Agreement
or any Secondary License (as applicable), including Contributors.
"Derivative Works" shall mean any work, whether in Source Code or other
form, that is based on (or derived from) the Program and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship.
"Modified Works" shall mean any work in Source Code or other form that
results from an addition to, deletion from, or modification of the
contents of the Program, including, for purposes of clarity any new file
in Source Code form that contains any contents of the Program. Modified
Works shall not include works that contain only declarations,
interfaces, types, classes, structures, or files of the Program solely
in each case in order to link to, bind by name, or subclass the Program
or Modified Works thereof.
"Distribute" means the acts of a) distributing or b) making available
in any manner that enables the transfer of a copy.
"Source Code" means the form of a Program preferred for making
modifications, including but not limited to software source code,
documentation source, and configuration files.
"Secondary License" means either the GNU General Public License,
Version 2.0, or any later versions of that license, including any
exceptions or additional permissions as identified by the initial
Contributor.
2. GRANT OF RIGHTS
a) Subject to the terms of this Agreement, each Contributor hereby
grants Recipient a non-exclusive, worldwide, royalty-free copyright
license to reproduce, prepare derivative works of, publicly display,
publicly perform, distribute and sublicense the Contribution of such
Contributor, if any, and such derivative works, in source code and
object code form.
a) Subject to the terms of this Agreement, each Contributor hereby
grants Recipient a non-exclusive, worldwide, royalty-free copyright
license to reproduce, prepare Derivative Works of, publicly display,
publicly perform, Distribute and sublicense the Contribution of such
Contributor, if any, and such Derivative Works.
b) Subject to the terms of this Agreement, each Contributor hereby
grants Recipient a non-exclusive, worldwide, royalty-free patent license
under Licensed Patents to make, use, sell, offer to sell, import and
otherwise transfer the Contribution of such Contributor, if any, in
source code and object code form. This patent license shall apply to the
combination of the Contribution and the Program if, at the time the
Contribution is added by the Contributor, such addition of the
Contribution causes such combination to be covered by the Licensed
Patents. The patent license shall not apply to any other combinations
which include the Contribution. No hardware per se is licensed
hereunder.
b) Subject to the terms of this Agreement, each Contributor hereby
grants Recipient a non-exclusive, worldwide, royalty-free patent
license under Licensed Patents to make, use, sell, offer to sell,
import and otherwise transfer the Contribution of such Contributor,
if any, in Source Code or other form. This patent license shall
apply to the combination of the Contribution and the Program if, at
the time the Contribution is added by the Contributor, such addition
of the Contribution causes such combination to be covered by the
Licensed Patents. The patent license shall not apply to any other
combinations which include the Contribution. No hardware per se is
licensed hereunder.
c) Recipient understands that although each Contributor grants the
licenses to its Contributions set forth herein, no assurances are
provided by any Contributor that the Program does not infringe the
patent or other intellectual property rights of any other entity. Each
Contributor disclaims any liability to Recipient for claims brought by
any other entity based on infringement of intellectual property rights
or otherwise. As a condition to exercising the rights and licenses
granted hereunder, each Recipient hereby assumes sole responsibility to
secure any other intellectual property rights needed, if any. For
example, if a third party patent license is required to allow Recipient
to distribute the Program, it is Recipient's responsibility to acquire
that license before distributing the Program.
c) Recipient understands that although each Contributor grants the
licenses to its Contributions set forth herein, no assurances are
provided by any Contributor that the Program does not infringe the
patent or other intellectual property rights of any other entity.
Each Contributor disclaims any liability to Recipient for claims
brought by any other entity based on infringement of intellectual
property rights or otherwise. As a condition to exercising the
rights and licenses granted hereunder, each Recipient hereby
assumes sole responsibility to secure any other intellectual
property rights needed, if any. For example, if a third party
patent license is required to allow Recipient to Distribute the
Program, it is Recipient's responsibility to acquire that license
before distributing the Program.
d) Each Contributor represents that to its knowledge it has sufficient
copyright rights in its Contribution, if any, to grant the copyright
license set forth in this Agreement.
d) Each Contributor represents that to its knowledge it has
sufficient copyright rights in its Contribution, if any, to grant
the copyright license set forth in this Agreement.
e) Notwithstanding the terms of any Secondary License, no
Contributor makes additional grants to any Recipient (other than
those set forth in this Agreement) as a result of such Recipient's
receipt of the Program under the terms of a Secondary License
(if permitted under the terms of Section 3).
3. REQUIREMENTS
A Contributor may choose to distribute the Program in object code form
under its own license agreement, provided that:
3.1 If a Contributor Distributes the Program in any form, then:
a) it complies with the terms and conditions of this Agreement; and
a) the Program must also be made available as Source Code, in
accordance with section 3.2, and the Contributor must accompany
the Program with a statement that the Source Code for the Program
is available under this Agreement, and informs Recipients how to
obtain it in a reasonable manner on or through a medium customarily
used for software exchange; and
b) its license agreement:
b) the Contributor may Distribute the Program under a license
different than this Agreement, provided that such license:
i) effectively disclaims on behalf of all other Contributors all
warranties and conditions, express and implied, including
warranties or conditions of title and non-infringement, and
implied warranties or conditions of merchantability and fitness
for a particular purpose;
i) effectively disclaims on behalf of all Contributors all warranties
and conditions, express and implied, including warranties or conditions
of title and non-infringement, and implied warranties or conditions of
merchantability and fitness for a particular purpose;
ii) effectively excludes on behalf of all other Contributors all
liability for damages, including direct, indirect, special,
incidental and consequential damages, such as lost profits;
ii) effectively excludes on behalf of all Contributors all liability for
damages, including direct, indirect, special, incidental and
consequential damages, such as lost profits;
iii) does not attempt to limit or alter the recipients' rights
in the Source Code under section 3.2; and
iii) states that any provisions which differ from this Agreement are
offered by that Contributor alone and not by any other party; and
iv) requires any subsequent distribution of the Program by any
party to be under a license that satisfies the requirements
of this section 3.
iv) states that source code for the Program is available from such
Contributor, and informs licensees how to obtain it in a reasonable
manner on or through a medium customarily used for software exchange.
3.2 When the Program is Distributed as Source Code:
When the Program is made available in source code form:
a) it must be made available under this Agreement, or if the
Program (i) is combined with other material in a separate file or
files made available under a Secondary License, and (ii) the initial
Contributor attached to the Source Code the notice described in
Exhibit A of this Agreement, then the Program may be made available
under the terms of such Secondary Licenses, and
a) it must be made available under this Agreement; and
b) a copy of this Agreement must be included with each copy of
the Program.
b) a copy of this Agreement must be included with each copy of the
Program.
Contributors may not remove or alter any copyright notices contained
within the Program.
Each Contributor must identify itself as the originator of its
Contribution, if any, in a manner that reasonably allows subsequent
Recipients to identify the originator of the Contribution.
3.3 Contributors may not remove or alter any copyright, patent,
trademark, attribution notices, disclaimers of warranty, or limitations
of liability ("notices") contained within the Program from any copy of
the Program which they Distribute, provided that Contributors may add
their own appropriate notices.
4. COMMERCIAL DISTRIBUTION
Commercial distributors of software may accept certain responsibilities
with respect to end users, business partners and the like. While this
license is intended to facilitate the commercial use of the Program, the
Contributor who includes the Program in a commercial product offering
should do so in a manner which does not create potential liability for
other Contributors. Therefore, if a Contributor includes the Program in
a commercial product offering, such Contributor ("Commercial
Contributor") hereby agrees to defend and indemnify every other
Contributor ("Indemnified Contributor") against any losses, damages and
costs (collectively "Losses") arising from claims, lawsuits and other
legal actions brought by a third party against the Indemnified
license is intended to facilitate the commercial use of the Program,
the Contributor who includes the Program in a commercial product
offering should do so in a manner which does not create potential
liability for other Contributors. Therefore, if a Contributor includes
the Program in a commercial product offering, such Contributor
("Commercial Contributor") hereby agrees to defend and indemnify every
other Contributor ("Indemnified Contributor") against any losses,
damages and costs (collectively "Losses") arising from claims, lawsuits
and other legal actions brought by a third party against the Indemnified
Contributor to the extent caused by the acts or omissions of such
Commercial Contributor in connection with its distribution of the
Program in a commercial product offering. The obligations in this
section do not apply to any claims or Losses relating to any actual or
alleged intellectual property infringement. In order to qualify, an
Indemnified Contributor must: a) promptly notify the Commercial
Contributor in writing of such claim, and b) allow the Commercial
Contributor to control, and cooperate with the Commercial Contributor
in, the defense and any related settlement negotiations. The Indemnified
Contributor may participate in any such claim at its own expense.
Commercial Contributor in connection with its distribution of the Program
in a commercial product offering. The obligations in this section do not
apply to any claims or Losses relating to any actual or alleged
intellectual property infringement. In order to qualify, an Indemnified
Contributor must: a) promptly notify the Commercial Contributor in
writing of such claim, and b) allow the Commercial Contributor to control,
and cooperate with the Commercial Contributor in, the defense and any
related settlement negotiations. The Indemnified Contributor may
participate in any such claim at its own expense.
For example, a Contributor might include the Program in a commercial
product offering, Product X. That Contributor is then a Commercial
@ -145,80 +181,97 @@ Contributor. If that Commercial Contributor then makes performance
claims, or offers warranties related to Product X, those performance
claims and warranties are such Commercial Contributor's responsibility
alone. Under this section, the Commercial Contributor would have to
defend claims against the other Contributors related to those
performance claims and warranties, and if a court requires any other
Contributor to pay any damages as a result, the Commercial Contributor
must pay those damages.
defend claims against the other Contributors related to those performance
claims and warranties, and if a court requires any other Contributor to
pay any damages as a result, the Commercial Contributor must pay
those damages.
5. NO WARRANTY
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED
ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
EITHER EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES
OR CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR
A PARTICULAR PURPOSE. Each Recipient is solely responsible for
determining the appropriateness of using and distributing the Program
and assumes all risks associated with its exercise of rights under this
Agreement, including but not limited to the risks and costs of program
errors, compliance with applicable laws, damage to or loss of data,
programs or equipment, and unavailability or interruption of operations.
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT
PERMITTED BY APPLICABLE LAW, THE PROGRAM IS PROVIDED ON AN "AS IS"
BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR
IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR CONDITIONS OF
TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE. Each Recipient is solely responsible for determining the
appropriateness of using and distributing the Program and assumes all
risks associated with its exercise of rights under this Agreement,
including but not limited to the risks and costs of program errors,
compliance with applicable laws, damage to or loss of data, programs
or equipment, and unavailability or interruption of operations.
6. DISCLAIMER OF LIABILITY
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR
ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING
WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR
DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED
HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, AND TO THE EXTENT
PERMITTED BY APPLICABLE LAW, NEITHER RECIPIENT NOR ANY CONTRIBUTORS
SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION LOST
PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE
EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
7. GENERAL
If any provision of this Agreement is invalid or unenforceable under
applicable law, it shall not affect the validity or enforceability of
the remainder of the terms of this Agreement, and without further action
by the parties hereto, such provision shall be reformed to the minimum
extent necessary to make such provision valid and enforceable.
the remainder of the terms of this Agreement, and without further
action by the parties hereto, such provision shall be reformed to the
minimum extent necessary to make such provision valid and enforceable.
If Recipient institutes patent litigation against any entity (including
a cross-claim or counterclaim in a lawsuit) alleging that the Program
itself (excluding combinations of the Program with other software or
hardware) infringes such Recipient's patent(s), then such Recipient's
If Recipient institutes patent litigation against any entity
(including a cross-claim or counterclaim in a lawsuit) alleging that the
Program itself (excluding combinations of the Program with other software
or hardware) infringes such Recipient's patent(s), then such Recipient's
rights granted under Section 2(b) shall terminate as of the date such
litigation is filed.
All Recipient's rights under this Agreement shall terminate if it fails
to comply with any of the material terms or conditions of this Agreement
and does not cure such failure in a reasonable period of time after
becoming aware of such noncompliance. If all Recipient's rights under
this Agreement terminate, Recipient agrees to cease use and distribution
of the Program as soon as reasonably practicable. However, Recipient's
obligations under this Agreement and any licenses granted by Recipient
relating to the Program shall continue and survive.
All Recipient's rights under this Agreement shall terminate if it
fails to comply with any of the material terms or conditions of this
Agreement and does not cure such failure in a reasonable period of
time after becoming aware of such noncompliance. If all Recipient's
rights under this Agreement terminate, Recipient agrees to cease use
and distribution of the Program as soon as reasonably practicable.
However, Recipient's obligations under this Agreement and any licenses
granted by Recipient relating to the Program shall continue and survive.
Everyone is permitted to copy and distribute copies of this Agreement,
but in order to avoid inconsistency the Agreement is copyrighted and may
only be modified in the following manner. The Agreement Steward reserves
the right to publish new versions (including revisions) of this
Agreement from time to time. No one other than the Agreement Steward has
the right to modify this Agreement. The Eclipse Foundation is the
initial Agreement Steward. The Eclipse Foundation may assign the
but in order to avoid inconsistency the Agreement is copyrighted and
may only be modified in the following manner. The Agreement Steward
reserves the right to publish new versions (including revisions) of
this Agreement from time to time. No one other than the Agreement
Steward has the right to modify this Agreement. The Eclipse Foundation
is the initial Agreement Steward. The Eclipse Foundation may assign the
responsibility to serve as the Agreement Steward to a suitable separate
entity. Each new version of the Agreement will be given a distinguishing
version number. The Program (including Contributions) may always be
distributed subject to the version of the Agreement under which it was
received. In addition, after a new version of the Agreement is
published, Contributor may elect to distribute the Program (including
its Contributions) under the new version. Except as expressly stated in
Sections 2(a) and 2(b) above, Recipient receives no rights or licenses
to the intellectual property of any Contributor under this Agreement,
whether expressly, by implication, estoppel or otherwise. All rights in
the Program not expressly granted under this Agreement are reserved.
Distributed subject to the version of the Agreement under which it was
received. In addition, after a new version of the Agreement is published,
Contributor may elect to Distribute the Program (including its
Contributions) under the new version.
This Agreement is governed by the laws of the State of New York and the
intellectual property laws of the United States of America. No party to
this Agreement will bring a legal action under this Agreement more than
one year after the cause of action arose. Each party waives its rights
to a jury trial in any resulting litigation.
Except as expressly stated in Sections 2(a) and 2(b) above, Recipient
receives no rights or licenses to the intellectual property of any
Contributor under this Agreement, whether expressly, by implication,
estoppel or otherwise. All rights in the Program not expressly granted
under this Agreement are reserved. Nothing in this Agreement is intended
to be enforceable by any entity that is not a Contributor or Recipient.
No third-party beneficiary rights are created under this Agreement.
Exhibit A - Form of Secondary Licenses Notice
"This Source Code may also be made available under the following
Secondary Licenses when the conditions for such availability set forth
in the Eclipse Public License, v. 2.0 are satisfied: {name license(s),
version(s), and exceptions or additional permissions here}."
Simply including a copy of this Agreement, including this Exhibit A
is not sufficient to license the Source Code under Secondary Licenses.
If it is not possible or desirable to put the notice in a particular
file, then You may include the notice in a location (such as a LICENSE
file in a relevant directory) where a recipient would be likely to
look for such a notice.
You may add additional accurate notices of copyright ownership.

172
Makefile Normal file
View file

@ -0,0 +1,172 @@
# jolt — Clojure on Chez Scheme. Single substrate, no Janet.
#
# bin/joltc runs jolt directly off the checked-in seed (host/chez/seed/); there is no
# build step. `make test` is the full gate. `make remint` rebuilds the seed after a
# source change.
.PHONY: test ci values corpus unit smoke buildsmoke staticnativesmoke selfhost sci cts certify ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline shakesmoke remint joltc joltc-release joltc-debug joltcsmoke submodules
# Every target needs the vendored submodules; fail with the fix, not a load error.
submodules:
@test -f vendor/irregex/irregex.scm || { \
echo "vendor submodules missing; run: git submodule update --init --recursive"; exit 1; }
# Full gate (dev machine). Includes the self-host byte-fixpoint, which only holds
# on the same Chez that minted the seed.
test: submodules selfhost ci
@echo "OK: all gates passed"
# CI gate: behavior only. The checked-in seed is a minted artifact (like a
# lockfile) — it RUNS correctly on any Chez, but `selfhost` rebuilds it and a
# different Chez version may emit byte-different (gensym/order) output, so the
# byte-fixpoint is a dev-machine check, not a CI one (jolt-8479).
ci: submodules values corpus unit smoke buildsmoke staticnativesmoke sci cts ffi transient infer wp devirt fieldread numwp fieldnum protoret narrow directlink numeric inline certify
@echo "OK: CI gates passed"
# Self-host fixpoint: bootstrap.ss rebuild == checked-in seed.
selfhost:
@sh host/chez/selfcheck.sh
# Value-model unit tests (nil/truthiness/collections on Chez).
values:
@chez --script test/chez/values-test.ss
# Corpus conformance vs JVM-sourced expecteds (allowlist + floor).
corpus:
@chez --script host/chez/run-corpus.ss
# Host-specific unit cases.
unit:
@chez --script host/chez/run-unit.ss
# Real-CLI smoke over bin/joltc.
smoke:
@sh host/chez/smoke.sh
# `jolt build` produces a working standalone binary.
buildsmoke:
@sh host/chez/build-smoke.sh
# `jolt build` cc-links a :jolt/native :static archive into the binary (the
# default), and --dynamic keeps the runtime load-shared-object path.
staticnativesmoke:
@sh host/chez/static-native-smoke.sh
# Build joltc as a self-contained native binary into target/<profile>/joltc. The
# binary bundles the runtime, compiler, jolt-core + stdlib source, the Chez boots,
# and a launcher stub, so it runs AND compiles jolt apps with no Chez or cc on the
# machine. Built on a dev/CI host that HAS Chez + cc. release = optimize-level 3,
# no inspector info, compressed; debug = optimize-level 0 + inspector + debug info.
joltc-release:
@chez --script host/chez/build-joltc.ss release target/release/joltc
joltc-debug:
@chez --script host/chez/build-joltc.ss debug target/debug/joltc
# Re-mint the seed first so the embedded compiler image is current, then both builds.
joltc: selfhost joltc-release joltc-debug
@echo "OK: target/release/joltc and target/debug/joltc built"
# Self-build smoke: the distributed joltc compiles an app with Chez + cc removed.
joltcsmoke:
@sh host/chez/joltc-selfbuild-smoke.sh
# SCI conformance: load borkdude/sci's source through joltc (floor-gated).
sci:
@chez --script host/chez/run-sci.ss
# clojure-test-suite conformance: run the vendored jank-lang/clojure-test-suite
# per-namespace under joltc, gated on the per-namespace baseline
# (test/chez/cts-known-failures.txt).
cts:
@bash host/chez/cts.sh
# FFI: bind native functions (typed foreign-procedure), memory, and that a
# :blocking call is collect-safe (a parked thread doesn't pin the collector).
ffi:
@chez --script test/chez/ffi-binding-test.ss
# Transients: mutable backing, snapshot on persistent!, and linear-time builds.
transient:
@chez --script test/chez/transient-test.ss
# Inference / success-type checking: drive jolt.passes.types directly and assert
# diagnostic counts + collected calls/escapes (the optimization pass the other
# gates don't exercise).
infer:
@chez --script host/chez/run-infer.ss
# Whole-program param-type fixpoint: record types flowing across fn boundaries
# (a callee's param picks up its callers' ctor return types), the foundation the
# bare-index field reads + protocol devirtualization build on.
wp:
@chez --script host/chez/run-wp.ss
# Protocol-call devirtualization: a monomorphic call resolves its impl by the
# inferred record tag (find-protocol-method) instead of routing through the
# protocol var; the result must match ordinary dispatch.
devirt:
@chez --script host/chez/run-devirt.ss
# Native record field reads: a keyword lookup on a statically-known record reads
# the field by its declared slot (jrec-field-at) instead of jolt-get; the value
# must match, and a non-field key / default-arg form keeps the generic path.
fieldread:
@chez --script host/chez/run-fieldread.ss
# Hintless whole-program double inference: a fn whose every call site passes a
# flonum has its param typed :double by the closed-world fixpoint and unboxed to
# fl-ops with no ^double hint; an integer caller leaves it generic, an escaped fn
# keeps :any.
numwp:
@chez --script host/chez/run-numwp.ss
# Double record fields: a ^double-tagged field reads back as a flonum (coerced at
# construction and set!), so hintless arithmetic over those fields unboxes to fl-ops;
# an untagged field stays generic.
fieldnum:
@chez --script host/chez/run-fieldnum.ss
# Protocol-method return inference: a method whose impls all return the same record
# type has a monomorphic return, so a (method recv ..) call types as that record and
# a field read off the result bare-indexes; a disagreeing impl keeps the generic path.
protoret:
@chez --script host/chez/run-protoret.ss
# Nilable record types + flow-sensitive narrowing: a record-or-nil types as a nilable
# record (some?/nil? don't fold, so a runtime guard stays); inside (if (some? x) ..)
# the then-branch narrows x to non-nil, so its field reads bare-index and unbox.
narrow:
@chez --script host/chez/run-narrow.ss
# Direct-linking emission: a closed-world build binds top-level app defs to jv$
# Scheme bindings and routes app->app calls/refs to them, skipping var-deref +
# jolt-invoke; ^:dynamic/^:redef and nested defs opt out.
directlink:
@chez --script test/chez/directlink-test.ss
# Hint-directed fast arithmetic: ^double/^long param hints (and float literals)
# lower arithmetic to Chez fl*/fx* ops; un-hinted integer code stays generic.
numeric:
@chez --script test/chez/numeric-test.ss
# IR inlining: a small single-arity defn is spliced at call sites (under optimize),
# with ^double/^long entry/return coercions carried through via :coerce nodes.
inline:
@chez --script test/chez/inline-test.ss
# Tree-shake soundness: build example apps (incl. deps.edn git-lib apps) default vs
# --tree-shake and require identical output. Slow (two builds per app); not in the
# default gate. Skips without the examples repo / Chez kernel dev files.
shakesmoke:
@sh host/chez/tree-shake-smoke.sh
# JVM oracle: certify the corpus against reference Clojure. Skips if clojure absent.
certify:
@if command -v clojure >/dev/null 2>&1; then \
clojure -M test/conformance/certify.clj; \
else \
echo "certify: clojure not on PATH — skipped"; \
fi
# Re-mint the seed after changing a seed source (reader/analyzer/backend/core).
remint:
@sh host/chez/remint.sh

420
README.md
View file

@ -2,255 +2,233 @@
[![tests](https://github.com/jolt-lang/jolt/actions/workflows/tests.yml/badge.svg)](https://github.com/jolt-lang/jolt/actions/workflows/tests.yml)
A Clojure implementation on top of [Janet](https://janet-lang.org). Jolt reads Clojure source and, by default, compiles each form to native Janet bytecode — falling back to a tree-walking interpreter for forms the compiler doesn't handle, so results always match the interpreter. It ships a Clojure-compatible standard library. The goal is a Janet-hosted [SCI](https://github.com/borkdude/sci)-style runtime with a minimal bootstrap.
A Clojure implementation on [Chez Scheme](https://cisco.github.io/ChezScheme/).
Jolt reads Clojure source, analyzes it to a host-neutral IR, emits Scheme, and
runs it on Chez. The compiler is self-hosted: it is written in Clojure
(`jolt-core/`) and compiles itself. It ships a Clojure-compatible standard library.
## Install
Grab the self-contained `joltc` binary (Linux/macOS/Windows) — it bundles the
runtime, compiler, and standard library, so there is nothing else to install.
Download the binary archive for your platform from the
[releases page](https://github.com/jolt-lang/jolt/releases) (`joltc-<ver>-<platform>.tar.gz`,
or the `.zip` on Windows). The "Source code" archives GitHub attaches to every
release are not binaries — see [Build](#build) before using one.
With Homebrew:
```bash
brew install jolt-lang/jolt/jolt
```
Or with the install script (installs to `/usr/local/bin` by default; `--dir <dir>`
and `--version <v>` override that):
```bash
curl -sL https://raw.githubusercontent.com/jolt-lang/jolt/main/install | bash
```
Then `joltc -e '(+ 1 2)'`. To run from source instead (needs Chez), see
[Build](#build).
## Requirements
Only [Chez Scheme](https://cisco.github.io/ChezScheme/) (the gate invokes it as
`chez`). The conformance gate additionally uses Clojure on the JVM as an oracle,
but running jolt does not.
## Build
There is no build step. The bootstrap seed (`host/chez/seed/{prelude,image}.ss`)
is checked in, so a fresh clone runs immediately:
```bash
git clone https://github.com/jolt-lang/jolt.git
git clone --recurse-submodules https://github.com/jolt-lang/jolt.git
cd jolt
git submodule update --init # pulls vendor/sci and vendor/clojure-test-suite
jpm build # builds build/jolt and build/jolt-deps
bin/joltc -e '(+ 1 2)' # => 3
```
Requires `jpm` and a recent Janet (CI-tested against 1.41). See
[doc/building-and-deps.md](doc/building-and-deps.md) for build details, the
`jpm clean` caveat, how namespaces are resolved (`JOLT_PATH`), and pulling
Clojure libraries from a `deps.edn` with the `jolt-deps` tool.
The `--recurse-submodules` matters: jolt vendors its regex engine and test
suites as git submodules. In a checkout that's missing them (a plain
`git clone`, or after pulling a commit that adds one), fetch them with:
```bash
git submodule update --init --recursive
```
Note that GitHub's auto-generated "Source code (zip/tar.gz)" archives on the
releases page do **not** contain submodules, so they can't run or build —
clone the repo instead (or grab a prebuilt binary from the same page).
After changing a compiler source — the reader (`host/chez/reader.ss`), the
analyzer/IR/backend (`jolt-core/jolt/*.clj`), or the `clojure.core` overlay
(`jolt-core/clojure/core/*.clj`) — re-mint the seed:
```bash
make remint # iterates host/chez/bootstrap.ss to a byte-fixpoint
```
## Run
```bash
bin/joltc -e EXPR # evaluate a Clojure expression and print the result
```
build/jolt # start a REPL
build/jolt file.clj [args] # run a file (binds *command-line-args* and *file*)
build/jolt -e EXPR [args] # evaluate EXPR and print the result
build/jolt -m NS [args] # require NS and call its -main
build/jolt nrepl-server [addr] # start an nREPL server ([host:]port, default 7888)
build/jolt --version # print the version
build/jolt -h | --help # help
```
The REPL accumulates multi-line forms until they balance:
```
user=> (defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
#'user/fib
user=> (map fib (range 10))
(0 1 1 2 3 5 8 13 21 34)
```
Running a file evaluates its top-level forms:
```
$ echo '(println "hello" (* 6 7))' > hello.clj
$ build/jolt hello.clj
hello 42
```
## Use as a library
```janet
(use jolt/api)
(def ctx (init))
(eval-string ctx "(+ 1 2)") # → 3
(eval-string ctx "(map inc [1 2 3])") # → (2 3 4) ; a lazy seq, like Clojure
```
`(init)` returns a context with `clojure.core` loaded. Each context is isolated; use separate contexts for separate environments.
### Evaluation pipeline: interpreted and compiled
Every form passes through one router (`loader/eval-toplevel`) that decides *per
form* whether to tree-walk it or compile it to Janet bytecode. The shipped
runtime **compiles by default**; set `JOLT_INTERPRET=1` to force the interpreter.
**Hybrid, always correct.** The compiler is incomplete by design: a form it can't
compile correctly throws `jolt/uncompilable`, and the router falls back to the
tree-walking interpreter (`eval-form`) for that form. So the result *always*
matches the interpreter — compilation is a transparent speedup, never a semantic
change. Only the compile step is guarded; runtime errors in compiled code
propagate normally (no double-evaluation, no hidden errors).
What compiles: `def`/`defn`, multi-arity / named / variadic fns, `recur` (in
`loop` and directly in `fn`), `let`/`if`/`do`/`try`/`throw`/`quote`, map and
vector literals, and calls. What falls back to the interpreter: context-modifying
and definitional forms (`ns`, `defmacro`, `deftype`, `defprotocol`,
`defmulti`/`defmethod`, `reify`, `require`, `binding`, …), destructuring, regex
literals, and the handful of interpreter-only special forms.
**Live redefinition.** Compiled global references deref through Jolt **var cells**
(Janet early-binds plain symbols, which would freeze redefinition), so redefining
a `def`/`defn` at the REPL is visible to already-compiled callers — Clojure's var
model. Hot numeric primitives (`+ - * < > <= >=`) emit native Janet ops, and
calls compile to direct Janet calls.
```janet
(def ctx (init {:compile? true}))
(eval-string ctx "(defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))")
(eval-string ctx "(fib 30)") ; → 832040, native Janet bytecode
```
For compute-heavy code the compiled path is dramatically faster than tree-walking,
at native Janet speed.
**Validated at parity.** The conformance suite passes 258/258 under *all three*
execution paths — interpreter, compiler, and the self-hosted compiler
(`conformance-test.janet` runs all three in CI) — and the full clojure-test-suite
matches its baseline across ~4.6k assertions — evidence the hybrid path doesn't
diverge.
**AOT.** `aot.janet` marshals a compiled namespace to a Janet bytecode image
(`save-ns`) and loads it back into a fresh context (`load-ns-image`), skipping
parse/analyze/emit/compile on reload. Core fns are referenced by name against the
baked-in runtime; only user bytecode and var cells are serialized.
## Host interop
Jolt exposes CLJS-style host interop through `.` on any Janet table or struct — a field holding a function is called with the receiver as the first argument:
```clojure
(def obj {:greet (fn [self name] (str "Hello " name))})
(. obj greet "Alice") ; → "Hello Alice"
(.-greet obj) ; field access (reader sugar for (. obj :greet))
```
### The `janet` interop bridge
The whole Janet standard library is reachable from Clojure through an explicit
`janet` namespace segment, which marks every crossing into host code (where
Clojure semantics no longer hold):
```clojure
(janet.os/clock) ; → a Janet module fn: os/clock
(janet.string/join ["a" "b"] ",") ; → janet `string/join` (NB: takes a Janet
; tuple, not a Jolt vector — convert first)
(janet/slurp "deps.edn") ; → a Janet root builtin: slurp
(janet/type [1 2]) ; → :table
```
The rule is `janet/<name>` for a Janet root binding and `janet.<module>/<name>`
for a module binding. Because the boundary is explicit, you can tell at the call
site that a form drops into the host — and that values cross the boundary as
their Janet representations (a Jolt vector is a Janet table, etc.), so a Janet
function expecting a tuple needs an explicit conversion. The `jolt.interop`,
`jolt.shell`, and `jolt.http` namespaces are thin Clojure wrappers built on this.
This bridge is what makes networking (and everything else in Janet's stdlib)
available to ordinary Clojure — for example, `jolt.nrepl` (below) is plain
Clojure over `janet.net/*`.
```clojure
(require '[jolt.interop :as j])
(j/janet-type [1 2]) ; → :tuple
(j/janet-table-keys {:a 1 :b 2}) ; → [:b :a]
```
## nREPL
Jolt ships an [nREPL](https://nrepl.org) server and client (`jolt.nrepl`),
written in Clojure on top of the `janet.net/*` bridge. Start a server from the
CLI — it writes `.nrepl-port` so editors (CIDER, Calva, …) auto-connect:
```bash
jolt nrepl-server # listen on 127.0.0.1:7888, write .nrepl-port
jolt nrepl-server 12345 # choose a port
jolt nrepl-server 0.0.0.0:12345 # choose host and port (alias: nrepl)
$ bin/joltc -e '(->> (range 10) (filter even?) (map (fn [x] (* x x))) (reduce +))'
120
$ bin/joltc -e '(/ 1 2)'
1/2
```
Supported ops: `clone`, `describe`, `eval`, `load-file`, `close`, `ls-sessions`,
`interrupt` (acknowledged; an in-flight eval can't actually be interrupted), and
`eldoc`. `eval` streams `out`, reports the current `ns`, evaluates each form in
the message, and returns an `eval-error` status (the session stays usable) on
failure. One Jolt runtime backs the server and sessions share it, so `def`s
persist across a connection like a normal dev REPL.
## REPL and editor integration
It's also usable as a library — embed a server, or drive another nREPL as a
client:
```bash
bin/joltc repl # a line REPL with the project's deps loaded
bin/joltc --nrepl-server [port] # an nREPL server (default 7888) for editors
```
Both resolve the `deps.edn` in the current directory first, so the project's
source roots and native libraries are loaded — `(require '[my.ns])` works live.
`--nrepl-server` writes a `.nrepl-port` file in the project dir, so CIDER / Calva / Cursive
auto-detect the port; override it with the argument or `JOLT_NREPL_PORT`.
The server runs in dev mode — calls deref their var, so redefining a function
takes effect on the next call without restarting the process. The built-in
handler speaks `clone`/`describe`/`eval`/`load-file`/`close`; heavier ops
(sessions, interruptible eval, completion) are added as nREPL middleware listed
in `deps.edn` under `:nrepl/middleware`.
```clojure
(require '[jolt.nrepl :as nrepl])
(def server (nrepl/start-server! {:port 7888}))
;; ... later ...
(nrepl/stop-server! server)
(def c (nrepl/connect {:port 7888}))
(def session (nrepl/client-clone c))
(nrepl/client-eval c "(+ 1 2)" session) ; → responses incl. {"value" "3"}
(nrepl/client-close c)
;; from your editor, against the running process:
(require '[myapp.core :as app])
(app/start!) ; bring the app up
;; edit a handler, re-evaluate the defn — the running app sees it, no restart
(app/stop!)
```
## Compile a binary
`bin/joltc build` ahead-of-time compiles a project into a single self-contained
executable — the runtime, `clojure.core`, the standard library, the app, and its
`deps.edn` dependencies are linked in, so the result needs no Chez install, no
JVM, and no source on disk to run.
```bash
bin/joltc build -m myapp.core -o myapp # compile myapp.core's -main into ./myapp
./myapp arg1 arg2 # runs anywhere; args reach -main
```
Modes trade dynamism for speed: the default (release) build uses the proven code
generator; `--opt` also runs the inference + inlining + scalar-replacement passes
over the closed-world program; `--dev` is unoptimized.
Two opt-in closed-world flags cut dispatch cost and binary size:
```bash
bin/joltc build -m myapp.core --direct-link # app->app calls bind directly (no var lookup)
bin/joltc build -m myapp.core --tree-shake # ship only code reachable from -main
```
`--tree-shake` walks the call graph across your app, its libraries, and
`clojure.core`, drops everything unreachable from `-main` (and the compiler itself
when the app never `eval`s), and typically removes 12 MB. It stays sound by bailing
out — keeping everything, and reporting which library is responsible — when reachable
code resolves vars by name at runtime (`eval`/`resolve`/`ns-resolve`/…). See
[docs/tools-deps.md](docs/tools-deps.md) and `docs/rfc/0007`.
This needs Chez's kernel development files (`libkernel.a`, `scheme.h`) and a C
compiler. They come with a from-source Chez install; a distro `chezscheme`
package ships only the runtime, so `build` won't link a binary there.
RFC 0007 (`docs/rfc/`) covers the design and the three-mode model.
## Standalone joltc binary
`make` builds joltc itself into a single self-contained native binary — the
runtime, compiler, `jolt-core`/`stdlib` source, and the Chez boots are baked in,
so the result runs and `build`s jolt apps on a machine with neither Chez nor a C
compiler. Build it on a host that *does* have both.
```bash
make joltc-release # => target/release/joltc (optimize-level 3, compressed)
make joltc-debug # => target/debug/joltc (optimize-level 0, inspector + debug info)
make joltc # re-mint the seed first, then both
```
`make joltc` re-mints the seed so the embedded compiler image is current before
linking; use `joltc-release`/`joltc-debug` directly to skip that when the seed is
already minted. Like `build`, both require Chez's kernel development files
(`libkernel.a`, `scheme.h`) and a C compiler.
## Architecture
A small Chez runtime (`host/chez/*.ss`: value model, persistent collections, seqs,
vars/namespaces, host interop) hosts a portable Clojure overlay split across two
source roots by *when* they load:
- **`jolt-core/`** is baked into the seed — the compiler (`jolt-core/jolt/`:
reader/analyzer/IR/backend, plus `jolt.main`/`jolt.deps`) and `clojure.core` in
dependency-ordered tiers (`jolt-core/clojure/core/NN-*.clj`). Changing anything
here means re-minting the seed.
- **`stdlib/`** loads lazily at runtime off the source roots — the rest of the
standard library (`clojure.string`/`set`/`walk`/`edn`/`pprint`/…) plus the
`jolt.ffi` host library. Editing these needs no re-mint.
`bin/joltc` loads the checked-in seed and the spine, then compiles and evaluates on
Chez (read → analyze → IR → emit → eval). `host/chez/bootstrap.ss` rebuilds that
seed from source on pure Chez; the build is a self-hosting fixpoint (a rebuild
reproduces the checked-in seed byte-for-byte).
## Differences from Clojure
Jolt targets Clojure semantics but runs on Janet, not the JVM. The notable divergences:
Jolt targets Clojure semantics but runs on Chez, not the JVM. Most portable
Clojure runs unchanged — persistent collections (32-way-trie vectors, HAMT
maps/sets), the numeric tower (exact integers, bignums, ratios, doubles), lazy
and infinite sequences, transducers, destructuring, multimethods with
hierarchies, protocols/records (`deftype`/`defrecord`/`reify`/`extend-protocol`),
metadata, namespaces, atoms, `future`/`promise`/`agent`/`pmap`,
`clojure.core.async`, runtime `eval`/`load-string`/`defmacro`, and the full
reader (`#()`, `#_`, `#?`, tagged literals, `#"…"`) all behave as on the JVM.
`=` is category-aware (`(= 3 3.0)``false`) and `==` is value-equality, as in
Clojure. The genuine divergences:
- **Host platform.** No JVM and no Java interop — `import`, `gen-class`, `proxy` of Java classes, and `java.*` are unavailable. `instance?` recognizes a small set of built-in types (`clojure.lang.Atom`, `Number`, `String`, …).
- **Numbers.** Janet integers and doubles. `(/ 1 3)` is `0.3333…` and large products lose precision. No ratios or `BigDecimal` (`ratio?` is always false, `bigdec` falls back to a double); `bigint`/`biginteger` use Janet's 64-bit `int/s64`, not arbitrary precision. The reader still accepts Clojure's numeric literal syntaxes — the BigInt/BigDecimal suffixes (`42N`, `1.5M`), ratios (`1/2`), radixed integers (`2r1010`, `16rFF`), and exponents (`1e3`) — but reads them as plain Janet numbers (a ratio becomes its double quotient). The auto-promoting `+'`/`-'`/`*'`/`inc'`/`dec'` are aliases for the plain ops, since Janet numbers don't overflow. `quot`/`rem`/`mod` follow Clojure's sign rules. The symbolic values `##Inf`/`##-Inf`/`##NaN` read, and `infinite?`/`NaN?` work. Janet represents an integer and an integer-valued double identically, so `1` and `1.0` are indistinguishable: `(float?/double? 1.0)` is `false` and `(int? 1.0)` is `true``float?`/`double?` are true only for values with a fractional part or `##Inf`/`##NaN`.
- **Collections.** By default Jolt uses immutable persistent data structures: vectors are 32-way branching tries (structural-sharing persistent vectors with O(log₃₂ n) `conj`/`assoc`/`nth`), lists are persistent singly-linked cons cells (O(1) `conj`/`cons` prepend with structural sharing), and maps/sets are persistent hash structures. Value equality and sequence operations are Clojure-compatible, but hash-map/hash-set iteration order is unspecified and differs from Clojure — use `sorted-map`/`sorted-set` when order matters.
- **Mutable build mode.** Jolt can be compiled to use fast Janet-native *mutable* collections instead, via a build-time flag: `JOLT_MUTABLE=1 jpm build` (default `jpm build` is immutable). In mutable mode vectors and lists share one mutable array representation (so `conj` mutates in place and appends, and `vector?`/`list?` no longer distinguish them) — a performance/looseness trade-off. The default immutable build has full Clojure value semantics.
- **Concurrency / STM.** No refs, `dosync`, agents, or `send`; `locking` evaluates its body without real locking. Atoms, volatiles, promises, and delays are supported.
- **Futures.** `future` runs its body on a *real* OS thread (Janet's `ev/thread`), so it can use a second core for CPU-bound work — unlike the cooperatively-scheduled `go` blocks. `deref`/`@` parks until the result is ready (with the optional `(deref f timeout-ms timeout-val)` arity); `future?`, `future-done?`, `realized?`, `future-cancel`, and `future-cancelled?` are supported. Two important divergences from the JVM: (1) **snapshot semantics** — Janet threads have separate heaps, so the body and the state it closes over are *copied* to the worker thread and only the return value is copied back; mutating a captured atom does not propagate to the parent (communicate via the return value). (2) **no thread interruption** — Janet OS threads can't be cancelled mid-run, so `future-cancel` marks the *future* cancelled (deref then throws and the predicates flip) but the underlying computation still runs to completion in the background. As on the JVM, a live future thread keeps the process alive until it finishes (the JVM's non-daemon future pool behaves the same).
- **core.async.** `clojure.core.async` runs on Janet fibers and channels (`chan`, `go`, `go-loop`, `<!`/`>!`/`<!!`/`>!!`, `close!`, `alts!`, `timeout`, `put!`/`take!`, `buffer`/`dropping-buffer`/`sliding-buffer`, and channel transducers via `(chan n xform)`). Because Janet fibers are stackful coroutines, a `go` block is just its body run in a fiber — no CPS/state-machine rewrite — so `<!`/`>!` work *anywhere*, including inside `try`, nested `fn`s, and loops (positions Clojure's `go` macro forbids). Go blocks are cooperatively scheduled on one OS thread, so parking (`<!`) and blocking (`<!!`) coincide; `thread` runs cooperatively too. Dynamic-var bindings are conveyed into `go` blocks (each go block sees the bindings in effect when it was spawned).
- **Regex.** Compiled to Janet's PEG engine (Janet has no regex). Supported: capturing groups (`[whole g1 …]`), greedy and lazy quantifiers with backtracking, `(?:…)`, lookahead `(?=…)`/`(?!…)`, alternation, anchors `^ $ \b \B`, character classes, and the `(?i)` flag. Not supported: lookbehind, backreferences (`\1`), named groups (`(?<name>…)`), and Unicode property classes (`\p{Lu}`).
- **Arrays.** Java-style arrays map onto Janet's native types: `byte-array` is a Janet buffer (contiguous, C-backed); `object-array`/`int-array`/`double-array`/etc. are Janet arrays. `aget`/`aset`/`alength`/`aclone` work over both.
- **Transients.** `transient`/`conj!`/`assoc!`/`dissoc!`/`disj!`/`pop!`/`persistent!` are real mutable scratch collections backed by Janet's native arrays and tables (vectors → arrays, maps/sets → tables), so building a collection with them avoids the per-step copying of the persistent path (notably for maps/sets). `persistent!` freezes back to a persistent value.
- **Not implemented.** JVM reflection, `proxy`, and the `clojure.repl`/`clojure.template` namespaces.
Supported and Clojure-compatible: chars as a distinct type, lazy/infinite sequences, transducers, destructuring, multimethods with hierarchies, protocols/records (`deftype`/`defrecord`/`reify`/`extend-protocol`), metadata, namespaces, and the reader (`#()`, `#_`, `#?`, tagged literals, `#"…"`).
- **No JVM, no Java interop.** No reflection, no `gen-class`/`proxy`. Interop
syntax (`Class.`, `Class/static`, `.method`) resolves only against a shimmed
subset of the `java.*` standard library; a class token is a name, not a loaded
class. See [docs/host-interop.md](docs/host-interop.md). To call C libraries
directly, use the `jolt.ffi` foreign-function interface (how the db and
http-client libraries bind SQLite/libpq and sockets/OpenSSL/zlib).
- **No `BigDecimal`.** `decimal?` is always false and there is no `M` literal;
the rest of the numeric tower matches the JVM.
- **No STM.** No `ref`/`dosync`/`alter`/`commute` — coordinated shared state uses
atoms (per-atom mutex, JVM-style CAS). The concurrency primitives above are
otherwise present and run on a shared heap.
- **Regex engine.** Patterns compile through
[irregex](https://github.com/ashinn/irregex) (vendored), not
`java.util.regex`; common patterns work, Java-specific features can differ.
- **Coverage.** `clojure.core` is implemented function by function against the
JVM-sourced conformance corpus — broad but not total; a namespace can load with
most functions working and a few not yet implemented.
## Test
```
jpm test # full suite (recurses test/)
janet test/spec/sequences-spec.janet # a single spec
janet test/integration/conformance-test.janet
```bash
make test # the full gate
make corpus # conformance corpus vs the JVM-sourced spec
make unit # host-specific unit cases
make selfhost # bootstrap fixpoint (rebuild == checked-in seed)
make smoke # bin/joltc CLI smoke
make sci # load borkdude/sci's source through joltc (compat stress)
make ffi # HTTP-server GC-safety + http-client temp paths
make transient # transient mutation + linear-time builds
make certify # JVM oracle (skips if clojure is absent)
```
Tests are organized in three layers:
- **`test/spec/`** — the contract. Black-box, behavior-defining tables (one file
per public API area) that collectively pin down Jolt's defined behavior. This
is the authoritative description of what Jolt promises.
- **`test/integration/`** — cross-cutting and regression batteries: the Clojure
conformance suite (run in all three execution modes), SCI bootstrap/runtime
loading, jank conformance, the cross-dialect
[clojure-test-suite](https://github.com/jank-lang/clojure-test-suite) (a git
submodule at `vendor/clojure-test-suite`, run via a minimal `clojure.test` shim
and baseline-guarded), compile-mode tests, the library API, and a broad
systematic-coverage net.
- **`test/unit/`** — white-box tests for individual components (reader,
evaluator, types, persistent collections, regex, compiler).
`test/support/harness.janet` provides the shared `defspec` table runner (cases
are `["label" expected actual]`, compared with Jolt's own `=`) plus
`expect=`/`expect-throws` for unit tests.
The syntactic half of the contract — the surface syntax the reader accepts — is
specified as an EBNF grammar in [`doc/grammar.ebnf`](doc/grammar.ebnf), with
Jolt-vs-Clojure deviations noted inline. `test/spec/reader-syntax-spec.janet`
exercises it.
### clojure-test-suite conformance
The [clojure-test-suite](https://github.com/jank-lang/clojure-test-suite) battery
(vendored as a git submodule) runs ~3980 assertions green. Jolt validates its
arguments like Clojure — arithmetic on non-numbers, comparisons against `nil`,
out-of-range indices, malformed `conj!`/`assoc!`/`merge`, non-seqable
`first`/`seq`/`vec`, and lazy transformers (`map`/`filter`/…) realized over a
non-seqable all throw. The lazy seq fns return seqs (not vectors), so
`seq?`/`vector?`/`sequential?` of their results match Clojure. The assertions
that remain failing are accounted for by the platform/design differences above,
not by missing behavior:
- **No bignum/ratio/BigDecimal**`bigint`/`numerator`/`denominator`/`bigdec`,
the `big-int?`/auto-promotion checks, and the `2N`/`1/2`/`1.0M` literals read
but don't carry those exact types.
- **Integer/float identity** — Janet represents `1` and `1.0` identically, so
`quot`/`rem`/`mod`'s `double?`/`int?` result-type assertions and many
`float?`/`double?` cases can't distinguish them (`(str 0.0)` is `"0"`).
- **64-bit integers / Unicode**`bit-and` etc. on full-width 64-bit constants
lose precision (doubles), and `subs`/`count` work on bytes, not code points.
The conformance corpus (`test/chez/corpus.edn`) is a host-neutral language spec
whose expected values are sourced from reference JVM Clojure. See
[test/conformance/SPEC.md](test/conformance/SPEC.md).
## License
[Eclipse Public License 1.0](https://opensource.org/licenses/EPL-1.0)
[Eclipse Public License 2.0](https://www.eclipse.org/legal/epl-2.0/)

1
bench/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
.cpcache/

120
bench/README.md Normal file
View file

@ -0,0 +1,120 @@
# jolt benchmark suite
Benchmarks that isolate the workload axes jolt's optimizing passes target. The
ray tracer (`examples/ray-tracer`) is **float-compute-bound** — its time is
irreducible algorithmic math (hit-testing + transcendentals), and devirt,
allocation removal, and type-proving all measured **flat** on it. So it can't
tell us whether those passes work. These benchmarks make each pass's target
workload the *dominant* cost.
Reference: the cross-language suites these draw from —
[Are We Fast Yet?](https://github.com/smarr/are-we-fast-yet) (Marr et al., DLS '16)
and the [Computer Language Benchmarks Game](https://benchmarksgame-team.pages.debian.net/benchmarksgame/).
The benchmarks are portable Clojure, so they also run on JVM Clojure for an
absolute reference.
## Benchmarks
| Benchmark | Axis | Pass it exercises | Source |
|---|---|---|---|
| `binary-trees` | allocation / GC pressure (escaping short-lived records) | scalar-replace, escape analysis | CLBG |
| `dispatch` | polymorphic (**megamorphic**) protocol dispatch | devirt, inline-cache | AWFY-style |
| `mono-dispatch` | **monomorphic** protocol dispatch (devirt/inline-cache *can* fire) | devirt, inline-cache | AWFY-style |
| `collections` | persistent map/vector churn (HAMT / 32-way tries) | persistent structures, transients | CLBG k-nucleotide-style |
| `mandelbrot` | pure float compute (tight arith loops, no alloc/dispatch) | native arith, loop codegen | CLBG |
| `fib` | recursion: function-call + integer-arith overhead | native arith, small-fn inlining | CLBG |
What the ray tracer does **not** capture and these do: allocation as the
bottleneck (~7% there), megamorphic *and* monomorphic dispatch (its dispatch is
monomorphic and cheap), persistent-collection throughput (it uses fixed records,
no collections in the hot loop), and isolated compute/call overhead.
Planned additions: Richards / DeltaBlue (heavier OO dispatch), NBody (float
control with record state), k-nucleotide proper.
## Holistic scorecard
`bench/run.sh` compiles each benchmark to an **optimized AOT binary** (`joltc build
--direct-link --opt`) and times it against JVM Clojure running the same portable
source — the jolt/JVM scorecard. jolt's optimizing passes fire only in a build;
`joltc run -m` is unoptimized, so the harness always builds.
Indicative ratios (M-series, single isolated run — numbers are machine-specific,
regenerate locally), ascending:
| benchmark | ratio | axis |
|---|---|---|
| `fib` | ~0.6× | call + integer arith |
| `collections` | ~3.5× | persistent map/vector churn |
| `mandelbrot` | ~7.5× | pure float compute |
| `binary-trees` | ~10× | escaping short-lived records (allocation/GC) |
| `dispatch` | ~12× | megamorphic protocol dispatch |
| `mono-dispatch` | ~15× | monomorphic protocol dispatch |
- **Compute (~0.67.5×)** is the substrate floor: Chez is a native-compiling AOT
Scheme, not a profiling JIT. With native arith + direct-linking + inlining jolt
is at parity here — `fib` runs *faster* than JVM Clojure (no JIT warmup over a
short run), `collections` is within ~3.5×, and `mandelbrot` (~7.5×) is the
pure-tight-loop float ceiling that only native codegen moves further.
- **Dispatch & allocation (~1015×)** are the remaining architectural gaps, though
the type-proving / native-record / bare-field-read work has collapsed them by an
order of magnitude (`binary-trees` ~140×→~10×, `mono-dispatch` ~330×→~15×). On a
*statically proven* monomorphic receiver — which whole-program inference now gives
for a record iterated out of a vector — devirt resolves the impl and a per-site
inline cache holds it (resolved once, not per call), so `mono-dispatch` is no
longer worse than megamorphic. The remaining lever is `dispatch`: a *megamorphic*
site has no static type, so it pays a full protocol-registry lookup every call
where the JVM uses a polymorphic inline cache — a runtime (receiver-type-keyed)
cache is the missing piece. `binary-trees`
nodes escape into the tree, so scalar-replace can't remove them — residual GC
pressure.
## 64-bit integer arithmetic & generators (test.check)
The AOT suite above is float-compute / dispatch / allocation bound; none of it
exercises **64-bit integer arithmetic**, which Chez can't hold in a fixnum
(61-bit), so genuine 64-bit values are heap bignums. The SplitMix PRNG behind
`clojure.test.check` is the worst case — every `rand-long` is ~8 bignum ops. These
were measured in **run mode** (`joltc run`, where per-site var-cell caching is on;
the AOT build keeps it off) against JVM Clojure on the same portable source. The
first two rows are isolating microbenchmarks; the rest are real test.check
generators.
| workload | jolt | JVM | ratio | bound by |
|---|---|---|---|---|
| SplitMix `mix-64` (×100k) | 45ms | 14ms | ~3.2× | 64-bit integer arithmetic |
| deftype alloc + protocol dispatch (×100k) | 41ms | 5ms | ~8× | open-world dispatch |
| raw `split` + `rand-long` (×20k) | 74ms | 6ms | ~12× | bignum 64-bit + dispatch |
| `gen/large-integer` (×2k) | 108ms | 23ms | ~4.7× | arithmetic + rose-tree machinery |
| `(gen/vector gen/large-integer)` (×500) | 1289ms | 88ms | ~14.6× | element gen + gen machinery |
Two no-C codegen levers collapsed the **arithmetic** half: emitting `bit-and`/
`bit-or`/`bit-xor`/`bit-not` as inlined Chez `bitwise-*` primitives (they had gone
through a var-deref'd variadic overlay), and caching the resolved var cell per
reference site (a name lookup was ~45ns/access). Together they took `mix-64` from
~18× → ~3.2× JVM and the raw PRNG from ~30× → ~12×, and the generators ~1.6× each.
The residual gap is **machinery, not arithmetic**: the open-world generator
deftype/protocol dispatch + rose-tree allocation (~810×) can't be devirtualized
without static types, and the raw 64-bit ops bottom out at the Chez bignum floor
(~20× a native long, substrate-inherent). A native SplitMix C/FFI shim would give
the PRNG ~27× but is the only path that needs C.
## Running
```sh
bench/run.sh # full suite + JVM scorecard
bench/run.sh fib # one benchmark, default size
bench/run.sh fib 32 # one benchmark, custom size
NO_JVM=1 bench/run.sh # jolt only (skip the JVM reference)
```
Needs Chez's kernel dev files (`libkernel.a` + `scheme.h`) and `cc` for the build,
like `jolt build`; set `JOLT_CHEZ_CSV` to override the detected csv dir.
## A/B against a change
To measure a pass, run the suite on `main`, then on the branch, back to back
(same machine, quiet). Each benchmark prints `runs: [...]` and `mean: N ms`;
compare the means. A pass is worth landing when it moves a benchmark whose axis it
targets, even if the ray tracer stays flat.

53
bench/binary_trees.clj Normal file
View file

@ -0,0 +1,53 @@
;; binary-trees (Computer Language Benchmarks Game) — an ALLOCATION/GC stress
;; test. Builds and discards millions of short-lived `Node` records; the nodes
;; ESCAPE (stored in the tree, walked later), so this is the regime escape analysis
;; targets and the ray tracer never exercises (~7% alloc).
;;
;; Portable Clojure: runs on jolt and JVM Clojure for cross-impl comparison.
;; bench/run.sh binary-trees 14
(ns binary-trees)
(defrecord Node [left right])
(defn make-tree [depth]
(if (zero? depth)
(->Node nil nil)
(->Node (make-tree (dec depth)) (make-tree (dec depth)))))
(defn check-tree [node]
(let [l (:left node)]
(if (nil? l)
1
(+ (+ 1 (check-tree l)) (check-tree (:right node))))))
(defn run [max-depth]
(let [min-depth 4
stretch-depth (inc max-depth)
_ (check-tree (make-tree stretch-depth))
long-lived (make-tree max-depth)]
(loop [d min-depth acc 0]
(if (<= d max-depth)
(let [iterations (bit-shift-left 1 (+ (- max-depth d) min-depth))
sum (loop [i 0 s 0]
(if (< i iterations)
(recur (inc i) (+ s (check-tree (make-tree d))))
s))]
(recur (+ d 2) (+ acc sum)))
;; touch the long-lived tree so it isn't dead-code-eliminated
(+ acc (check-tree long-lived))))))
(defn -main [& args]
(let [max-depth (if (seq args) (Integer/parseInt (first args)) 14)]
(dotimes [_ 2] (run (min max-depth 10))) ; warmup
(let [runs 3
times (mapv (fn [_]
(let [t0 (System/nanoTime)
r (run max-depth)
ms (/ (- (System/nanoTime) t0) 1000000.0)]
[ms r]))
(range runs))
mss (mapv first times)
mean (/ (reduce + mss) runs)]
(println "binary-trees depth" max-depth "checksum" (second (first times)))
(println "runs:" (mapv (fn [t] (/ (Math/round (* t 10.0)) 10.0)) mss))
(println "mean:" (/ (Math/round (* mean 10.0)) 10.0) "ms"))))

46
bench/collections.clj Normal file
View file

@ -0,0 +1,46 @@
;; collections — PERSISTENT-COLLECTION churn. Builds and reads persistent maps
;; and vectors (32-way hash/array tries) under heavy assoc/update/conj/lookup, a
;; word-count-style workload (cf. CLBG k-nucleotide). Exercises jolt's persistent
;; data structures and (eventually) transients — an axis the ray tracer (fixed
;; records, no collections in the hot loop) doesn't touch.
;;
;; Portable Clojure (jolt + JVM Clojure).
;; bench/run.sh collections 200000
(ns collections)
;; map churn: accumulate a frequency map over a stream of keys, then sum it back
(defn freq-map [n buckets]
(loop [i 0 m {}]
(if (< i n)
(recur (inc i)
(let [k (mod (* i 2654435761) buckets)]
(assoc m k (+ 1 (get m k 0)))))
m)))
(defn sum-vals [m]
(reduce (fn [acc k] (+ acc (get m k))) 0 (keys m)))
;; vector churn: conj many, then reduce
(defn vec-sum [n]
(let [v (loop [i 0 v []] (if (< i n) (recur (inc i) (conj v (mod i 1000))) v))]
(reduce + 0 v)))
(defn run [n]
(let [m (freq-map n 4096)]
(+ (sum-vals m) (vec-sum (quot n 4)))))
(defn -main [& args]
(let [n (if (seq args) (Integer/parseInt (first args)) 200000)]
(dotimes [_ 2] (run (quot n 4))) ; warmup
(let [runs 3
times (mapv (fn [_]
(let [t0 (System/nanoTime)
r (run n)
ms (/ (- (System/nanoTime) t0) 1000000.0)]
[ms r]))
(range runs))
mss (mapv first times)
mean (/ (reduce + mss) runs)]
(println "collections n" n "result" (second (first times)))
(println "runs:" (mapv (fn [t] (/ (Math/round (* t 10.0)) 10.0)) mss))
(println "mean:" (/ (Math/round (* mean 10.0)) 10.0) "ms"))))

1
bench/deps.edn Normal file
View file

@ -0,0 +1 @@
{:paths ["."]}

56
bench/dispatch.clj Normal file
View file

@ -0,0 +1,56 @@
;; dispatch — a POLYMORPHIC-DISPATCH stress test. A protocol method is called in
;; a hot loop over a heterogeneous (megamorphic) collection of record types, with
;; minimal per-call work, so protocol dispatch dominates. This is the regime
;; devirtualization and the inline-cache target, and the one the ray
;; tracer can't reveal — its dispatch is monomorphic and a small fraction of the
;; float-math cost (devirt measured FLAT there).
;;
;; Portable Clojure (jolt + JVM Clojure).
;; bench/run.sh dispatch 20000
(ns dispatch)
(defprotocol Shape
(area [s])
(sides [s]))
(defrecord Circle [r] Shape (area [_] (* (* 3.14159 r) r)) (sides [_] 0))
(defrecord Square [s] Shape (area [_] (* s s)) (sides [_] 4))
(defrecord Triangle [b h] Shape (area [_] (* (* 0.5 b) h)) (sides [_] 3))
(defrecord Rect [w h] Shape (area [_] (* w h)) (sides [_] 4))
(defn build-shapes [n]
(mapv (fn [i]
(let [k (mod i 4)]
(cond
(= k 0) (->Circle (+ 1 (mod i 7)))
(= k 1) (->Square (+ 1 (mod i 5)))
(= k 2) (->Triangle (+ 1 (mod i 3)) (+ 2 (mod i 6)))
:else (->Rect (+ 1 (mod i 4)) (+ 1 (mod i 8))))))
(range n)))
;; megamorphic: every element may be a different type -> the call site sees all 4
(defn sum-area [shapes]
(reduce (fn [acc s] (+ (+ acc (area s)) (sides s))) 0.0 shapes))
(defn run [iters]
(let [shapes (build-shapes 1000)]
(loop [i 0 acc 0.0]
(if (< i iters)
(recur (inc i) (+ acc (sum-area shapes)))
acc))))
(defn -main [& args]
(let [iters (if (seq args) (Integer/parseInt (first args)) 20000)]
(dotimes [_ 2] (run (quot iters 4))) ; warmup
(let [runs 3
times (mapv (fn [_]
(let [t0 (System/nanoTime)
r (run iters)
ms (/ (- (System/nanoTime) t0) 1000000.0)]
[ms r]))
(range runs))
mss (mapv first times)
mean (/ (reduce + mss) runs)]
(println "dispatch iters" iters "result" (second (first times)))
(println "runs:" (mapv (fn [t] (/ (Math/round (* t 10.0)) 10.0)) mss))
(println "mean:" (/ (Math/round (* mean 10.0)) 10.0) "ms"))))

29
bench/fib.clj Normal file
View file

@ -0,0 +1,29 @@
;; fib — naive recursive Fibonacci: pure function-call + integer-arithmetic
;; throughput, with no allocation, dispatch, or collections. Isolates call
;; overhead and native integer arith, and is the natural target for
;; single-call-site / small-fn inlining and self-call direct-linking.
;;
;; Portable Clojure (jolt + JVM Clojure).
;; bench/run.sh fib 32
(ns fib)
(defn fib [n]
(if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
(defn run [n] (fib n))
(defn -main [& args]
(let [n (if (seq args) (Integer/parseInt (first args)) 32)]
(dotimes [_ 2] (run (- n 6))) ; warmup
(let [runs 3
times (mapv (fn [_]
(let [t0 (System/nanoTime)
r (run n)
ms (/ (- (System/nanoTime) t0) 1000000.0)]
[ms r]))
(range runs))
mss (mapv first times)
mean (/ (reduce + mss) runs)]
(println "fib n" n "result" (second (first times)))
(println "runs:" (mapv (fn [t] (/ (Math/round (* t 10.0)) 10.0)) mss))
(println "mean:" (/ (Math/round (* mean 10.0)) 10.0) "ms"))))

52
bench/mandelbrot.clj Normal file
View file

@ -0,0 +1,52 @@
;; mandelbrot — pure floating-point compute: for each point of an NxN grid,
;; iterate z = z^2 + c up to a cap and count iterations. No allocation, no
;; dispatch, no collections in the hot loop — just double arithmetic and tight
;; recur loops. This isolates the irreducible-math axis the ray tracer is bound
;; on (where devirt/alloc passes measured flat), so it tracks native-arith codegen
;; and loop quality directly.
;;
;; Portable Clojure (jolt + JVM Clojure). The jolt.png picture demo lives in
;; mandelbrot_png.clj so this file stays portable for the JVM reference run.
;; bench/run.sh mandelbrot 1000
(ns mandelbrot)
(defn count-point [cr ci cap]
(loop [i 0 zr 0.0 zi 0.0]
(if (or (>= i cap) (> (+ (* zr zr) (* zi zi)) 4.0))
i
(recur (inc i)
(+ (- (* zr zr) (* zi zi)) cr)
(+ (* 2.0 (* zr zi)) ci)))))
(defn run [n]
(let [cap 200
nd (* 1.0 n)]
(loop [y 0 acc 0]
(if (< y n)
(let [ci (- (/ (* 2.0 y) nd) 1.0)
row (loop [x 0 a 0]
(if (< x n)
(let [cr (- (/ (* 2.0 x) nd) 1.5)]
(recur (inc x) (+ a (count-point cr ci cap))))
a))]
(recur (inc y) (+ acc row)))
acc))))
(defn- run-bench [args]
(let [n (if (seq args) (Integer/parseInt (first args)) 1000)]
(dotimes [_ 2] (run (quot n 2))) ; warmup
(let [runs 3
times (mapv (fn [_]
(let [t0 (System/nanoTime)
r (run n)
ms (/ (- (System/nanoTime) t0) 1000000.0)]
[ms r]))
(range runs))
mss (mapv first times)
mean (/ (reduce + mss) runs)]
(println "mandelbrot n" n "result" (second (first times)))
(println "runs:" (mapv (fn [t] (/ (Math/round (* t 10.0)) 10.0)) mss))
(println "mean:" (/ (Math/round (* mean 10.0)) 10.0) "ms"))))
(defn -main [& args]
(run-bench args))

36
bench/mandelbrot_png.clj Normal file
View file

@ -0,0 +1,36 @@
;; mandelbrot picture demo — renders a real image of the set to a PNG via
;; jolt.png (FFI), reusing mandelbrot/count-point as the kernel. jolt-only (the
;; benchmark in mandelbrot.clj stays portable for the JVM reference).
;; joltc run -m mandelbrot-png [path] [size]
(ns mandelbrot-png
(:require [mandelbrot :as m]
[jolt.png :as png]))
(defn- color
"Escape-iteration count -> RGB. In-set points (n>=cap) are black; faster
escapes run through a warm gradient."
[n cap]
(if (>= n cap)
[0 0 0]
(let [t (/ (double n) cap)]
[(int (* 255 (min 1.0 (* 3.0 t))))
(int (* 255 (min 1.0 (max 0.0 (* 3.0 (- t 0.33))))))
(int (* 255 (min 1.0 (max 0.0 (* 3.0 (- t 0.66))))))])))
(defn render!
"Render a size×size view of the Mandelbrot set to a PNG at path."
[path size]
(let [w size h size cap 1000
img (png/image w h)]
(doseq [py (range h)]
(doseq [px (range w)]
(let [cr (- (* 3.5 (/ (double px) w)) 2.5) ; real ∈ [-2.5, 1.0]
ci (- (* 2.8 (/ (double py) h)) 1.4) ; imag ∈ [-1.4, 1.4]
[r g b] (color (m/count-point cr ci cap) cap)]
(png/put! img r g b))))
(png/write img w h path)
(println "wrote" path (str w "×" h ", cap " cap))))
(defn -main [& args]
(render! (or (first args) "mandelbrot.png")
(if (second args) (Integer/parseInt (second args)) 600)))

45
bench/mono_dispatch.clj Normal file
View file

@ -0,0 +1,45 @@
;; mono-dispatch — protocol dispatch where every call site sees ONE record type
;; (monomorphic). This is the regime where devirtualization and a
;; call-site inline cache CAN fire — the megamorphic `dispatch` bench deliberately
;; defeats them, so this is its complement: it measures how close a proven/cached
;; monomorphic dispatch gets to a direct call. Same per-call work as `dispatch`.
;;
;; Portable Clojure (jolt + JVM Clojure).
;; bench/run.sh mono-dispatch 20000
(ns mono-dispatch)
(defprotocol Shape
(area [s])
(sides [s]))
(defrecord Circle [r] Shape (area [_] (* (* 3.14159 r) r)) (sides [_] 0))
;; homogeneous: every element is a Circle -> the call site is monomorphic
(defn build-shapes [n]
(mapv (fn [i] (->Circle (+ 1 (mod i 7)))) (range n)))
(defn sum-area [shapes]
(reduce (fn [acc s] (+ (+ acc (area s)) (sides s))) 0.0 shapes))
(defn run [iters]
(let [shapes (build-shapes 1000)]
(loop [i 0 acc 0.0]
(if (< i iters)
(recur (inc i) (+ acc (sum-area shapes)))
acc))))
(defn -main [& args]
(let [iters (if (seq args) (Integer/parseInt (first args)) 20000)]
(dotimes [_ 2] (run (quot iters 4))) ; warmup
(let [runs 3
times (mapv (fn [_]
(let [t0 (System/nanoTime)
r (run iters)
ms (/ (- (System/nanoTime) t0) 1000000.0)]
[ms r]))
(range runs))
mss (mapv first times)
mean (/ (reduce + mss) runs)]
(println "mono-dispatch iters" iters "result" (second (first times)))
(println "runs:" (mapv (fn [t] (/ (Math/round (* t 10.0)) 10.0)) mss))
(println "mean:" (/ (Math/round (* mean 10.0)) 10.0) "ms"))))

70
bench/run.sh Executable file
View file

@ -0,0 +1,70 @@
#!/bin/sh
# Run the jolt benchmark suite against JVM Clojure and print a jolt/JVM scorecard.
#
# jolt's optimizing passes (direct-linking, inlining, scalar-replace, whole-program
# inference) fire only in an AOT BUILD — `joltc run -m` is unoptimized — so each
# benchmark is compiled to an optimized standalone binary and timed. JVM Clojure
# runs the same portable source for the absolute reference. Each benchmark prints
# `runs: [...]` and `mean: N ms`; the table shows the means and the jolt/JVM ratio.
#
# bench/run.sh # full suite + JVM scorecard
# bench/run.sh fib # one benchmark, default size
# bench/run.sh fib 32 # one benchmark, custom size
# NO_JVM=1 bench/run.sh # jolt only (skip the JVM reference)
#
# Building needs Chez's kernel dev files (libkernel.a + scheme.h) and a C compiler,
# the same as `jolt build`; set JOLT_CHEZ_CSV to override the detected csv dir.
set -e
cd "$(dirname "$0")"
root="$(cd .. && pwd)"
joltc="$root/bin/joltc"
export JOLT_PWD="$PWD"
# Locate Chez's kernel dev files for the optimized build (as build-smoke.sh does).
csv="$JOLT_CHEZ_CSV"
if [ -z "$csv" ]; then
chez_bin="$(command -v chez || command -v scheme || command -v petite || true)"
if [ -n "$chez_bin" ]; then
base="$(cd "$(dirname "$chez_bin")/.." 2>/dev/null && pwd)"
for d in "$base"/lib/csv*/*/; do
[ -f "${d}libkernel.a" ] && csv="${d%/}" && break
done
fi
fi
if [ -z "$csv" ] || [ ! -f "$csv/libkernel.a" ] || [ ! -f "$csv/scheme.h" ] || ! command -v cc >/dev/null 2>&1; then
echo "error: the optimized build needs Chez kernel dev files (libkernel.a + scheme.h) and cc." >&2
echo " set JOLT_CHEZ_CSV to the csv dir, e.g. \$(brew --prefix chezscheme)/lib/csv*/<machine>." >&2
exit 1
fi
export JOLT_CHEZ_CSV="$csv"
bindir="$(mktemp -d)"
trap 'rm -rf "$bindir"' EXIT
# name:default-arg, each sized to run in a few seconds. Axes: see README.md.
BENCHES="fib:30 mandelbrot:200 collections:30000 mono-dispatch:2000 dispatch:2000 binary-trees:14"
run_one() {
ns="${1%%:*}"; arg="${2:-${1##*:}}"
if ! "$joltc" build -m "$ns" -o "$bindir/$ns" --direct-link --opt >/dev/null 2>&1; then
printf '%-16s jolt build FAILED\n' "$ns"; return
fi
jmean=$("$bindir/$ns" "$arg" 2>/dev/null | awk '/^mean:/{print $2}')
if [ -z "$NO_JVM" ]; then
vmean=$(clojure -Sdeps '{:paths ["."]}' -M -m "$ns" "$arg" 2>/dev/null | awk '/^mean:/{print $2}')
ratio=$(awk "BEGIN{ if (\"$vmean\"+0>0 && \"$jmean\"+0>0) printf \"%.1fx\", (\"$jmean\"+0)/(\"$vmean\"+0); else printf \"-\" }")
printf '%-16s jolt %9s ms jvm %8s ms %s\n' "$ns" "${jmean:--}" "${vmean:--}" "$ratio"
else
printf '%-16s jolt %9s ms\n' "$ns" "${jmean:--}"
fi
}
if [ -n "$1" ]; then
spec=""
for s in $BENCHES; do [ "${s%%:*}" = "$1" ] && spec="$s"; done
[ -n "$spec" ] || { echo "unknown benchmark: $1 (have: ${BENCHES})" >&2; exit 1; }
run_one "$spec" "$2"
else
echo "jolt benchmark suite — optimized AOT binaries${NO_JVM:+ }${NO_JVM:-, vs JVM Clojure}"
for spec in $BENCHES; do run_one "$spec"; done
fi

41
bin/joltc Executable file
View file

@ -0,0 +1,41 @@
#!/bin/sh
# joltc — the pure-Chez jolt runtime. NO Janet.
#
# Compiles and evaluates jolt (Clojure) on Chez using the checked-in bootstrap
# seed (host/chez/seed/). With only Chez installed it runs jolt end to end:
#
# joltc -e "(+ 1 2)" evaluate an expression
# joltc run -m app.core [args] resolve deps.edn, run a namespace's -main
# joltc -M:alias [args] run an alias's :main-opts
# joltc repl | path | <task> REPL, print roots, or a deps.edn task
#
# The launcher cd's to the jolt repo root so the runtime's relative loads work;
# the user's original cwd (the project dir, where deps.edn lives) is passed in
# JOLT_PWD.
root="$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)"
export JOLT_PWD="${JOLT_PWD:-$PWD}"
# Identify the Chez Scheme executable
while read -r CHEZ
do
if [ `which ${CHEZ}` ]
then
break;
fi
done <<EOF
chez
chezscheme
EOF
# If we failed to find one, whinge and exit.
if [ ! `which ${CHEZ}` ]
then
echo "No valid Chez Scheme executable found: please install Chez Scheme."
exit 1
fi
# Version for --version / banners: git describe of this checkout, else "dev".
export JOLT_VERSION="${JOLT_VERSION:-$(git -C "$root" describe --tags --always --dirty 2>/dev/null || echo dev)}"
cd "$root" || exit 1
exec ${CHEZ} --script host/chez/cli.ss "$@"

View file

@ -1,117 +0,0 @@
# Building and dependencies
How to build Jolt from source and how to pull Clojure libraries into a project.
## Building
```bash
git clone https://github.com/jolt-lang/jolt.git
cd jolt
git submodule update --init # vendor/sci (used by the SCI bootstrap tests)
jpm build
```
This produces two executables under `build/`:
- **`jolt`** — the runtime: REPL, file/expr runner, nREPL server. The whole `.clj`
standard library (`clojure.string`/`set`/`walk`/`edn`/`zip`, `jolt.http`/
`interop`/`shell`/`nrepl`) is baked into this binary at build time, so it loads
from any directory — the build artifact is self-contained. (`clojure.core` is
built into the runtime in Janet and auto-referred, so it's always available.)
- **`jolt-deps`** — a separate tool that resolves a `deps.edn` (see below). It
sits beside the runtime the way `jpm` sits beside `janet`; the runtime itself
knows nothing about deps.edn.
Needs `jpm` and a recent Janet — developed and CI-tested against **1.41**. The
futures and core.async layers use Janet's threaded `ev/` channels (`ev/thread`,
`ev/thread-chan`), so older Janets may not run the full suite.
`jpm build` doesn't always notice source changes; run `jpm clean && jpm build`
after editing `src/` to be sure the binaries are current. `jpm test` runs against
the source directly, so it never goes stale.
## How namespaces are found
`(require ...)` resolves a namespace to a file by searching an ordered list of
source roots — the stdlib first, then any extra roots — trying `<ns>.clj` then
`<ns>.cljc` (dots become directories, dashes become underscores). Extra roots
come from:
- `JOLT_PATH` — a colon-separated list of directories (like a classpath), applied
at runtime;
- the `:paths` option to `init` when embedding Jolt as a library.
If a namespace isn't found on any root, the loader falls back to the stdlib baked
into the binary — that's how `clojure.string` and friends resolve when you run
the binary outside the source tree.
So you can point Jolt at a directory of Clojure source with no deps machinery at
all:
```bash
JOLT_PATH=/path/to/lib/src build/jolt myfile.clj
```
## Dependencies via deps.edn
`jolt-deps` reads a `deps.edn` in the current directory, fetches its
dependencies, and runs `jolt` with the resolved source directories on
`JOLT_PATH`.
```bash
jolt-deps path # print the resolved roots (':'-joined)
jolt-deps run FILE [args] # resolve, then run `jolt FILE …`
jolt-deps repl # resolve, then start a REPL
jolt-deps -e EXPR [args] # resolve, then evaluate EXPR
```
`jolt-deps` launches the `jolt` binary it finds on `PATH` (override with
`$JOLT_BIN`).
Example `deps.edn`:
```clojure
{:paths ["src"]
:deps {weavejester/medley {:git/url "https://github.com/weavejester/medley"
:git/tag "1.0.0"}
my/helpers {:local/root "../helpers"}}}
```
```bash
jolt-deps run -m myapp.main
```
### What's supported
- **git deps**`{:git/url … :git/tag …}` or `{:git/url … :git/sha …}` (use a
full SHA; `git fetch` can't resolve a short one). Transitive deps from each
dependency's own `deps.edn` are resolved too.
- **local deps**`{:local/root "../path"}`.
- The project's own `:paths` (default `["src"]`) are included.
Resolution reuses jpm's git fetch and cache (a dependency is cloned once into
`jpm_tree/.cache` and reused). Resolved roots are cached on a hash of `deps.edn`,
so an unchanged `deps.edn` doesn't re-fetch.
### What's not
- **No Maven.** `:mvn/version` deps are ignored — git and local only.
- **Pure `clj`/`cljc` only.** A library that needs the JVM (Java interop, host
classes) or a `clojure.core` feature Jolt doesn't implement will fail to load
or fail at a call. Coverage is per-function: a namespace can load with most
functions working and a few not.
### Bundling into one file
`jolt uberscript OUT.clj -m NS` (or `jolt-deps uberscript …`, which resolves deps
first) bundles `NS` and every namespace it requires — your code plus its
dependencies — into a single `.clj` in dependency order, ending with a call to
`NS/-main`. The result runs on a plain `jolt` with no `JOLT_PATH`, no deps
fetched, and no jpm:
```bash
jolt-deps uberscript app.clj -m myapp.main
jolt app.clj arg1 arg2
```
See [`tools-deps.md`](tools-deps.md) for the design rationale.

View file

@ -1,138 +0,0 @@
# Self-hosting architecture: portable jolt-core over a host runtime
Design for splitting Jolt into a **portable Clojure-in-Clojure core** and a
**host runtime** (Janet today, another runtime tomorrow), so the language is
truly self-hosted and `jolt-core` can be lifted out and re-hosted.
This is the design that must be right *before* writing the compiler in Clojure —
see [[self-hosting-compiler]] for the staged plan it plugs into.
## What "truly self-hosted + portable" requires
Two independent properties:
1. **Self-hosted** — the compiler and most of `clojure.core` are written in
Clojure and compiled by Jolt itself.
2. **Portable** — that Clojure code (`jolt-core`) depends only on a small,
explicit **host contract**, never on Janet directly. Re-hosting means
implementing the contract for a new runtime; `jolt-core` is reused verbatim.
The enemy is `jolt-core` calling `janet/tuple`, `make-vec`, `ns-find`, etc.
directly — that welds it to Janet. Every host dependency must go through the
contract.
## Prior art (the seam everyone uses)
- **Clojure (JVM).** `clojure.lang.*` (Java) is the host: `RT`/`Numbers` runtime
helpers, the `Compiler` (form → JVM bytecode), persistent data structures,
`Var`/`Namespace`, the reader. `clojure/core.clj` is the language, in Clojure.
Seam: ~20 primitive special forms + `RT` static methods. Everything else is
Clojure.
- **ClojureScript (self-hosted).** Two portable passes — `cljs.analyzer`
(form → AST **as data**, reading a **compiler-state map** of
namespaces/defs/macros, *not* host objects) and `cljs.compiler` (AST → JS, the
host-specific back end). `cljs.core` is Clojure compiled to JS. Platform splits
live in `.cljc` reader conditionals. This is the closest model to what we want:
**the analyzer is host-agnostic; only the back end and the runtime are
host-specific.**
- **Nanopass / Guile Tree-IL.** A high-level IR is the portability seam; multiple
back ends consume it.
- **ClojureCLR / ClojureDart / jank.** Same shape every time: portable analyzer +
host back end + host runtime.
The invariant across all of them: **the IR (analyzer output) and a small runtime
protocol are the contract; the front end is portable, the back end and runtime
are per-host.**
## Decisions (locked)
- **Seam = a minimal host protocol.** `jolt-core` calls a small documented set of
host fns (in ns `jolt.host`): `resolve-sym`, `macro?`, `macroexpand-1`,
`current-ns`, `intern!`, plus the `RT` primitives. Each host provides `jolt.host`
(+ RT). Re-hosting = reimplement that handful of fns. The protocol *is* the
boundary; `jolt-core` never touches Janet directly.
- **Physical split now.** Portable Clojure lives under `jolt-core/` (a new source
root, embedded into the binary like the rest of the stdlib); host Janet code for
the new pipeline under `host/janet/`. Legacy host modules under `src/jolt/*.janet`
are the existing Janet host and get relocated under `host/janet/` in a later
mechanical pass (tracked) — not moved big-bang now, to keep the suite green.
## The Jolt split
```
jolt-core/ PORTABLE Clojure — no Janet. Depends only on the contract.
ir the IR spec (data shapes the analyzer emits)
analyzer form -> IR (macroexpands; resolves via host protocol)
macros when/cond/->/defn/... (the macro library, in Clojure)
core clojure.core fns expressible in Clojure, over RT primitives
host/janet/ THE HOST — Janet. Implements the contract.
reader text -> jolt forms
rt data structures + RT primitive fns (cons/first/+/get/apply…)
backend IR -> Janet forms -> Janet compile -> bytecode (the emitter)
cenv the compile-time host protocol impl (resolve/macro?/intern)
bootstrap load jolt-core, wire analyzer+backend into the loader
interop janet.* bridge
```
Two contracts cross the seam:
### 1. The IR (analyzer → back end)
The existing `:op`-tagged AST, made **host-neutral**:
- `{:op :const :val v}`, `:if`, `:do`, `:let`, `:fn` (arities), `:invoke`,
`:vector`/`:map`/`:set`, `:quote`, `:throw`/`:try`, `:loop`/`:recur`.
- **Globals reference vars by NAME, not by host cell:**
`{:op :var :ns "clojure.core" :name "map"}`. (compiler.janet today embeds the
Janet var cell as a constant — that's a host leak and breaks AOT. Name-based
refs are both portable and AOT-friendly; the back end resolves the cell.)
- No embedded host function values. Calls to runtime primitives are
`{:op :rt :name "cons"}` resolved by the back end to the host's RT fn.
### 2. The host contract (two protocols)
- **Compile-time (`cenv`)** — what the analyzer needs from the host while
analyzing: `(current-ns)`, `(resolve-sym sym) -> {:kind :var|:macro|:local|:special|:host, :ns, :name}`,
`(macroexpand-1 form)`, `(intern! ns sym meta)`. The analyzer calls only these;
it never touches Janet ns/var tables. (CLJS keeps this as pure data; we use a
small protocol — a minimal, documented boundary — because Jolt already has live
ns/var objects. The protocol *is* the seam.)
- **Runtime (`RT`)** — the primitive fns emitted code and `jolt-core` call by
stable name: arithmetic/compare, `cons/first/rest/seq/conj/get/assoc/count`,
`apply`, `=`, vector/map/set constructors, var deref/bind, keyword/symbol
construction. The back end maps each to the host (on Janet, mostly the existing
`core-*`). To re-host, implement this set.
## Why name-based vars (not embedded cells)
`compiler.janet` compiles a global ref to a closure over the Janet var cell. That
(a) is a Janet value baked into the IR — not portable, and (b) can't be marshaled
for AOT without the runtime-dict trick. Compiling instead to *resolve var by
(ns,name) at call time* through an RT primitive keeps redefinition live, makes the
IR host-neutral, and makes images trivially portable. The per-call lookup is the
cost; it can be cached/direct-linked later as an opt-in optimization.
## Bootstrap & staging (keeps the suite green throughout)
`compiler.janet` stays as the **bootstrap back end** until the Clojure pipeline is
proven. Order:
1. **Freeze the IR** spec and refactor `compiler.janet`'s emit to consume
name-based `:var` (no behavior change; bootstrap still works).
2. **Define the host contract** (`cenv` + `RT`) and implement it on Janet,
exposed under a stable namespace the Clojure core can call.
3. **Write `jolt.analyzer` in Clojure** producing IR, against `cenv`. Diff its IR
against the Janet analyzer on the conformance corpus until identical.
4. **Janet back end consumes IR** from the Clojure analyzer; wire into the loader
behind a flag. Validate at parity (dual-mode conformance + clojure-test-suite).
5. **Flip** the loader to the Clojure analyzer + Janet back end; `compiler.janet`
shrinks to the back end only.
6. **Move `clojure.core`** macros then fns into `jolt-core` incrementally, each
compiled by the prior stage, isolating host bits behind `RT`.
Guards at every step: the dual-mode conformance harness (interpret vs compile)
and the clojure-test-suite baseline.
## The portability test
When done, re-hosting Jolt to runtime X means writing only: `host/X/{reader, rt,
backend, cenv, bootstrap}`. `jolt-core/{ir, analyzer, macros, core}` is reused
unchanged. That is the concrete bar for "truly self-hosted and portable."

View file

@ -1,172 +0,0 @@
# Toward a self-hosting Jolt compiler
Research and design notes for evolving Jolt from "interpreter + opt-in ad-hoc
compiler" toward a self-hosting Clojure-in-Clojure compiler that emits Janet
bytecode, keeps full REPL live-redefinition, and rests on a minimal Janet
bootstrap. This is a design doc, not a changelog — it describes where we are, the
prior art, the constraints we verified, and a recommended path.
## The goal
- **Self-hosting, Clojure-in-Clojure.** A small kernel in the host (Janet) is
enough to start; the rest of Clojure — including the compiler — is written in
Clojure and compiled by Jolt itself, growing the language as it compiles more
of itself.
- **Janet bytecode out.** Compiled code runs as native Janet bytecode (fast),
not tree-walking.
- **Full runtime flexibility.** `def`/`defn` redefinition, vars, protocols,
multimethods, and everything else stay live and redefinable at the REPL even
for compiled code.
- **Minimal host requirement.** Shrink what must exist in Janet to the
irreducible base.
## Where Jolt is today
- ~5,500 lines of **Janet** implement `clojure.core` (`core.janet`) and a
tree-walking interpreter (`evaluator.janet`); ~1k lines of **Clojure** are the
stdlib (`clojure.string/set/walk/…`, `jolt.*`). So the language is mostly in
the host, inverted from the Clojure-in-Clojure ideal.
- The interpreter (`eval-form`) is the complete reference path.
- The compiler (`compiler.janet`) — `analyze-form` (reader form → `:op` AST) →
`emit` (AST → Janet form) → Janet `compile`/`eval` — is now **on by default**
in the shipped runtime (`JOLT_INTERPRET=1` opts out). It is a *hybrid*: forms
it can't compile correctly throw `jolt/uncompilable` and fall back to the
interpreter (`loader/eval-toplevel`), so results always match the interpreter.
Validated at parity — conformance 218/218 under both interpret and compile, and
the clojure-test-suite under compile passes 3932 (vs the 3913 interpreter
baseline) across ~4.6k assertions.
- Done so far: var-indirection (globals deref through var cells, so compiled code
is REPL-redefinable); hybrid fallback; compilation of multi-arity / named /
variadic fns and `recur` inside `fn`; map and vector literal compilation
(mode-correct via `make-vec` / `build-map-literal`); resolution that mirrors
the interpreter (current ns → `clojure.core` → Janet-env fallback); and AOT
(`aot.janet`) that marshals a compiled namespace to a Janet bytecode image
against the baked-in runtime dictionary and loads it back.
- Still open — the actual self-hosting: the compiler and most of `clojure.core`
are still Janet. Rewriting them in Clojure (compiled by Jolt) is the remaining
Clojure-in-Clojure work.
## What the host gives us (verified)
Janet already is the backend and the AOT story — we don't need a custom bytecode
emitter:
- `(compile form env source)` → a **function** (compiled bytecode). Jolt's job is
Clojure form → correct Janet form → `compile`.
- `marshal`/`unmarshal`, `make-image`/`load-image` → serialize a compiled
environment to a **bytecode image** and load it back: this is Phase 4 AOT.
- `asm`/`disasm` → bytecode assembler/disassembler if we ever want to bypass the
form layer (we shouldn't need to).
**The catch we verified:** Janet *early-binds* top-level references. Compile
`(defn caller [] foo)`, then redefine `foo` — the compiled `caller` still returns
the old value. So emitting Jolt globals as plain Janet symbols (what the current
compiler largely does) is fundamentally incompatible with REPL redefinition. This
is the single most important design constraint below.
## Prior art
- **Clojure (JVM).** A Java runtime + compiler bootstraps `clojure.core`, which is
written in Clojure; thereafter Clojure compiles Clojure to JVM bytecode. Only
~20 special forms are primitive; everything else is macros/functions. Crucially,
compiled call sites go **through Var objects** (a deref), so redefining a var is
visible to existing compiled callers — that's how speed and live redefinition
coexist. Clojure 1.8 added opt-in **direct linking** (inline the call, drop the
var indirection) for speed where you don't need redefinition (used for core in
production). AOT compiles namespaces to `.class` files.
- **ClojureScript self-hosting.** Two stages: an **analyzer** (source → AST plus
a "compiler state" map of namespaces/defs/macros) and a **compiler** (AST → JS).
`cljs.js` exposes compile/eval at runtime; bootstrapped CLJS compiles CLJS at
~2× the JVM compiler. The host VM (JS engine) is the backend — the same shape we
want with Janet as the backend.
- **Nanopass (Chez Scheme).** A compiler as *many small passes* over *formally
specified* intermediate languages, with autogenerated boilerplate to recur
through unchanged forms and checks that each pass's output matches its grammar.
The lesson for "grow the language as it compiles itself": keep passes small and
IRs explicit so adding a form is local and verifiable.
- **Guile.** A Lisp on a bytecode VM: source → Tree-IL (high-level IR) → CPS
(optimization IR) → VM bytecode, with several front-end languages targeting
Tree-IL. The closest analog to "Lisp → bytecode on a VM."
## Assessment: is the current approach the right one?
The overall *shape* is right and matches ClojureScript: front-end (analyze →
emit) with the host VM as the backend, emitting host forms that the host compiles
to bytecode. Two things need to change to reach the goal:
1. **Late binding for globals.** Compile a reference to a Jolt var as a **deref
through the var cell**, not as a Janet symbol. Jolt vars are already cells
(`{:jolt/type :jolt/var :root …}`); a compiled global call becomes roughly
`((var-root cell) args…)` instead of `(janet-symbol args…)`. Redefinition
updates the cell's root, so compiled callers see it — exactly Clojure's model.
One indirection per global call; locals and control flow stay direct and fast.
Offer opt-in **direct linking** for hot/AOT code that doesn't need redefinition.
2. **Move the compiler and core into Clojure.** Today both are Janet. Self-hosting
means the compiler is Clojure compiled by Jolt, and most of `clojure.core` is
Clojure. That's the bulk of the work and where the "language builds itself"
payoff lives.
So: keep the emit-to-Janet target (it's correct and gives us bytecode + AOT for
free), fix global binding, and progressively self-host.
## Recommended architecture
**Pipeline (nanopass-lite).** Keep the data-driven `:op` AST and grow it as small,
named passes rather than one big walker:
1. *read* — reader → forms (already have it).
2. *macroexpand* — fully expand to special forms + calls (the interpreter already
expands; share one expander).
3. *analyze* — forms → AST, resolving locals vs vars and tagging ops.
4. *(optional) optimize* — constant-fold, direct-link hot calls, etc.
5. *emit* — AST → Janet form, with globals as var-cell derefs.
6. *compile* — Janet `compile` → bytecode; `make-image` for AOT.
Make each pass total over the IR so an unhandled node is an explicit gap, not a
silent miss.
**The kernel (minimal Janet bootstrap).** The irreducible base that must exist in
the host before any Clojure can run: the reader; the value/representation layer
(vars, namespaces, symbols, keywords, persistent collections, chars); host
interop (the `janet.*` bridge); `fn`/`if`/`do`/`let`/`quote`/`def`/`loop`/`recur`
evaluation; and `compile`/`eval`. Everything else — the rest of `clojure.core`,
the macros, and the compiler — is Clojure loaded and (eventually) compiled by the
kernel. Today the kernel is far larger than this; shrinking it is a long game.
**Hybrid interpret/compile (Phase 3, and a bootstrap safety net).** When a pass
can't yet compile a sub-form, emit a call back into the interpreter (`eval-form`)
for that sub-form instead of erroring. This lets the compiler be incomplete and
still correct (hot paths compile, cold/unsupported paths interpret), lets us grow
coverage incrementally, and de-risks the self-hosting bootstrap.
**Live flexibility.** Vars stay first-class cells; compiled code derefs them;
`def` updates the root; protocol/multimethod dispatch stays dynamic. Direct
linking is opt-in, never the default, so the REPL is always live.
## A staged path
1. **Var-indirection in the emitter***done*. Global refs compile as var-cell
derefs, so a compiled `defn` is redefinable at the REPL.
2. **Hybrid fallback + coverage** (`jolt-1bj`) — *done*. Forms the compiler can't
compile throw `jolt/uncompilable` and fall back to the interpreter, so compile
mode is always correct. Covered: multi-arity/named/variadic fns, `recur` in
`fn`, map/vector literals, and resolution matching the interpreter.
Destructuring compiles via the shared `destructure` expander: the `fn`/`let`/
`loop`/`defn` macros desugar to plain-symbol `fn*`/`let*`/`loop*`, so it no
longer falls back — and the primitives reject patterns outright, matching
Clojure (`jolt-f79`).
5. **Compile-by-default + AOT** (`jolt-7j9`) — *done, done out of order*. Once the
hybrid path was validated at parity, compilation was flipped on by default and
AOT images (`aot.janet`) landed. Done before 34 because it's the runtime
payoff and only needed the hybrid path to be correct, not self-hosting.
3. **Self-host the compiler** (`jolt-lcn`) — *open*. Rewrite `compiler.janet` as
Clojure (`jolt.compiler`) that Jolt compiles. Now the compiler is part of the
language it compiles.
4. **Shrink the kernel / core-in-Clojure** (`jolt-uqi`) — *open*. Move
`clojure.core` from Janet to Clojure incrementally, each piece compiled by the
previous stage — the language building itself — leaving a minimal Janet kernel.
What remains (3 and 4) is the actual Clojure-in-Clojure rewrite: the largest part
of the work and where the "language builds itself" payoff lives. The correctness
and runtime foundations it needs — redefinable compiled code, an always-correct
hybrid path, compile-by-default, and AOT — are now in place.

View file

@ -1,91 +0,0 @@
# deps.edn support — design notes
How Jolt loads pure-Clojure libraries from a `deps.edn`, and why it's built the
way it is. For how to *use* it, see [building-and-deps.md](building-and-deps.md).
Scope, decided up front:
- **git + local deps only** — no Maven/`~/.m2` resolution.
- **pure `clj`/`cljc`** — anything needing the JVM won't load or run; expected.
- **no classpath abstraction**`require` just needs to find a dep's namespaces;
"the classpath" is an ordered list of source directories.
- **piggyback on jpm** — reuse jpm's git fetch + cache; don't write a package
manager.
- **separate tool** — resolution lives in `jolt-deps`, beside the runtime, the
way `jpm` sits beside `janet`. The `jolt` runtime knows nothing about deps.edn.
## How jpm handles dependencies
jpm's package code (`jpm/pm.janet`) splits into a fetch half and a build half,
and we use only the first:
- **`resolve-bundle`** normalizes a dep spec to `{:url :tag :type :shallow}`,
accepting `:url`/`:repo` + `:tag`/`:sha`/`:commit`/`:ref`. A deps.edn
`{:git/url … :git/sha …}` maps straight onto it.
- **`download-bundle url :git tag shallow`** clones into a content-addressed cache
(`<modpath>/.cache/git_<tag>_<sanitized-url>`) and returns the path —
`git init` + `remote add` + fetch + reset, plus submodules. No build step.
- **`bundle-install`** is the half we skip: it then runs `project.janet` build
rules, which a Clojure lib doesn't have. It's cleanly separable from the clone.
So jpm gives us git resolution and a cache for free; calling `download-bundle`
needs `jpm/config/load-default` first (it sets `gitpath` and the cache dyns).
## How it works
`src/jolt/deps.janet` reads `deps.edn` (Janet parses it directly — EDN and Janet
syntax overlap for the `:deps`/`:paths` subset), then walks `:deps`:
- `:git/url` (+ `:git/sha` or `:git/tag`) → `resolve-bundle` + `download-bundle`
into `jpm_tree/.cache`;
- `:local/root` → the path as-is;
- `:mvn/*` and anything else → ignored.
Each resolved dependency contributes its own `:paths` (default `["src"]`) as
source roots, and we recurse into its `deps.edn` for transitive deps. The result
is a de-duplicated, ordered list of directories. `resolve-deps-cached` memoizes
that list in the tree keyed on a hash of `deps.edn`, so an unchanged file doesn't
re-fetch. jpm is loaded lazily (`require`, not `import`) so it's pulled in only
when resolving — never embedded in a built binary.
The loader (`evaluator.janet/find-ns-file`) resolves a namespace by searching the
context's `:source-paths` in order (the stdlib `src/jolt` first), trying `<ns>.clj`
then `<ns>.cljc`. Extra roots come from `JOLT_PATH` or `init`'s `:paths` option.
`jolt-deps` (`src/jolt/deps_cli.janet`, its own `declare-executable`) ties it
together: it resolves the roots and runs the `jolt` binary with them on
`JOLT_PATH`. The runtime's only dependency interface is that env var.
`jolt uberscript` bundles a namespace and everything it requires into one
standalone `.clj`. It requires the entry namespace and uses the order in which
the loader finishes loading files — a dependency finishes before the file that
required it, so the order is topological — then concatenates that source. The
baked-in stdlib is excluded (it's part of the runtime, not bundled).
Gotcha worth remembering: the `jolt` CLI's context is built into its image at
build time, so `JOLT_PATH` is applied at runtime in `main`, not in `init` (whose
env read would be frozen at build).
## Limitations
- Pure `clj`/`cljc` only — JVM interop, host classes, and unimplemented
`clojure.core` corners fail. Coverage is per-function: a namespace can load with
most functions working and a few not.
- Source only; compiled `.class` files in a git dep are ignored.
- git `:git/sha` must be a full SHA (`git fetch` can't resolve a short one).
## Conformance
`test/integration/deps-conformance-test.janet` resolves a few real pure-`cljc`
git libraries and reports whether their namespaces load and a sample call works.
It's network-gated behind `JOLT_CONFORMANCE=1` so CI stays offline. Use it to
check a library against the current interpreter, and to drive fixes for whatever
gap a failure points at (the same loop as the clojure-test-suite battery). A
library fails when it relies on something Jolt doesn't provide — JVM interop, or
a regex feature like Unicode property classes (`\p{…}`).
## Not yet
- **Compiling deps into a binary image.** `uberscript` already produces a
standalone `.clj`; baking a project's dependencies directly into a custom
executable image is a heavier variant that isn't implemented.

97
docs/MODULES.md Normal file
View file

@ -0,0 +1,97 @@
# Module map
Where things live and what to read before changing them. Start here to answer
"where does feature X live?" and "what else do I need to touch?"
## Areas
| Area | Directory | Responsibility | Re-mint? |
| --- | --- | --- | --- |
| Chez runtime | `host/chez/*.ss` | The substrate: value model, persistent collections, seqs, vars/namespaces, host interop, native `clojure.core` shims, regex, FFI, IO, the **reader**. Composed by `rt.ss`. | only `reader.ss` |
| Compiler | `jolt-core/jolt/*.clj` | analyzer → IR → backend, the optimization passes, the CLI, the deps resolver, nREPL. Baked into the seed. | **yes** |
| `clojure.core` overlay | `jolt-core/clojure/core/NN-*.clj` | Portable `clojure.core` in dependency-ordered tiers (`00-syntax``50-io`); the `NN` prefix *is* the load order. | **yes** |
| Stdlib | `stdlib/clojure/*.clj` | Lazily-loaded portable namespaces (string/set/walk/edn/pprint/zip/test/data). | no |
| Build & tooling | `host/chez/build.ss`, `emit-image.ss`, `compile-eval.ss`, `loader.ss`, `cli.ss`, `bootstrap.ss` | AOT binary build, cross-compile, runtime eval/load, CLI spine, seed mint. | no (except via `reader.ss`) |
| Tests & gate | `test/chez/`, `test/conformance/`, `host/chez/run-*.ss`, `Makefile` | Corpus (JVM oracle), unit, per-feature tests. Every `make` target has a comment. | no |
**The reader is in `host/chez/reader.ss`** (Scheme, a seed source) — *not* in
`jolt-core/jolt/` with the rest of the compiler. Re-mint applies to it.
`rt.ss` is the runtime's load-order manifest: it `(load …)`s every shim in
dependency order with a per-file comment. Read it to see how the runtime is
composed and where a given `.ss` fits.
## `host/chez/*.ss` by family
- **Value model**: `values.ss` (nil/numbers/keywords/symbols), `collections.ss`
(persistent vec + HAMT map/set), `seq.ss` + `lazy-bridge.ss` (seqs, lazy-seqs),
`transients.ss`, `records.ss` + `records-interop.ss`.
- **Native `clojure.core` shims**: `natives-*.ss` (array/coll/format/meta/misc/num/
queue/reader/seq/str/transduce), plus `predicates.ss`, `converters.ss`, `printing.ss`.
- **Vars / namespaces / dynamics**: `vars.ss`, `ns.ss`, `dyn-binding.ss` (the
thread-local binding stack), `dynamic-var-defaults.ss` (a few `*…*` constant defaults),
`atoms.ss`, `multimethods.ss`.
- **Host interop**: `host-class.ss` (class tokens + method dispatch),
`host-static.ss` (interop registry core) + `host-static-methods.ss` (`Class/member`
statics) + `host-static-classes.ss` (instantiable object classes), `host-table.ss`,
`host-contract.ss` (the `jolt.host` seam the compiler resolves against),
`dot-forms.ss`, `records-interop.ss`.
- **Scalars / misc**: `regex.ss` (vendored irregex), `math.ss`, `inst-time.ss`,
`bigdec.ss`, `syntax-quote.ss`.
- **IO / system / concurrency / FFI**: `io.ss`, `png.ss`, `concurrency.ss`,
`async.ss`, `ffi.ss`.
- **Compiler entry on Chez**: `reader.ss`, `compile-eval.ss`, `emit-image.ss`,
`loader.ss`, `cli.ss`, `build.ss`, `bootstrap.ss`.
## Where is a `clojure.core` fn implemented?
Two homes, with a defined precedence:
1. **Native shim** — a `(def-var! "clojure.core" "name" …)` in a `host/chez/*.ss`
(hot/representation-coupled fns: `first`, `get`, `=`, the predicates).
2. **Overlay** — a `defn` in a `jolt-core/clojure/core/NN-*.clj` tier (most of
`clojure.core`, in portable Clojure).
3. **`post-prelude.ss`** re-asserts a handful of natives *after* the overlay loads,
so the native version wins (the overlay's value-reading versions are wrong for
Chez-native chars/atoms/etc.). Each entry there says why.
`grep 'def-var! "clojure.core" "frequencies"' host/chez` and
`grep -rn 'defn frequencies' jolt-core/clojure/core` to find a given fn. See
[seed-overlay-registry.md](seed-overlay-registry.md) for the shadowing mechanism.
## Cross-cutting features — touch points
A feature's *core* lives in one file; these are the other files you must keep in
sync when changing it.
- **Tree-shaking / DCE** (`--tree-shake`): `emit-image.ss` (the `dce-*` helpers +
record producers) and `build.ss` (`bld-shake-all` reachability + the manifest
splice in `bld-emit-runtime`); the flag in `main.clj`; validated by
`host/chez/tree-shake-smoke.sh` (`make shakesmoke`) and `build-smoke.sh`. See
[tools-deps.md](tools-deps.md#tree-shaking).
- **Direct-linking** (`--direct-link`): `backend_scheme.clj` (`direct-link?`,
`emit-top-form`, the `jv$<fqn>` bindings); `build.ss` turns it on; `main.clj` the
flag; `test/chez/directlink-test.ss`.
- **Numeric fl*/fx\*** (`^double`/`^long` hints): `jolt-core/jolt/passes/numeric.clj`
(the hint-directed pass + loop-counter + `:coerce`); `backend_scheme.clj`
(`dbl-ops`/`lng-ops` op strings, `emit-numeric`, entry/return coercion);
`analyzer.clj` (`nhint-of`, `:nhints`, `with-ret-nhint`); `host-contract.ss`
(`:num-ret` on resolve); `rt.ss` (`jolt->fx`); `test/chez/numeric-test.ss`.
- **IR inlining** (under `--opt`): `passes/inline.clj` (splice) + `passes.clj`
(stash) + `host-contract.ss` (`inline-ir`/`stash-inline!`); `test/chez/inline-test.ss`.
- **Multimethods**: `host/chez/multimethods.ss` (dispatch) + the overlay
`defmulti`/`defmethod` macros + `host-contract.ss` late-bind.
- **AOT namespace context** (`jolt build`): `build.ss` (`bld-ns-prelude`) emits
`(set-chez-ns! ns)` + `chez-register-alias!` per app namespace (both the normal
and tree-shake emit paths), matching the loader's per-file ns context;
`test/chez/build-app` (`make buildsmoke`).
- **Deps resolution**: `jolt-core/jolt/deps.clj` (the only file) + `main.clj`
(applies the roots) + `loader.ss` (the `require` path).
## Conventions you must preserve
See **CLAUDE.md → "Conventions & Patterns"** for the load-bearing rules: the
re-mint trigger, the tier macro-ordering rule, the `get`-on-your-own-wrapper trap,
`:jolt/type`-as-a-key parsing, the `var-deref` calling convention (the compiler is
reached from the `.ss` runtime by string lookup, so a public `defn` with no
in-Clojure callers can still be live), and the writing style.

113
docs/building-and-deps.md Normal file
View file

@ -0,0 +1,113 @@
# Building and dependencies
How to run Jolt from source and how to pull Clojure libraries into a project.
## Running
```bash
git clone https://github.com/jolt-lang/jolt.git
cd jolt
git submodule update --init # vendor/sci (used by the SCI bootstrap tests)
bin/joltc -e '(println "hello")'
```
There is **no build step**. `bin/joltc` (`host/chez/cli.ss`) loads the
checked-in bootstrap seed (`host/chez/seed/{prelude,image}.ss`) plus the spine
and compiles+evals on Chez (read → analyze → IR → emit → eval), so a fresh
clone runs immediately. The whole `.clj` standard library
(`clojure.string`/`set`/`walk`/`edn`/`pprint`/…) and `clojure.core` are part of
the overlay, so they're always available.
`bin/joltc` is both the runtime (REPL, file/expr runner) and the dependency
front-end (`deps.edn` resolution, see below). A run with no `deps.edn` never
touches the resolver.
The bootstrap seed is **checked in**. After changing a seed source — the reader
(`host/chez/reader.ss`), the analyzer/IR/backend (`jolt-core/jolt/*.clj`), or the
`clojure.core` overlay (`jolt-core/clojure/core/*.clj`) — re-mint the seed with
`make remint` (it iterates `host/chez/bootstrap.ss` to a byte-fixpoint), or
`make selfhost` fails. Runtime-only `host/chez/*.ss` shims don't need a re-mint.
## How namespaces are found
`(require ...)` resolves a namespace to a file by searching an ordered list of
source roots — the stdlib first, then any extra roots — trying `<ns>.clj` then
`<ns>.cljc` (dots become directories, dashes become underscores). Extra roots
come from:
- `JOLT_PATH` — a colon-separated list of directories (like a classpath), applied
at runtime;
- the `:paths` option to `init` when embedding Jolt as a library.
If a namespace isn't found on any root, the loader falls back to the stdlib in
the overlay — that's how `clojure.string` and friends resolve when you run
outside the source tree.
So you can point Jolt at a directory of Clojure source with no deps machinery at
all:
```bash
JOLT_PATH=/path/to/lib/src bin/joltc run myfile.clj
```
## Dependencies via deps.edn
`bin/joltc` reads a `deps.edn` in the current directory, fetches its
dependencies, and prepends the resolved source directories to the source roots
for the run. The CLI commands (`jolt.deps` + `jolt.main`):
```bash
bin/joltc run -m NS [args] # resolve deps.edn, load NS, call its -main
bin/joltc run FILE # resolve deps.edn, load a Clojure file
bin/joltc -M:alias [args] # run the alias's :main-opts
bin/joltc -A:alias [args] # add the alias's paths/deps, then run the rest
bin/joltc repl # start a line REPL (project deps + native libs loaded)
bin/joltc --nrepl-server [port] # start an nREPL server (default 7888) for editors
bin/joltc path # print the resolved source roots (':'-joined)
bin/joltc <task> # run a deps.edn :tasks entry
```
Example `deps.edn`:
```clojure
{:paths ["src"]
:deps {weavejester/medley {:git/url "https://github.com/weavejester/medley"
:git/sha "<full-sha>"}
my/helpers {:local/root "../helpers"}}}
```
```bash
bin/joltc run -m myapp.main
```
### What's supported
- **git deps**`{:git/url … :git/sha …}` (use a full SHA; `git fetch` can't
resolve a short one), with an optional `:deps/root` for a subdirectory.
Transitive deps from each dependency's own `deps.edn` are resolved too.
- **local deps**`{:local/root "../path"}`.
- The project's own `:paths` (default `["src"]`) are included.
- **aliases** — `:aliases {:dev {:extra-paths ["dev"] :extra-deps {…}
:main-opts ["-e" "…"]}}`, selected with `-A:dev` (or several: `-A:dev:test`).
`:extra-paths`/`:extra-deps` accumulate across selected aliases;
`:main-opts` is last-wins and runs via `-M:alias`.
- **tasks**`:tasks {clean "rm -rf target" test {:main-opts ["-m" "…"]}}`.
A string task is a shell command; a map task runs jolt with its `:main-opts`.
Run one with `bin/joltc <taskname>`.
Resolution is breadth-first, so a top-level coordinate always beats a transitive
one for the same lib.
Git clones land in a global, sha-immutable cache shared across projects —
`$JOLT_GITLIBS`, else `~/.jolt/gitlibs`.
### What's not
- **No Maven.** `:mvn/version` deps are skipped with a warning — git and local
only.
- **Pure `clj`/`cljc` only.** A library that needs the JVM (Java interop, host
classes) or a `clojure.core` feature Jolt doesn't implement will fail to load
or fail at a call. Coverage is per-function: a namespace can load with most
functions working and a few not.
See [`tools-deps.md`](tools-deps.md) for the design rationale.

View file

@ -3,10 +3,11 @@
===========================================================================
This grammar specifies the surface syntax accepted by Jolt's reader
(src/jolt/reader.janet) the text that `read`/`parse-string`/`load-string`
turn into data/forms. It is the syntactic half of Jolt's contract; the
behavioural half lives in test/spec/. Where Jolt diverges from Clojure the
difference is called out in a comment.
(host/chez/reader.ss, with the portable half in jolt-core/jolt/reader.clj)
the text that `read`/`read-string`/`load-string` turn into data/forms. It is
the syntactic half of Jolt's contract; the behavioural half lives in the
conformance corpus (test/chez/corpus.edn, see docs/spec/02-reader.md). Where
Jolt diverges from Clojure the difference is called out in a comment.
Notation (ISO-ish EBNF):
= definition | alternation
@ -50,11 +51,12 @@ collection = list | vector | map ;
nil = "nil" ;
boolean = "true" | "false" ;
(* Numbers. Jolt accepts Clojure's numeric literal syntaxes, but since Jolt
numbers are Janet ints/doubles it has no distinct bignum, ratio or
BigDecimal types: the BigInt suffix N and BigDecimal suffix M are read as the
plain number, a ratio a/b is read as its double quotient, and radixed
integers are computed by base. The symbolic floats ##Inf/##-Inf/##NaN are
(* Numbers. Jolt carries a real numeric tower (JVM parity): an integer literal
reads as an exact integer (arbitrary precision), a ratio a/b as an exact
Ratio, a decimal/exponent literal as a double. The BigDecimal suffix M reads
as a real BigDecimal (unscaled x 10^-scale) 1.5M, 0.0M, 3M; class is
java.math.BigDecimal. The BigInt suffix N reads as an exact integer. Radixed
integers are computed by base; the symbolic floats ##Inf/##-Inf/##NaN are
also read. (No octal-with-leading-0 literal.) *)
number = symbolic-value
| [ sign ] , ( radix-int | ratio | hex-int | decimal ) ;
@ -62,10 +64,10 @@ sign = "+" | "-" ;
integer = digit , { digit } ;
hex-int = "0" , ( "x" | "X" ) , hex-digit , { hex-digit } , [ "N" ] ;
radix-int = integer , ( "r" | "R" ) , alnum , { alnum } ; (* base 2..36: 2r1010, 16rFF, 36rZ *)
ratio = integer , "/" , integer ; (* read as a double quotient *)
ratio = integer , "/" , integer ; (* exact Ratio *)
decimal = integer , [ "." , digit , { digit } ] , [ exponent ] , [ num-suffix ] ;
exponent = ( "e" | "E" ) , [ sign ] , digit , { digit } ;
num-suffix = "N" | "M" ; (* BigInt / BigDecimal in Clojure; plain number in Jolt *)
num-suffix = "N" | "M" ; (* N = exact integer (BigInt); M = BigDecimal *)
symbolic-value = "##Inf" | "##-Inf" | "##NaN" ;
digit = "0".."9" ;
hex-digit = digit | "a".."f" | "A".."F" ;
@ -109,7 +111,7 @@ map-entry = form , ws , form ; (* an even number of forms *)
(* --------------------------------------------------------------------------
Reader macros (prefix sugar). Each expands to a 2-element form
(op operand), e.g. 'x -> (quote x), @a -> (deref a).
(op operand), e.g. 'x -> (quote x), @a -> (clojure.core/deref a).
-------------------------------------------------------------------------- *)
reader-macro = quote | syntax-quote | unquote | unquote-splice
@ -119,13 +121,17 @@ quote = "'" , form ; (* (quote form) *)
syntax-quote = "`" , form ; (* (syntax-quote form) *)
unquote-splice = "~@" , form ; (* (unquote-splicing form) *)
unquote = "~" , form ; (* (unquote form) *)
deref = "@" , form ; (* (deref form) *)
deref = "@" , form ; (* (clojure.core/deref form) qualified, *)
(* so it derefs even where deref is shadowed *)
metadata = "^" , meta-form , ws , form ; (* attach metadata to form *)
meta-form = map | keyword | symbol | string ;
(* Normalized like Clojure: a symbol or string is a type hint -> {:tag ...};
a keyword -> {keyword true}; a map is used as-is. On a symbol the metadata
rides on the symbol (it stays a bare symbol, so a hint like ^String is
transparent in params/lets/bodies); other targets use a runtime with-meta. *)
a keyword -> {keyword true}; a map is used as-is. A keyword/symbol/string
meta-form on a symbol rides ON the symbol (it stays a bare symbol, so a hint
like ^String is transparent in params/lets/bodies). A MAP meta-form routes
through a runtime (with-meta form ...) even on a symbol, so a name
with ^{:map} metadata reads as a form, not a bare symbol def/defn/defmacro/ns
unwrap that to the bare name (and attach the metadata). *)
(* --------------------------------------------------------------------------
Dispatch forms introduced by "#".
@ -147,7 +153,7 @@ anon-arg = "%" | "%" , digit , { digit } | "%&" ;
var-quote = "#'" , symbol ; (* (var symbol) *)
(* Regex literal -> a Janet PEG-backed regex value.
(* Regex literal -> an irregex-backed regex value.
Supported: groups, greedy/lazy quantifiers, (?:..), lookahead (?=..)/(?!..),
alternation, anchors ^ $ \b \B, classes, (?i). NOT: lookbehind,
backreferences, named groups. *)

271
docs/host-interop.md Normal file
View file

@ -0,0 +1,271 @@
# Host interop and JVM standard-library shims
Jolt runs on Chez Scheme, not the JVM, so there are no real Java classes behind
interop forms. Instead the runtime ships shims for the slice of the JVM standard
library that portable Clojure code reaches for, so libraries written against
`clojure.core` and common `java.*` classes run unchanged. The Clojure interop
syntax works against these shims:
```clojure
(Math/sqrt 2) ; static call
Math/PI ; static field
(StringBuilder.) ; constructor
(.append sb "x") ; instance method
(instance? String "hi") ; class token
```
A class token (`String`, `java.util.UUID`, …) resolves to a name; there is no
reflection and no class hierarchy. `(class x)` returns the JVM class name for the
scalar/collection types Clojure programs compare against (`"java.lang.Long"`,
`"java.lang.String"`, and so on).
## Source layering: JVM-specific code lives in the java layer
Keep anything JVM-specific in `host/chez/java/`. The rest of the runtime stays
JVM-free, and the compiler in `jolt-core/` is JVM-free by construction.
- `host/chez/java/` holds the JVM model: the `java.*` mirrors, the class tokens
and class hierarchy, `(class x)`/`(type x)`/`instance?`, exception classes, the
interop dispatch for `.method`/`Class/static`/`(Class.)`. If a value or name
only means something because the JVM has it, it belongs here.
- The rest of `host/chez/` is the host-neutral runtime — the value model
(`values.ss`, `collections.ss`, `seq.ss`), reader, vars, multimethods, meta. It
speaks jolt's own taxonomy (`:string`, `:vector`, `:jolt/inst`), never JVM class
names.
- `jolt-core/` (the Clojure compiler + `clojure.core` overlay) emits and reasons
in that taxonomy only. The JVM mapping happens *after*, in the java layer.
The worked example is `type`. The core layer (`natives-meta.ss`) computes the
keyword taxonomy and binds it as `__type-tag` — that's what `print-method` and the
reader dispatch on, with no JVM in scope. The java layer (`java/host-class.ss`)
then rebinds the public `clojure.core/type` to Clojure's `(or (:type meta) (class
x))`, mapping `:jolt/inst` → `java.util.Date` and so on, right next to `(class
…)`. So the compiler keeps emitting `:jolt/inst`; the java layer remaps it.
When you add interop behaviour, prefer registering it through the generic hooks a
java-layer file already uses — `register-class-arm!` for `(class x)`,
`register-instance-check-arm!` for `instance?`, `register-eq-arm!` for value
equality — rather than threading a JVM concept back into a host-neutral file. A
new `java.*` shim is a new file under `host/chez/java/` loaded from `rt.ss`, not a
branch added to `collections.ss` or `seq.ss`.
## What's shimmed
This is the surface today, not the whole JVM. Methods not listed generally
aren't implemented; a few are accepted but no-ops (noted inline).
### Numbers and language
- **`java.lang.Math`** — `sqrt` `cbrt` `pow` `exp` `log` `log10` `floor` `ceil`
`round` `abs` `max` `min` `sin` `cos` `tan` `asin` `acos` `atan` `signum`
`random`; fields `PI`, `E`. (`clojure.math` mirrors these as functions.)
- **`Long` / `Integer`** — `parseLong`/`parseInt`/`valueOf` (optional radix),
`MAX_VALUE`, `MIN_VALUE`; `(Integer. x)`.
- **`Double` / `Float`** — `parseDouble`, `valueOf`, `toString`, `isNaN`,
`isInfinite`, the `*_VALUE`/`*_INFINITY`/`NaN` fields; `(Double. s)`.
- **`Boolean`** — `parseBoolean`, `TRUE`, `FALSE`.
- **`Character`** — `isUpperCase` `isLowerCase` `isDigit` `isWhitespace` (ASCII).
- **Boxed-number methods** — every number answers `.intValue` `.longValue`
`.doubleValue` `.floatValue` `.byteValue` `.shortValue` `.toString`
`.hashCode` (integer projections wrap modulo their width, as on the JVM).
- **`java.lang.System`** — `currentTimeMillis` `nanoTime` `exit` `getProperty`
`setProperty` `clearProperty` `getProperties` `getenv` `gc` (a full Chez
collection — clears weak references and fires their queues).
- **`java.lang.Thread`** — real OS threads over Chez `fork-thread`, sharing the
one heap (a captured atom/var is shared): `(Thread. thunk)` + `start` / `join` /
`run` / `isAlive`; plus `sleep` (real), `yield`/`interrupted`/`interrupt`
(no-ops), `currentThread`.
- **`java.util.concurrent.CountDownLatch`** — `(CountDownLatch. n)` + `countDown`
/ `await` / `getCount`, a real counting barrier (mutex + condition).
- **`java.lang.ref.SoftReference` / `WeakReference` + `ReferenceQueue`** — genuine
GC reclamation: the referent is held through a Chez weak pair, so the collector
reclaims it once unreachable (`.get` then returns nil) and a guardian enqueues
the reference on its `ReferenceQueue` (`poll`). Chez has no reference softer than
weak, so a `SoftReference` clears on unreachability, not memory pressure — eager,
but real eviction (core.cache's SoftCache).
- **`java.lang.Object`** — `(Object.)` as a fresh-identity sentinel; `.toString`
`.hashCode` `.equals` `.getClass` work on any value.
- **`java.lang.Class`** — `forName` (throws a catchable `ClassNotFoundException`
for a class jolt can't back, so `(try (Class/forName "opt.Dep") (catch …))`
dependency probes work). There is no reflection, but a few common interfaces
carry a modeled ancestry so `(supers c)` / `(ancestors c)` answer like the JVM —
e.g. `(ancestors (class f))` for a function yields `Runnable` and `Callable`,
the check `core.memoize` uses to validate a memoizable argument.
### Strings and text
- **`java.lang.String`** statics — `valueOf`, `format` (the `clojure.core/format`
engine; `String/format` with a leading locale is accepted). Instance methods
go through `clojure.string` / the native string ops.
- **`StringBuilder`** — `append` `toString` `length` `charAt` `setLength`.
- **`java.text.NumberFormat`** — `getInstance` `getNumberInstance`
`getIntegerInstance`; `.format`, `.setGroupingUsed`,
`.setMinimum/MaximumFractionDigits`.
- **`java.util.StringTokenizer`** — `hasMoreTokens` `countTokens` `nextToken`.
- **`java.util.regex.Pattern`** — `compile` (with `Pattern/MULTILINE`), `quote`;
`.split`, `.pattern`. (`#"…"` literals and `clojure.string` regex fns are the
usual entry points.)
### Collections (mutable)
- **`java.util.ArrayList`** — `add` `get` `set` `size` `isEmpty` `remove` `clear`
`contains` `toArray` `iterator`.
- **`java.util.HashMap`** / **`java.util.concurrent.ConcurrentHashMap`** — `put`
`get` `getOrDefault` `containsKey` `containsValue` `size` `isEmpty` `remove`
`clear` `putAll` `keySet` `values` `entrySet`; `clojure.core`'s `get` / `count` /
`contains?` also read them. (One shared heap, so the plain mutable map serves the
concurrent one.)
### I/O
- **`java.io.File`** — `(File. path)` / `(File. parent child)`. A File keeps the
path as given (`(.getPath (File. "rel"))` is `"rel"`, `.isAbsolute` false); a
relative path resolves against `JOLT_PWD` only when the filesystem is touched.
Methods: `getPath` `getName` `getParent` `getParentFile` `getAbsolutePath`
`getAbsoluteFile` `getCanonicalPath` `getCanonicalFile` `toURI` `toURL`
`exists` `isDirectory` `isFile` `isAbsolute` `isHidden` `length` `lastModified`
`canRead` `canWrite` `canExecute` `list` `listFiles` `mkdir` `mkdirs` `delete`
`createNewFile` `renameTo` `compareTo` `equals` `hashCode`. Statics:
`File/separator` `File/separatorChar` `File/pathSeparator` `File/createTempFile`
`File/listRoots`.
- **Byte streams**`FileInputStream` / `FileOutputStream` (over a path/File,
`append` arg), `ByteArrayInputStream` / `ByteArrayOutputStream`
(`toByteArray`/`toString`/`size`/`reset`), `BufferedInputStream` /
`BufferedOutputStream`. `read`/`read(byte[])`, `write(int)`/`write(byte[])`,
`flush`, `close`. Each is a Chez binary port underneath.
- **Char streams**`FileReader` / `InputStreamReader` (read a byte stream as
UTF-8), `FileWriter` / `OutputStreamWriter`, `BufferedReader` (`readLine`,
`lines`) / `BufferedWriter` (`newLine`), `StringReader` / `StringWriter` /
`PushbackReader`.
- **`clojure.java.io`** — `file` `as-file` `reader` `writer` `input-stream`
`output-stream` `copy` (byte-exact for byte sources) `make-parents`
`delete-file` `resource` `as-url`. `slurp`/`spit`/`line-seq`/`with-open` work
over all of the above.
- **`java.lang.ClassLoader`** — `getSystemClassLoader`, `.getResource`,
`.getResourceAsStream` (resolved against the source roots).
### Time and date
- **`java.util.Date`** — `(Date.)` / `(Date. ms)`; `getTime` `toInstant`
`toLocalDate(Time)` `before` `after` `equals` `toString` (RFC 3339).
- **`java.time`** — `Instant` (`now`, `ofEpochMilli`, `toEpochMilli`, `atZone`),
`LocalDateTime`, `ZoneId`, `DateTimeFormatter` (`ofPattern`, `ISO_LOCAL_*`,
localized styles), `FormatStyle`.
- **`java.text.SimpleDateFormat`** — `(SimpleDateFormat. pattern)`; `parse`
`format` `toPattern` `applyPattern` (`setTimeZone`/`setLenient` accepted but
ignored — formatting is UTC).
- **`java.util.TimeZone`** / **`java.util.Locale`** — constructed and passed
through; only UTC is honored for formatting.
### Net, encoding, misc
- **`java.net.URL`** — `(URL. spec)`; `toString` `toExternalForm` `getProtocol`
`getPath` `getFile`.
- **`java.net.URI`** — full component accessors (`getScheme` `getHost` `getPort`
`getPath` `getQuery` `getFragment`, raw variants, `isAbsolute`).
- **`java.util.Base64`** — `getEncoder`/`getDecoder` with `encode`,
`encodeToString`, `decode`.
- **`java.nio.charset.Charset`** — `forName`.
- **`java.util.UUID`** — `randomUUID`, `fromString`; `(UUID. s)`.
- **Exceptions**`Throwable` `Exception` `RuntimeException`
`IllegalArgumentException` `IllegalStateException` `IOException`
`NumberFormatException` `ArithmeticException` `NullPointerException`
`ClassCastException` `IndexOutOfBoundsException` `FileNotFoundException`
`UnsupportedOperationException` `Error` `AssertionError` and the common network
exceptions, each with the `(E.)` / `(E. msg)` / `(E. msg cause)` / `(E. cause)`
constructors. `try` dispatches its `catch` clauses by class in order, respecting
the exception supertype hierarchy (`(catch Exception e …)` catches a
`RuntimeException` but not an `Error`); a thrown value matching no clause
re-throws. An untyped host condition (e.g. from `(/ 1 0)`) is caught by a
`RuntimeException`/`Exception`/`Throwable` clause.
What's deliberately absent: STM (`clojure.lang.LockingTransaction/isRunning`
returns `false`), reflection, `gen-class`/`proxy` of Java classes, and
`BigDecimal`.
## Adding your own shim from a library
The built-in shims above are baked into the seed. A library or project can
register its **own** host classes at load time — no seed re-mint, no host edits.
Put the registration calls at the top level of a namespace your code requires.
Four functions (in `clojure.core`) plus the tagged-table seam (in `jolt.host`)
cover it.
`__register-class-ctor!` makes `(Name. …)` work; `__register-class-statics!`
makes `Name/field` and `(Name/method …)` work; `__register-class-methods!`
attaches instance methods to a tagged value; `__register-instance-check!` teaches
`instance?` about your class. **Method and static names are strings** (they match
the literal name in the interop form).
A stateful object is a *tagged table*`jolt.host/tagged-table` creates one,
`ref-put!`/`ref-get` set and read its fields. Read the tag back with
`jolt.host/ref-get` (or test it with `jolt.host/table?`); a plain `get` /
keyword lookup deliberately can't see a wrapper's own `:jolt/type`.
```clojure
(ns mylib.greeter
(:require [jolt.host :as host]))
;; (Greeter. name) -> a tagged value carrying its name
(__register-class-ctor! "Greeter"
(fn [name] (-> (host/tagged-table :greeter)
(host/ref-put! :name name))))
;; (.hello g) -> instance method, keyed by the literal method name
(__register-class-methods! :greeter
{"hello" (fn [self] (str "hi " (host/ref-get self :name)))})
;; Greeter/VERSION (field) and (Greeter/make x) (static method)
(__register-class-statics! "Greeter"
{"VERSION" "1.0"
"make" (fn [name] (Greeter. name))})
;; (instance? Greeter x)
(__register-instance-check!
(fn [class-name v]
(when (= class-name "Greeter")
(and (host/table? v) (= :greeter (host/ref-get v :jolt/type))))))
```
```clojure
(.hello (Greeter. "ada")) ;=> "hi ada"
Greeter/VERSION ;=> "1.0"
(.hello (Greeter/make "bob")) ;=> "hi bob"
(instance? Greeter (Greeter. "x")) ;=> true
```
An instance-check predicate returns `true`/`false` to decide, or `nil` to defer
to the next registered check and the built-ins — so several libraries can
register checks without clobbering each other. This is the mechanism jolt's
HTTP client library uses to emulate `java.net.URL` and `HttpURLConnection` so
`clj-http-lite` runs unchanged.
`__register-instance-check!` answers one `(instance? Foo x)` question. When a
class belongs to a *hierarchy* — a custom exception that should be caught as an
`IOException`, or a value that should match `(instance? SomeInterface x)` across
its whole supertype chain and dispatch a protocol extended to any of those
supertypes — declare its direct supers once with `jolt.host/register-class-supers!`
instead. `instance?`, `isa?`, `supers`/`ancestors`, and `extend-protocol`
dispatch all derive from the one declaration (supers are given by canonical name;
transitivity is computed):
```clojure
;; a library's exception type that catch/instance? should treat as an IOException
(jolt.host/register-class-supers! "com.acme.RetryExhaustedException"
["java.io.IOException"])
(throw (jolt.host/throwable "com.acme.RetryExhaustedException" "gave up"))
;; (catch java.io.IOException e …) now matches it; (instance? java.lang.Exception e) is true
```
deftype/defrecord classes join the same graph automatically at definition: a
record's ancestry carries the record interfaces (`clojure.lang.IRecord`,
`IPersistentMap`, `Associative`, …), a bare deftype carries
`clojure.lang.IType`, and every protocol the type implements inline appears as
an implemented interface — so `(ancestors MyRecord)`, `(isa? MyRecord
clojure.lang.IPersistentMap)`, and hierarchy relationships `derive`d on a
class's supers all answer like the JVM.
Extending a *built-in* class instead (adding a method to core's `String` shim,
say) means editing the relevant `host/chez/*.ss` file and running `make remint`
— see [building-and-deps.md](building-and-deps.md).

80
docs/libraries.md Normal file
View file

@ -0,0 +1,80 @@
# Clojure libraries known to work with Jolt
Libraries confirmed to load and pass their conformance checks on Jolt. A library
listed here works. See the [examples](https://github.com/jolt-lang/examples),
e.g. the [ring-app example](https://github.com/jolt-lang/examples/tree/main/ring-app).
* [aero](https://github.com/juxt/aero) — EDN configuration with tag literals
(`#ref`/`#env`/`#or`/`#profile`/`#long`/…)
* [config](https://github.com/yogthos/config) — environment configuration
* [Selmer](https://github.com/yogthos/Selmer) — Django-style templates
* [medley](https://github.com/weavejester/medley) — collection utilities
* [cuerdas](https://github.com/funcool/cuerdas) — string manipulation
* [ring-core](https://github.com/ring-clojure/ring) — via `:deps/root "ring-core"`,
on the ring-app example
* [ring-codec](https://github.com/ring-clojure/ring-codec) — URL/form encoding
* [ring-defaults](https://github.com/ring-clojure/ring-defaults) — the standard
middleware stack (params, static resources + content-type, session, security
headers); its session/CSRF crypto comes from
[jolt-lang/jolt-crypto](https://github.com/jolt-lang/jolt-crypto) (OpenSSL)
* [reitit-core](https://github.com/metosin/reitit) — data-driven routing; the
`reitit.Trie` Java class is mirrored by
[jolt-lang/router](https://github.com/jolt-lang/router).
* [integrant](https://github.com/weavejester/integrant) — data-driven system
configuration (`#ig/ref`), with its
[dependency](https://github.com/weavejester/dependency) and
[meta-merge](https://github.com/weavejester/meta-merge) deps
* [honeysql](https://github.com/seancorfield/honeysql) — SQL formatter and helpers
* [clojure.jdbc](https://github.com/yogthos/clojure.jdbc) — via
[jolt-lang/db](https://github.com/jolt-lang/db)'s `jdbc.core`, over the built-in
SQLite access (libsqlite3 via Chez's FFI)
* [tools.logging](https://github.com/clojure/tools.logging) — runs verbatim over a
native `clojure.tools.logging.impl` stderr backend
* [migratus](https://github.com/yogthos/migratus) — database migrations over
[jolt-lang/db](https://github.com/jolt-lang/db)
* [malli](https://github.com/metosin/malli) — data schema validation, on the
malli-app example.
* [markdown-clj](https://github.com/yogthos/markdown-clj) — Markdown → HTML, on the
markdown-app example
* [hiccup](https://github.com/weavejester/hiccup) — HTML from Clojure data, on the
hiccup-app example
* [clojure.data.json](https://github.com/clojure/data.json) — JSON reading and writing
* [clojure.spec.alpha](https://github.com/clojure/spec.alpha) — data specs
* [core.match](https://github.com/clojure/core.match) — pattern matching.
* [core.cache](https://github.com/clojure/core.cache) — caching (Basic/FIFO/LRU/
LU/TTL/Soft + the wrapped atom API), over
[data.priority-map](https://github.com/clojure/data.priority-map).
* [core.memoize](https://github.com/clojure/core.memoize) — function memoization
over [core.cache](https://github.com/clojure/core.cache).
* [core.async](https://github.com/clojure/core.async) — CSP channels and `go` blocks
(`<!`/`>!`/`alts!`, `pipeline`, `mult`/`mix`/`pub`/`sub`) on real OS threads.
* [core.logic](https://github.com/clojure/core.logic) — relational logic programming
(unification, `run`/`fresh`/`conde`, finite domains).
* [math.combinatorics](https://github.com/clojure/math.combinatorics) — permutations,
combinations, subsets, selections, cartesian products, partitions.
* [core.contracts](https://github.com/clojure/core.contracts) — programming by
contract (`contract`/`with-constraints`/`provide`), over
[core.unify](https://github.com/clojure/core.unify).
* [data.zip](https://github.com/clojure/data.zip) — zipper navigation, including
`clojure.data.zip.xml`; XML parsing via [jolt-lang/xml](https://github.com/jolt-lang/xml)
(which now ships `clojure.xml/parse`).
* [data.csv](https://github.com/clojure/data.csv) — reading and writing CSV.
* [data.codec](https://github.com/clojure/data.codec) — base64 encode/decode over
byte arrays.
* [data.priority-map](https://github.com/clojure/data.priority-map) — priority
maps (incl. keyfn / custom comparator), with `subseq`/`rsubseq`.
* [tools.macro](https://github.com/clojure/tools.macro) — local macros
(`macrolet`/`symbol-macrolet`), `mexpand`/`mexpand-all`.
* [algo.monads](https://github.com/clojure/algo.monads) — monad macros and
monads (maybe/seq/state/writer/reader/…), over
[tools.macro](https://github.com/clojure/tools.macro).
* [test.check](https://github.com/clojure/test.check) — property-based testing
(generators, `quick-check`, shrinking).
* [tools.reader](https://github.com/clojure/tools.reader) — a Clojure reader in
Clojure (edn + full reader, indexing/pushback reader types).
* [rewrite-clj](https://github.com/clj-commons/rewrite-clj) — parse/rewrite Clojure
source while preserving whitespace and comments (nodes + zipper), over
[tools.reader](https://github.com/clojure/tools.reader).
* [tick](https://github.com/juxt/tick) — date/time over Jolt's `java.time`;
`#time/…` literals via `time-literals`.
* [transit-jolt](https://github.com/jolt-lang/transit-jolt) — Transit (JSON) read/write

View file

@ -0,0 +1,179 @@
# RFC 0001 — A Specification for the Clojure Language
- **Status**: Draft
- **Champions**: jolt maintainers
- **Created**: 2026-06-10
## Summary
Produce a normative, implementation-independent specification of the Clojure
language — the reader, the evaluation model, the special forms, the data types
and their equality/hashing/ordering contracts, sequences and laziness, and the
`clojure.core` library — to the standard set by R7RS Scheme and the Racket
reference. The specification is developed *in this repository*, validated
continuously by jolt's executable conformance suite, and intended to be useful
to every alternative implementation (ClojureScript, jank, babashka/sci,
Basilisp, ClojureCLR, jolt).
## Motivation
Clojure has no specification. The language is defined by:
1. the reference JVM implementation's source,
2. docstrings (frequently silent on edge cases),
3. community folklore (ClojureDocs examples, mailing-list threads),
4. each alternative implementation's reverse-engineering effort.
Every alternative implementation independently re-derives answers to the same
questions — *what does `(nth coll nil)` do? is `(first "")` an error? does
`conj` on `nil` produce a list or vector? in what order does `reduce-kv` visit
a map?* — and they routinely diverge. The cross-dialect
[clojure-test-suite](https://github.com/jank-lang/clojure-test-suite) exists
precisely because these divergences are real and frequent: it currently
encodes hundreds of edge-case assertions that no normative document captures.
Building jolt's self-hosted compiler forced us to answer these questions
one at a time (the conformance harness runs every behavior through three
independent execution paths and demands agreement). That work product — over
300 three-way-validated conformance assertions, ~1,500 behavioral spec cases,
and a frozen catalog of which forms are language vs. host — is the seed of a
specification, currently trapped in test files. This RFC proposes promoting it
into prose with normative force.
### Why us / why now
A useful spec needs an implementation that can *afford* to be strict. The
reference implementation can't adopt a spec retroactively without breaking
changes; an alternative implementation chasing drop-in compatibility can't
deviate from the reference even where the reference is accidental. jolt's
goals (self-hosted, minimal seed, multiple execution paths that must agree)
already require us to decide, for every form, *what the contract is* — we are
writing the spec anyway, in test form. The marginal cost of writing it down
properly is small; the value to the ecosystem is large.
## Goals
1. **Normative core**: reader grammar, evaluation model, all special forms,
data types with equality/hashing/ordering contracts, seq/laziness
contracts, namespaces/vars, and per-var entries for the portable
`clojure.core` surface.
2. **Executable**: every normative statement is paired with at least one
conformance test. The spec and the suite are maintained together; a spec
claim without a test is marked `unverified`.
3. **Host classification**: every `clojure.core` var is classified
**portable** (specified normatively), **host-dependent** (interface
specified, behavior host-defined — e.g. `slurp`, `*out*`), or
**JVM-specific** (documented as outside the portable language — e.g.
`bases`, `definline`, agents/STM as currently scoped).
4. **Versioned against reference Clojure**: each spec edition states the
reference version it describes (initially 1.12) and records *deliberate*
divergences (e.g. where reference behavior is accidental — these become
labeled "implementation-defined" with the reference behavior noted).
5. **Useful to other implementations**: no jolt-specific concepts in
normative text. jolt appears only in conformance-suite references.
## Non-goals
- Specifying the JVM interop surface (`proxy`, `gen-class`, `.`-forms beyond
their syntax), agents, STM refs, or the Java class hierarchy mapping.
These are catalogued as host/JVM surface, not specified.
- Specifying `clojure.spec`, `core.async`, or other contrib libraries
(candidates for later, separate documents).
- Changing the language. The spec describes Clojure as it is; divergence
decisions document reality, they don't invent semantics.
- Replacing clojure-test-suite — we contribute to it and cite it.
## The specification document
Lives in `docs/spec/`. Shape (mirroring R7RS chapters):
| § | Document | Content |
|---|---|---|
| 0 | `00-front-matter.md` | conformance terms (RFC 2119), entry format, host classification |
| 1 | `01-evaluation.md` | evaluation model: forms, environments, vars, macroexpansion order |
| 2 | `02-reader.md` | lexical syntax: formal grammar, all reader macros, reader conditionals |
| 3 | `03-special-forms.md` | the special forms, one normative entry each |
| 4 | `04-data-types.md` | nil/booleans/numbers/strings/chars/keywords/symbols/colls; equality, hashing, ordering |
| 5 | `05-sequences.md` | the seq abstraction, laziness contract, realization boundaries |
| 6 | `06-namespaces-vars.md` | namespaces, vars, dynamic binding, resolution |
| 7 | `07-polymorphism.md` | protocols, records/types, multimethods, hierarchies |
| 8 | `08-macros.md` | defmacro, syntax-quote/hygiene, `&env`/`&form` |
| 9 | `09-core-library.md` | normative per-var entries for the portable surface |
| A | `coverage.md` | generated status dashboard: 694 vars × {specified, tested, implemented, classification} |
### The normative entry format
Every special form and library var gets an entry with these fields
(exemplars in `03-special-forms.md` and `09-core-library.md`):
```
### name
Signature(s), since-version
1. Semantics — numbered MUST/SHOULD statements
2. Edge cases — nil, empty, bounds, wrong-type behavior (normative)
3. Errors — what MUST throw, and when error type is implementation-defined
4. Examples — executable, drawn from ClojureDocs where community-validated
5. Conformance — test IDs that verify each numbered statement
```
### Evidence sources, in priority order
1. **Differential testing** against reference Clojure 1.12 (the ground truth
for behavior questions).
2. **clojure-test-suite** (cross-dialect agreement = portable semantics;
dialect splits = host-dependent candidates).
3. **ClojureDocs export** (`clojuredocs-export.edn`, 694 core vars, 648 with
community examples) — examples become spec examples after verification.
4. **jank's language test corpus** (~800 per-form tests under
`test/jank/{form,call,metadata,reader-macro,syntax-quote,var}`) — the
per-construct granularity model for §2§3 conformance.
5. Reference implementation source — last resort, for intent.
## Current baseline (measured 2026-06-10)
- ClojureDocs inventory: **694** `clojure.core` vars (648 with examples).
- jolt implements **572**; **373 (66%)** are exercised by the behavioral
spec/conformance suites; 139 implemented-but-untested.
- Initial classification of the 182 unimplemented: ~31 dynamic vars, ~20
agents/taps, ~11 STM, ~15 special-form docs, ~105 to adjudicate
(genuinely-portable gaps spotted already: `compare`, `any?`, `update-keys`,
`update-vals`, `parse-long`, `parse-double`, `parse-boolean`,
`partitionv`, `splitv-at`, `macroexpand`, `time`, `with-redefs`).
- Conformance: 302 assertions × 3 execution paths; ~1,500 behavioral cases;
clojure-test-suite ≥ 4081/4707 assertions.
## Process
1. **Section by section**, in chapter order. §2 (reader) and §3 (special
forms) first — they are the smallest closed sets and jank's corpus gives
per-construct conformance shape immediately.
2. Each PR that adds/edits normative text MUST add or cite the conformance
tests for every numbered statement, and update `coverage.md`.
3. Divergences from reference Clojure discovered during writing get filed,
then either fixed in jolt or recorded as a labeled divergence — never
silently spec'd to jolt's behavior.
4. Editions: spec snapshots versioned independently of jolt releases
(`Clojure Language Specification, Draft N`).
5. When a chapter stabilizes, solicit review from other implementations
(jank, babashka, Basilisp maintainers) before marking it Stable.
## Alternatives considered
- **Contribute prose to clojure-test-suite instead**: the suite is the right
*conformance* home but tests can't express rationale, classification, or
grammar; both are needed and they cross-reference.
- **Spec only what jolt implements**: rejected — the host classification of
the *full* 694-var surface is half the value.
- **EDN/data-format spec only** (edn already has a loose spec): far too
narrow; the evaluation model and core library are where divergence lives.
## Open questions
1. Numerics: the reference has longs/doubles/ratios/BigInt with promotion
rules; CLJS has JS numbers. Resolved: jolt carries the Scheme numeric tower
(exact integers/bignums, exact ratios, flonum doubles), matching the
reference's tower — see the numerics note in §4.
2. Where do `*print-length*`-style dynamic vars land — host-dependent
interface or portable with defaults?
3. License/venue if the spec outgrows this repo (likely CC-BY; separate repo
once §1§3 stabilize).

View file

@ -0,0 +1,95 @@
# RFC 0002 — Reader-Conditional Feature Set
- **Status**: Superseded (2026-06-25) — jolt now includes `:clj` in the default
set; see the note below.
- **Created**: 2026-06-10
- **Spec**: `docs/spec/02-reader.md` §2.3 S18
> **Update (2026-06-25).** The default set is now **`#{:jolt :clj :default}`** —
> `:clj` is satisfied by default. The clj ecosystem's `.cljc` libraries gate
> their host code behind `#?(:clj …)` with no `:jolt`/`:default` fallback, so
> the conformance libraries (core.cache, core.match, tick, malli, …) only load
> with `:clj` present; requiring an opt-in for each was friction with no payoff
> once jolt's `clojure.lang.*`/`java.*` emulation was broad enough to run those
> `:clj` branches. Matching is still by **clause order**, so a library can place
> a `:jolt` branch first to override. There is no `JOLT_FEATURES` environment
> variable; a loading context overrides the set at runtime with
> `reader-features-set!`. The rest of this RFC is the original (reverted)
> design.
## Summary
jolt's reader-conditional feature set is **`#{:jolt :default}`**, matched in
**clause order** (the first clause whose key the platform satisfies wins).
A loading context may opt a foreign, clj-targeted library into `:clj`
compatibility via `reader-features-set!` (or process-wide via the
`JOLT_FEATURES` environment variable). jolt does **not** satisfy `:clj` by
default.
## Background
`#?(:clj … :cljs … :default …)` selects a branch by platform feature at read
time. Until now jolt satisfied `:clj` — a compatibility shortcut inheriting
the JVM branches of `.cljc` files, on the theory that the `:clj` branch is
usually the "main" implementation. Each dialect chooses its own policy:
ClojureScript satisfies only `:cljs`; jank uses `:jank`; babashka includes
`:clj` because it genuinely is JVM-Clojure-compatible to a deep degree.
Two defects forced the decision:
1. jolt is *not* JVM-compatible where it matters for `:clj` branches: they
contain interop (`java.util.*`, `deftype` over JVM classes) and encode
JVM-specific *expectations* in tests (e.g. `parse-uuid`'s reference
permissiveness), both of which jolt fails.
2. The old implementation also matched by **key priority** (`:clj` first,
then `:default`) rather than clause order — `#?(:default 5 :clj 6)` read
as `6`, diverging from Clojure on all platforms.
## Decision and evidence
Measured A/B over the cross-dialect clojure-test-suite (identical tree,
2026-06-10):
| Feature set | Assertions reached | Pass | Fail | Error | Clean files |
|---|---|---|---|---|---|
| `clj, default` (old) | 4967 | 4324 | 524 | 119 | 78 |
| `jolt, default` (new) | **5069** | **4470** | **518** | **81** | **86** |
The portable convention reads *more* of the suite (`:default` branches were
being shadowed by `:clj` ones jolt can't satisfy) and improves every metric:
+146 passes, 38 errors, +8 clean files. The `:clj` shortcut was a net
liability, not a compatibility win.
The opposing case — loading real-world clj-targeted libraries — is real:
SCI's `.cljc` sources select their implementation via `#?(:clj …)`/`:cljs`
with no `:jolt` branches, and fail to load under the portable set. That is a
property of the **loading context**, not of the platform: the resolution is
per-context opt-in, exactly how the SCI bootstrap now loads
(`(reader-features-set! ["jolt" "clj" "default"])`).
## Specification (normative, mirrored in spec §2.3 S18)
1. The platform feature set is implementation-defined and MUST be
documented. jolt's is `#{:jolt :default}`.
2. Matching MUST be by clause order: the first clause whose key is in the
feature set wins. `:default` matches on every platform.
`#?(:default 5 :clj 6)` is `5` everywhere.
3. An unmatched conditional reads as nothing (no form); an unmatched
`#?@(…)` splices nothing.
4. Implementations SHOULD provide a per-loading-context override so foreign
libraries written for other dialects can be read under a compatibility
set; using it is a deliberate, scoped decision (jolt:
`reader-features-set!` / `JOLT_FEATURES`).
## Consequences
- Suite baselines re-measured and raised: `baseline-pass` 4324 → 4470,
`baseline-clean-files` 78 → 86.
- Reader tests assert the portable set + clause-order semantics, plus one
opt-in round-trip through `reader-features-set!`.
- Loading clj-ecosystem libraries via deps requires deciding their feature
set; the deps loader currently inherits the process default — a future
refinement is per-dependency feature configuration (filed with the deps
work).
- `.cljc` authors targeting jolt can write `:jolt` branches and rely on
`:default` fallbacks.

108
docs/rfc/0003-transients.md Normal file
View file

@ -0,0 +1,108 @@
# RFC 0003: Transients — semantics and the Chez mutable backing
Status: accepted (design note)
This note pins down what transients *are* in Jolt, where their behavior
deviates from JVM Clojure and why, and how the transient machinery is
represented in the Chez runtime. It exists so the design doesn't revisit
transients every round.
## What a transient is in Jolt
A transient is a Chez record (`jolt-transient`, `host/chez/transients.ss`)
wrapping *true mutable* host backing, snapshotted to the immutable collection on
`persistent!`. The backing is per kind:
- transient vector — a growable Scheme vector (a capacity buffer plus a fill
count `n`). `conj!`/`pop!` are in-place, amortized O(1); the buffer doubles on
growth.
- transient map — a Chez hashtable keyed by `key-hash` / `jolt=`
(value-equality, nil-safe). Hashing by value keeps collection keys comparing
across representations.
- transient set — a Chez hashtable of elements.
- `cow` — a copy-on-write fallback for anything else (e.g. a sorted coll).
`transient` accepts pvecs, pmaps, psets, and the exotic colls handled by the
`cow` path. Each kind copies its source into the matching mutable backing once.
The bang ops (`conj!`, `assoc!`, `dissoc!`, `disj!`, `pop!`) mutate that backing
in place and return the transient — O(1) per op (amortized for the vector push).
`persistent!` snapshots a persistent value from the backing (folding the
hashtable into a pmap/pset, handing off the buffer as a pvec) and invalidates the
transient (the record's active flag clears; any further bang op or a second
`persistent!` throws "transient used after persistent!", matching Clojure's
invalidation contract).
Read ops work on an active transient where Clojure supports them: `get`,
`contains?`, `count`, and `nth` (vector kind) see through the transient.
`seq` on a transient is not supported, as in Clojure.
## Deviations from JVM Clojure (deliberate)
**O(n) edges, O(1) middle.** Clojure's `(transient v)` is O(1) — the transient
*shares* the persistent trie and marks nodes editable; `persistent!` is O(1)
too. Jolt's `transient` copies the source into a mutable buffer/hashtable (O(n))
and `persistent!` snapshots back (O(n)). The bang ops in between are host-mutable
O(1), which is *faster* per-op than trie editing. So the asymptotics of the usual
pattern
(persistent! (reduce conj! (transient []) coll))
are identical (O(n) total either way) with a better constant in the loop and a
worse constant at the two edges. The pattern transients exist for — batch
construction — is fully served. What is NOT served is transient-editing a
*large* collection to change a few keys: that's O(n) in Jolt vs O(log n) in
Clojure, because `transient` copies the source into a growable Scheme vector /
Chez hashtable and `persistent!` snapshots it back.
**No thread-ownership check.** JVM Clojure ≥1.7 also dropped the owner-thread
assertion (for fork/join), keeping only "don't use after persistent!", which
Jolt enforces. A transient handed across threads is a data race exactly as in
Clojure — documented, not checked, same as the JVM.
**`(conj!)` / `(conj! t)` arities** follow Clojure's transducer-era contract:
zero args makes a fresh `(transient [])`, one arg returns it untouched.
`assoc!` tolerates a dangling final key (treated as `k nil`), matching the
lenient kvs walk of Jolt's `assoc`.
**No transient sorted variants** — same as Clojure. One leniency: Clojure
throws on `(transient '(1))`, but Jolt routes a list through the `cow` fallback
path, yielding a transient. Harmless but non-Clojure; tighten if it ever
bites.
## Why transients live in the host
Transients are part of the value/representation layer in the Chez runtime
(`host/chez/transients.ss`), not the portable `clojure.core` overlay, on three
grounds:
1. **They are the mutation kernel.** A transient's entire value is direct
mutation of a host buffer/hashtable. The overlay has no mutation seam of its
own. Re-expressing the bang ops in Clojure would mean either growing the host
surface one-for-one (a host-vector-push, a host-hashtable-put, …, i.e. moving
the same code behind more indirection) or simulating mutation over persistent
values (defeating the point of transients).
2. **They sit under the collection dispatch.** `conj`/`assoc`/`get`/`count`/
`contains?` see through a transient. Hoisting the transient ops above that
dispatch would put a compiled-Clojure call inside the hottest paths for no
semantic gain — transients have no semantics to *fix*.
3. **The value layer is the host's job.** The persistent collections and, with
them, their mutable scratch counterparts, live in the Chez runtime alongside
the value model. Transients are representation, not library.
What lives in the overlay: anything *derived* — e.g. `into`'s transient-using
fast path, or `update!`-style conveniences — is plain Clojure over
`transient`/bang-ops/`persistent!`.
## Future work
- The persistent map/set are a bitmap HAMT with structural sharing
(`host/chez/collections.ss`), so Clojure-style O(1) `transient`/`persistent!`
via editable nodes is a real option there — an internal change behind the same
surface, not a semantics change. The persistent vector is a flat
copy-on-write Scheme vector rather than a trie, so the transient surface for
it stays the copy-to-growable-vector path.
- `transient?` (Jolt extension, useful in tests) stays; Clojure has no public
predicate, so it must not leak into portability-sensitive code.

147
docs/rfc/0004-type-hints.md Normal file
View file

@ -0,0 +1,147 @@
# RFC 0004: Type hints and keyword-lookup specialization
Status: accepted (design note)
This note describes how Jolt treats Clojure type hints, and the one place it
uses them: a `^:struct` or `^Record` hint on a local lets a constant-keyword
lookup skip its runtime representation guard. It records the rationale, the
soundness contract, the checked mode for catching inaccurate hints, and the
measured effect, so later work does not relitigate it.
## Background: why the lookup carries a guard
A Jolt map value has several runtime representations (see RFC on collections and
`host/chez/collections.ss`): a persistent hash map (a bitmap HAMT) for the
general case, plus sorted maps, transients, and record/deftype instances. A
record instance is a Chez record (`jrec`) whose fields are read directly off the
record's storage, while a HAMT lookup runs the full `jolt=`/`jolt-hash`-keyed
collection path.
A constant-keyword lookup `(:k m)` compiles to a guarded form: it inspects the
subject's representation and routes a HAMT/sorted/transient/lazy-seq value to the
full `jolt-get` semantics, while a record/raw-get-safe value takes the direct
field read, which matches `jolt-get` for keyword keys. The guard is correct and
cheap, but on a raw-get-safe value it is wasted work: profiling the ray tracer (a
naive all-maps program) found keyword lookups are about half of a render, and the
guard is the only avoidable part of each one.
Dropping the guard is only safe when the subject is known to be a plain
struct/record rather than a tagged collection. Jolt does not infer that
inter-procedurally (it would be unsound across a dynamic language's call
boundaries). A type hint supplies the same fact soundly, as a programmer
assertion.
## What the hints mean
Two hints on a local resolve to the "plain struct/record" assertion, which we
call the `:struct` hint internally:
- `^:struct` — the value is a plain struct or record map. There is no Clojure
keyword with this meaning (Clojure's type hints are class names), so this is a
Jolt-specific metadata flag, analogous to `^:dynamic`.
- `^Name` where `Name` is a `defrecord`/`deftype`. Both forms define a `->Name`
positional constructor, so the analyzer treats a tag whose `->Name` resolves
as a record type. Record instances are raw-get-safe, so the lookup drops the
guard. A `^String`, `^long`, or any other non-record tag is not a record and
is ignored, exactly as before.
Every other hint parses and is inert, matching Clojure (S12b in the reader
spec). A hint never changes a program's result; it only permits an
optimization.
## How it flows
The reader already keeps `^hint` metadata on the binding symbol and is otherwise
transparent (`host/chez/reader.ss`). The change threads that fact to the lookup
site:
1. The analyzer (`jolt-core/jolt/analyzer.clj`) records a `:struct` hint per
local in its env when a param or `let` binding carries `^:struct` or a
record-type tag, and attaches `:hint :struct` to that local's `:local` IR
node. Resolving a record-type tag uses the host contract function
`record-type?` (`jolt.host`, backed by `host/chez/host-contract.ss`), which
checks for the `->Name` constructor.
2. The back end (`jolt-core/jolt/backend_scheme.clj`) emits the direct field read
when the lookup subject is a `:local` carrying the hint, and the guarded form
otherwise. The unhinted path is identical to before.
3. The inline pass (`jolt-core/jolt/passes.clj`) propagates the hint: when it
binds a non-trivial call argument to a fresh local, it carries the called
function's parameter hint onto that local, so lookups inside the spliced body
keep the direct path. Without this, inlining a hinted function would erase the
benefit, because the hinted parameter is replaced by an unhinted temporary.
The same machinery covers both `(:k m)` and `(get m :k [default])` when the key
is a constant keyword. A `get` with a variable, numeric, or string key falls
through to `jolt-get` unchanged.
## Record hints across namespaces, and as inference seeds
A `^RecordType` hint does two things beyond dropping the lookup guard.
**It carries the specific type, not just "a struct".** The guard-skip only needs
to know the value is raw-get-safe (`:struct`), but the structural inference (RFC
0005) wants the actual record type so a field read gets the field's type —
`(:origin ray)` on a `^Ray ray` is a `Vec3`, not `:any`. A record hint on a
parameter is resolved to the record's constructor key and used to **seed the
inference's parameter type**. That is what keeps a record parameter's reads typed
across a namespace boundary *without* whole-program inference (RFC 0005,
"Cross-namespace inference") — the open-world counterpart to the whole-program
pass. Hinting only the public entry point is not enough; the hint has to be on
the function where the hot reads actually happen.
**It resolves across namespaces.** A hint may name a record defined in another
namespace, in either spelling — `^Vec3` where the type is `:refer`-ed, or
`^v/Vec3` where the namespace is `:as`-aliased. Resolution (`record-ctor-key`,
a `jolt.host` contract function backed by `host/chez/host-contract.ss`) runs
against the *compile* namespace and maps the type to its home constructor key
through a constructor-value index — keyed by the constructor value, not a var's
namespace, so a `:refer`-interned var (whose namespace is the referring one)
still resolves home. The reader keeps a tag's namespace qualifier (`^v/Vec3`
`"v/Vec3"`, not `"Vec3"`) so the aliased spelling has something to resolve. Both
`defrecord` field hints and function parameter hints use this resolution.
## Soundness and the checked mode
An accurate hint is correctness-preserving by construction: for a struct or
record the bare get equals the guarded result. An inaccurate hint (asserting
`^:struct` for a value that is actually a phm) makes the raw get return the wrong
thing. This is the same contract as a wrong Clojure `^String`, except that a
wrong Jolt hint fails silently rather than throwing.
To make a lie visible without taxing the fast path, `JOLT_CHECK_HINTS=1` keeps
the guard but throws on the tagged arm with a message naming the local and key:
```
type hint violated on `m`: (:a m) — value is a
phm/sorted/transient/lazy-seq, not the plain struct/record the
^:struct/^Record hint asserts
```
This is a development aid, off by default, with zero cost to normal builds (the
flag is read when the lookup is compiled, and the bare get is emitted when it is
off). The flag is part of the image-cache fingerprint.
## Coverage
Type hints parse in every position Clojure accepts them and are inert except for
the optimization above. This matches Clojure's "parse and otherwise do nothing"
model, with the difference that Clojure additionally uses hints to avoid
reflection and select primitive arithmetic, which do not apply to the Chez host.
## Measured effect
On the ray tracer (`~/src/examples/ray-tracer`, all values are `{:r :g :b}`-style
maps), with inlining on and the hot parameters hinted, a render goes from 13.3s
to 10.9s, about 1.22x, taking it to roughly 7.8x the JVM from 9.4x after the
inline pass. A seeded render produces an identical pixel checksum hinted and
unhinted, confirming the hints are correctness-preserving on the full pipeline.
## Status and non-goals
Implemented. Not pursued: inter-procedural shape inference (unsound in a dynamic
language without a guard, which costs as much as the one being removed) and a
shape-based "hidden class" representation (profiling showed allocation is about
1% of the workload, so a cheaper allocation would not help, and an escaping-map
lookup through a runtime shape check costs about the same as the guard it would
replace). The hint is the sound, opt-in lever on the part of the cost that can
move.

View file

@ -0,0 +1,303 @@
# RFC 0005 — Structural collection-type inference
- **Status**: Implemented. Ray tracer 12.8s to 11.0s hint-free,
matching the explicit `^:struct` version; render checksum unchanged.
- **Champions**: jolt maintainers
- **Created**: 2026-06-13
## Summary
Replace jolt's ad-hoc inference lattice with a single recursive **structural
type**, so that the type of a value mirrors the tree shape of the data it
describes. A struct-map carries its field types, a vector its element type, a
function its parameter and return types, recursively. A keyword lookup returns
the looked-up field's type, so nested access like `(:r (:direction ray))` is
typed end to end. This unifies the two facts the current inference tracks
inconsistently (a vector's element type, but not a map's field types), subsumes
the existing inference passes as special cases, and
closes the remaining ray-tracer gap without a hint. The system is a
soft-typing-style inference: it never rejects a program, it assigns a concrete
type only when it can prove one, and it falls back to `:any` (and the existing
runtime guard) everywhere else.
## Motivation
The existing inference specializes a collection access (drops the
`:jolt/type` guard, emits `pv-count`, and so on) when it can prove the
collection's type. It works, it is sound, and it is fully dynamic-fallback
safe. But its type lattice grew ad hoc:
- `:struct-map` means "a raw-get-safe map" but carries **no field types**.
- `{:vec ELEM}` carries its **element type**.
These are the same idea applied to two kinds of child in the data tree, but
only one is tracked. The cost is concrete: in the ray tracer a lookup result
like `(:direction ray)` is typed `:any`, so `(:r (:direction ray))` keeps its
guard, and the `vec3` functions (called all day with such values) cannot be
typed, so the inference reaches only about 3% where the explicit `^:struct`
hint reaches 22%. The hint wins precisely because it asserts the field/param
shape the inference fails to derive.
The fix is to make the type a structural tree, tagged as precisely as provable.
Then `:struct` tracking and field tracking are one mechanism, the special cases
collapse into one signature table, and nested access is typed by construction.
## The type lattice
A type `T` is one of:
- A scalar tag: `:num`, `:str`, `:kw`, `:bool`, `:char`. (Optionally a coarser
`:nonnil` for "provably not nil and not false", which is what the struct-vs-phm
decision needs; see below.)
- `:nil`.
- `{:struct {field -> T}}` — a raw-get-safe map (a record) whose
field `k` has type `(fields k)` or `:any` if absent. The degenerate
`{:struct {}}` is "a struct, fields unknown" and replaces today's
`:struct-map`.
- `{:vec T}` — a vector whose elements have type `T`.
- `{:set T}` — a set of `T`.
- `:phm` — a persistent hash map (NOT raw-get-safe; distinct from `:struct`).
- `{:fn {:params [T...] :ret T}}` — a function (optional precision; the current
flat param/return inference is the zero-arity-detail version of this).
- `:any` — the top. Anything not provably more specific.
- `:bottom` (represented as the absence of a type / `nil` internally) — the
identity for join, used to seed the fixpoint.
Types are immutable values comparable by structural equality, exactly like the
current `{:vec ELEM}` representation, so they flow across the portable
inference and the host unchanged.
### Join (least upper bound)
```
join(T, T) = T
join(bottom, T) = T
join({:struct a}, {:struct b}) = {:struct {k -> join(a[k]?:any, b[k]?:any) for k in keys(a) keys(b)}}
join({:vec a}, {:vec b}) = {:vec join(a, b)}
join({:set a}, {:set b}) = {:set join(a, b)}
join(_, _) = :any ; different constructors
```
Two struct types join field-wise; a field present in only one side becomes
`:any` in the result (it might be absent, so a lookup of it is not provably
typed). This is the standard record lattice.
### Termination: depth cap
Structural types of recursive data (a tree node that contains a tree node, a
cons cell) would be infinite. To keep types finite and the inter-procedural
fixpoint terminating, structural types are **depth-capped**: beyond a small
depth `D` (proposed `D = 4`) a child type is `:any`. Construction and join both
truncate at `D`. With the cap the lattice has finite height, so the monotone
fixpoint converges. The ray tracer's shapes (vec3 inside ray inside hit-info)
are depth 2 to 3, well inside the cap.
## Inference rules
Inference is a forward pass producing `[type node']` for each IR node (the
existing shape), threaded with a local type environment and the
inter-procedural state. The rules are uniform over the structural
type:
- **Literals.** `{:k v ...}` with constant scalar keys and struct-safe values
builds `{:struct {:k type(v) ...}}`; otherwise `:phm`. `[a b ...]` builds
`{:vec (join type(a) type(b) ...)}`. `#{...}` builds `{:set ...}`. Scalars
build their scalar tag. (The struct-vs-phm condition is the same as the back
end's: scalar keys, and every value provably non-nil and non-false.)
- **Lookup returns the field type.** `(:k m)` / `(get m :k)` where
`m : {:struct fs}` returns `(fs :k)` or `:any`. This is the single rule that
makes nesting work and that unifies field tracking with `:struct` tracking.
- **Indexing returns the element type.** `(nth v i)` / `(v i)` where
`v : {:vec T}` returns `T`. `(first v)` / `(peek v)` likewise.
- **Flow.** `let`/`loop` bind init types; `if` joins the branch types; `do`
takes the tail type. (As today.)
- **Calls use signatures.** Every call result type comes from the callee's
signature: core fns from a fixed signature table (below), user fns from the
inter-procedural fixpoint's inferred signature.
The inter-procedural fixpoint, recompile, escape gate, and closed-world
assumption are unchanged. They now propagate structural types instead of flat
tags.
## Core function signatures
The current special cases (`truthy-ret-fns`, `vector-ret-fns`, `elem-fns`,
`hof-table`, and the `conj`/`range`/`reduce`/`mapv` branches) collapse into one
table of **type schemes**, possibly parametric:
```
inc, dec, +, -, *, /, mod, ... : (... :num) -> :num
count : (Coll) -> :num
nth : ∀T. ({:vec T}, :num) -> T (3-arg adds a default: -> join(T, default))
get : ∀T. ({:struct fs}, :k) -> (fs :k) ; const key
first,peek : ∀T. ({:vec T}) -> T
conj : ∀T. ({:vec T}, x) -> {:vec join(T, type(x))}
assoc : ({:struct fs}, :k, v) -> {:struct (assoc fs :k type(v))} ; const key
vec, mapv : ... -> {:vec ...}
range : (...) -> {:vec :num}
rand-nth : ∀T. ({:vec T}) -> T
map, filter, mapv, filterv, reduce, ... ; see HOFs
```
Parametric schemes (the `∀T`) are where polymorphism actually matters, and they
give the element/field propagation for free. **Higher-order functions are just
schemes whose parameter is itself a function type**: `reduce`'s scheme says its
function argument is `(Acc, Elem) -> Acc` applied to the collection's element
type, so the closure's element parameter is typed by applying the scheme,
replacing the hand-written `hof-table`.
## Hints as seeds
`^:struct x` seeds `x : {:struct {}}` (a struct, fields unknown) at a unit
boundary the inference cannot see across. A future extension could allow a shape
hint `^{:r :num :g :num :b :num}` to seed field types, but once inference is
structural this is rarely needed; the hint stays the escape hatch for genuinely
dynamic boundaries, exactly as today.
## Soundness
Unchanged in spirit from the current system: a concrete type is assigned only
when proven (a literal genuinely has those fields; a fn provably returns that
shape), and everything unprovable is `:any`, which keeps the dynamic guard. A
wrong specialization is therefore impossible. The inter-procedural part keeps
the closed-world (optimization-mode) assumption already adopted, which is sound
under whole-program / source-distribution compilation.
## Compilation modes and defaults
Direct-linking — and the inference and specialization it enables — is the
**default for running a program** and stays **off for interactive work**, chosen
by the CLI run mode rather than a global opt-in flag:
| mode | linking | whole-program |
|---|---|---|
| `-m` / `-M NS` (program entry) | direct (default) | **auto** (closed world) |
| `FILE` / `-f` / stdin (`-`) | direct (default) | no (per-namespace) |
| `repl`, `-e`, `nrepl-server` | indirect / open | no |
A program run is a closed world — every namespace is required, then the code
runs to completion — so it direct-links: user code gets inlining, record shapes,
and the inference's specialization. A `-m` / `-M` entry is the exact point where
all requires are done and `-main` is about to run, so the whole-program
cross-namespace pass (below) runs there automatically. Interactive modes stay
open: a REPL, `-e`, and the nREPL server must let you redefine vars — which
direct-linking seals against — so they keep the indirect, live-deref path.
Env overrides, all winning over the mode default:
- `JOLT_NO_DIRECT_LINK=1` — force the open/indirect path even for a program run
(runtime redefinition, hot-reload, self-modifying code).
- `JOLT_NO_WHOLE_PROGRAM=1` — keep direct-linking but skip the whole-program
pass (per-namespace inference only).
- `JOLT_DIRECT_LINK=1` — force direct-linking on even in an interactive mode.
- `JOLT_WHOLE_PROGRAM=1` — force the whole-program pass on in any direct-linked
mode.
- `JOLT_NO_SHAPE=1` — disable the record/shape representation under direct-linking.
What direct-linking gives up is what Clojure's `:direct-linking` and jank's
`-Odirect-call` give up: a direct call embeds its callee, so redefining the
callee is not seen by already-compiled callers. Whole-program additionally
const-links stable vars (data defs, record types, `^:redef`), extending the same
trade. That is why the interactive modes stay open and the opt-outs exist.
### Cross-namespace inference
Per-namespace inference (a `FILE` run, or any namespace under
`JOLT_NO_WHOLE_PROGRAM`) types a function's parameters from the call sites it can
see **within that namespace**. A function whose record parameter is supplied by a
caller in *another* namespace is left `:any`, its field reads keep the guard, and
the values derived from it widen — so a decomposed program is markedly slower
than the same code in one namespace (measured at ~3.7× on the ray tracer split
across five namespaces). The information exists in the program; per-namespace
compilation just can't see a caller in a not-yet-loaded namespace. Two ways to
supply it:
1. **Whole-program** (auto for `-m` / `-M`) runs one closed-world inference
fixpoint over every loaded namespace before `-main`, typing each parameter
from its call sites wherever they live. Namespaces required later (inside
`-main`) fall back to per-namespace inference.
2. **Parameter type hints** (`^RecordType`, RFC 0004) declare the type directly,
so it also works in the open world — REPL, library code that must be fast for
any caller, and hot-reloading servers — where the world cannot be closed.
## Relationship to Hindley-Milner and soft typing
This is HM-shaped with two deliberate departures, which is the textbook
definition of **soft typing** (Wright and Cartwright, "A Practical Soft Type
System for Scheme", 1997 — HM extended with union types and a dynamic type).
Taken from HM:
- The **structural type language** (records, vectors, functions as type
constructors). This is the "tree of types".
- **Constraint propagation** and **type schemes** for the core library (the
`∀T` signatures). That parametric polymorphism is exactly what HM provides,
and it is where it matters (generic collection functions like `nth`,
`reduce`, `map`).
Changed, on purpose:
- Replace "unify or **fail**" with "**join over a lattice whose top is `:any`**".
The inference never rejects a program; an unprovable spot becomes `:any` and
keeps the runtime guard. This is the "fall back to dynamic when in doubt"
policy made principled.
- **Monovariant** for user functions (the inter-procedural fixpoint plus
inlining cover the practical polymorphism); parametric schemes are kept only
for core functions.
So: HM structural types and constraint propagation and core-fn schemes, solved
by lattice join with a dynamic top instead of unification-or-fail. Other AOT
inferencers for dynamic languages do the whole-program version of the same
thing (RPython's annotator, Crystal's global inference, Shed Skin), all with a
union/dynamic fallback.
## Implementation and migration
This is a refactor that **simplifies** the current code: it deletes the ad-hoc
tag soup and the per-op special cases and replaces them with one recursive type
plus a signature table.
1. Define the structural type, `join`, the depth cap, and the predicates
(`struct-safe?`, `field-type`, `elem-type`) in `jolt.passes`.
2. Rewrite `infer` so each op produces/consumes structural types: literals
build shapes; `(:k m)` returns the field type; calls consult the signature
table.
3. Move the core-fn knowledge into a signature table (subsumes the existing
tables and HOF handling).
4. The back end keeps reading the use-site type to specialize (guard drop for
`{:struct}`, `pv-count`/`pv-nth` for `{:vec}`), now uniformly.
5. Keep the inter-procedural fixpoint, recompile, escape gate, and triggering as
is; they propagate structural types.
The phases land incrementally behind the same optimization-mode gate, each
verified against conformance (three modes), the full test gate, and the
ray-tracer benchmark, exactly as the current phases were.
## Design problems and open questions
- **Recursion / termination.** Handled by the depth cap (`D = 4`). Open
question: is a fixed cap better than proper recursive (mu) types? A cap is
simpler and sound; mu-types are more precise but add complexity. Proposed:
start with the cap.
- **Compile-time cost.** Structural types are larger and the fixpoint does more
work. Mitigations: the depth cap bounds type size; inference runs only in
optimization mode; the fixpoint iteration count stays bounded. Needs
measurement on a large namespace (clojure.core itself) to confirm acceptable.
- **Heterogeneous data.** `[1 "a"]` joins to `{:vec :any}`; a map whose field
varies across branches joins that field to `:any`. Correct degradation, not a
problem, but worth stating.
- **Non-constant keys.** `(assoc m k v)` / `(:k m)` with a non-constant `k`
cannot track a specific field; the result degrades to `{:struct {}}` or
`:phm` as appropriate. Field tracking only applies to constant scalar keys.
- **`false`/`nil` field values.** A map literal is `{:struct ...}` only when
every value is provably non-nil and non-false (the back end stores such maps
as a phm). The `:nonnil` tag (or a per-type "provably truthy" predicate) is
what the literal rule needs; this must be carried correctly or struct
inference is unsound.
- **Function-type precision.** `{:fn ...}` is optional. The current flat
param/return inference is enough for the collection-specialization goal;
full function types matter more for the type-checker (RFC 0006) and could be
deferred.
- **Closed-world boundary.** Inherited from the inter-procedural pass:
param/return inference assumes the compiled unit is the whole program.
Documented there; unchanged.

View file

@ -0,0 +1,232 @@
# RFC 0006 — Compile-time detection of provably-wrong code (success typing)
- **Status**: Implemented. Core-fn error domains (arithmetic on non-numbers,
count/first/rest/next/seq/nth on non-seqable scalars), `JOLT_TYPE_CHECK=
off|warn|error`. Follow-ups landed: bounded scalar **unions** so a
use is reported only when every member is in the error domain; **user-fn
error domains** behind `JOLT_TYPE_CHECK_USER` (closed-world);
precise **file:line:col** locations. The checker is now one
inference walk (folded into `infer`), and is **on by default in direct-link
builds** — where it piggybacks on the specialization inference for ~free —
and opt-in (`JOLT_TYPE_CHECK`) in plain builds.
- **Champions**: jolt maintainers
- **Created**: 2026-06-13
- **Depends on**: RFC 0005 (structural collection-type inference)
## Summary
Reuse the structural type inference of RFC 0005 as a **loose type checker**: at
compile time, flag code that is *provably* wrong, accept everything that is
merely ambiguous, and never produce a false positive. Concretely, when an
expression's inferred type is concrete and the operation applied to it would
throw at runtime for that type (for example passing a string where a function
only ever operates on numbers), report a clear compile-time error pointing at
the offending form, with the inferred type and what was expected. When the type
is `:any`, a union that includes a valid case, or beyond the inference's depth
cap, accept it silently. This is **success typing** (the discipline behind
Erlang's Dialyzer), applied to jolt for free on top of the inference we already
need for optimization.
## Motivation
Once the compiler tracks concrete types for many values (RFC 0005), it can see
some programs that cannot possibly be correct: `(inc "x")`, `(first 5)`,
`(count :k)`, `(/ 1 "two")`. Today these compile and fail at runtime, often far
from the cause. Reporting them at compile time, with a precise location and
message, turns a class of runtime crashes into immediate, actionable feedback,
at no extra inference cost.
The design constraint the user set is the right one and is exactly success
typing's contract: **accept ambiguous cases, reject only provably-wrong ones.**
A checker that never lies about errors is one developers trust and that does not
get in the way of correct-but-untypeable dynamic code.
## Principle: success typing, never a false positive
Success typing (Lindahl and Sagonas, "Practical Type Inference Based on Success
Typings", 2006; the basis of Dialyzer) inverts the usual type-checker stance.
A normal checker accepts only what it can prove correct and rejects the rest
(false positives on dynamic code). A success typer accepts everything that
*could* be correct and rejects only what *cannot* be correct under any
execution. It is sound for **rejection**: if it reports an error, the code is
genuinely wrong. It is intentionally incomplete: it misses errors it cannot
prove. That is the correct trade for a dynamic language, and it matches the
user's "accept ambiguous, reject provably wrong".
Mapped onto jolt:
- The inference assigns a value a concrete type only when it can prove it
(RFC 0005). Unprovable is `:any`.
- A use site is reported **iff** the argument's inferred type is concrete and
lies entirely outside the operation's accepted domain, where the operation
*throws* on that domain (not merely returns a benign default).
- `:any`, a depth-capped child, or a union that includes an accepted type is
**never** reported.
## What "provably wrong" means
The checker needs, per operation it understands, an **error domain**: the set
of argument types for which the operation throws at runtime. This is narrower
than "the types it is documented to accept", because Clojure is lenient in many
places and flagging a benign case would be a false positive:
- `(get 5 :k)` returns `nil`, it does not throw. NOT reported.
- `(:k 5)` returns `nil`. NOT reported.
- `(count 5)` throws ("count not supported on number"). Reported when the
argument is provably a non-countable scalar.
- `(first 5)` throws (not seqable). Reported for a provably non-seqable scalar.
- `(inc "x")`, `(+ 1 "x")` throw. Reported when an argument is provably a
non-number (`:str`, `:kw`, `:struct`, `:vec`, ...).
- `(nth 5 0)` throws. Reported for a provably non-indexable scalar.
So the checker ships a curated table of the clearest throwing operations with
their error domains. It starts small (arithmetic on non-numbers, seq/`count`/
`nth`/`first` on non-seqables) and grows conservatively. Anything not in the
table is not checked, which is safe (no false positive).
A use site is reported only when:
1. the argument's inferred type `T` is concrete (not `:any`, not a union that
includes an accepted type, not truncated by the depth cap), and
2. `T` is in the operation's error domain (the operation provably throws on `T`).
## Examples
```clojure
(inc "x") ; ERROR: inc expects a number, got a string
(let [n "x"] (inc n)) ; ERROR: same, n inferred :str
(count :foo) ; ERROR: count not supported on :kw
(first 42) ; ERROR: 42 is not seqable
(:k 5) ; accepted (returns nil, not an error)
(inc (rand-nth coll)) ; accepted if the element type is :any/unknown
(inc (if c 1 "x")) ; accepted: union {:num, :str} includes :num (ambiguous)
(defn f [n] (inc n)) ... ; if f is ALWAYS called with strings in-unit, ERROR at the call;
; if its callers are unknown/varied, accepted
```
## Error reporting
A reported error includes:
- the source location (`file:line:col`) of the offending form;
- the operation and the parameter position;
- the inferred type of the argument, rendered readably (`:str`,
`{:struct {:r :num}}`, `{:vec :any}`);
- what the operation requires (`a number`, `a seqable`).
Example:
```
type error scene.clj:42:18: `inc` requires a number, but argument 1 is a string
```
Errors are attributed to the form the user wrote. For macro-expanded code, the
checker reports at the original form's recorded position (the loader already
tracks `:error-pos`), never at synthesized internals.
## Strictness levels
`JOLT_TYPE_CHECK` controls behavior:
- **off** — no checking.
- **warn** — report to stderr, do not fail compilation. **The default in
direct-link builds**, where checking rides the specialization inference for
~free; opt-in elsewhere.
- **error** — fail compilation on a provable type error. Opt-in for CI / strict
builds.
When `JOLT_TYPE_CHECK` is unset, checking is **on (`warn`) in direct-link
builds** and **off in plain REPL/dev builds** (where it would cost a standalone
inference pass, ~2.6× compile). `JOLT_TYPE_CHECK_USER` additionally enables
reporting against inferred user-function domains (closed-world; see below).
Because the checker only fires on provable errors, even `error` mode cannot
break a correct program: a correct program has no provable type errors to
report. (A correct-but-untypeable program is simply not reported, since its
types degrade to `:any`.)
## Soundness of rejection (no false positives)
The whole value of this feature is that a reported error is real. The
guarantees:
- The inference assigns concrete types only when provable (RFC 0005). So a
concrete `T` at a use site is a genuine lower bound on what flows there in the
analyzed world.
- The error-domain table lists only operations that genuinely throw on the
listed types, verified against the runtime.
- Ambiguity is always accepted: `:any`, unions containing an accepted type, and
depth-capped children are never reported.
Two boundaries need care and bound where the checker is allowed to fire:
- **Closed-world / redefinition.** Inter-procedural argument types assume the
compiled unit is the whole program (inherited from RFC 0005). For the checker,
this means a reported error on a *user* function's parameter is only as sound
as that assumption. The conservative initial policy: only report against
**core-function** error domains (stable, not redefinable) and against types
derived without crossing an open boundary. Reporting against inferred user-fn
signatures is a later, opt-in escalation.
- **Macros / generated code.** Check post-expansion IR but report at the user's
source location, and suppress reports inside expansions the user did not
write (or attribute them to the macro call site).
## Relationship to other systems
- **Dialyzer / success typing** (Erlang): the direct model — sound for
rejection, no false positives, accepts the ambiguous.
- **Typed Clojure / core.typed**: opt-in *sound* gradual typing that rejects
what it cannot prove correct; the opposite trade (false positives on dynamic
code), which is why we do not follow it.
- **clj-kondo**: a popular Clojure linter that flags some obvious type misuses
syntactically; this RFC subsumes the type-driven subset with inference-backed
precision and no false positives.
## Implementation
The checker is a thin pass over the same inference results:
1. After (or during) inference, walk the IR. At each call to an operation in
the error-domain table, look at the inferred type of each checked argument.
2. If concrete and in the error domain, record a diagnostic with location, the
inferred type, and the expected domain.
3. Emit diagnostics per the strictness level.
It adds no new inference; it consumes RFC 0005's types and a small curated
table. It can ship after RFC 0005 lands, starting in `warn` mode with the
smallest high-confidence table (arithmetic and seq/count/nth/first), and grow.
## Design problems and open questions
- **Curating the error domain.** The table must list only genuinely-throwing
cases. Getting it wrong (listing a lenient op) yields false positives, which
destroys trust. Mitigation: start tiny, test each entry against the runtime,
grow slowly. Open question: derive the table from the same machinery the
runtime uses, to avoid drift?
- **Unions.** *Resolved.* The lattice has a bounded scalar union
`{:union #{T...}}` (cap 4); differing if-branches form a union instead of
collapsing to `:any`, and a use is reported only when *every* member is in the
error domain. Unions are opaque to structural specialization, so codegen is
unchanged.
- **User-function signatures.** *Resolved, opt-in.* Behind
`JOLT_TYPE_CHECK_USER`: the checker re-checks a registered non-redefinable
user fn's body with one parameter bound to its concrete argument type; a
diagnostic the all-`:any` body did not have means that argument is provably
wrong. Monotonic, so still no false positives; closed-world, hence opt-in.
- **Negative/never types.** *Resolved.* Calling a provably
non-callable value (`:num`/`:str` — keywords/maps/vectors/sets are IFn) is
reported at the default level; wrong-arity to a registered single-fixed-arity
user fn is reported under the `JOLT_TYPE_CHECK_USER` opt-in. A union callee is
flagged only when every member is non-callable.
- **Position vs intent.** *Resolved.* The reader records each list
form's absolute offset (identity-keyed, so positions survive macroexpansion
exactly when the user's sub-form is spliced through); the analyzer stamps it
onto `:invoke` nodes, the checker carries it into each diagnostic, and the
back end renders `file:line:col`. Inlining/scalar-replace preserve it via
`assoc`.
- **Interaction with the optimization gate.** *Resolved (jolt audit).* The
checker is one inference walk folded into `infer`. In direct-link builds it
piggybacks on the specialization inference that already runs (~free, default
on); in plain builds it runs as a standalone pass only when `JOLT_TYPE_CHECK`
is set. "Run inference for checking" and "specialize from inference" are the
same walk now, gated by a `checking?` flag.

View file

@ -0,0 +1,186 @@
# RFC 0007 — Compilation modes and binary output
- **Status**: Draft. No code yet; this fixes the design before Phase 4 work
(beads `jolt-cf1q.5`) starts.
- **Champions**: jolt maintainers
- **Created**: 2026-06-22
## Summary
Give jolt a `jolt build` command that emits a standalone executable, and a
three-mode model that trades dynamism for speed:
- **dev** — open/indirect linking, redefinition works, no perf focus. What
`repl`/`-e`/`nrepl` already are.
- **release** (default for a built program) — direct-linked, closed-world,
per-namespace inference. Fast, still a recognizable Clojure runtime.
- **optimized** — whole-program inference, `fl*`/`fx*` typed emission, Chez
whole-program optimization. Fastest, sacrifices dynamic redefinition.
All three already have their machinery in the tree — the inference and inline
passes were ported into `jolt-core/jolt/passes/`. What is missing is (a) a code
path that writes emitted Scheme to disk and AOT-compiles it instead of
eval'ing it in process, and (b) a switch that turns the dormant passes on. This
RFC specifies both.
## Motivation
The Janet host could produce binaries (`jolt uberscript` with dead-code
elimination, `jolt cgen-build` for a single native binary). The Chez rehost
dropped that machinery with the Janet host — it was Janet-specific (IR→C made
sense when the host was Janet). On Chez the natural target is Chez's own native
compiler, so the old emitters were deleted rather than re-pointed.
The result today: `bin/joltc` only ever loads the checked-in seed and
compile-evals in process. `jolt.main/-main` dispatches `run / -M / -A / repl /
nrepl / task` and nothing else. There is no way to ship an app as a binary, and
the optimization passes are inert — `jolt.host/inline-enabled?` is a stub
returning `#f` (`host/chez/host-contract.ss:283`), so every call links
indirectly and nothing inlines. Jolt on Chez runs only in what this RFC calls
dev mode.
The passes themselves survived intact:
- `jolt/passes/types.clj` — structural collection-type inference (RFC 0005) +
success-type checking (RFC 0006).
- `jolt/passes/inline.clj` — inline + flatten-lets + scalar-replace, already
gated "direct-link only".
- `jolt/passes/fold.clj` — const-fold, including predicate folding.
So this is not a port of lost code. It is wiring: a build front-end, a
file-emitting back-end path, and a mode switch over passes that already exist.
## The three modes
| Mode | Linking | Inference | Redefinition | Driver |
|---|---|---|---|---|
| **dev** | indirect (var-deref per call) | off | yes | `repl`, `-e`, `nrepl`, `run` of a file by default |
| **release** | direct, closed-world | per-namespace | no (closed world) | `jolt build` default |
| **optimized** | direct + whole-program | whole-program fixpoint, `fl*`/`fx*` | no | `jolt build --opt` / `-M`-style entry |
The modes are points on one axis (how much the back end may assume is fixed),
not three code paths. Each mode is a setting of two independent knobs the passes
already understand:
- **direct-link?** — may a call to a var compile to a direct procedure
reference instead of a `var-deref`? Enables inlining and call-site folding.
Opt-out is per-target: a `^:redef` or `^:dynamic` var always links indirect.
- **whole-program?** — does inference see the whole reachable program at once
(closed world), so a record param's callers in other namespaces are visible
and its field reads specialize? Without it, inference is per-namespace and a
cross-ns param de-specializes to `:any` (the cross-ns penalty documented in
the `cross-ns-param-penalty` memory; declared `^RecordType` hints are the
open-world escape hatch).
```
dev: direct-link? = false whole-program? = false
release: direct-link? = true whole-program? = false
optimized: direct-link? = true whole-program? = true
```
`fl*`/`fx*` typed emission (unchecked flonum/fixnum Scheme ops) rides on
optimized: only whole-program inference proves the types that make dropping the
numeric-tower dispatch sound. Release keeps the tower.
## CLI surface
```
jolt build [-m NS | FILE] [-o OUT] [--opt] [--dev]
```
- Resolves `deps.edn` exactly as `run` does (reuse `jolt.deps`).
- Default mode is **release**. `--opt` selects optimized; `--dev` builds an
unoptimized binary (useful to ship a debuggable build, not for the REPL).
- `-o` names the output (default the entry ns / file stem).
- Output is a single executable: a Chez boot file plus the compiled program,
launched by a thin wrapper, or a fully linked image where the platform allows.
App libraries are baked in — no source roots needed at runtime.
Env opt-outs for the build (mirrors the Janet knobs, now keyed off the mode
rather than the run): `JOLT_NO_DIRECT_LINK` forces open linking even in a build,
`JOLT_NO_WHOLE_PROGRAM` keeps direct-link but per-namespace, `JOLT_WHOLE_PROGRAM=1`
forces whole-program. These already name the two knobs above.
## Emission pipeline
The in-process spine today (`host/chez/compile-eval.ss`) is, per form:
```
source → read → analyze (→ IR) → emit (→ Scheme string) → (eval (read …))
```
`jolt build` keeps everything up to `emit` and replaces the per-form `eval` with
accumulate-then-compile:
1. **Assemble the program.** Starting from the entry ns's `-main`, load the
transitive `require` graph (the loader already does this) and collect every
reachable top-level form, in dependency order, with its compile namespace.
2. **Dead-code elimination.** Re-target the uberscript DCE idea: compute
reachability from `-main` plus non-prunable forms, drop dead `defn`/`defn-`.
Bail to keep-all on `resolve`/`ns-resolve`/`requiring-resolve`/`find-var`/
`intern`/`eval`/`load-string` (anything that defeats static reachability);
keep and scan `defmethod`/`defrecord`/`extend` bodies so dispatch targets
stay live.
3. **Emit to a file.** Run `analyze → emit` for each surviving form under the
mode's knobs, concatenating the Scheme strings into one program source (the
core overlay prelude first, exactly as the seed image is built today).
4. **Compile.** Feed that source to Chez `compile-program` (release) or
`compile-whole-program` (optimized, which also lets Chez cross-module
inline), producing a compiled object, then link a boot file / wrapper into
the final executable.
Steps 34 are the only genuinely new back-end code. Step 2 is a re-port of a
deleted pass. Steps before them already run on every `joltc` invocation.
## Turning the passes on
`inline-enabled?` is the existing gate. Today `host-contract.ss` hardwires it to
`#f`. Under this RFC the build sets it (and a parallel `whole-program?` flag)
from the chosen mode before compiling, so:
- release: `inline-enabled?` → true, whole-program off. Per-ns inference and
inlining light up; `fl*`/`fx*` stays off.
- optimized: both on; the types pass runs its whole-program fixpoint and the
back end may emit unchecked numeric ops where a flonum/fixnum is proven.
No new pass is required to reach release — it is the ported passes, ungated.
## Staging
1. **Spike (de-risk Chez AOT).** Emit a trivial whole program to disk and prove
`compile-program` + boot/static link yields a standalone binary that runs.
This is the only real unknown.
2. **`jolt build` release.** Front-end + file-emitting back-end path + flip
`inline-enabled?` from the mode. Gate against the bench/corpus suites; binary
output must pass the corpus a `run` passes.
3. **DCE.** Re-port the reachability pass; gate with a test like the old
`uberscript-dce` case.
4. **Optimized.** Whole-program flag, `compile-whole-program`, `fl*`/`fx*`
emission. Gate on the bench suite (ray tracer, binary-trees) for size and
speed vs the spike baseline.
Each stage is TDD against the existing gates (`make test`, `make corpus`, the
`bench/` programs). Modes land behind the build command, so dev — the only mode
today — is unaffected until a stage proves out.
## Open questions
- **Static vs. boot-file linking.** A fully static Chez image is the smallest,
most portable artifact but the most work to link; a boot file plus a stub
launcher is the easy first cut. Spike decides which step 1 targets.
- **FFI in a built binary.** `jolt.ffi` loads native libraries at runtime; a
closed-world build still needs that to work. The build must bake the FFI
Clojure side and keep dynamic `dlopen` at run time.
- **Macro and `eval` at runtime.** Release/optimized are closed-world, but an
app that calls `eval`/`load-string` needs the compiler present. Either ship
the compiler image in the binary (larger) or reject those builds (the DCE
bail-out already detects the calls).
## Prior art in this repo
The optimization design these modes turn on is RFC 0004 (type hints), RFC 0005
(structural inference), RFC 0006 (success checking). The linking model — direct
linking as a per-unit property, `^:redef`/`^:dynamic` as the only opt-out — and
the cross-ns specialization penalty are recorded in beads memories
(`jolt-linking-model`, `cross-ns-param-penalty`). Phase 4 (`jolt-cf1q.5`) is the
tracking issue.

21
docs/rfc/README.md Normal file
View file

@ -0,0 +1,21 @@
# RFCs
Design notes for non-obvious language and compiler decisions. An RFC records *why*
a thing is built the way it is; the code is the source of truth for *how*.
| # | Title | Status | Governs |
| --- | --- | --- | --- |
| [0001](0001-language-specification.md) | A Specification for the Clojure Language | Draft | The conformance target — what "is Clojure" means for jolt. |
| [0002](0002-reader-conditional-features.md) | Reader-Conditional Feature Set | Accepted | `#?(...)` feature keys (`:jolt`, `:clj`, `:default`). |
| [0003](0003-transients.md) | Transients | Accepted | `transient`/`persistent!` semantics + the Chez mutable backing. |
| [0004](0004-type-hints.md) | Type hints + keyword-lookup specialization | Accepted | `^Type`/`^:struct` hints → the bare-`get` fast path. |
| [0005](0005-structural-type-inference.md) | Structural collection-type inference | Implemented | The `:struct`/`:vec`/`:set` lattice in `passes/types`. |
| [0006](0006-success-type-checking.md) | Success typing (provably-wrong-code detection) | Implemented | The error-domain checker in `passes/types`. |
| [0007](0007-compilation-modes-and-binary-output.md) | Compilation modes + binary output | Implemented (doc lags) | `release`/`--opt`/`--dev`, `--direct-link`, `--tree-shake`. |
RFC 0007's own status line still says "Draft, no code yet" — that is stale:
direct-linking and tree-shaking shipped (see [tools-deps.md](../tools-deps.md) and
`backend_scheme.clj` / `build.ss`). Two compiler features that grew alongside it —
**IR inlining** (`passes/inline.clj`, under `--opt`) and **numeric `fl*`/`fx*`
lowering** from `^double`/`^long` hints (`passes/numeric.clj`) — are not yet written
up as RFCs; their touch points are in [../MODULES.md](../MODULES.md).

View file

@ -0,0 +1,45 @@
# Seed ↔ Overlay Registry
Jolt is Clojure on Chez Scheme. `clojure.core` is built from two tiers that both
define `clojure.core`-facing vars, and for a handful of names *both* tiers carry
a definition. This document records how the two tiers relate and which copy is
authoritative.
## The two tiers
- **Native shims** (`host/chez/natives-*.ss`) bind a set of `clojure.core` vars
directly to Scheme runtime values via `def-var!` — collection constructors,
seq fns, numeric/string ops, and so on. These cover names the overlay assumes
exist as bare `clojure.core` vars but does not define itself.
- **The Clojure overlay** (`jolt-core/clojure/core/NN-*.clj`) defines the rest of
`clojure.core` in dependency-ordered tiers, loaded in order: `00-syntax`,
`00-kernel`, `10-seq`, `20-coll`, `25-sorted`, `30-macros`, `40-lazy`, `50-io`.
The overlay loads after the native shims. When an overlay tier `(defn X …)` for a
name a native shim already bound, the **overlay def shadows the native binding**
user code sees the overlay copy. The native binding then survives only if some
other native/runtime code still calls the Scheme value directly.
So a name's *home* is determined by two facts:
1. is it bound by a native shim? (the Scheme value is reachable from the runtime)
2. does an overlay tier `(defn X …)`? (the overlay copy is what user code sees)
## The compiled seed
`clojure.core` is compiled ahead of time into the checked-in seed
(`host/chez/seed/{prelude,image}.ss`) as Scheme `def-var!` forms. The seed's
source twin is the overlay (`jolt-core/clojure/core/*.clj` plus the stdlib
namespaces under `stdlib/clojure/`); `host/chez/emit-image.ss` re-emits the
prelude from those sources on Chez. The build is a byte-fixpoint: rebuilding from
an up-to-date seed reproduces it exactly.
## Consistency guard
There is no separate drift-check test for the registry. The self-hosting
fixpoint is the guard: after changing a seed source (a core tier, the compiler
namespaces, the host contract, the reader, or `emit-image.ss`) you must re-mint
the seed (`make remint`), and `make selfhost` fails if the checked-in seed and
its sources have drifted. So if the overlay's shadowing relationship changes, the
re-minted prelude changes with it, and the fixpoint check keeps source and seed
in agreement.

View file

@ -0,0 +1,88 @@
# Clojure Language Specification — Front Matter
**Edition**: Draft 1 · **Describes**: Clojure 1.12 (reference) · **Status**: in progress
This document specifies the Clojure programming language independently of any
implementation. See `docs/rfc/0001-language-specification.md` for motivation,
process, and scope.
## 1. Conformance terminology
The key words **MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT**, and **MAY**
are to be interpreted as described in RFC 2119.
- A statement marked **MUST** is normative: a conforming implementation
exhibits exactly this behavior, and the conformance suite tests it.
- **implementation-defined** marks behavior a conforming implementation must
document but may choose (e.g. the concrete error type thrown where the
reference throws a JVM exception class).
- **host-defined** marks behavior delegated to the host platform (e.g. what
`slurp` accepts as a source).
- **⚠ reference-divergence** marks a place where this spec deliberately
differs from observed reference behavior, with rationale; the reference
behavior is always recorded alongside.
## 2. Classification of the core surface
Every `clojure.core` var carries exactly one classification (dashboard:
`coverage.md`):
| Class | Meaning | Spec treatment |
|---|---|---|
| **portable** | semantics independent of host | full normative entry (§9) |
| **host-dependent** | portable *interface*, host-defined behavior | interface entry; behavior host-defined |
| **JVM-specific** | meaningful only on the JVM | catalogued in Appendix; not specified |
Initial classifications are mechanical and reviewable; reclassification is an
ordinary spec change.
## 3. The normative entry format
Each special form (§3) and portable var (§9) is specified as:
```
### name — since <version>
(signature ...) (signature ...)
Semantics
S1. <numbered normative statement, MUST/SHOULD/MAY>
S2. ...
Edge cases
E1. <nil / empty / bounds / wrong-type behavior normative>
Errors
X1. <what MUST throw; error TYPE is implementation-defined unless stated>
Examples
<executable; verified against the reference; sourced from ClojureDocs
where community-validated>
Conformance
S1 → <suite>/<test id>; E1 → ... (statements without a test: UNVERIFIED)
```
The **Conformance** field is load-bearing: every numbered statement names the
test(s) that verify it. A normative statement with no test is labeled
`UNVERIFIED` and is a defect in the spec.
## 4. Evidence and verification
Behavioral questions are settled in this order: differential testing against
the reference implementation → cross-dialect agreement in clojure-test-suite
→ ClojureDocs community examples (verified before inclusion) → reference
source (for intent). Conformance tests live in this repository
(the corpus `test/chez/corpus.edn`, run on Chez via `host/chez/run-corpus.ss`
and certified against reference JVM Clojure by `test/conformance/certify.clj`)
and in the cross-dialect clojure-test-suite.
## 5. Chapter plan
| § | File | Status |
|---|---|---|
| 1 | `01-evaluation.md` | planned |
| 2 | `02-reader.md` | **drafted** (grammar + reader-macro catalog; 2 divergences open) |
| 3 | `03-special-forms.md` | **exemplars written** (`if`, `let*`); catalog complete |
| 4 | `04-data-types.md` | planned (numeric-tower design note required) |
| 5 | `05-sequences.md` | planned (laziness contract from jolt Phase-5 work) |
| 6 | `06-namespaces-vars.md` | planned |
| 7 | `07-polymorphism.md` | planned |
| 8 | `08-macros.md` | planned |
| 9 | `09-core-library.md` | **exemplars written** (`first`, `reduce`, `parse-uuid`) |
| A | `coverage.md` | **generated** (regenerate: `python3 tools/spec_coverage.py`) |

255
docs/spec/02-reader.md Normal file
View file

@ -0,0 +1,255 @@
# §2 The Reader (Lexical Syntax)
**Status**: token grammar drafted; reader-macro catalog complete with
normative entries; #inst and literal-collapse divergences resolved.
Conformance: jolt `reader-forms-spec` + `reader-syntax-spec` (granularity
model: jank's per-construct corpus, 62 files under
`test/jank/{reader-macro,syntax-quote}` — adapted rows cited per entry).
The reader maps a stream of characters to *forms* (data). Reading is
independent of evaluation: every form the reader produces is a value of the
language (§4), and `read-string` exposes the reader as a function. Evaluation
of forms is §1's concern; only `quote`-family reader macros reference it here.
## 2.1 Tokens
Whitespace is space, tab, newline, return, **and comma** (`,` is whitespace —
S1). A `;` begins a comment to end of line (S2). Tokens:
```
form := literal | symbol | keyword | list | vector | map | set
| reader-macro-form
list := '(' form* ')'
vector := '[' form* ']'
map := '{' (form form)* '}'
literal := nil | boolean | number | string | character
nil := 'nil' boolean := 'true' | 'false'
```
- S3. A map literal MUST contain an even number of forms; duplicate keys
MUST be an error at read time.
- S4. A set literal (`#{…}`, §2.3) with duplicate elements MUST be an error
at read time.
### Numbers
```
integer := ['+'|'-'] (digits | '0' [xX] hexdigits | '0' octdigits | radixR digits)
float := ['+'|'-'] digits '.' digits? exponent? | ['+'|'-'] digits exponent
ratio := ['+'|'-'] digits '/' digits ; host-numeric-tower (§4 note)
exponent := [eE] ['+'|'-'] digits
```
- S5. Trailing `N` (BigInt) and `M` (BigDecimal) suffixes are part of the
grammar; their value semantics are the §4 numeric-tower question.
Implementations without those towers SHOULD read them as the nearest
numeric type and MUST document the choice. The Chez host carries the full
tower: `N` reads as an exact integer (arbitrary precision) and `M` as a real
BigDecimal — `1.5M`, `0.0M`, `3M` — with value equality ignoring scale
(`1.0M = 1.00M`), `(class 1.5M)``java.math.BigDecimal`, and `decimal?` true.
### Symbols and keywords
```
symbol := name | ns '/' name ; '/' alone names the division fn
keyword := ':' name | ':' ns '/' name | '::' name | '::' alias '/' name
```
- S6. Symbol constituent characters: alphanumerics and `* + ! - _ ' ? < > =
. $ & %` (with `%` and `&` further constrained inside `#()`); a symbol
MUST NOT begin with a digit; `.` and `/` have positional restrictions.
- S7. `::kw` MUST resolve to the current namespace at *read* time
(`::k` in ns `user` reads as `:user/k`); `::alias/k` resolves `alias` through
the current namespace's aliases. (Clojure raises a read error for an unknown
alias; jolt reads it as `:alias/k`.)
### Strings and characters
- S8. Strings are `"…"` with escapes `\" \\ \n \t \r \b \f \uNNNN \oNNN`.
- S9. Character literals: `\c`, the named set `\newline \space \tab
\return \backspace \formfeed`, unicode `\uNNNN`, octal `\oNNN`.
**Conformance** (2.1): jolt `reader-syntax-spec` "dispatch & sugar";
clojure-test-suite reader files; jank `form/*` literal dirs. S3/S4 duplicate
checks → UNVERIFIED (rows to add).
## 2.2 Quote-family reader macros
| Sugar | Reads as | |
|---|---|---|
| `'form` | `(quote form)` | S10 |
| `@form` | `(clojure.core/deref form)` | S11 |
| `^meta form` | form with metadata attached (see below) | S12 |
| `#'sym` | `(var sym)` | S13 |
| `` `form `` | syntax-quote (§2.4) | |
| `~form`, `~@form` | unquote / unquote-splicing — only within syntax-quote (S14: MUST error outside) | |
- S11. `@form` reads as `(clojure.core/deref form)` — the operator is the
fully-qualified `clojure.core/deref`, not a bare `deref`, so `@x` still
dereferences in a namespace that excludes and rebinds `deref`
(`(ns … (:refer-clojure :exclude [deref]))`), matching Clojure.
- S12a. `^:kw form``^{:kw true} form`; `^Sym form``^{:tag Sym} form`;
`^"str"``^{:tag "str"} form`. Multiple `^` stack, rightmost innermost,
merged left-over-right.
- S12b. Type hints are semantically transparent: a hint MUST NOT change a
program's result. Hints parse in every position they do in Clojure (params,
`let` bindings, `def` names, return position, arbitrary forms) and are
otherwise inert. As a non-normative optimization, jolt recognizes two hints
on a local as an assertion that a constant-keyword lookup may skip its
runtime representation guard: `^:struct` (a plain struct/record map) and
`^Name` where `Name` is a `defrecord`/`deftype`. The assertion is the
programmer's (an inaccurate hint yields a wrong lookup, like a wrong Clojure
`^String`); `JOLT_CHECK_HINTS=1` turns a violated hint into an error at no
cost to unchecked builds. See RFC 0004.
- S13a. `#'ns/sym` MUST denote the same var as `(var ns/sym)`:
`(= (var clojure.core/str) #'clojure.core/str)` is true.
**Conformance**: jolt `reader-forms-spec` "var-quote #'", "metadata ^",
"syntax-quote"; jank `var-quote/pass-qualified.jank`, `metadata/*`.
## 2.3 Dispatch (`#`) reader macros
| Form | Meaning | Entry |
|---|---|---|
| `#{…}` | set literal | S4 above |
| `#"…"` | regex literal — reads to a regex value; escaping is regex-level, not string-level (single `\d`) | S15 |
| `#(…)` | anonymous fn | S16 below |
| `#_form` | discard | S17 below |
| `#?(…)` / `#?@(…)` | reader conditional (+splicing) | S18 below |
| `##Inf ##-Inf ##NaN` | symbolic floats | S19 |
| `#tag form` | tagged literal | S20 below |
| `#! …` | shebang comment line (implementations SHOULD accept) | |
### S16 — anonymous function `#(…)`
- `#(body)` reads as `(fn [args…] (body))` with parameters derived from the
`%`-symbols appearing in body: `%``%1`, `%n` positional, `%&` the rest
parameter. Arity = highest `%n` mentioned (plus rest if `%&`).
- The `%`-symbols are collected from the WHOLE body, recursing through every
nested form including vector, map and set literals — `#(assoc {} :k %)`,
`#(hash-set % %2)` and `#(get {:t %} :t)` all see their `%`s. (A reader that
scanned only call forms would miscompile `#(identity {:text %})` as a 0-arg fn.)
- The synthesized parameters are auto-gensyms (their names carry the `#` suffix,
like Clojure's `p1__N#`), so an `#()` written inside a syntax-quote survives:
the params are mapped consistently and left unqualified rather than being
qualified to the current namespace (a qualified symbol is not a valid
parameter). E.g. `` `(map #(inc %) xs) `` expands correctly inside a macro.
- `#()` literals MUST NOT nest.
```clojure
(#(+ %1 %2) 1 2) ;=> 3
(apply #(apply + %&) [1 2 3]) ;=> 6
(map #(* % %) [1 2]) ;=> (1 4)
```
### S17 — discard `#_`
- `#_form` reads and discards the next form entirely (it is never evaluated).
- Discards compose: `#_ #_ a b` discards two following forms.
- `#_` inside collection literals removes the element: `[1 #_2 3]``[1 3]`.
### S18 — reader conditionals
- `#?(:feat₁ f₁ :feat₂ f₂ …)` reads as the form of the first feature key the
platform satisfies, else nothing. `:default` matches any platform.
`#?@(…)` splices a sequential form into the surrounding context.
- Feature keys are implementation-defined; each implementation MUST document
its feature set, and SHOULD follow the portable convention *own dialect key
+ `:default`*. Matching MUST be by **clause order** — the first clause whose
key the platform satisfies wins (`#?(:default 5 :clj 6)` is `5` everywhere)
— not by key priority. Implementations SHOULD provide a per-loading-context
compatibility override for foreign-dialect libraries. (jolt:
`#{:jolt :clj :default}` — jolt emulates `clojure.lang.*`/`java.*`, so it
reads the `:clj` branch of a `.cljc` library by default; a library can put a
`:jolt` branch first to override, or a loading context can call
`reader-features-set!`. History in RFC 0002.)
- Reader conditionals MUST be an error outside `.cljc`-style reading unless
the implementation documents otherwise.
### S19 — symbolic values
`##Inf`, `##-Inf`, `##NaN` read as the IEEE-754 values. `(= ##NaN ##NaN)` is
false; `(NaN? ##NaN)` is true.
### S20 — tagged literals
- `#tag form`: the reader resolves `tag` in the data-reader table and MUST
apply the reader function to the *read* form, yielding its result as the
read value. An unknown tag MUST be a read error (jank
`fail-unsupported-tag`).
- Built-in tags every implementation MUST provide: `#uuid "…"` → a UUID
value (§9 `parse-uuid` semantics — round-trips through printing), and
`#inst "…"` → an instant value: RFC3339 with partial-timestamp defaults
(`#inst "2020"``#inst "2020-01-01T00:00:00.000-00:00"`), equality by
instant (offset-normalized), `inst?`/`inst-ms` (epoch milliseconds), printed
canonically as `#inst "yyyy-MM-ddThh:mm:ss.fff-00:00"` and round-tripping. A
malformed timestamp MUST be an error.
**Conformance** (2.3): jolt `reader-forms-spec` "#() (% %N %&)" + new rows
(symbolic values, stacked discard, conditionals); `uuid-spec` reader-literal
group; jank `reader-macro/{function,regex,uuid,symbolic-value}/*`,
`fail-unsupported-tag.jank`.
## 2.4 Syntax-quote
Syntax-quote (`` ` ``) is read-level template construction with namespace
resolution:
- S21. Inside syntax-quote, an unqualified symbol that resolves in
`clojure.core` MUST be qualified to `clojure.core/sym`; a symbol resolving
through a namespace alias MUST be qualified to the aliased namespace; an
unresolved symbol MUST be qualified to the current namespace. Special-form
names stay bare.
- S22. `sym#` generates a fresh symbol, stable *within one syntax-quote
template* (all `sym#` in the same template denote the same generated
symbol; distinct templates generate distinct symbols).
- S23. `~form` inserts the value of `form`; `~@form` splices a sequential
value; `~'sym` is the idiom for an intentionally-unqualified symbol.
- S24. Syntax-quote distributes through collection literals (vectors, maps,
sets) — qualification and unquoting apply inside them.
- S25. A syntax-quoted self-evaluating literal is the literal, collapsed at
read time — so nested/adjacent backticks over literals are inert:
`(= "meow" ```"meow")` is true. General nested syntax-quote over symbols
and collections expands recursively (quasiquote semantics) — that general
case remains UNVERIFIED pending dedicated conformance rows.
**Conformance**: jolt `reader-forms-spec` "syntax-quote" (gensym, unquote,
splice) + conformance "syntax-quote fully-qualifies"; jank
`syntax-quote/{pass-gensym,pass-namespace-resolution,pass-resolve-alias,
unquote,unquote-splice}/*`. S25 → UNVERIFIED.
## 2.5 What the reader is not
The reader performs **no macroexpansion and no evaluation** (tagged-literal
reader functions are the deliberate exception, S20). Forms read identically
whether or not they will be evaluated; `read-string` of any printable value
`v` followed by evaluation yields a value equal to `v` for the
self-evaluating types (§4 print/read round-trip contract).
## Strict tokens and edn mode
The reader rejects what the reference rejects (corpus `edn / strictness`,
`reader / strict tokens`):
- A token that starts like a number but doesn't parse as one is
NumberFormatException, never a symbol: `1a`, `08` (a leading zero demands
octal digits; `042` is 34), `0x2g`, `2r2`. A ratio's parts are plain digit
runs (`1/-1` is invalid); a zero denominator is ArithmeticException.
- Empty ns/name parts are invalid tokens: `:`, `::`, `foo/`, `/foo`, `:/foo`.
`/` (division), `ns//` and `:/` (a name of exactly `/`) are valid.
- Map literals with duplicate keys and set literals with duplicate elements
throw IllegalArgumentException at read.
- An unsupported string escape (`"\q"`) and an octal escape past `\377`
(string or `\o` char) throw. A stray close delimiter at top level is
"Unmatched delimiter". `\r` terminates a line comment like `\n`.
- `#inst` validates its calendar fields progressively (month 112, day valid
for the month including leap years, hour < 24, minute < 60); `#uuid`
demands canonical 8-4-4-4-12 hex.
clojure.edn adds on top of that (`__read-form-edn` seam): auto-resolved
keywords (`::k`) are invalid (no resolution context), each `#_` discarded
form is validated through the same `:readers`/`:default` pipeline (an
unreadable tagged element throws even when discarded), `M` literals
construct BigDecimals, lists satisfy `list?`, and end-of-input honors the
`:eof` option — an opts map without `:eof` makes EOF an error, while the
no-opts arity returns nil.

View file

@ -0,0 +1,155 @@
# §3 Special Forms
**Status**: catalog complete; normative exemplars for `if` and `let*`; the
remaining entries follow the same format (tracked in `coverage.md`).
A *special form* is a form whose head symbol is evaluated by rule rather than
by function application or macroexpansion. The special forms of Clojure are:
> `def` · `if` · `do` · `let*` · `fn*` · `loop*` · `recur` · `quote` · `var`
> · `throw` · `try`/`catch`/`finally` · `set!` · `monitor-enter` ·
> `monitor-exit` (host) · the interop forms `.` and `new` (host)
`let`, `fn`, `loop`, `and`, `or`, `when`, … are **macros** over these (§8);
implementations MUST treat them as redefinable macros, not additional special
forms. `monitor-enter`/`monitor-exit`, `.` and `new` are host forms: their
syntax is specified here, their behavior is host-defined.
Special-form head symbols are not shadowable: a binding named `if` does not
change the meaning of `(if ...)` in operator position. ⚠ This matches the
reference; it differs from Scheme. A local *may* legally be named like a special
form and used in value position (`(let [if 5] if)``5`); only operator
position is reserved. **Macros, unlike special forms, ARE shadowable** by a local
(`(let [when (fn ...)] (when 1 2))` calls the local).
A list form in operator position is resolved in this order (the canonical
read → **macroexpand** → analyze pipeline): a local binding shadows everything;
otherwise a macro head is expanded and the result re-analyzed; otherwise a
special-form head is parsed by rule; otherwise the form is a function
application. Macroexpansion therefore happens *before* special-form dispatch, so
a macro is never mistaken for a special form (and vice versa).
---
### if — since 1.0
```
(if test then)
(if test then else)
```
**Semantics**
- S1. `test` MUST be evaluated first, exactly once.
- S2. Every value other than `nil` and `false` is *logically true*. If the
value of `test` is logically true, `then` MUST be evaluated and its value
returned; otherwise `else` (or `nil` when absent) MUST be evaluated and its
value returned.
- S3. The branch not taken MUST NOT be evaluated.
- S4. `if` MUST be usable in tail position with respect to `recur` (§3
`recur`): an `if` whose branch is a `recur` form is a valid recur target
path.
**Edge cases**
- E1. `(if test then)` with a logically false `test` evaluates to `nil`.
- E2. The empty collections (`()`, `[]`, `{}`, `#{}`), the number `0`, and
the empty string `""` are logically **true** (only `nil`/`false` are
false). ⚠ This differs from several Lisps and is a frequent divergence
source in alternative implementations.
**Errors**
- X1. `(if)` and `(if test)` with fewer than two argument forms, or more
than three, MUST be a compile-time error.
**Examples**
```clojure
(if 0 :t :f) ;=> :t
(if "" :t :f) ;=> :t
(if nil :t :f) ;=> :f
(if false :t) ;=> nil
```
**Conformance**
S1S3, E1E2 → jolt `forms-spec` "if/do/def" group; truthiness group in
`truthiness-spec`; clojure-test-suite `core_test/if.cljc`. S4 → `forms-spec`
fn/loop recur cases. X1 → `forms-spec` "if arity (X1)" (0/1/4-arg forms throw
in both the analyzer and the interpreter).
---
### let* — since 1.0
```
(let* [sym₁ init₁ … symₙ initₙ] body…)
```
`let*` is the primitive sequential-binding form. The user-facing `let` macro
adds destructuring and expands to `let*` (§8); `let*` itself accepts **only
simple symbols** in binding positions.
**Semantics**
- S1. Each `initᵢ` MUST be evaluated in order, exactly once, in an
environment where `sym₁…symᵢ₋₁` are bound to their values (sequential
scope, as Scheme `let*`).
- S2. The body forms MUST be evaluated in order with all bindings in scope;
the value of the last body form is the value of the `let*` form. An empty
body evaluates to `nil`.
- S3. A later binding MAY rebind the same symbol; each binding creates a new
lexical binding visible from the next init onward (no mutation of the
earlier binding is implied).
- S4. Bindings are lexical and immutable: there is no form that assigns to a
`let*`-bound local. (Closures capture bindings by value; see §3 `fn*`.)
- S5. The binding vector MUST be a vector literal with an even number of
forms.
**Edge cases**
- E1. `(let* [] body)` is valid and equivalent to `(do body…)`.
- E2. Binding a symbol that names a var shadows the var for the lexical
extent of the body; `(var sym)` within that extent still denotes the var.
**Errors**
- X1. An odd number of binding forms MUST be a compile-time error.
- X2. A non-symbol in a binding position (e.g. a destructuring pattern) MUST
be a compile-time error for `let*` — destructuring belongs to the `let`
macro. ("Bad binding form, expected symbol" in the reference.)
**Examples**
```clojure
(let* [a 1 b (+ a 1)] (* a b)) ;=> 2
(let* [x 1 x (inc x)] x) ;=> 2
(let* [] 42) ;=> 42
```
**Conformance**
S1S3, E1 → jolt `forms-spec` let group; clojure-test-suite
`core_test/let.cljc`; jank corpus `form/let/*`. X2 → jolt
`destructuring-spec` "primitives reject patterns". S4, X1 → UNVERIFIED
(cases to add).
---
## Remaining entries (format above; status in coverage.md)
| Form | Notes for the entry author |
|---|---|
| `def` | var creation vs re-binding; metadata on the name; `(def x)` unbound; return value is the var |
| `do` | empty `(do)` → nil; top-level `do` splices for compilation units (important and under-documented) |
| `fn*` | arities, variadic `&`, closure capture, self-name, simple-symbol params only, recur target |
| `loop*` | recur arity must match bindings; recur rebinds in place |
| `letfn` | mutually-recursive local fns (`letrec*` semantics — a fn body sees every binding, not only earlier ones). jolt treats `letfn` as a primitive special, not the reference's `letfn` macro → `letfn*` indirection; behavior is identical |
| `recur` | tail-position rule (normative definition of tail position needed), across `if`/`do`/`let*`/`try` interactions |
| `quote` | self-evaluation table: which literals are self-evaluating unquoted |
| `var` | `#'` reader sugar; resolution at compile time |
| `throw` | any value vs Throwable — host question; jolt/cljs allow data, reference requires Throwable → classification needed |
| `try/catch/finally` | catch dispatch order, `:default`-style catch-all is a dialect extension (⚠ divergence note), finally evaluation guarantees, value of try |
| `set!` | three targets, all implemented: `(set! *var* val)` sets the var's innermost thread binding (else root); `(set! field val)` inside a `deftype` method mutates a `^:unsynchronized-mutable`/`^:volatile-mutable` field in place; `(set! (.-field obj) val)` does the same via interop syntax. Returns val |
| `.` / `new` | syntax only; behavior host-defined |

View file

@ -0,0 +1,366 @@
# §9 The Core Library
**Status**: entry format fixed; exemplars for `first`, `reduce`, `parse-uuid`.
The full portable surface (≈500 vars after classification, dashboard in
`coverage.md`) is filled in chapter-by-chapter using this format.
Entries specify *behavioral contracts*, not implementations. Performance
characteristics are specified only where the language community relies on
them (e.g. vector `nth` is "effectively constant time" — SHOULD-level).
---
## Collection return types & laziness (cross-cutting)
Two contracts hold across the sequence library and are not restated per entry.
**Return-type fidelity.** A function returns the same *kind* of collection the
reference does — value equality is not enough, since `(= [0 1] '(0 1))`.
- Sequence transformations return **seqs** (lazy unless noted): `map`, `filter`,
`remove`, `keep`, `mapcat`, `take`/`drop` and their `-while` forms, `partition`,
`partition-all`, `partition-by`, `interpose`, `dedupe`, `distinct`, `concat`,
`reductions`, `cons`, `rest`, `sequence`. The *elements* of `partition` /
`partition-all` / `partition-by` are themselves seqs, not vectors.
- The vector variants return **vectors**: `mapv`, `filterv`, `vec`, `subvec`,
`partitionv`, `partitionv-all`, `splitv-at`. `split-at` / `split-with` return a
2-vector `[take drop]`. A transducer applied eagerly (`into []`, the
`partition-all` transducer's chunks) yields vectors.
- Type-preserving functions return the input's type: `replace` over a vector is a
vector, over any other seqable a (lazy) seq; `empty`/`into (empty coll)` keep the
collection kind; `set`/`into #{}` return sets; `into {}`/`select-keys`/`zipmap`/
`frequencies`/`group-by`/`merge` return maps (`group-by` values are vectors).
**Laziness.** The lazy sequence functions — including `sequence`, `eduction`, and
`mapcat` — MUST consume their source incrementally and so terminate on an infinite
or unbounded source when only a prefix is demanded: `(first (sequence (map inc)
(range)))` and `(take n (mapcat f (range)))` return without realizing the whole
source. `(apply concat coll-of-colls)` is likewise lazy in its argument seq. The
eager consumers (`reduce`, `into`, `count`, `vec`, `doall`) realize the demanded
portion fully.
These are exercised by the `seq / lazy over infinite` and the per-fn type-predicate
rows in the conformance corpus.
---
### first — since 1.0
```
(first coll)
```
**Semantics**
- S1. MUST return the first element of `(seq coll)`.
- S2. If `(seq coll)` is `nil` (i.e. `coll` is empty or `nil`), MUST return
`nil`.
- S3. MUST accept anything *seqable* (§5): seqs, lists, vectors, maps
(yielding map entries), sets, strings (yielding characters), `nil`.
- S4. On a lazy sequence, MUST realize at most the first element (§5
laziness contract).
**Edge cases**
- E1. `(first nil)``nil`; `(first [])``nil`; `(first "")``nil`.
- E2. A `nil` or `false` first *element* is returned as-is — callers cannot
distinguish "empty" from "first element is nil" via `first` alone (that is
what `seq` is for).
- E3. On a map, the element is a map entry; on an unordered collection (map,
set) *which* element is first is implementation-defined but MUST be
consistent with that collection's seq order for the same collection value.
**Errors**
- X1. A non-seqable argument (e.g. a number) MUST throw.
**Examples**
```clojure
(first [1 2 3]) ;=> 1
(first '()) ;=> nil
(first "ab") ;=> \a
(first {:a 1}) ;=> [:a 1]
(first [nil 2]) ;=> nil
```
**Conformance**
S1S3, E1E2 → jolt `sequences-spec` "seq / access"; clojure-test-suite
`core_test/first.cljc`. S4 → jolt `lazy-seqs-spec` counter cases. X1 →
clojure-test-suite `core_test/first.cljc` (throwing cases).
---
### reduce — since 1.0
```
(reduce f coll)
(reduce f init coll)
```
**Semantics**
- S1. With `init`: MUST return `init` if `(seq coll)` is nil; otherwise MUST
return `(f … (f (f init e₁) e₂) … eₙ)`, applying `f` left-to-right over the
elements, exactly once each.
- S2. Without `init`: if `coll` is empty, MUST return `(f)` (f called with
no arguments); if `coll` has one element, MUST return that element
*without calling `f`*; otherwise as S1 with `init = e₁` over `e₂…eₙ`.
- S3. **Reduced short-circuit**: if any intermediate result is a `reduced`
value, iteration MUST stop and the dereferenced value MUST be returned
immediately; `f` MUST NOT be called again.
- S4. `reduce` is eager: it MUST fully realize the consumed portion of a
lazy `coll` (to the end, or to the `reduced` point).
**Edge cases**
- E1. `(reduce f nil)``(f)`; `(reduce f init nil)``init`.
- E2. A `reduced` value as the *initial* `init` is NOT unwrapped before the
first call in the reference — ⚠ under-documented; differential result to
pin down and test before this entry is marked verified.
- E3. Visit order over maps is entry order of the map's seq;
over vectors/lists/seqs it is sequential order (normative).
**Errors**
- X1. Without `init`, on an empty coll, if `f` has no zero-arg arity the
call `(f)` MUST throw (arity error).
**Examples**
```clojure
(reduce + [1 2 3 4]) ;=> 10
(reduce + 10 [1 2 3 4]) ;=> 20
(reduce + []) ;=> 0 ; (+) is 0
(reduce + [5]) ;=> 5 ; f not called
(reduce (fn [a x] (if (> a 2) (reduced a) (+ a x))) 0 [1 2 3 4 5]) ;=> 3
```
**Conformance**
S1S3, E1 → jolt `sequences-spec` "map filter reduce" group +
`transducers-spec` "reduce honors reduced"; clojure-test-suite
`core_test/reduce.cljc`. S2 (single-element, f-not-called) → jolt conformance
"reduce single no init". E2 → UNVERIFIED (differential test to add). S4 →
`lazy-seqs-spec`.
---
### parse-uuid — since 1.11
```
(parse-uuid s)
```
**Semantics**
- S1. If `s` is a string in canonical UUID form — five groups of hex digits
of lengths 8, 4, 4, 4, 12 separated by `-` — MUST return a UUID value `u`
such that `(uuid? u)` is true and `(str u)` is the lowercase form of `s`.
- S2. Parsing MUST be case-insensitive and equality on the results
case-insensitive: `(= (parse-uuid s) (parse-uuid (upper-case s)))` is true.
- S3. If `s` is a string not in canonical form, MUST return `nil`.
⚠ reference-divergence: reference Clojure (java.util.UUID) additionally
accepts non-canonical forms like `"0-0-0-0-0"`; ClojureScript and other
dialects are strict. This spec adopts **strict** (the cross-dialect
behavior); the reference's permissiveness is recorded as host leniency.
- S4. UUID values MUST support value equality, hashing (usable as map keys
and set members), `str` (lowercase canonical form), and print as the
tagged literal `#uuid "…"` such that the printed form reads back equal
(§2 tagged literals).
**Edge cases**
- E1. `""`, over-long, truncated, non-hex characters, and misplaced dashes
`nil`.
**Errors**
- X1. A non-string argument MUST throw.
**Examples**
```clojure
(parse-uuid "b6883c0a-0342-4007-9966-bc2dfa6b109e") ;=> #uuid "b6883c0a-…"
(uuid? *1) ;=> true
(parse-uuid "df0993") ;=> nil
(parse-uuid 1000) ;; throws
```
**Conformance**
S1S4, E1, X1 → jolt `uuid-spec` (30 cases) + 6 three-path conformance
cases; clojure-test-suite `core_test/parse_uuid.cljc`,
`core_test/uuid_qmark.cljc`, `core_test/random_uuid.cljc`.
---
### clojure.template/apply-template, clojure.test/are — since 1.1
```
(apply-template argv expr values)
(are argv expr & args)
```
**Semantics**
- S1. `apply-template` MUST replace every occurrence of each `argv` symbol
in `expr` with its corresponding value by structural walk (postwalk symbol
substitution), not by lexical binding. Occurrences inside `quote` and at
any nesting depth substitute: `(apply-template '[x] '(f 'x) '[if])`
`(f 'if)`.
- S2. `do-template` MUST partition `args` by `(count argv)` and expand to a
`do` of one substituted `expr` per group.
- S3. `clojure.test/are` MUST expand through `do-template` with `expr`
wrapped in `is`. Consequently `(are [x] (special-symbol? 'x) if def)`
asserts `(special-symbol? 'if)` and `(special-symbol? 'def)` — a
let-binding implementation is non-conforming (the quoted symbol would not
substitute).
**Errors**
- X1. `are` MUST throw at macroexpansion when `(count args)` is not a
positive multiple of a non-empty `(count argv)` (empty/empty is allowed).
- X2. `apply-template` MUST throw when `argv` is not a vector of symbols.
**Conformance**
S1S3 → `test/chez/clojure-test.clj` (are with quoted template var);
clojure-test-suite `core_test/special_symbol_qmark.cljc` and every
`are`-based suite namespace.
---
### make-hierarchy, derive, underive, isa?, parents, ancestors, descendants — since 1.0
```
(make-hierarchy)
(derive tag parent) (derive h tag parent)
(underive tag parent) (underive h tag parent)
(isa? child parent) (isa? h child parent)
(parents tag) (ancestors tag) (descendants tag) ; + (f h tag) forms
```
**Semantics**
- S1. A hierarchy is a pure value `{:parents {tag #{...}} :ancestors {...}
:descendants {...}}`; the 3-arity forms are pure, the shorter arities read and
mutate the global hierarchy.
- S2. `isa?` is true when `(= child parent)`, when the host type system says
parent is assignable from child (both classes), when the relationship was
`derive`d — including a relationship derived on one of a class child's
supers — or component-wise for equal-length vectors.
- S3. Class tags answer through the host type hierarchy: `(parents c)` includes
the class's direct supers (`bases` — a concrete class's chain roots at
`java.lang.Object`, an interface's does not); `(ancestors c)` is the
transitive set plus anything `derive`d on the class or its supers. A
deftype/defrecord class's ancestry includes its implemented protocol
interfaces and, for records, the record interfaces
(`clojure.lang.IRecord`/`IPersistentMap`/`Associative`/…; `clojure.lang.IType`
for a bare deftype).
- S4. `derive` returns the updated hierarchy (3-arity) or nil (2-arity);
deriving a relationship that already holds transitively, or one that would
create a cycle, throws.
**Errors**
- X1. `derive` asserts its argument shapes: parent must be a namespaced Named
value; tag must be a class or a Named value (namespaced in the 2-arity
global form); `(derive h tag tag)` fails the `not=` assert. AssertionError.
- X2. `underive`/`derive` with a non-hierarchy `h` throw at the parents
lookup (the map is called as a function, like the reference).
- X3. `(descendants h SomeClass)` throws UnsupportedOperationException
("Can't get descendants of classes") — Java type inheritance is not
enumerable downward.
**Conformance**
S1S4, X1X3 → corpus `hierarchy / *` rows; clojure-test-suite
`core_test/{derive,underive,isa_…,parents,ancestors,descendants}.cljc`
(all fully passing).
---
### atom, add-watch, remove-watch, set-validator!, get-validator — since 1.0
```
(atom x & {:keys [meta validator]})
(add-watch iref key f) (remove-watch iref key)
(set-validator! iref f) (get-validator iref)
```
**Semantics**
- S1. Watches, validators, and reference metadata are one contract (the JVM's
ARef/IRef) shared by atoms, vars, and agents. `add-watch`/`remove-watch`
return the reference; re-adding a key replaces that watch in place.
- S2. A watch is called `(f key ref old new)` after a state change: atom
swap!/reset!/compare-and-set!, var ROOT changes (`def` on a watched var,
`var-set` outside a thread binding, `alter-var-root` — a thread-binding set
does not notify), and each agent action's state change.
- S3. A validator gates every state change and, via the `:validator` ctor
option, the initial value — an invalid initial value never constructs the
reference.
- S4. The `:meta` ctor option attaches reference metadata (`meta` reads it,
`alter-meta!`/`reset-meta!` update it); nil is allowed.
**Errors**
- X1. A rejected value (validator returns logical false or the ctor option
fails on the initial value) throws IllegalStateException "Invalid reference
state".
- X2. A non-map `:meta` ctor option throws ClassCastException.
**Conformance**
S1S4, X1X2 → corpus `iref / *` rows; clojure-test-suite
`core_test/{atom,add-watch,remove-watch}.cljc` (the remaining baselined error
in the watch namespaces is their STM `ref` section — refs are out of scope,
`stm-refs` in `coverage.md`).
---
### clojure.string coercion, some-fn, ifn? — since 1.2/1.3
```
(clojure.string/upper-case s) … (some-fn p & ps) (ifn? x)
```
**Semantics**
- S1. The clojure.string case fns and searches (`upper-case`, `lower-case`,
`capitalize`, `starts-with?`, `ends-with?`, `includes?`, `index-of`,
`replace`) take any Object `s` through its `toString`, like the reference's
`^CharSequence`+`.toString` signatures: `(upper-case :kw)` is `":KW"`,
`(capitalize 1)` is `"1"`. nil throws (method call on null); a nil `substr`
throws.
- S2. `some-fn` follows the reference arities: at least one predicate
(`(some-fn)` is an arity error) and the returned fn chains with `or`, so a
no-match result is the last predicate's own falsy value (`false` stays
`false`).
- S3. `ifn?` covers fns, keywords, symbols, maps, sets, vectors, vars,
multimethods, promises (invoking a promise delivers it), and a
deftype/defrecord implementing `clojure.lang.IFn`'s `invoke`.
- S4. A `defmulti`/`defmethod` deferred inside a fn body interns/resolves in
the namespace it was WRITTEN in (the macros bake their expansion ns), not
whatever namespace is current when it runs.
**Conformance**
S1S4 → corpus `string / toString coercion`, `core / some-fn`, `core / ifn?`,
`multimethods / deferred definition`; clojure-test-suite string/some-fn/
ifn-qmark/boolean-qmark/reduce namespaces (all fully passing).
---
## Authoring notes
- Source examples from the ClojureDocs export (`clojuredocs-export.edn`,
648 core vars have community examples) — but every example is verified
against the reference before inclusion.
- When writing an entry surfaces a behavior question, settle it by
differential test first; if dialects split, that's a classification
decision (host-dependent / divergence note), not a coin flip.
- An entry is **Verified** when no field carries UNVERIFIED; `coverage.md`
tracks per-var status.

55
docs/spec/README.md Normal file
View file

@ -0,0 +1,55 @@
# The Clojure Language Specification (Draft)
A normative, implementation-independent specification of the Clojure
language, developed alongside jolt's self-hosted compiler and validated by
its executable conformance suites. **Why**: Clojure has no spec — every
alternative implementation re-derives semantics from the reference
implementation and folklore. See the RFC for motivation, scope, evidence
sources, and process: [`../rfc/0001-language-specification.md`](../rfc/0001-language-specification.md).
## Documents
| Doc | Content | Status |
|---|---|---|
| [`00-front-matter.md`](00-front-matter.md) | conformance terms, entry format, host classification | drafted |
| [`02-reader.md`](02-reader.md) | token grammar + reader-macro catalog | drafted |
| `01`, `04``08` | see chapter plan in front matter | planned |
| [`03-special-forms.md`](03-special-forms.md) | special-form catalog + normative exemplars (`if`, `let*`) | exemplars |
| [`09-core-library.md`](09-core-library.md) | per-var entry format + exemplars (`first`, `reduce`, `parse-uuid`) | exemplars |
| [`coverage.md`](coverage.md) | generated dashboard over the 694-var surface | generated |
| [`../grammar.ebnf`](../grammar.ebnf) | reader surface syntax (EBNF), companion to `02-reader.md` | reference |
Regenerate the dashboard after surface changes:
`python3 tools/spec_coverage.py` (reads `tools/clojuredocs-export.json` and
probes a working jolt checkout via `bin/joltc`).
## Current numbers (2026-06-22)
Of the 694 `clojure.core` vars in the ClojureDocs inventory, jolt interns 574.
Broadly:
- **568** implemented in jolt *and* exercised by the behavioral suites
- **6** implemented but not directly tested — each gets a test with its spec entry
- **6** portable but absent from jolt's resolvable surface (the REPL history
vars `*1`/`*2`/`*3`/`*e`, plus `letfn`/`re-groups`, which work but aren't
interned where `resolve` can see them) — tracked as gaps
- the rest classified host/JVM/concurrency (see the dashboard for the full
per-var breakdown — it is the source of truth)
## How this connects to the test suites
- `test/chez/corpus.edn` — the host-neutral behavioral corpus, one row per
case (`{:suite :label :expected :actual}`). The Chez compiler evaluates each
case via `host/chez/run-corpus.ss` (run with `make corpus`), and
`test/conformance/certify.clj` certifies every `:expected` against reference
JVM Clojure (run with `make certify`). Spec entries cite these cases.
- `test/conformance/` — the certification tooling and classified divergences
(`certify.clj`, `known-divergences.edn`); see its `README.md` and `SPEC.md`.
- `vendor/clojure-test-suite` — the cross-dialect suite (≥4081 assertions
passing); dialect splits there are classification evidence.
- jank's per-construct corpus (`~/src/jank/compiler+runtime/test/jank`) is
the granularity model for §2/§3 conformance.
The invariant: **every numbered normative statement names its conformance
test**, or is marked UNVERIFIED. The spec cannot drift from the
implementations that check it.

721
docs/spec/coverage.md Normal file
View file

@ -0,0 +1,721 @@
# Appendix A — Coverage Dashboard (generated)
Generated 2026-06-26 by `tools/spec_coverage.py` — do not edit by hand.
Surface: **694** clojure.core vars (ClojureDocs export; 648 with
community examples). jolt interns 594 of them.
| Status | Count | Meaning |
|---|---|---|
| implemented+tested | 590 | in jolt and exercised by spec/conformance |
| implemented-untested | 4 | in jolt, no direct test — spec entries will add them |
| resolvable-not-interned | 0 | works in code but invisible to ns introspection (conformance finding) |
| missing-portable | 0 | portable semantics, jolt lacks it — implementation gap |
| special-form | 16 | specified in §3, not a library var |
| dynamic-var | 11 | classification needed: portable default vs host-dependent |
| agents-taps | 16 | out of scope pending concurrency design note |
| stm-refs | 11 | out of scope pending concurrency design note |
| jvm-specific | 46 | catalogued, not specified |
Classifications are initial and mechanical — reclassifying is an ordinary
spec change. A var is *Verified* only when its §9 entry exists and carries no
UNVERIFIED field; that column will be added as entries land.
## Per-var status
| Var | Status | ClojureDocs examples |
|---|---|---|
| `*` | implemented+tested | ✓ |
| `*'` | implemented+tested | ✓ |
| `*1` | implemented+tested | ✓ |
| `*2` | implemented+tested | ✓ |
| `*3` | implemented+tested | ✓ |
| `*agent*` | dynamic-var | ✓ |
| `*allow-unresolved-vars*` | dynamic-var | ✓ |
| `*assert*` | implemented+tested | ✓ |
| `*clojure-version*` | implemented+tested | ✓ |
| `*command-line-args*` | implemented-untested | ✓ |
| `*compile-files*` | implemented+tested | ✓ |
| `*compile-path*` | dynamic-var | ✓ |
| `*compiler-options*` | dynamic-var | ✓ |
| `*data-readers*` | implemented+tested | ✓ |
| `*default-data-reader-fn*` | implemented+tested | ✓ |
| `*e` | implemented+tested | ✓ |
| `*err*` | implemented+tested | ✓ |
| `*file*` | implemented-untested | ✓ |
| `*flush-on-newline*` | implemented+tested | |
| `*fn-loader*` | dynamic-var | |
| `*in*` | implemented+tested | |
| `*math-context*` | implemented+tested | |
| `*ns*` | implemented+tested | ✓ |
| `*out*` | implemented+tested | ✓ |
| `*print-dup*` | implemented+tested | ✓ |
| `*print-length*` | implemented+tested | ✓ |
| `*print-level*` | implemented+tested | ✓ |
| `*print-meta*` | implemented+tested | ✓ |
| `*print-namespace-maps*` | implemented-untested | ✓ |
| `*print-readably*` | implemented+tested | ✓ |
| `*read-eval*` | implemented+tested | ✓ |
| `*reader-resolver*` | dynamic-var | |
| `*repl*` | dynamic-var | |
| `*source-path*` | dynamic-var | ✓ |
| `*suppress-read*` | dynamic-var | |
| `*unchecked-math*` | implemented+tested | ✓ |
| `*use-context-classloader*` | dynamic-var | ✓ |
| `*verbose-defrecords*` | dynamic-var | |
| `*warn-on-reflection*` | implemented+tested | ✓ |
| `+` | implemented+tested | ✓ |
| `+'` | implemented+tested | ✓ |
| `-` | implemented+tested | ✓ |
| `-'` | implemented+tested | ✓ |
| `->` | implemented+tested | ✓ |
| `->>` | implemented+tested | ✓ |
| `->ArrayChunk` | jvm-specific | |
| `->Eduction` | implemented+tested | |
| `->Vec` | jvm-specific | |
| `->VecNode` | jvm-specific | |
| `->VecSeq` | jvm-specific | |
| `-cache-protocol-fn` | jvm-specific | |
| `-reset-methods` | jvm-specific | |
| `.` | special-form | ✓ |
| `..` | implemented+tested | ✓ |
| `/` | implemented+tested | ✓ |
| `<` | implemented+tested | ✓ |
| `<=` | implemented+tested | ✓ |
| `=` | implemented+tested | ✓ |
| `==` | implemented+tested | ✓ |
| `>` | implemented+tested | ✓ |
| `>=` | implemented+tested | ✓ |
| `EMPTY-NODE` | jvm-specific | |
| `Inst` | jvm-specific | |
| `NaN?` | implemented+tested | ✓ |
| `PrintWriter-on` | jvm-specific | ✓ |
| `StackTraceElement->vec` | jvm-specific | ✓ |
| `Throwable->map` | jvm-specific | ✓ |
| `abs` | implemented+tested | ✓ |
| `accessor` | jvm-specific | ✓ |
| `aclone` | implemented+tested | ✓ |
| `add-classpath` | jvm-specific | ✓ |
| `add-tap` | agents-taps | ✓ |
| `add-watch` | implemented+tested | ✓ |
| `agent` | implemented+tested | ✓ |
| `agent-error` | implemented+tested | ✓ |
| `agent-errors` | agents-taps | |
| `aget` | implemented+tested | ✓ |
| `alength` | implemented+tested | ✓ |
| `alias` | implemented+tested | ✓ |
| `all-ns` | implemented+tested | ✓ |
| `alter` | stm-refs | ✓ |
| `alter-meta!` | implemented+tested | ✓ |
| `alter-var-root` | implemented+tested | ✓ |
| `amap` | jvm-specific | ✓ |
| `ancestors` | implemented+tested | ✓ |
| `and` | implemented+tested | ✓ |
| `any?` | implemented+tested | ✓ |
| `apply` | implemented+tested | ✓ |
| `areduce` | jvm-specific | ✓ |
| `array-map` | implemented+tested | ✓ |
| `as->` | implemented+tested | ✓ |
| `aset` | implemented+tested | ✓ |
| `aset-boolean` | implemented+tested | ✓ |
| `aset-byte` | implemented+tested | ✓ |
| `aset-char` | implemented+tested | ✓ |
| `aset-double` | implemented+tested | ✓ |
| `aset-float` | implemented+tested | ✓ |
| `aset-int` | implemented+tested | ✓ |
| `aset-long` | implemented+tested | ✓ |
| `aset-short` | implemented+tested | ✓ |
| `assert` | implemented+tested | ✓ |
| `assoc` | implemented+tested | ✓ |
| `assoc!` | implemented+tested | ✓ |
| `assoc-in` | implemented+tested | ✓ |
| `associative?` | implemented+tested | ✓ |
| `atom` | implemented+tested | ✓ |
| `await` | implemented+tested | ✓ |
| `await-for` | agents-taps | ✓ |
| `await1` | agents-taps | |
| `bases` | jvm-specific | ✓ |
| `bean` | implemented+tested | ✓ |
| `bigdec` | implemented+tested | ✓ |
| `bigint` | implemented+tested | ✓ |
| `biginteger` | implemented+tested | ✓ |
| `binding` | implemented+tested | ✓ |
| `bit-and` | implemented+tested | ✓ |
| `bit-and-not` | implemented+tested | ✓ |
| `bit-clear` | implemented+tested | ✓ |
| `bit-flip` | implemented+tested | ✓ |
| `bit-not` | implemented+tested | ✓ |
| `bit-or` | implemented+tested | ✓ |
| `bit-set` | implemented+tested | ✓ |
| `bit-shift-left` | implemented+tested | ✓ |
| `bit-shift-right` | implemented+tested | ✓ |
| `bit-test` | implemented+tested | ✓ |
| `bit-xor` | implemented+tested | ✓ |
| `boolean` | implemented+tested | ✓ |
| `boolean-array` | implemented+tested | ✓ |
| `boolean?` | implemented+tested | ✓ |
| `booleans` | implemented+tested | ✓ |
| `bound-fn` | implemented+tested | ✓ |
| `bound-fn*` | implemented+tested | ✓ |
| `bound?` | implemented+tested | ✓ |
| `bounded-count` | implemented+tested | ✓ |
| `butlast` | implemented+tested | ✓ |
| `byte` | implemented+tested | ✓ |
| `byte-array` | implemented+tested | ✓ |
| `bytes` | implemented+tested | ✓ |
| `bytes?` | implemented+tested | ✓ |
| `case` | implemented+tested | ✓ |
| `cast` | jvm-specific | ✓ |
| `cat` | implemented+tested | ✓ |
| `catch` | special-form | ✓ |
| `char` | implemented+tested | ✓ |
| `char-array` | implemented+tested | ✓ |
| `char-escape-string` | implemented+tested | ✓ |
| `char-name-string` | implemented+tested | ✓ |
| `char?` | implemented+tested | ✓ |
| `chars` | implemented+tested | ✓ |
| `chunk` | implemented+tested | ✓ |
| `chunk-append` | implemented+tested | ✓ |
| `chunk-buffer` | implemented+tested | ✓ |
| `chunk-cons` | implemented+tested | ✓ |
| `chunk-first` | implemented+tested | ✓ |
| `chunk-next` | implemented+tested | ✓ |
| `chunk-rest` | implemented+tested | ✓ |
| `chunked-seq?` | implemented+tested | ✓ |
| `class` | implemented+tested | ✓ |
| `class?` | implemented+tested | ✓ |
| `clear-agent-errors` | agents-taps | |
| `clojure-version` | implemented+tested | ✓ |
| `coll?` | implemented+tested | ✓ |
| `comment` | implemented+tested | ✓ |
| `commute` | stm-refs | ✓ |
| `comp` | implemented+tested | ✓ |
| `comparator` | implemented+tested | ✓ |
| `compare` | implemented+tested | ✓ |
| `compare-and-set!` | implemented+tested | ✓ |
| `compile` | jvm-specific | ✓ |
| `complement` | implemented+tested | ✓ |
| `completing` | implemented+tested | ✓ |
| `concat` | implemented+tested | ✓ |
| `cond` | implemented+tested | ✓ |
| `cond->` | implemented+tested | ✓ |
| `cond->>` | implemented+tested | ✓ |
| `condp` | implemented+tested | ✓ |
| `conj` | implemented+tested | ✓ |
| `conj!` | implemented+tested | ✓ |
| `cons` | implemented+tested | ✓ |
| `constantly` | implemented+tested | ✓ |
| `construct-proxy` | implemented+tested | ✓ |
| `contains?` | implemented+tested | ✓ |
| `count` | implemented+tested | ✓ |
| `counted?` | implemented+tested | ✓ |
| `create-ns` | implemented+tested | ✓ |
| `create-struct` | jvm-specific | ✓ |
| `cycle` | implemented+tested | ✓ |
| `dec` | implemented+tested | ✓ |
| `dec'` | implemented+tested | ✓ |
| `decimal?` | implemented+tested | ✓ |
| `declare` | implemented+tested | ✓ |
| `dedupe` | implemented+tested | ✓ |
| `def` | special-form | ✓ |
| `default-data-readers` | implemented+tested | ✓ |
| `definline` | jvm-specific | |
| `definterface` | implemented+tested | ✓ |
| `defmacro` | special-form | ✓ |
| `defmethod` | implemented+tested | ✓ |
| `defmulti` | implemented+tested | ✓ |
| `defn` | implemented+tested | ✓ |
| `defn-` | implemented+tested | ✓ |
| `defonce` | implemented+tested | ✓ |
| `defprotocol` | implemented+tested | ✓ |
| `defrecord` | implemented+tested | ✓ |
| `defstruct` | jvm-specific | ✓ |
| `deftype` | implemented+tested | ✓ |
| `delay` | implemented+tested | ✓ |
| `delay?` | implemented+tested | ✓ |
| `deliver` | implemented+tested | ✓ |
| `denominator` | implemented+tested | ✓ |
| `deref` | implemented+tested | ✓ |
| `derive` | implemented+tested | ✓ |
| `descendants` | implemented+tested | ✓ |
| `destructure` | implemented+tested | ✓ |
| `disj` | implemented+tested | ✓ |
| `disj!` | implemented+tested | ✓ |
| `dissoc` | implemented+tested | ✓ |
| `dissoc!` | implemented+tested | ✓ |
| `distinct` | implemented+tested | ✓ |
| `distinct?` | implemented+tested | ✓ |
| `do` | special-form | ✓ |
| `doall` | implemented+tested | ✓ |
| `dorun` | implemented+tested | ✓ |
| `doseq` | implemented+tested | ✓ |
| `dosync` | stm-refs | ✓ |
| `dotimes` | implemented+tested | ✓ |
| `doto` | implemented+tested | ✓ |
| `double` | implemented+tested | ✓ |
| `double-array` | implemented+tested | ✓ |
| `double?` | implemented+tested | ✓ |
| `doubles` | implemented+tested | ✓ |
| `drop` | implemented+tested | ✓ |
| `drop-last` | implemented+tested | ✓ |
| `drop-while` | implemented+tested | ✓ |
| `eduction` | implemented+tested | ✓ |
| `empty` | implemented+tested | ✓ |
| `empty?` | implemented+tested | ✓ |
| `ensure` | stm-refs | ✓ |
| `ensure-reduced` | implemented+tested | ✓ |
| `enumeration-seq` | implemented+tested | ✓ |
| `error-handler` | agents-taps | ✓ |
| `error-mode` | agents-taps | ✓ |
| `eval` | implemented+tested | ✓ |
| `even?` | implemented+tested | ✓ |
| `every-pred` | implemented+tested | ✓ |
| `every?` | implemented+tested | ✓ |
| `ex-cause` | implemented+tested | ✓ |
| `ex-data` | implemented+tested | ✓ |
| `ex-info` | implemented+tested | ✓ |
| `ex-message` | implemented+tested | ✓ |
| `extend` | implemented+tested | ✓ |
| `extend-protocol` | implemented+tested | ✓ |
| `extend-type` | implemented+tested | ✓ |
| `extenders` | implemented+tested | ✓ |
| `extends?` | implemented+tested | ✓ |
| `false?` | implemented+tested | ✓ |
| `ffirst` | implemented+tested | ✓ |
| `file-seq` | implemented+tested | ✓ |
| `filter` | implemented+tested | ✓ |
| `filterv` | implemented+tested | ✓ |
| `finally` | special-form | ✓ |
| `find` | implemented+tested | ✓ |
| `find-keyword` | implemented+tested | ✓ |
| `find-ns` | implemented+tested | ✓ |
| `find-protocol-impl` | jvm-specific | |
| `find-protocol-method` | jvm-specific | |
| `find-var` | implemented+tested | ✓ |
| `first` | implemented+tested | ✓ |
| `flatten` | implemented+tested | ✓ |
| `float` | implemented+tested | ✓ |
| `float-array` | implemented+tested | ✓ |
| `float?` | implemented+tested | ✓ |
| `floats` | implemented+tested | ✓ |
| `flush` | implemented+tested | ✓ |
| `fn` | implemented+tested | ✓ |
| `fn?` | implemented+tested | ✓ |
| `fnext` | implemented+tested | ✓ |
| `fnil` | implemented+tested | ✓ |
| `for` | implemented+tested | ✓ |
| `force` | implemented+tested | ✓ |
| `format` | implemented+tested | ✓ |
| `frequencies` | implemented+tested | ✓ |
| `future` | implemented+tested | ✓ |
| `future-call` | implemented+tested | ✓ |
| `future-cancel` | implemented+tested | ✓ |
| `future-cancelled?` | implemented+tested | ✓ |
| `future-done?` | implemented+tested | ✓ |
| `future?` | implemented+tested | ✓ |
| `gen-class` | jvm-specific | ✓ |
| `gen-interface` | jvm-specific | ✓ |
| `gensym` | implemented+tested | ✓ |
| `get` | implemented+tested | ✓ |
| `get-in` | implemented+tested | ✓ |
| `get-method` | implemented+tested | ✓ |
| `get-proxy-class` | implemented+tested | ✓ |
| `get-thread-bindings` | implemented+tested | ✓ |
| `get-validator` | implemented+tested | ✓ |
| `group-by` | implemented+tested | ✓ |
| `halt-when` | implemented+tested | ✓ |
| `hash` | implemented+tested | ✓ |
| `hash-combine` | implemented+tested | ✓ |
| `hash-map` | implemented+tested | ✓ |
| `hash-ordered-coll` | implemented+tested | ✓ |
| `hash-set` | implemented+tested | ✓ |
| `hash-unordered-coll` | implemented+tested | ✓ |
| `ident?` | implemented+tested | ✓ |
| `identical?` | implemented+tested | ✓ |
| `identity` | implemented+tested | ✓ |
| `if` | special-form | ✓ |
| `if-let` | implemented+tested | ✓ |
| `if-not` | implemented+tested | ✓ |
| `if-some` | implemented+tested | ✓ |
| `ifn?` | implemented+tested | ✓ |
| `import` | implemented+tested | ✓ |
| `in-ns` | implemented+tested | ✓ |
| `inc` | implemented+tested | ✓ |
| `inc'` | implemented+tested | ✓ |
| `indexed?` | implemented+tested | ✓ |
| `infinite?` | implemented+tested | ✓ |
| `init-proxy` | implemented+tested | ✓ |
| `inst-ms` | implemented+tested | ✓ |
| `inst-ms*` | implemented+tested | |
| `inst?` | implemented+tested | ✓ |
| `instance?` | implemented+tested | ✓ |
| `int` | implemented+tested | ✓ |
| `int-array` | implemented+tested | ✓ |
| `int?` | implemented+tested | ✓ |
| `integer?` | implemented+tested | ✓ |
| `interleave` | implemented+tested | ✓ |
| `intern` | implemented+tested | ✓ |
| `interpose` | implemented+tested | ✓ |
| `into` | implemented+tested | ✓ |
| `into-array` | implemented+tested | ✓ |
| `ints` | implemented+tested | ✓ |
| `io!` | stm-refs | ✓ |
| `isa?` | implemented+tested | ✓ |
| `iterate` | implemented+tested | ✓ |
| `iteration` | jvm-specific | ✓ |
| `iterator-seq` | implemented+tested | ✓ |
| `juxt` | implemented+tested | ✓ |
| `keep` | implemented+tested | ✓ |
| `keep-indexed` | implemented+tested | ✓ |
| `key` | implemented+tested | ✓ |
| `keys` | implemented+tested | ✓ |
| `keyword` | implemented+tested | ✓ |
| `keyword?` | implemented+tested | ✓ |
| `last` | implemented+tested | ✓ |
| `lazy-cat` | implemented+tested | ✓ |
| `lazy-seq` | implemented+tested | ✓ |
| `let` | implemented+tested | ✓ |
| `letfn` | implemented+tested | ✓ |
| `line-seq` | implemented+tested | ✓ |
| `list` | implemented+tested | ✓ |
| `list*` | implemented+tested | ✓ |
| `list?` | implemented+tested | ✓ |
| `load` | implemented+tested | ✓ |
| `load-file` | implemented-untested | ✓ |
| `load-reader` | jvm-specific | ✓ |
| `load-string` | implemented+tested | ✓ |
| `loaded-libs` | jvm-specific | ✓ |
| `locking` | implemented+tested | ✓ |
| `long` | implemented+tested | ✓ |
| `long-array` | implemented+tested | ✓ |
| `longs` | implemented+tested | ✓ |
| `loop` | implemented+tested | ✓ |
| `macroexpand` | implemented+tested | ✓ |
| `macroexpand-1` | implemented+tested | ✓ |
| `make-array` | implemented+tested | ✓ |
| `make-hierarchy` | implemented+tested | ✓ |
| `map` | implemented+tested | ✓ |
| `map-entry?` | implemented+tested | ✓ |
| `map-indexed` | implemented+tested | ✓ |
| `map?` | implemented+tested | ✓ |
| `mapcat` | implemented+tested | ✓ |
| `mapv` | implemented+tested | ✓ |
| `max` | implemented+tested | ✓ |
| `max-key` | implemented+tested | ✓ |
| `memfn` | implemented+tested | ✓ |
| `memoize` | implemented+tested | ✓ |
| `merge` | implemented+tested | ✓ |
| `merge-with` | implemented+tested | ✓ |
| `meta` | implemented+tested | ✓ |
| `method-sig` | jvm-specific | ✓ |
| `methods` | implemented+tested | ✓ |
| `min` | implemented+tested | ✓ |
| `min-key` | implemented+tested | ✓ |
| `mix-collection-hash` | jvm-specific | |
| `mod` | implemented+tested | ✓ |
| `monitor-enter` | special-form | |
| `monitor-exit` | special-form | |
| `munge` | implemented+tested | ✓ |
| `name` | implemented+tested | ✓ |
| `namespace` | implemented+tested | ✓ |
| `namespace-munge` | implemented+tested | ✓ |
| `nat-int?` | implemented+tested | ✓ |
| `neg-int?` | implemented+tested | ✓ |
| `neg?` | implemented+tested | ✓ |
| `new` | special-form | ✓ |
| `newline` | implemented+tested | ✓ |
| `next` | implemented+tested | ✓ |
| `nfirst` | implemented+tested | ✓ |
| `nil?` | implemented+tested | ✓ |
| `nnext` | implemented+tested | ✓ |
| `not` | implemented+tested | ✓ |
| `not-any?` | implemented+tested | ✓ |
| `not-empty` | implemented+tested | ✓ |
| `not-every?` | implemented+tested | ✓ |
| `not=` | implemented+tested | ✓ |
| `ns` | implemented+tested | ✓ |
| `ns-aliases` | implemented+tested | ✓ |
| `ns-imports` | implemented+tested | ✓ |
| `ns-interns` | implemented+tested | ✓ |
| `ns-map` | implemented+tested | ✓ |
| `ns-name` | implemented+tested | ✓ |
| `ns-publics` | implemented+tested | ✓ |
| `ns-refers` | implemented+tested | ✓ |
| `ns-resolve` | implemented+tested | ✓ |
| `ns-unalias` | implemented+tested | ✓ |
| `ns-unmap` | implemented+tested | ✓ |
| `nth` | implemented+tested | ✓ |
| `nthnext` | implemented+tested | ✓ |
| `nthrest` | implemented+tested | ✓ |
| `num` | implemented+tested | ✓ |
| `number?` | implemented+tested | ✓ |
| `numerator` | implemented+tested | ✓ |
| `object-array` | implemented+tested | ✓ |
| `odd?` | implemented+tested | ✓ |
| `or` | implemented+tested | ✓ |
| `parents` | implemented+tested | ✓ |
| `parse-boolean` | implemented+tested | ✓ |
| `parse-double` | implemented+tested | ✓ |
| `parse-long` | implemented+tested | ✓ |
| `parse-uuid` | implemented+tested | ✓ |
| `partial` | implemented+tested | ✓ |
| `partition` | implemented+tested | ✓ |
| `partition-all` | implemented+tested | ✓ |
| `partition-by` | implemented+tested | ✓ |
| `partitionv` | implemented+tested | |
| `partitionv-all` | implemented+tested | |
| `pcalls` | implemented+tested | ✓ |
| `peek` | implemented+tested | ✓ |
| `persistent!` | implemented+tested | ✓ |
| `pmap` | implemented+tested | ✓ |
| `pop` | implemented+tested | ✓ |
| `pop!` | implemented+tested | ✓ |
| `pop-thread-bindings` | implemented+tested | |
| `pos-int?` | implemented+tested | ✓ |
| `pos?` | implemented+tested | ✓ |
| `pr` | implemented+tested | ✓ |
| `pr-str` | implemented+tested | ✓ |
| `prefer-method` | implemented+tested | ✓ |
| `prefers` | implemented+tested | ✓ |
| `primitives-classnames` | jvm-specific | ✓ |
| `print` | implemented+tested | ✓ |
| `print-ctor` | jvm-specific | ✓ |
| `print-dup` | implemented+tested | ✓ |
| `print-method` | implemented+tested | ✓ |
| `print-simple` | jvm-specific | ✓ |
| `print-str` | implemented+tested | ✓ |
| `printf` | implemented+tested | ✓ |
| `println` | implemented+tested | ✓ |
| `println-str` | implemented+tested | ✓ |
| `prn` | implemented+tested | ✓ |
| `prn-str` | implemented+tested | ✓ |
| `promise` | implemented+tested | ✓ |
| `proxy` | implemented+tested | ✓ |
| `proxy-call-with-super` | implemented+tested | |
| `proxy-mappings` | implemented+tested | ✓ |
| `proxy-name` | jvm-specific | |
| `proxy-super` | implemented+tested | ✓ |
| `push-thread-bindings` | implemented+tested | |
| `pvalues` | implemented+tested | ✓ |
| `qualified-ident?` | implemented+tested | ✓ |
| `qualified-keyword?` | implemented+tested | ✓ |
| `qualified-symbol?` | implemented+tested | ✓ |
| `quot` | implemented+tested | ✓ |
| `quote` | special-form | ✓ |
| `rand` | implemented+tested | ✓ |
| `rand-int` | implemented+tested | ✓ |
| `rand-nth` | implemented+tested | ✓ |
| `random-sample` | implemented+tested | ✓ |
| `random-uuid` | implemented+tested | ✓ |
| `range` | implemented+tested | ✓ |
| `ratio?` | implemented+tested | ✓ |
| `rational?` | implemented+tested | ✓ |
| `rationalize` | implemented+tested | ✓ |
| `re-find` | implemented+tested | ✓ |
| `re-groups` | implemented+tested | ✓ |
| `re-matcher` | implemented+tested | ✓ |
| `re-matches` | implemented+tested | ✓ |
| `re-pattern` | implemented+tested | ✓ |
| `re-seq` | implemented+tested | ✓ |
| `read` | implemented+tested | ✓ |
| `read+string` | implemented+tested | ✓ |
| `read-line` | implemented+tested | ✓ |
| `read-string` | implemented+tested | ✓ |
| `reader-conditional` | implemented+tested | ✓ |
| `reader-conditional?` | implemented+tested | ✓ |
| `realized?` | implemented+tested | ✓ |
| `record?` | implemented+tested | ✓ |
| `recur` | special-form | ✓ |
| `reduce` | implemented+tested | ✓ |
| `reduce-kv` | implemented+tested | ✓ |
| `reduced` | implemented+tested | ✓ |
| `reduced?` | implemented+tested | ✓ |
| `reductions` | implemented+tested | ✓ |
| `ref` | stm-refs | ✓ |
| `ref-history-count` | stm-refs | ✓ |
| `ref-max-history` | stm-refs | |
| `ref-min-history` | stm-refs | ✓ |
| `ref-set` | stm-refs | ✓ |
| `refer` | implemented+tested | ✓ |
| `refer-clojure` | implemented+tested | ✓ |
| `reify` | implemented+tested | ✓ |
| `release-pending-sends` | agents-taps | ✓ |
| `rem` | implemented+tested | ✓ |
| `remove` | implemented+tested | ✓ |
| `remove-all-methods` | implemented+tested | ✓ |
| `remove-method` | implemented+tested | ✓ |
| `remove-ns` | implemented+tested | ✓ |
| `remove-tap` | agents-taps | |
| `remove-watch` | implemented+tested | ✓ |
| `repeat` | implemented+tested | ✓ |
| `repeatedly` | implemented+tested | ✓ |
| `replace` | implemented+tested | ✓ |
| `replicate` | implemented+tested | ✓ |
| `require` | implemented+tested | ✓ |
| `requiring-resolve` | jvm-specific | ✓ |
| `reset!` | implemented+tested | ✓ |
| `reset-meta!` | implemented+tested | ✓ |
| `reset-vals!` | implemented+tested | ✓ |
| `resolve` | implemented+tested | ✓ |
| `rest` | implemented+tested | ✓ |
| `restart-agent` | implemented+tested | ✓ |
| `resultset-seq` | jvm-specific | ✓ |
| `reverse` | implemented+tested | ✓ |
| `reversible?` | implemented+tested | ✓ |
| `rseq` | implemented+tested | ✓ |
| `rsubseq` | implemented+tested | ✓ |
| `run!` | implemented+tested | ✓ |
| `satisfies?` | implemented+tested | ✓ |
| `second` | implemented+tested | ✓ |
| `select-keys` | implemented+tested | ✓ |
| `send` | implemented+tested | ✓ |
| `send-off` | implemented+tested | ✓ |
| `send-via` | agents-taps | ✓ |
| `seq` | implemented+tested | ✓ |
| `seq-to-map-for-destructuring` | implemented+tested | ✓ |
| `seq?` | implemented+tested | ✓ |
| `seqable?` | implemented+tested | ✓ |
| `seque` | implemented+tested | ✓ |
| `sequence` | implemented+tested | ✓ |
| `sequential?` | implemented+tested | ✓ |
| `set` | implemented+tested | ✓ |
| `set!` | special-form | ✓ |
| `set-agent-send-executor!` | agents-taps | ✓ |
| `set-agent-send-off-executor!` | agents-taps | ✓ |
| `set-error-handler!` | agents-taps | ✓ |
| `set-error-mode!` | agents-taps | ✓ |
| `set-validator!` | implemented+tested | ✓ |
| `set?` | implemented+tested | ✓ |
| `short` | implemented+tested | ✓ |
| `short-array` | implemented+tested | ✓ |
| `shorts` | implemented+tested | ✓ |
| `shuffle` | implemented+tested | ✓ |
| `shutdown-agents` | agents-taps | ✓ |
| `simple-ident?` | implemented+tested | ✓ |
| `simple-keyword?` | implemented+tested | ✓ |
| `simple-symbol?` | implemented+tested | ✓ |
| `slurp` | implemented+tested | ✓ |
| `some` | implemented+tested | ✓ |
| `some->` | implemented+tested | ✓ |
| `some->>` | implemented+tested | ✓ |
| `some-fn` | implemented+tested | ✓ |
| `some?` | implemented+tested | ✓ |
| `sort` | implemented+tested | ✓ |
| `sort-by` | implemented+tested | ✓ |
| `sorted-map` | implemented+tested | ✓ |
| `sorted-map-by` | implemented+tested | ✓ |
| `sorted-set` | implemented+tested | ✓ |
| `sorted-set-by` | implemented+tested | ✓ |
| `sorted?` | implemented+tested | ✓ |
| `special-symbol?` | implemented+tested | ✓ |
| `spit` | implemented+tested | ✓ |
| `split-at` | implemented+tested | ✓ |
| `split-with` | implemented+tested | ✓ |
| `splitv-at` | implemented+tested | |
| `str` | implemented+tested | ✓ |
| `stream-into!` | jvm-specific | |
| `stream-reduce!` | jvm-specific | |
| `stream-seq!` | jvm-specific | |
| `stream-transduce!` | jvm-specific | |
| `string?` | implemented+tested | ✓ |
| `struct` | jvm-specific | ✓ |
| `struct-map` | jvm-specific | ✓ |
| `subs` | implemented+tested | ✓ |
| `subseq` | implemented+tested | ✓ |
| `subvec` | implemented+tested | ✓ |
| `supers` | implemented+tested | ✓ |
| `swap!` | implemented+tested | ✓ |
| `swap-vals!` | implemented+tested | ✓ |
| `symbol` | implemented+tested | ✓ |
| `symbol?` | implemented+tested | ✓ |
| `sync` | stm-refs | |
| `tagged-literal` | implemented+tested | ✓ |
| `tagged-literal?` | implemented+tested | |
| `take` | implemented+tested | ✓ |
| `take-last` | implemented+tested | ✓ |
| `take-nth` | implemented+tested | ✓ |
| `take-while` | implemented+tested | ✓ |
| `tap>` | agents-taps | ✓ |
| `test` | implemented+tested | ✓ |
| `the-ns` | implemented+tested | ✓ |
| `thread-bound?` | implemented+tested | ✓ |
| `throw` | special-form | ✓ |
| `time` | implemented+tested | ✓ |
| `to-array` | implemented+tested | ✓ |
| `to-array-2d` | implemented+tested | ✓ |
| `trampoline` | implemented+tested | ✓ |
| `transduce` | implemented+tested | ✓ |
| `transient` | implemented+tested | ✓ |
| `tree-seq` | implemented+tested | ✓ |
| `true?` | implemented+tested | ✓ |
| `try` | special-form | ✓ |
| `type` | implemented+tested | ✓ |
| `unchecked-add` | implemented+tested | ✓ |
| `unchecked-add-int` | implemented+tested | ✓ |
| `unchecked-byte` | implemented+tested | ✓ |
| `unchecked-char` | implemented+tested | |
| `unchecked-dec` | implemented+tested | ✓ |
| `unchecked-dec-int` | implemented+tested | ✓ |
| `unchecked-divide-int` | implemented+tested | ✓ |
| `unchecked-double` | implemented+tested | ✓ |
| `unchecked-float` | implemented+tested | ✓ |
| `unchecked-inc` | implemented+tested | ✓ |
| `unchecked-inc-int` | implemented+tested | ✓ |
| `unchecked-int` | implemented+tested | ✓ |
| `unchecked-long` | implemented+tested | ✓ |
| `unchecked-multiply` | implemented+tested | ✓ |
| `unchecked-multiply-int` | implemented+tested | |
| `unchecked-negate` | implemented+tested | ✓ |
| `unchecked-negate-int` | implemented+tested | ✓ |
| `unchecked-remainder-int` | implemented+tested | |
| `unchecked-short` | implemented+tested | ✓ |
| `unchecked-subtract` | implemented+tested | ✓ |
| `unchecked-subtract-int` | implemented+tested | ✓ |
| `underive` | implemented+tested | ✓ |
| `unquote` | jvm-specific | ✓ |
| `unquote-splicing` | jvm-specific | ✓ |
| `unreduced` | implemented+tested | ✓ |
| `unsigned-bit-shift-right` | implemented+tested | ✓ |
| `update` | implemented+tested | ✓ |
| `update-in` | implemented+tested | ✓ |
| `update-keys` | implemented+tested | ✓ |
| `update-proxy` | implemented+tested | ✓ |
| `update-vals` | implemented+tested | ✓ |
| `uri?` | implemented+tested | ✓ |
| `use` | implemented+tested | ✓ |
| `uuid?` | implemented+tested | ✓ |
| `val` | implemented+tested | ✓ |
| `vals` | implemented+tested | ✓ |
| `var` | special-form | ✓ |
| `var-get` | implemented+tested | ✓ |
| `var-set` | implemented+tested | ✓ |
| `var?` | implemented+tested | ✓ |
| `vary-meta` | implemented+tested | ✓ |
| `vec` | implemented+tested | ✓ |
| `vector` | implemented+tested | ✓ |
| `vector-of` | jvm-specific | ✓ |
| `vector?` | implemented+tested | ✓ |
| `volatile!` | implemented+tested | ✓ |
| `volatile?` | implemented+tested | ✓ |
| `vreset!` | implemented+tested | ✓ |
| `vswap!` | implemented+tested | ✓ |
| `when` | implemented+tested | ✓ |
| `when-first` | implemented+tested | ✓ |
| `when-let` | implemented+tested | ✓ |
| `when-not` | implemented+tested | ✓ |
| `when-some` | implemented+tested | ✓ |
| `while` | implemented+tested | ✓ |
| `with-bindings` | implemented+tested | ✓ |
| `with-bindings*` | implemented+tested | ✓ |
| `with-in-str` | implemented+tested | ✓ |
| `with-loading-context` | jvm-specific | |
| `with-local-vars` | implemented+tested | ✓ |
| `with-meta` | implemented+tested | ✓ |
| `with-open` | implemented+tested | ✓ |
| `with-out-str` | implemented+tested | ✓ |
| `with-precision` | implemented+tested | ✓ |
| `with-redefs` | implemented+tested | ✓ |
| `with-redefs-fn` | implemented+tested | ✓ |
| `xml-seq` | implemented+tested | ✓ |
| `zero?` | implemented+tested | ✓ |
| `zipmap` | implemented+tested | ✓ |

216
docs/tools-deps.md Normal file
View file

@ -0,0 +1,216 @@
# deps.edn support — design notes
How Jolt loads pure-Clojure libraries from a `deps.edn`, and why it's built the
way it is. For how to *use* it, see [building-and-deps.md](building-and-deps.md).
Scope, decided up front:
- **git + local deps only** — no Maven/`~/.m2` resolution.
- **pure `clj`/`cljc`** — anything needing the JVM won't load or run; expected.
- **no classpath abstraction**`require` just needs to find a dep's namespaces;
"the classpath" is an ordered list of source directories.
- **own resolver, own reader**`deps.edn` is read by jolt's own reader, and git
fetch/cache is a thin shell-out to `git`; no external package manager.
- **deps-agnostic runtime core** — resolution is a CLI front-end concern, not a
runtime one. The runtime knows nothing about `deps.edn`; it only consumes a
list of source roots. The CLI resolves a `deps.edn` into those roots before
running.
## How resolution works
`jolt.deps` (`jolt-core/jolt/deps.clj`) reads `deps.edn` (jolt's own reader
parses the EDN), then walks `:deps`:
- `:git/url` + `:git/sha` (+ optional `:deps/root`) → clone the sha into the git
cache and contribute the checkout (or its `:deps/root` subdir);
- `:local/root` → the path as-is;
- `:mvn/*` → skipped with a warning;
- anything else → ignored.
git resolution shells out to `git` through `jolt.host/sh``git init` + remote
add + fetch + reset at the requested sha. Clones land in a global, sha-immutable
cache (`$JOLT_GITLIBS`, else `~/.jolt/gitlibs`) shared across projects, the
`tools.gitlibs` `~/.gitlibs` model.
Each resolved dependency contributes its own `:paths` (default `["src"]`) as
source roots; the walk is **breadth-first** so every top-level coordinate
registers before any transitive one — a top-level pin always wins, matching
tools.deps. The result is a de-duplicated, ordered list of directories.
Two tools.deps features are mirrored in reduced form. **Aliases**: `:aliases`
entries supply `:extra-paths`/`:extra-deps` (accumulate across the aliases
selected with `-A:a:b`) and `:main-opts` (last-wins, run with `-M:alias`).
**Tasks**: the honest subset of babashka's — a string task is a shell command, a
map task is `{:main-opts […]}`; bare Clojure expressions aren't a separate task
form.
## How the CLI ties it together
`jolt.main` (`jolt-core/jolt/main.clj`) is the CLI dispatch. Driven by `cli.ss`,
it resolves the project (`jolt.deps/resolve-project`), prepends the resolved
roots, and de-sugars the argv into a run:
- `run -m NS args` → load `NS`, call its `-main`;
- `run FILE` → load the file;
- `-M:alias` → run the alias's `:main-opts`;
- `-A:alias` → add the alias's paths/deps, then run the rest;
- `repl` → a line REPL;
- `path` → print the resolved roots;
- `build -m NS [-o OUT] [--opt|--dev]` → AOT-compile the app into a standalone binary;
- `<task>` → run a `deps.edn` `:tasks` entry.
The resolver lives in the overlay alongside the runtime, but the runtime's only
dependency interface is the list of source roots it's handed.
## Native libraries
A library that binds C declares the shared objects it needs under `:jolt/native`,
so `jolt.main` loads them before the namespace is required and its `foreign-fn`
bindings resolve. Each entry is a map — `{:name "sqlite3" :darwin
["libsqlite3.0.dylib" …] :linux ["libsqlite3.so.0" …]}` — with optional
`:optional true` (absence is fine, a feature-gated dep) and `:process true` (use
the running process's own symbols, e.g. libc sockets, no external file). A
project inherits its dependencies' `:jolt/native`.
### Static vs dynamic linking
When you `joltc build`, a native lib is **statically linked** into the binary by
default if the spec carries a `:static` archive — so the executable calls the C
code with no shared object present at runtime. Add `:static` alongside the runtime
candidates:
```clojure
{:name "sqlite3"
:static {:archive "/opt/homebrew/lib/libsqlite3.a"} ; or {:lib "sqlite3" :libdir "/usr/lib"}
:darwin ["libsqlite3.0.dylib"] ; still used by `run`/`repl` and by --dynamic
:linux ["libsqlite3.so.0"]}
```
`:static {:archive PATH}` force-loads the whole `.a` and is the reliable
cross-platform form. `:static {:lib NAME :libdir DIR}` links `-lNAME` (with a
`-Bstatic` preference on Linux); on macOS, which has no `-Bstatic`, prefer the
archive form. A spec with no `:static` (or a build passed `--dynamic`, or
`:jolt/build {:dynamic-natives true}`) keeps the old behavior — the shared object
is loaded at startup via `load-shared-object`.
Static linking needs a C compiler (`cc`) on `PATH` at build time (plus the C libs
the Chez kernel links — lz4, zlib, ncurses). The distributed `joltc` bundles the
Chez kernel, so it re-links the launcher stub with the archive baked in — no
external Chez, just `cc`. Without a `cc`, a `:static` lib fails with a message
pointing you to install one or pass `--dynamic`. Keep a `:darwin`/`:linux`
candidate on any `:static` spec so `run`/`repl` (which have no static binary) can
still load it.
## Standalone binaries
`joltc build -m NS` compiles the app and every library into one executable (the
runtime + compiler are baked in). Resolved `:jolt/native` libs are statically
linked in (or loaded at startup — see [Native libraries](#native-libraries)), so
an FFI app — sockets, SQLite — runs with no jolt or Chez on the path.
Output goes under the project's `target/`, cargo-style: `target/release/<project>`
by default and with `--opt`, `target/debug/<project>` with `--dev` (the
`<name>.build` scratch dir sits beside it). `-o PATH` overrides — absolute as-is,
relative against the project dir. Paths resolve against the project (`JOLT_PWD`),
not the CLI's cwd, since `bin/joltc` runs from the jolt repo.
`:jolt/build {:embed ["resources" …]}` bakes those directories' files into the
binary; `io/resource` serves them from the image with no files on disk. Resources
not embedded resolve at runtime against `JOLT_PWD` (or the cwd), so the
ship-the-binary-with-its-`resources/`-dir model also works. Files read through
`io/file` (e.g. a `config.edn` a config library loads) stay external by design —
edit them without rebuilding.
A standalone build needs Chez's kernel dev files (`libkernel.a`, `scheme.h`) and
a C compiler; `JOLT_CHEZ_CSV` overrides the auto-detected `csv<ver>/<machine>`
dir. `--opt` turns on the inference/flatten/scalar-replace passes; the default
`release` mode is const-fold only.
`--direct-link` (or `:jolt/build {:direct-link true}`) opts into a closed world: a
call between the app's own functions binds to its target directly, skipping the var
lookup and generic dispatch a runtime call pays — at the cost of runtime
redefinition of those vars and `eval`/`load-string`. It's off by default, so
ordinary builds (including `release` and `--opt`) stay dynamically linked. A var
marked `^:redef` or `^:dynamic` stays indirect even under `--direct-link`, and calls
into `clojure.core` stay indirect in every mode.
## Tree-shaking
`--tree-shake` (or `:jolt/build {:tree-shake true}`) ships only the code reachable
from `-main`. The build constructs one call graph spanning the app, every resolved
library, and the `clojure.core`/stdlib prelude, then keeps `-main`, every
side-effecting top-level form (so a `defmethod`/`defrecord`/protocol registration
keeps its targets live), and everything reachable from those — dropping the rest. A
reference counts whether it's a call or a value (`#'x`, a fn passed to `map`, a fn
stored in a map): any reference keeps its target live, so nothing reachable is ever
dropped. An app that never compiles at runtime (no reachable `eval`/`load-string`)
also drops the analyzer and back end from the binary. Typical savings are 12 MB;
behaviour is unchanged.
**It bails — keeps everything — when reachable code resolves a var by name at
runtime** (`eval`, `resolve`, `ns-resolve`, `requiring-resolve`, `find-var`,
`intern`, `load-string`, `load-file`). A static call graph can't follow a runtime
`resolve`, so dropping anything would be unsound. The build prints which definitions
forced the bail:
```
jolt build: tree-shake skipped (reachable code resolves vars at runtime):
selmer.filters/generate-json -> clojure.core/resolve
clojure.tools.logging/call-str -> clojure.core/ns-resolve
```
These are almost always libraries, not your code — `resolve` is how mature Clojure
libraries implement plugin systems and optional integrations (a logging backend
chosen at runtime, a template filter that lazily loads an optional dependency). On
the JVM that costs nothing; in a closed-world binary it defeats reachability. To make
an app tree-shakeable, keep runtime resolution off the *reachable* path: a backend
that's fixed on jolt can be referenced directly rather than resolved (the jolt
`tools.logging` port dropped the JVM's dynamic factory selection for exactly this),
and an optional integration you don't use can be dropped or hard-wired. Unreached
`resolve`-using code is shaken away like anything else — only resolution on the live
path triggers the bail.
The closed-world soundness model follows Stalin's dead-code analysis: in a program
with no `eval`, a definition is live iff it is referenced (called or as a value) from
a root, transitively.
## Limitations
- Pure `clj`/`cljc` only — JVM interop, host classes, and unimplemented
`clojure.core` corners fail. Coverage is per-function: a namespace can load with
most functions working and a few not.
- Source only; compiled `.class` files in a git dep are ignored.
- git `:git/sha` must be a full SHA (`git fetch` can't resolve a short one).
## Stack traces
An uncaught error prints the message, the top-level source location, and — when
frames are available — a `trace:` backtrace. In an AOT `jolt build --direct-link`
binary the frames map to `ns/name (file:line)`; on the runtime eval path they are
the surviving fn names. Tail-call optimization erases tail-called frames, so the
default trace shows only the non-tail spine.
A fuller **tail-frame history** recovers the frames TCO erases: each compiled fn
records itself on entry into a bounded ring-of-rings buffer, so the trace shows
TCO-elided frames (including the immediate error site) while a tight tail loop
stays bounded and its non-tail caller context is preserved.
It is **on by default in REPL-driven development** — a `repl` or nREPL session
turns it on, so an error in code you evaluate or reload shows a tail-frame trace
with no setup. Because the recording is baked in at compile time, only code
compiled while a session is live is traced; reload a namespace to trace code that
was already loaded (e.g. an app's initial `-M:run` load before its nREPL started).
Elsewhere it is off (a small per-call cost, and never emitted into a `jolt build`
binary). Override with the environment: `JOLT_TRACE=1` forces it on for a whole
run — including a plain `-M:run`, so the app's own load is traced — and
`JOLT_TRACE=0` forces it off, even in a REPL/nREPL session.
## Conformance
The known-working libraries (see [libraries.md](libraries.md)) and the
[examples](https://github.com/jolt-lang/examples) exercise real pure-`cljc` git
libraries end to end — resolving them from git, loading their namespaces, and
running sample calls. A library fails when it relies on something Jolt doesn't
provide — JVM interop, or a regex feature like Unicode property classes
(`\p{…}`).

221
host/chez/atoms.ss Normal file
View file

@ -0,0 +1,221 @@
;; atoms — host-coupled mutable reference cells for the Chez host.
;;
;; atom/deref/swap!/reset! are host primitives (not the clojure.core overlay),
;; so the runtime provides native shims, def-var!'d into clojure.core. They
;; lower to var-deref in prelude mode. The hierarchy machinery
;; (global-hierarchy = (atom (make-hierarchy))) calls `atom` at the prelude's
;; LOAD time, so without this shim the whole prelude fails to load.
;;
;; compare-and-set!/swap-vals!/reset-vals! are overlay fns over the native kernel
;; in the live system; provided here natively too so the host is self-sufficient
;; for atoms without the full prelude (the overlay versions, when the full prelude
;; loads, override these but compose the same native kernel).
;; watches is an alist of (key . watch-fn); validator is a jolt fn or jolt-nil.
;; The peripheral ops + the notify/validate behaviour live natively here, and
;; post-prelude.ss re-asserts them over the overlay's def-var!.
;; `lock` is a per-atom mutex guarding the read-modify-write critical sections,
;; so swap!/reset!/compare-and-set! are atomic under real OS threads
;; (futures/go blocks share the heap). The user fn in swap! runs OUTSIDE the lock
;; (a CAS retry loop, like the JVM) so it never deadlocks on re-entrant access and
;; a watch/validator can deref the same atom.
(define-record-type jolt-atom
(fields (mutable val) (mutable watches) (mutable validator) lock)
(nongenerative jolt-atom-v3))
;; a rejected reference value is IllegalStateException, like ARef.validate.
(define (jolt-iref-state-throw)
(jolt-throw (jolt-host-throwable "java.lang.IllegalStateException" "Invalid reference state")))
;; (atom init :meta m :validator f) — the ARef ctor contract: the validator runs
;; against the initial value (an invalid init never constructs), :meta must be a
;; map (anything else is the JVM's IPersistentMap cast failure).
(define (jolt-atom-new v . opts)
(let loop ((o opts) (validator jolt-nil) (m #f))
(cond
((or (null? o) (null? (cdr o)))
(let ((a (make-jolt-atom v '() validator (make-mutex))))
(jolt-atom-validate a v)
(when (and m (not (jolt-nil? m)))
(unless (jolt-map? m)
(jolt-throw (jolt-host-throwable
"java.lang.ClassCastException"
(string-append "class " (jolt-class-name m)
" cannot be cast to class clojure.lang.IPersistentMap"))))
(hashtable-set! meta-table a m))
a))
((and (keyword-t? (car o)) (string=? (keyword-t-name (car o)) "validator"))
(loop (cddr o) (cadr o) m))
((and (keyword-t? (car o)) (string=? (keyword-t-name (car o)) "meta"))
(loop (cddr o) validator (cadr o)))
(else (loop (cddr o) validator m)))))
;; validate a candidate value: a non-nil validator that returns falsey rejects.
(define (jolt-atom-validate a v)
(let ((vf (jolt-atom-validator a)))
(when (and (not (jolt-nil? vf)) (jolt-not (jolt-invoke vf v)))
(jolt-iref-state-throw))))
;; notify each watch (k ref old new), in insertion order (alist is reverse-built,
;; so walk it reversed to match add order).
(define (jolt-atom-notify a old new)
(for-each (lambda (kv) (jolt-invoke (cdr kv) (car kv) a old new))
(reverse (jolt-atom-watches a))))
;; deref reads an atom; it also unwraps a `reduced` (Clojure @(reduced x) => x,
;; which the overlay's `unreduced` relies on). The reduced record is in seq.ss.
(define (jolt-deref x)
(cond
((jolt-atom? x) (jolt-atom-val x))
((jolt-reduced? x) (jolt-reduced-val x))
(else (error #f "deref: unsupported reference type" x))))
;; CAS the val from `old` to `nv` by identity (eq?), atomically. Returns #t on
;; success. The compute step (f) runs outside this, so we re-check under the lock.
(define (jolt-atom-cas! a old nv)
(with-mutex (jolt-atom-lock a)
(if (eq? (jolt-atom-val a) old)
(begin (jolt-atom-val-set! a nv) #t)
#f)))
;; (swap! a f arg*): JVM-style CAS loop — read, compute f OUTSIDE the lock, then
;; atomically compare-and-set; retry if another thread changed it. Validate the
;; new value before storing, notify watches after.
(define (jolt-swap! a f . args)
(let retry ()
(let* ((old (jolt-atom-val a))
(nv (apply jolt-invoke f old args)))
(jolt-atom-validate a nv)
(if (jolt-atom-cas! a old nv)
(begin (jolt-atom-notify a old nv) nv)
(retry)))))
(define (jolt-reset! a v)
(jolt-atom-validate a v)
(let ((old (with-mutex (jolt-atom-lock a)
(let ((o (jolt-atom-val a))) (jolt-atom-val-set! a v) o))))
(jolt-atom-notify a old v)
v))
;; compare-and-set! keeps jolt= (value) semantics, done atomically under the lock.
(define (jolt-compare-and-set! a oldv newv)
(jolt-atom-validate a newv)
(let ((swapped (with-mutex (jolt-atom-lock a)
(if (jolt= (jolt-atom-val a) oldv)
(begin (jolt-atom-val-set! a newv) #t)
#f))))
(when swapped (jolt-atom-notify a oldv newv))
swapped))
(define (jolt-swap-vals! a f . args)
(let retry ()
(let* ((old (jolt-atom-val a))
(nv (apply jolt-invoke f old args)))
(jolt-atom-validate a nv)
(if (jolt-atom-cas! a old nv)
(begin (jolt-atom-notify a old nv) (jolt-vector old nv))
(retry)))))
(define (jolt-reset-vals! a v)
(jolt-atom-validate a v)
(let ((old (with-mutex (jolt-atom-lock a)
(let ((o (jolt-atom-val a))) (jolt-atom-val-set! a v) o))))
(jolt-atom-notify a old v)
(jolt-vector old v)))
;; --- watches / validators: the IRef seam --------------------------------------
;; On the JVM these are the ARef contract shared by atom/var/agent/ref. The atom
;; keeps its record slots (the hot swap!/reset! path); every OTHER watchable
;; reference type registers a predicate here and stores its watches/validator in
;; identity-keyed side tables. A ref type makes itself notify by calling
;; iref-notify at its mutation points (vars do at root set).
(define iref-arms '())
(define (register-iref-arm! pred) (set! iref-arms (cons pred iref-arms)))
(define (iref? r)
(let loop ((as iref-arms))
(cond ((null? as) #f) (((car as) r) #t) (else (loop (cdr as))))))
(define iref-watch-tbl (make-weak-eq-hashtable))
(define iref-validator-tbl (make-weak-eq-hashtable))
(define (iref-notify r old new)
(for-each (lambda (kv) (jolt-invoke (cdr kv) (car kv) r old new))
(reverse (hashtable-ref iref-watch-tbl r '()))))
(define (iref-validate r v)
(let ((vf (hashtable-ref iref-validator-tbl r jolt-nil)))
(when (and (not (jolt-nil? vf)) (jolt-not (jolt-invoke vf v)))
(jolt-iref-state-throw))))
;; add-watch interns (key . fn) (replacing any existing key, keeping order);
;; remove-watch drops it; both return the reference. set-validator! installs a
;; validator and validates the CURRENT value immediately (Clojure throws if it's
;; already invalid); get-validator reads the slot.
(define (jolt-watch-add alist key f)
(cons (cons key f) (remp (lambda (kv) (jolt=2 (car kv) key)) alist)))
(define (jolt-add-watch a key f)
(cond
((jolt-atom? a)
(jolt-atom-watches-set! a (jolt-watch-add (jolt-atom-watches a) key f))
a)
((iref? a)
(hashtable-set! iref-watch-tbl a (jolt-watch-add (hashtable-ref iref-watch-tbl a '()) key f))
a)
(else (error #f "add-watch: not a watchable reference" a))))
(define (jolt-remove-watch a key)
(cond
((jolt-atom? a)
(jolt-atom-watches-set! a
(remp (lambda (kv) (jolt=2 (car kv) key)) (jolt-atom-watches a)))
a)
((iref? a)
(hashtable-set! iref-watch-tbl a
(remp (lambda (kv) (jolt=2 (car kv) key)) (hashtable-ref iref-watch-tbl a '())))
a)
(else (error #f "remove-watch: not a watchable reference" a))))
(define (jolt-set-validator! a f)
(let ((vf (if (jolt-nil? f) jolt-nil f)))
(cond
((jolt-atom? a)
(when (and (not (jolt-nil? vf)) (jolt-not (jolt-invoke vf (jolt-atom-val a))))
(jolt-iref-state-throw))
(jolt-atom-validator-set! a vf))
((iref? a)
(when (and (not (jolt-nil? vf)) (jolt-not (jolt-invoke vf (jolt-deref a))))
(jolt-iref-state-throw))
(hashtable-set! iref-validator-tbl a vf))
(else (error #f "set-validator!: not a reference" a)))
jolt-nil))
(define (jolt-get-validator a)
(cond ((jolt-atom? a) (jolt-atom-validator a))
((iref? a) (hashtable-ref iref-validator-tbl a jolt-nil))
(else jolt-nil)))
;; vars are watchable IRefs: a root change (def / var-set on the root /
;; alter-var-root) validates and notifies like Var.bindRoot. The def-var! wrap
;; pays two weak-table probes per def and only does IRef work on a watched var.
(register-iref-arm! var-cell?)
(define def-var!-pre-iref def-var!)
(set! def-var!
(lambda (ns name v)
(let ((c (jolt-var ns name)))
(if (or (pair? (hashtable-ref iref-watch-tbl c '()))
(not (jolt-nil? (hashtable-ref iref-validator-tbl c jolt-nil))))
(let ((old (var-cell-root c)))
(iref-validate c v)
(let ((r (def-var!-pre-iref ns name v)))
(iref-notify c old v)
r))
(def-var!-pre-iref ns name v)))))
(def-var! "clojure.core" "atom" jolt-atom-new)
(def-var! "clojure.core" "deref" jolt-deref)
(def-var! "clojure.core" "swap!" jolt-swap!)
(def-var! "clojure.core" "reset!" jolt-reset!)
(def-var! "clojure.core" "compare-and-set!" jolt-compare-and-set!)
(def-var! "clojure.core" "swap-vals!" jolt-swap-vals!)
(def-var! "clojure.core" "reset-vals!" jolt-reset-vals!)
(def-var! "clojure.core" "atom?" jolt-atom?)
;; peripheral ops: the overlay (20-coll) re-defs these over jolt.host/ref-put!,
;; which fails on an atom record — post-prelude.ss re-asserts the natives.
(def-var! "clojure.core" "add-watch" jolt-add-watch)
(def-var! "clojure.core" "remove-watch" jolt-remove-watch)
(def-var! "clojure.core" "set-validator!" jolt-set-validator!)
(def-var! "clojure.core" "get-validator" jolt-get-validator)

41
host/chez/bootstrap.ss Normal file
View file

@ -0,0 +1,41 @@
;; bootstrap.ss — the pure-Chez self-build.
;;
;; Given a SEED (prelude, image) pair — the bootstrap compiler, checked in under
;; host/chez/seed/ — it loads them, then rebuilds the clojure.core prelude AND the
;; compiler image from the .clj/.ss sources using the on-Chez compiler
;; (emit-image.ss), writing fresh artifacts: read -> analyze -> emit all run on
;; Chez. The seed is a JOINT fixpoint, so a rebuild from an up-to-date seed
;; reproduces it byte-for-byte (`make selfhost` checks this); when the sources
;; change, run it twice to reconverge and re-mint the seed.
;;
;; Run from the repo root:
;; chez --script host/chez/bootstrap.ss SEED-PRELUDE SEED-IMAGE OUT-PRELUDE OUT-IMAGE
(import (chezscheme))
(define bs-args (cdr (command-line))) ; drop the script name
(when (< (length bs-args) 4)
(display "usage: bootstrap.ss SEED-PRELUDE SEED-IMAGE OUT-PRELUDE OUT-IMAGE\n")
(exit 2))
(define bs-seed-prelude (list-ref bs-args 0))
(define bs-seed-image (list-ref bs-args 1))
(define bs-out-prelude (list-ref bs-args 2))
(define bs-out-image (list-ref bs-args 3))
;; Load the runtime + the SEED compiler (prelude for macros, image for the
;; analyzer/emitter), exactly as the spine assembles a program.
(load "host/chez/rt.ss")
(set-chez-ns! "clojure.core")
(load bs-seed-prelude)
(load "host/chez/post-prelude.ss")
(set-chez-ns! "user")
(load "host/chez/host-contract.ss")
(load bs-seed-image)
(load "host/chez/compile-eval.ss")
(load "host/chez/emit-image.ss")
;; Rebuild both artifacts from source ON CHEZ and write them out.
(let ((p (open-output-file bs-out-prelude 'replace)))
(put-string p (jolt-emit-prelude)) (close-port p))
(let ((p (open-output-file bs-out-image 'replace)))
(put-string p (jolt-emit-image)) (close-port p))
(display "bootstrap: rebuilt prelude + compiler image on Chez\n")

264
host/chez/build-joltc.ss Normal file
View file

@ -0,0 +1,264 @@
;; build-joltc.ss — build joltc itself as a self-contained native binary (jolt-eaj).
;;
;; chez --script host/chez/build-joltc.ss <profile> <out-path>
;; profile: "release" | "debug" out-path: e.g. target/release/joltc
;;
;; Runs on a dev/CI machine that HAS Chez + cc. Produces a binary that needs
;; NEITHER: it bakes the full runtime + compiler image + all jolt-core/stdlib
;; source + the Chez petite/scheme boots + a prebuilt launcher stub into one
;; cc-linked executable, so the resulting joltc can run AND `build` jolt apps on
;; its own. joltc itself is cc-linked (not appended) so its signature stays clean
;; for Homebrew/codesign, like dirge's binaries; only the apps it later builds use
;; the appended-stub path (host/chez/build.ss build-self-contained).
;;
;; Pipeline:
;; 0. cc-compile host/chez/stub/launcher.c against the Chez kernel.
;; 1. emit flat.ss = runtime + compiler image (cli.ss load order) + inlined
;; build.ss + every jolt-core/stdlib file as a baked string literal + the
;; joltc launcher.
;; 2. in-process compile-file + make-boot-file (profile Chez settings), error
;; restored around the call (the runtime shadows it; regex.ss/%chez-error).
;; 3. xxd the joltc boot + petite/scheme boots + stub into C arrays, generate
;; main.c, cc-link -> out-path. The launcher reads the petite/scheme/stub
;; arrays via FFI on `build` (jolt-materialize-bundles!).
(import (chezscheme))
(load "host/chez/rt.ss")
(set-chez-ns! "clojure.core")
(load "host/chez/seed/prelude.ss")
(load "host/chez/post-prelude.ss")
(set-chez-ns! "user")
(load "host/chez/host-contract.ss")
(load "host/chez/seed/image.ss")
(load "host/chez/compile-eval.ss")
(load "host/chez/png.ss")
(load "host/chez/loader.ss")
(load "host/chez/java/ffi.ss")
(set-source-roots! (list "jolt-core" "stdlib"))
(load "host/chez/build.ss") ; bld-* helpers, ei-* (emit-image), dce
(define jb-args (cdr (command-line)))
(define jb-profile (if (pair? jb-args) (car jb-args) "release"))
(define jb-out (if (and (pair? jb-args) (pair? (cdr jb-args))) (cadr jb-args)
(string-append "target/" jb-profile "/joltc")))
(define jb-release? (string=? jb-profile "release"))
(unless (or jb-release? (string=? jb-profile "debug"))
(error 'build-joltc "profile must be \"release\" or \"debug\"" jb-profile))
;; Version baked into the binary's saved heap. Prefer $JOLT_VERSION (CI sets it to
;; the release tag); else derive it from git in this checkout; else "dev".
(define jb-version
(let ((env (getenv "JOLT_VERSION")))
(if (and env (> (string-length env) 0))
env
(let ((s (bld-sh-capture "git describe --tags --always --dirty 2>/dev/null")))
(if (> (string-length s) 0) s "dev")))))
(define jb-build (string-append jb-out ".build"))
(bld-check-toolchain)
(bld-system (string-append "mkdir -p '" (path-parent jb-out) "' '" jb-build "'"))
;; --- 0. compile the launcher stub -------------------------------------------
(define jb-stub (string-append jb-build "/launcher"))
(display "build-joltc: compiling launcher stub\n")
(bld-system (string-append
"cc -O2 -I'" bld-csv-dir "' 'host/chez/stub/launcher.c' '"
bld-csv-dir "/libkernel.a' -o '" jb-stub "' " (bld-link-libs)))
;; --- 1. emit flat.ss --------------------------------------------------------
(define jb-flat-ss (string-append jb-build "/flat.ss"))
(define (str-suffix? s suf)
(let ((n (string-length s)) (m (string-length suf)))
(and (>= n m) (string=? (substring s (- n m) n) suf))))
;; Bake every jolt-core/stdlib source file as an in-heap string literal keyed by
;; its root-relative path ("jolt/main.clj", "clojure/string.clj") — exactly what
;; resolve-on-roots probes. Literals (not read-file-string at startup) because
;; flat.ss top-level forms run at every startup, with no source on disk.
(define (jb-emit-source-embeds out)
(for-each
(lambda (root)
(for-each
(lambda (rp)
(let ((rel (car rp)) (abs (cdr rp)))
(when (or (str-suffix? rel ".clj") (str-suffix? rel ".cljc"))
(put-string out (string-append
"(register-embedded-resource! " (ei-str-lit rel) " "
(ei-str-lit (read-file-string abs)) ")\n")))))
(bld-walk-files root "" '())))
(list "jolt-core" "stdlib")))
;; Embed every runtime .ss the build inlines into an app (the transitive closure of
;; the manifest's loads: rt.ss + all it loads, the seed, compile-eval, loader, ffi,
;; png, vendored irregex). Keyed by the exact path the (load "…") forms use, so
;; build.ss's bld-source-string reads them from the binary with no jolt source on
;; disk. Traversal mirrors bld-emit-runtime/bld-inline-line via the same
;; bld-file-lines + bld-load-path, so the embedded set is exactly what build reads.
(define (jb-collect-load-paths)
(let ((seen (make-hashtable string-hash string=?)) (order '()))
(define (walk path)
(when (and path (not (hashtable-ref seen path #f)))
(hashtable-set! seen path #t)
(set! order (cons path order))
(for-each (lambda (l) (walk (bld-load-path l))) (bld-file-lines path))))
(for-each (lambda (entry) (when (string? entry) (walk (bld-load-path entry))))
bld-runtime-manifest)
(for-each (lambda (kv) (walk (bld-load-path (cdr kv)))) bld-tagged-loads)
(reverse order)))
(define (jb-emit-runtime-embeds out)
(for-each
(lambda (path)
(put-string out (string-append
"(register-embedded-resource! " (ei-str-lit path) " "
(ei-str-lit (read-file-string path)) ")\n")))
(jb-collect-load-paths)))
;; The launcher (Chez scheme-start): replicates host/chez/cli.ss but reads argv
;; from the scheme-start lambda and has no repo root to cd into (all source is
;; embedded; JOLT_PWD defaults to cwd via io/jolt.main). build.ss is already
;; inlined, so `build` dispatches straight to jolt.host/build-binary after the
;; bundled boots/stub are materialized from the binary's own C arrays.
(define (jb-emit-launcher out)
(put-string out "
;; Materialize the bundled Chez boots + launcher stub (cc-linked into this binary
;; as C arrays) into the embedded-bytes store, so build-self-contained can spill
;; them. Done lazily on `build` only.
(define (jolt-materialize-bundles!)
(load-shared-object #f)
(let ((memcpy (foreign-procedure \"memcpy\" (u8* uptr uptr) void*)))
(for-each
(lambda (spec)
(let* ((len (foreign-ref 'unsigned-int (foreign-entry (caddr spec)) 0))
(bv (make-bytevector len)))
(memcpy bv (foreign-entry (cadr spec)) len)
(register-embedded-bytes! (car spec) bv)))
'((\"csv/petite.boot\" \"jolt_petite_boot\" \"jolt_petite_boot_len\")
(\"csv/scheme.boot\" \"jolt_scheme_boot\" \"jolt_scheme_boot_len\")
(\"stub/launcher\" \"jolt_stub\" \"jolt_stub_len\")
(\"csv/scheme.h\" \"jolt_scheme_h\" \"jolt_scheme_h_len\")
(\"csv/libkernel.a\" \"jolt_libkernel_a\" \"jolt_libkernel_a_len\")
(\"stub/launcher.c\" \"jolt_launcher_c\" \"jolt_launcher_c_len\")))))
(suppress-greeting #t)
(scheme-start
(lambda args
(set-source-roots! (list \"jolt-core\" \"stdlib\"))
;; JOLT_TRACE at RUNTIME (the env is unset at heap-build), before any app ns
;; compiles, so a `-M:run` traces the app's own code.
(jolt-trace-init-from-env!)
(guard (v (#t (jolt-report-throwable v (current-error-port)) (exit 1)))
(cond
((and (= (length args) 2) (string=? (car args) \"-e\"))
(let ((result (jolt-final-str
(jolt-compile-eval (string-append \"(do \" (cadr args) \")\") \"user\"))))
(unless (string=? result \"\") (display result) (newline))))
(else
(when (and (pair? args) (string=? (car args) \"build\"))
(jolt-materialize-bundles!))
(load-namespace \"jolt.main\")
(apply jolt-invoke (var-deref \"jolt.main\" \"-main\") args))))
(exit 0)))
"))
(display "build-joltc: emitting flat source\n")
(let ((out (open-output-file jb-flat-ss 'replace)))
;; full runtime + compiler image: keep the compiler (joltc evals at runtime).
(bld-emit-runtime out #f #f)
(put-string out "\n;; === build driver (inlined for self-contained `jolt build`) ===\n")
(bld-inline-line "(load \"host/chez/build.ss\")" out 0)
(put-string out "\n;; === embedded runtime source (self-contained `build` reads these) ===\n")
(jb-emit-runtime-embeds out)
(put-string out "\n;; === embedded jolt-core + stdlib source ===\n")
(jb-emit-source-embeds out)
;; Bake the version into the saved heap (runs at heap-build; loader.ss defined
;; jolt-baked-version above, so this set! resolves).
(put-string out (string-append "\n;; === baked version ===\n(set! jolt-baked-version "
(ei-str-lit jb-version) ")\n"))
(put-string out "\n;; === joltc launcher ===\n")
(jb-emit-launcher out)
(close-port out))
;; --- 2. compile + boot in a FRESH Chez (profile Chez settings) --------------
;; joltc is a compiler/REPL: it evals jolt-compiled Scheme at runtime, which must
;; resolve the runtime's top-level procedures (var-deref, jolt-inc, …) through the
;; boot's interaction-environment. compile-file's top-level defines are visible
;; there only when compiled in the REAL interaction-environment, and `error` (and
;; other primitives the inlined runtime references before redefining) bind to the
;; kernel primitive only when compiled against a clean chezscheme env. A fresh
;; Chez process gives both at once — exactly the legacy build-with-cc pass. The
;; in-process compile in build.ss/build-self-contained is for the distributed
;; joltc building (non-eval) apps, where no Chez is available.
(define jb-flat-so (string-append jb-build "/flat.so"))
(define jb-boot (string-append jb-build "/joltc.boot"))
(define jb-bool (lambda (b) (if b "#t" "#f")))
(display (string-append "build-joltc: compiling (" jb-profile " profile)\n"))
(let ((cs (string-append jb-build "/compile.ss")))
(let ((p (open-output-file cs 'replace)))
(put-string p
(string-append
"(import (chezscheme))\n"
"(optimize-level " (if jb-release? "3" "0") ")\n"
"(generate-inspector-information " (jb-bool (not jb-release?)) ")\n"
"(generate-procedure-source-information " (jb-bool (not jb-release?)) ")\n"
"(debug-on-exception " (jb-bool (not jb-release?)) ")\n"
"(fasl-compressed " (jb-bool jb-release?) ")\n"
"(compile-file " (ei-str-lit jb-flat-ss) " " (ei-str-lit jb-flat-so) ")\n"
"(make-boot-file " (ei-str-lit jb-boot) " '()\n "
(ei-str-lit (string-append bld-csv-dir "/petite.boot")) "\n "
(ei-str-lit (string-append bld-csv-dir "/scheme.boot")) "\n "
(ei-str-lit jb-flat-so) ")\n"))
(close-port p))
(bld-system (string-append bld-chez " --script '" cs "'")))
;; --- 3. embed boots/stub as C arrays + cc-link ------------------------------
;; xxd a file into header H and rename its symbol to NAME / NAME_len.
(define (jb-c-array file h name)
(bld-system (string-append "xxd -i '" file "' > '" h "'"))
(bld-system (string-append
"sed -i.bak -E 's/unsigned char [A-Za-z0-9_]+\\[\\]/unsigned char " name "[]/; "
"s/unsigned int [A-Za-z0-9_]+_len/unsigned int " name "_len/' '" h "'")))
(display "build-joltc: embedding boots + stub, linking\n")
(jb-c-array jb-boot (string-append jb-build "/boot_data.h") "jolt_boot")
(jb-c-array (string-append bld-csv-dir "/petite.boot") (string-append jb-build "/petite_data.h") "jolt_petite_boot")
(jb-c-array (string-append bld-csv-dir "/scheme.boot") (string-append jb-build "/scheme_data.h") "jolt_scheme_boot")
(jb-c-array jb-stub (string-append jb-build "/stub_data.h") "jolt_stub")
;; Also bundle the Chez kernel (libkernel.a + scheme.h) and the launcher source,
;; so a `build` with :static native libs can re-link a custom stub with those
;; archives baked in — the appended-stub path can't add object code to a prebuilt
;; stub, so it relinks (build.ss bld-relink-stub). Needs the system cc at build.
(jb-c-array (string-append bld-csv-dir "/scheme.h") (string-append jb-build "/schemeh_data.h") "jolt_scheme_h")
(jb-c-array (string-append bld-csv-dir "/libkernel.a") (string-append jb-build "/libkernel_data.h") "jolt_libkernel_a")
(jb-c-array "host/chez/stub/launcher.c" (string-append jb-build "/launcherc_data.h") "jolt_launcher_c")
(define jb-main-c (string-append jb-build "/main.c"))
(let ((mc (open-output-file jb-main-c 'replace)))
(put-string mc
(string-append
"#include \"scheme.h\"\n"
"#include \"boot_data.h\"\n"
"#include \"petite_data.h\"\n"
"#include \"scheme_data.h\"\n"
"#include \"stub_data.h\"\n"
"#include \"schemeh_data.h\"\n"
"#include \"libkernel_data.h\"\n"
"#include \"launcherc_data.h\"\n"
"int main(int argc, char *argv[]) {\n"
" Sscheme_init(0);\n"
" Sregister_boot_file_bytes(\"jolt\", jolt_boot, jolt_boot_len);\n"
" Sbuild_heap(0, 0);\n"
" int status = Sscheme_start(argc, (const char **)argv);\n"
" Sscheme_deinit();\n return status;\n}\n"))
(close-port mc))
;; -rdynamic puts the embedded jolt_* boot/stub symbols in the dynamic symbol
;; table so `build` can foreign-entry them to spill the bundled Chez boots. On
;; Linux dlsym can't see executable symbols otherwise (macOS exports them anyway).
(bld-system (string-append
;; the embedded jolt_* arrays must be foreign-entry-visible at runtime:
;; -rdynamic on ELF; on Windows an exe needs an export table (GetProcAddress).
"cc -O2 " (if bld-nt? "-Wl,--export-all-symbols " "-rdynamic ") "-I'" bld-csv-dir "' -I'" jb-build "' '" jb-main-c "' '"
bld-csv-dir "/libkernel.a' -o '" jb-out "' " (bld-link-libs)))
(display (string-append "build-joltc: wrote " jb-out "\n"))

170
host/chez/build-smoke.sh Executable file
View file

@ -0,0 +1,170 @@
#!/bin/sh
# build smoke: `jolt build` compiles a multi-namespace app (macro + cross-ns +
# clojure.string) into a standalone binary, which then runs with no jolt source
# or Chez install on the path — args reach -main, output matches.
root="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)"
cd "$root"
# Preflight: a standalone build needs Chez's kernel dev files (libkernel.a +
# scheme.h) and a C compiler. A distro chezscheme package ships neither, so on
# such hosts (CI included) skip — like `certify` skips without Clojure. Pin the
# csv dir we validate so the build uses exactly it.
csv="$JOLT_CHEZ_CSV"
if [ -z "$csv" ]; then
chez_bin="$(command -v chez || command -v scheme || command -v petite || true)"
if [ -n "$chez_bin" ]; then
base="$(cd "$(dirname "$chez_bin")/.." 2>/dev/null && pwd)"
for d in "$base"/lib/csv*/*/; do
[ -f "${d}libkernel.a" ] && csv="${d%/}" && break
done
fi
fi
if ! command -v cc >/dev/null 2>&1 || [ -z "$csv" ] || [ ! -f "$csv/scheme.h" ] || [ ! -f "$csv/libkernel.a" ]; then
echo "build smoke: skipped (Chez kernel dev files or C compiler not available)"
exit 0
fi
export JOLT_CHEZ_CSV="$csv"
app="$root/test/chez/build-app"
out="$(mktemp -d)/app-bin"
trap 'rm -rf "$(dirname "$out")"' EXIT
echo "build smoke: compiling app.core -> $out"
if ! JOLT_PWD="$app" bin/joltc build -m app.core -o "$out" >/dev/null 2>&1; then
echo " FAIL: jolt build exited non-zero"
exit 1
fi
[ -x "$out" ] || { echo " FAIL: no executable produced"; exit 1; }
# Run from a neutral cwd with args. The first line is an embedded resource
# (deps.edn :jolt/build :embed), proving io/resource resolves from the binary with
# no resources/ dir on disk; the rest exercise a macro, cross-ns, and args.
got="$(cd / && "$out" alpha bb ccc 2>&1)"
want='embedded resource ok
HELLO FROM A BUILT BINARY!
HELLO FROM A BUILT BINARY!
args: [alpha bb ccc]
sum: 10
greet-default: greet:default
greet-loud: greet:loud
greet-soft: greet:soft'
if [ "$got" != "$want" ]; then
echo " FAIL: binary output mismatch"
echo "--- want ---"; echo "$want"
echo "--- got ----"; echo "$got"
exit 1
fi
# Optimized mode (inference + flatten + scalar-replace) must produce the same
# result — a sanity check that the passes don't miscompile this app.
if ! JOLT_PWD="$app" bin/joltc build -m app.core -o "$out" --opt >/dev/null 2>&1; then
echo " FAIL: jolt build --opt exited non-zero"; exit 1
fi
got_opt="$(cd / && "$out" alpha bb ccc 2>&1)"
if [ "$got_opt" != "$want" ]; then
echo " FAIL: --opt binary output mismatch"
echo "--- got ----"; echo "$got_opt"
exit 1
fi
# Closed-world direct-linking (opt-in): same result, and the cross-namespace call
# (app.core -> app.util/shout) must lower to a direct jv$ binding, not var-deref.
if ! JOLT_PWD="$app" bin/joltc build -m app.core -o "$out" --direct-link >/dev/null 2>&1; then
echo " FAIL: jolt build --direct-link exited non-zero"; exit 1
fi
got_dl="$(cd / && "$out" alpha bb ccc 2>&1)"
if [ "$got_dl" != "$want" ]; then
echo " FAIL: --direct-link binary output mismatch"
echo "--- got ----"; echo "$got_dl"
exit 1
fi
if ! grep -q '(jv\$app.util\$shout' "$out.build/flat.ss"; then
echo " FAIL: --direct-link did not emit a direct app->app call"; exit 1
fi
# A direct-link build registers fn sources, so an uncaught throw prints a Clojure
# stack trace mapping each native frame back to ns/name (file:line).
if ! grep -q 'jolt-register-source!' "$out.build/flat.ss"; then
echo " FAIL: --direct-link did not emit source registrations"; exit 1
fi
boom_err="$(cd / && "$out" --boom 2>&1 >/dev/null)"
for frame in 'app.util/deep-boom' 'app.util/mid-boom' 'app.core/-main'; do
if ! printf '%s' "$boom_err" | grep -q "$frame"; then
echo " FAIL: stack trace missing frame $frame"
echo "--- got ----"; echo "$boom_err"
exit 1
fi
done
# A built binary runs -main with *ns* = user, like clojure.main — so a runtime
# resolve of an aliased symbol is nil (the alias lives in the entry ns, not user),
# matching the JVM and interpreted joltc rather than the entry ns's alias table. A
# separate app: `resolve` defeats tree-shaking, so keep it out of the shake test's
# app above.
nsp="$(dirname "$out")/nsparity"
mkdir -p "$nsp/src/nsp"
printf '{:paths ["src"]}\n' > "$nsp/deps.edn"
printf '(ns nsp.lib)\n(defn thing [] 1)\n' > "$nsp/src/nsp/lib.clj"
printf '(ns nsp.main (:require [nsp.lib :as l]))\n(defn -main [& _]\n (println "ns:" (str *ns*))\n (println "resolve:" (pr-str (resolve (quote l/thing))))\n (println "ns-resolve:" (pr-str (ns-resolve (quote nsp.lib) (quote thing)))))\n' > "$nsp/src/nsp/main.clj"
nspout="$(dirname "$out")/nsparity-bin"
if ! JOLT_PWD="$nsp" bin/joltc build -m nsp.main -o "$nspout" >/dev/null 2>&1; then
echo " FAIL: jolt build of the ns-parity app exited non-zero"; exit 1
fi
nsp_out="$(cd / && "$nspout" 2>&1)"
if ! printf '%s' "$nsp_out" | grep -q 'ns: user' \
|| ! printf '%s' "$nsp_out" | grep -q '^resolve: nil' \
|| ! printf '%s' "$nsp_out" | grep -q "ns-resolve: #'nsp.lib/thing"; then
echo " FAIL: built binary -main ns parity — want 'ns: user', 'resolve: nil', ns-resolve found"
echo "--- got ----"; echo "$nsp_out"
exit 1
fi
# Tree-shaking (opt-in): same result, and an unreachable def (the `twice` macro,
# expanded at AOT and never called at runtime) is dropped.
if ! JOLT_PWD="$app" bin/joltc build -m app.core -o "$out" --tree-shake >/dev/null 2>&1; then
echo " FAIL: jolt build --tree-shake exited non-zero"; exit 1
fi
got_ts="$(cd / && "$out" alpha bb ccc 2>&1)"
if [ "$got_ts" != "$want" ]; then
echo " FAIL: --tree-shake binary output mismatch"
echo "--- got ----"; echo "$got_ts"
exit 1
fi
if grep -q 'def-var! "app.util" "twice"' "$out.build/flat.ss"; then
echo " FAIL: --tree-shake did not drop the unreachable twice macro"; exit 1
fi
# The app never evals, so the compiler image (analyzer/back end) is dropped.
if grep -q 'def-var! "jolt.analyzer"' "$out.build/flat.ss"; then
echo " FAIL: --tree-shake kept the compiler image in a no-eval app"; exit 1
fi
# Core is shaken: a clojure.core overlay fn this app never uses is dropped.
if grep -q 'def-var! "clojure.core" "group-by"' "$out.build/flat.ss"; then
echo " FAIL: --tree-shake kept an unreachable clojure.core fn (group-by)"; exit 1
fi
# A registered data reader that returns a CODE form must be compiled into the
# binary (the emit path applies it too, not just the interpreted loader): the
# datareader-app's #code literal builds to 42, not the literal list.
drapp="$root/test/chez/datareader-app"
drout="$(dirname "$out")/dr-bin"
if ! JOLT_PWD="$drapp" bin/joltc build -m drtest.main -o "$drout" >/dev/null 2>&1; then
echo " FAIL: jolt build of a data-reader app exited non-zero"; exit 1
fi
got_dr="$(cd / && "$drout" 2>&1 | tail -1)"
if [ "$got_dr" != "42" ]; then
echo " FAIL: built #code data reader — want 42, got \`$got_dr\`"; exit 1
fi
# A script namespace with no -main (just top-level side effects) must build and
# run its top-level forms, then exit cleanly — not crash calling a nil -main.
nomain="$(dirname "$out")/nomain"
mkdir -p "$nomain/src"
printf '{:paths ["src"]}\n' > "$nomain/deps.edn"
printf '(ns script)\n(println "no-main script ran")\n' > "$nomain/src/script.clj"
nmout="$(dirname "$out")/nomain-bin"
if ! JOLT_PWD="$nomain" bin/joltc build -m script -o "$nmout" >/dev/null 2>&1; then
echo " FAIL: jolt build of a no-main script exited non-zero"; exit 1
fi
got_nm="$(cd / && "$nmout" 2>&1)"; rc_nm=$?
if [ "$got_nm" != "no-main script ran" ] || [ "$rc_nm" != "0" ]; then
echo " FAIL: no-main script binary — want 'no-main script ran' rc 0, got \`$got_nm\` rc $rc_nm"
exit 1
fi
echo "build smoke: passed (release + optimized + direct-link + tree-shake + compiler+core shake + data-reader + no-main)"

751
host/chez/build.ss Normal file
View file

@ -0,0 +1,751 @@
;; build.ss — `jolt build`: AOT-compile an app into a standalone executable.
;;
;; Loaded on demand by cli.ss when the command is `build`. Defines the host
;; primitive jolt.host/build-binary, which jolt.main's build command calls after
;; resolving the project's deps + source roots.
;;
;; The pipeline (Phase 4 stage 2):
;; 1. load the entry namespace — registers its macros/vars and follows requires,
;; recording the app namespaces in dependency order (loader's ns-loaded-hook).
;; 2. re-emit each app namespace to Scheme (the emit-image cross-compile path),
;; now that its macros are registered.
;; 3. textually inline the cli.ss runtime load sequence into one flat source,
;; append the app emission + a launcher that calls the entry's -main.
;; 4. compile-file -> make-boot-file -> embed the boot as C bytes -> cc-link
;; against libkernel.a into a single self-contained binary.
;;
;; emit-image.ss supplies the cross-compiler (ei-* helpers); it's loaded here so a
;; normal run never pays for it.
(load "host/chez/emit-image.ss")
(load "host/chez/dce.ss")
;; --- shell helpers ----------------------------------------------------------
;; Run a command, return its stdout as one trimmed string ("" on no output).
(define (bld-sh-capture cmd)
(let* ((p (process (bld-sh-wrap cmd))) (in (car p)))
(let loop ((acc '()))
(let ((l (get-line in)))
(if (eof-object? l)
(begin (close-port in)
;; rejoin with newlines (get-line stripped them). Callers use
;; single-line output; this just avoids silently concatenating
;; two lines into one corrupt token if a command emits more.
(let ((ls (reverse acc)))
(if (null? ls) ""
(fold-left (lambda (s x) (string-append s "\n" x)) (car ls) (cdr ls)))))
(loop (cons l acc)))))))
(define (bld-system cmd)
(let ((rc (system (bld-sh-wrap cmd))))
(unless (zero? rc)
(error 'jolt-build (string-append "command failed (" (number->string rc) "): " cmd)))))
;; mkdir -p without a subprocess (the self-contained build shells out to nothing).
(define (bld-mkdir-p dir)
(unless (or (string=? dir "") (string=? dir "/") (string=? dir ".") (file-exists? dir))
(bld-mkdir-p (path-parent dir))
(guard (e (#t #f)) (mkdir dir))))
(define (bld-contains? s sub)
(let ((ns (string-length s)) (nsub (string-length sub)))
(let loop ((i 0))
(cond ((> (+ i nsub) ns) #f)
((string=? (substring s i (+ i nsub)) sub) #t)
(else (loop (+ i 1)))))))
;; --- toolchain discovery ----------------------------------------------------
(define bld-machine (symbol->string (machine-type)))
(define bld-osx? (bld-contains? bld-machine "osx"))
(define bld-nt? (bld-contains? bld-machine "nt"))
;; Chez's system/process run through cmd.exe on Windows; every build command
;; here is written for sh (MSYS2 provides it). On nt, spill the command to a
;; script and run `sh <file>` — workspace paths carry no spaces, and the
;; script file sidesteps cmd's quoting entirely. Identity elsewhere.
(define bld-shell-counter 0)
(define (bld-sh-wrap cmd)
(if bld-nt?
(let* ((tmp (or (getenv "TEMP") (getenv "TMP") "."))
(f (begin (set! bld-shell-counter (+ bld-shell-counter 1))
(string-append tmp "\\jolt-sh-"
(number->string bld-shell-counter) ".sh"))))
(let ((p (open-output-file f 'replace)))
(put-string p cmd)
(close-port p))
(string-append "sh " f))
cmd))
;; The Chez executable, for the isolated compile pass (see build-binary step 4).
(define bld-chez
(let ((p (bld-sh-capture "command -v chez || command -v scheme || command -v petite")))
(if (> (string-length p) 0) p "chez")))
;; Chez version off (scheme-version) "Chez Scheme Version X.Y.Z" — last token.
(define bld-version
(let* ((s (scheme-version)) (n (string-length s)))
(let loop ((i n))
(if (or (= i 0) (char=? (string-ref s (- i 1)) #\space))
(substring s i n)
(loop (- i 1))))))
;; The csv<ver>/<machine> dir holding scheme.h, libkernel.a, *.boot. Derived from
;; the chez executable's location; JOLT_CHEZ_CSV overrides.
(define bld-csv-dir
(let ((env (getenv "JOLT_CHEZ_CSV")))
(or (and env (> (string-length env) 0) env)
(let* ((bindir (bld-sh-capture "dirname \"$(command -v chez || command -v scheme || command -v petite)\""))
(cand (string-append bindir "/../lib/csv" bld-version "/" bld-machine)))
cand))))
(define (bld-have-cc?)
(> (string-length (bld-sh-capture "command -v cc")) 0))
(define (bld-check-toolchain)
(for-each
(lambda (f)
(let ((p (string-append bld-csv-dir "/" f)))
(unless (file-exists? p)
(error 'jolt-build (string-append "Chez build file missing: " p
"\nSet JOLT_CHEZ_CSV to the csv<ver>/<machine> dir.")))))
'("scheme.h" "libkernel.a" "petite.boot" "scheme.boot")))
;; Link flags. macOS Homebrew layout for the kernel's lz4/zlib/ncurses deps.
(define (bld-link-libs)
(cond
(bld-osx?
(let ((lz4 (bld-sh-capture "brew --prefix lz4 2>/dev/null")))
(string-append
(if (> (string-length lz4) 0) (string-append "-L" lz4 "/lib ") "")
"-llz4 -lz -lncurses -framework Foundation -liconv -lm")))
;; Windows (ta6nt, MinGW-w64 under MSYS2): the Chez kernel pulls in
;; compression, winsock, COM/UUID, and the registry.
(bld-nt?
;; -static: a single-file exe (no libwinpthread/libgcc/lz4 DLL deps) —
;; required for a distributable binary and for TLS init consistency.
"-static -llz4 -lz -lws2_32 -lrpcrt4 -lole32 -luuid -ladvapi32 -luser32 -lshell32 -lm")
;; Linux: the Chez kernel pulls in compression (lz4/z), the expression
;; editor (ncurses + terminfo), threads, dlopen, libuuid, and clock_gettime.
(else "-llz4 -lz -lncurses -ltinfo -ldl -lm -lpthread -luuid -lrt")))
;; --- runtime manifest (mirrors host/chez/cli.ss's load order) ---------------
;; A line is either literal Scheme text to inline, or a tag whose emission the build
;; controls: 'prelude (the clojure.core blob, replaced by the shaken core under
;; tree-shake), 'image + 'compile-eval (the compiler, dropped for a no-eval app).
;; Tagging keeps the splice/drop decisions off fragile substring matching.
(define bld-runtime-manifest
(list
"(load \"host/chez/rt.ss\")"
"(set-chez-ns! \"clojure.core\")"
'prelude
"(load \"host/chez/post-prelude.ss\")"
"(set-chez-ns! \"user\")"
"(load \"host/chez/host-contract.ss\")"
'image
'compile-eval
"(load \"host/chez/png.ss\")"
"(load \"host/chez/loader.ss\")"
"(load \"host/chez/java/ffi.ss\")"
"(set-source-roots! (list \"jolt-core\" \"stdlib\"))"))
(define bld-tagged-loads
'((prelude . "(load \"host/chez/seed/prelude.ss\")")
(image . "(load \"host/chez/seed/image.ss\")")
(compile-eval . "(load \"host/chez/compile-eval.ss\")")))
;; A single-line top-level `(load "PATH")` -> PATH, else #f.
(define (bld-load-path line)
(let ((s (let trim ((i 0))
(if (and (< i (string-length line))
(memv (string-ref line i) '(#\space #\tab)))
(trim (+ i 1))
(substring line i (string-length line))))))
(and (>= (string-length s) 7)
(string=? (substring s 0 6) "(load ")
(let* ((q1 (let scan ((i 6)) (if (char=? (string-ref s i) #\") i (scan (+ i 1)))))
(q2 (let scan ((i (+ q1 1))) (if (char=? (string-ref s i) #\") i (scan (+ i 1))))))
(substring s (+ q1 1) q2)))))
;; runtime source for PATH: from the binary's embedded store if present (a
;; self-contained joltc building an app, with no jolt checkout on disk), else read
;; from disk (running from a source checkout). build-joltc embeds every runtime
;; .ss the manifest inlines, so `build` never touches the filesystem for them.
(define (bld-source-string path)
(let ((emb (hashtable-ref embedded-resources path #f)))
(if (string? emb) emb (read-file-string path))))
(define (bld-string-lines s)
(let ((n (string-length s)))
(let loop ((i 0) (start 0) (acc '()))
(cond ((>= i n) (reverse (if (> i start) (cons (substring s start i) acc) acc)))
((char=? (string-ref s i) #\newline)
(loop (+ i 1) (+ i 1) (cons (substring s start i) acc)))
(else (loop (+ i 1) start acc))))))
(define (bld-file-lines path) (bld-string-lines (bld-source-string path)))
;; Emit one line to OUT, recursively inlining a `(load ...)` of a repo file.
(define (bld-inline-line line out depth)
(when (> depth 50) (error 'jolt-build "load nesting too deep"))
(let ((p (bld-load-path line)))
(if p
(for-each (lambda (l) (bld-inline-line l out (+ depth 1))) (bld-file-lines p))
(begin (put-string out line) (put-string out "\n")))))
;; Inline the runtime manifest, dispatching on the manifest tags. core-strs (the
;; shaken clojure.core defs, or #f) replaces the 'prelude blob; drop-compiler? (a
;; closed AOT app that never compiles from source) omits 'image + 'compile-eval —
;; the analyzer/back end are dead weight in the binary (~0.8MB).
(define (bld-emit-runtime out drop-compiler? core-strs)
(for-each
(lambda (entry)
(cond
((eq? entry 'prelude)
(if core-strs
(for-each (lambda (s) (put-string out s) (put-string out "\n")) core-strs)
(bld-inline-line (cdr (assq 'prelude bld-tagged-loads)) out 0)))
((memq entry '(image compile-eval))
(unless drop-compiler? (bld-inline-line (cdr (assq entry bld-tagged-loads)) out 0)))
(else (bld-inline-line entry out 0))))
bld-runtime-manifest))
;; --- app emission -----------------------------------------------------------
;; Re-emit one app namespace to a list of Scheme strings: optimize (run-passes)
;; and stay strict — a form that fails to emit must fail the build, not vanish.
;; The loop itself is emit-image's ei-emit-ns* (optimize? #t, guard? #f).
(define (bld-emit-ns ns-name src) (ei-emit-ns* ns-name src #t #f))
;; --- whole-program inference pre-pass ---------------------------------------
;; Analyze every app form (all namespaces, deps-first) to IR and run the
;; closed-world param-type fixpoint, so each fn's param types pick up the record
;; types its callers pass. The per-ns emit below then bare-indexes field reads and
;; devirtualizes protocol calls at those sites (the back end reads the resulting
;; :hint/:devirt annotations). Optimized builds only; registries come from the
;; runtime tables populated as the app loaded.
(define jolt-wp-infer! (var-deref "jolt.passes.types" "wp-infer!"))
(define jolt-wp-set-record-shapes! (var-deref "jolt.passes.types" "set-record-shapes!"))
(define jolt-wp-set-proto-methods! (var-deref "jolt.passes.types" "set-protocol-methods!"))
(define jolt-wp-host-record-shapes (var-deref "jolt.host" "record-shapes"))
(define jolt-wp-host-proto-methods (var-deref "jolt.host" "protocol-methods"))
(define (bld-wp-infer! ordered)
(jolt-wp-set-record-shapes! (jolt-wp-host-record-shapes #f))
(jolt-wp-set-proto-methods! (jolt-wp-host-proto-methods #f))
(let ((nodes '()))
(for-each
(lambda (nf)
(set-chez-ns! (car nf))
(let ((src (ldr-read-source (cdr nf))))
(parameterize ((rdr-source-file (cdr nf)))
(for-each
(lambda (f)
(ce-scan-requires! f (car nf))
(unless (or (ei-ns-form? f) (ce-macro-form? f))
(guard (e (#t #f))
(set! nodes (cons (jolt-ce-analyze (make-analyze-ctx (car nf)) f) nodes)))))
(ei-read-all src)))))
ordered)
(jolt-wp-infer! (apply jolt-vector (reverse nodes)))))
;; Strings emitted before each app ns's forms, replaying what the source loader
;; does per file: (1) set chez-current-ns so runtime ns-sensitive setup forms
;; (defmulti/defmethod resolve their target var through it) land in the right ns;
;; (2) register the ns's :as aliases so a quoted alias resolves at runtime — a
;; (defmethod ig/foo …) passes 'ig/foo to defmethod-setup, which needs ig -> the
;; real ns, but the build strips the (ns …) form that would register it.
(define (bld-scan-spec! ns-name spec emit!)
(let ((items (cond ((pvec? spec) (seq->list spec))
((and (cseq? spec) (cseq-list? spec)) (seq->list spec))
(else '()))))
(when (and (pair? items) (symbol-t? (car items)))
(let ((target (symbol-t-name (car items))))
(let loop ((xs (cdr items)))
(when (and (pair? xs) (pair? (cdr xs)))
(let ((k (car xs)) (v (cadr xs)))
(when (keyword? k)
(cond
((and (string=? (keyword-t-name k) "as") (symbol-t? v))
(emit! (string-append "(chez-register-alias! " (ei-str-lit ns-name)
" " (ei-str-lit (symbol-t-name v))
" " (ei-str-lit target) ")")))
;; :refer [a b] / :refer :all — a defmethod on a referred multifn
;; resolves the bare name through the refer table at runtime.
((or (string=? (keyword-t-name k) "refer") (string=? (keyword-t-name k) "only"))
(cond
((and (keyword? v) (string=? (keyword-t-name v) "all"))
(emit! (string-append "(chez-register-refer-all! " (ei-str-lit ns-name)
" " (ei-str-lit target) ")")))
((or (pvec? v) (and (cseq? v) (cseq-list? v)))
(for-each (lambda (n)
(when (symbol-t? n)
(emit! (string-append "(chez-register-refer! " (ei-str-lit ns-name)
" " (ei-str-lit (symbol-t-name n))
" " (ei-str-lit target) ")"))))
(seq->list v))))))))
(loop (cddr xs))))))))
(define (bld-ns-prelude ns-name src)
(let ((acc (list (string-append "(set-chez-ns! " (ei-str-lit ns-name) ")")))
(nsf (let loop ((fs (ei-read-all src)))
(cond ((null? fs) #f)
((ei-ns-form? (car fs)) (car fs))
(else (loop (cdr fs)))))))
(when nsf
(for-each
(lambda (clause)
(when (and (cseq? clause) (cseq-list? clause))
(let ((citems (seq->list clause)))
(when (and (pair? citems) (keyword? (car citems))
(let ((kn (keyword-t-name (car citems))))
(or (string=? kn "require") (string=? kn "use"))))
(for-each (lambda (spec)
(bld-scan-spec! ns-name spec
(lambda (s) (set! acc (cons s acc)))))
(cdr citems))))))
(seq->list nsf)))
(reverse acc)))
;; --- bundling: native libs + resources --------------------------------------
;; A jolt seq of jolt strings -> a Scheme list of Scheme strings.
(define (bld-strs x) (map jolt-str-render-one (seq->list x)))
;; Emit native-library loads. `natives` is the encoded jolt seq jolt.main/
;; encode-natives produced: each entry is ["process"] | ["static" form…] |
;; ["req" cand…] | ["opt" cand…]. `which` selects 'required (process + static +
;; req) or 'optional. Required loads are emitted before the app forms (the app's
;; defcfn foreign-procedures resolve their symbols at top-level eval during
;; startup, so the libs must be loaded first); a load-shared-object failure there
;; is fatal — correct for a required lib. A "static" lib is cc-linked into the
;; binary (see bld-native-link-flags), so its symbols are already in the process:
;; it loads them the same way a "process" lib does. Optional loads run in the
;; scheme-start launcher, where guard catches a missing lib (an optional lib's
;; namespace is only present when the app requires it, so its foreign-procedures
;; aren't among the baked top-level forms).
(define (bld-emit-natives out natives which)
(for-each
(lambda (entry)
(let* ((parts (bld-strs entry)) (kind (car parts)) (cands (cdr parts))
(cand-lits (fold-left (lambda (s c) (string-append s (ei-str-lit c) " ")) "" cands)))
(cond
((and (eq? which 'required) (or (string=? kind "process") (string=? kind "static")))
(put-string out "(jolt-build-load-native '() #f #t)\n"))
((and (eq? which 'required) (string=? kind "req"))
(put-string out (string-append "(jolt-build-load-native (list " cand-lits ") #f #f)\n")))
((and (eq? which 'optional) (string=? kind "opt"))
(put-string out (string-append "(jolt-build-load-native (list " cand-lits ") #t #f)\n"))))))
(seq->list natives)))
;; The cc link fragment for the "static" natives: each archive must be FORCE-loaded
;; (the linker would otherwise drop an archive member main.c never references) and,
;; on Linux, the executable's symbols exported into the dynamic table so the
;; startup (load-shared-object #f) + foreign-procedure can resolve them (-rdynamic,
;; added by build-with-cc when this fragment is non-empty). Returns "" when no lib
;; is statically linked. Entry forms: ["static" "archive" path] | ["static" "lib"
;; name libdir].
(define (bld-native-link-flags natives)
(fold-left
(lambda (acc entry)
(let ((parts (bld-strs entry)))
(if (string=? (car parts) "static")
(string-append acc " " (bld-one-static-link (cdr parts)))
acc)))
"" (seq->list natives)))
;; A statically-linked native is only in the OUTPUT binary, but build step 1
;; evaluates the app's `foreign-procedure` forms in THIS process (to register its
;; macros/vars), and Chez resolves a foreign entry eagerly. So make the archive's
;; symbols resolvable here: build a throwaway shared object from it (force-loading
;; every member) and load it. The output binary still cc-links the static archive;
;; this temp .so is build-time only. Only the "archive" form is preloaded — the
;; "lib" form names a system library the OS loader already finds by soname.
(define (bld-preload-static-natives! natives builddir)
(let ((n 0))
(for-each
(lambda (entry)
(let ((parts (bld-strs entry)))
(when (and (string=? (car parts) "static") (string=? (cadr parts) "archive"))
(let* ((archive (caddr parts))
(so (string-append builddir "/native-" (number->string n)
(if bld-osx? ".dylib" ".so"))))
(set! n (+ n 1))
(bld-system
(if bld-osx?
(string-append "cc -dynamiclib -undefined dynamic_lookup -Wl,-all_load '"
archive "' -o '" so "'")
(string-append "cc -shared -Wl,--whole-archive '" archive
"' -Wl,--no-whole-archive -Wl,--unresolved-symbols=ignore-all -o '" so "'")))
(load-shared-object so)))))
(seq->list natives))))
(define (bld-one-static-link form)
(let ((kind (car form)))
(cond
((string=? kind "archive")
(let ((path (cadr form)))
(if bld-osx?
(string-append "-Wl,-force_load," path)
(string-append "-Wl,--whole-archive " path " -Wl,--no-whole-archive"))))
((string=? kind "lib")
(let* ((lib (cadr form)) (dir (caddr form))
(L (if (> (string-length dir) 0) (string-append "-L" dir " ") "")))
;; -Bstatic forces the .a over a .so of the same -l name (GNU ld). macOS's
;; ld64 has no -Bstatic; there an :archive path is the reliable form.
(if bld-osx?
(string-append L "-l" lib)
(string-append L "-Wl,-Bstatic -l" lib " -Wl,-Bdynamic"))))
(else ""))))
;; Walk an embed root recursively; return (resource-name . abspath) pairs, where
;; resource-name is the "/"-joined path under the root (what io/resource is asked for).
(define (bld-walk-files root rel acc)
(let ((dir (if (string=? rel "") root (string-append root "/" rel))))
(fold-left
(lambda (acc name)
(let* ((relpath (if (string=? rel "") name (string-append rel "/" name)))
(full (string-append root "/" relpath)))
(if (file-directory? full)
(bld-walk-files root relpath acc)
(cons (cons relpath full) acc))))
acc
(directory-list dir))))
;; Emit register-embedded-resource! per file under each embed dir. Emitted BEFORE
;; the app forms so the (read-file-string ABSPATH) runs at heap build — the file's
;; contents bake into the boot image and io/resource serves them with no file on
;; disk. ABSPATH only has to exist at build time.
(define (bld-emit-embeds out embed-dirs)
(for-each
(lambda (root)
(when (file-directory? root)
(for-each
(lambda (rp)
(put-string out (string-append
"(register-embedded-resource! " (ei-str-lit (car rp))
" (read-file-string " (ei-str-lit (cdr rp)) "))\n")))
(bld-walk-files root "" '()))))
(bld-strs embed-dirs)))
;; --- the build --------------------------------------------------------------
;; entry-ns: the app's main namespace (a string). out-path: the binary to write.
;; mode: "dev" | "release" | "optimized". Every form runs through jolt.passes/
;; run-passes (const-fold always; inline + type inference when optimized turns on
;; direct-linking). Deps + source roots are already applied by the caller.
;; natives: encoded :jolt/native libs to load at startup. embed-dirs: dirs whose
;; files bake into the binary (single-file). ext-roots: project-relative io/resource
;; roots resolved at runtime against JOLT_PWD (ship-alongside resources).
;; direct-link?: opt-in closed-world direct-linking (app->app calls bind directly,
;; no runtime redefinition). Off by default in every mode — release stays
;; dynamically linked.
(define (bld-suffix? s suf)
(let ((n (string-length s)) (m (string-length suf)))
(and (>= n m) (string=? (substring s (- n m) n) suf))))
(define (build-binary entry-ns out-path mode natives embed-dirs ext-roots direct-link? tree-shake?)
;; Windows executables carry .exe; normalize here so the append-payload and
;; cc paths agree and the shell can run the result.
(let ((out-path (if (and bld-nt? (not (bld-suffix? out-path ".exe")))
(string-append out-path ".exe")
out-path)))
;; The self-contained path (jolt-embedded-bytes "stub/launcher") needs no csv
;; kernel files, no Chez, no cc — only the legacy cc path does.
(unless (jolt-embedded-bytes "stub/launcher") (bld-check-toolchain))
(when (> (string-length (bld-native-link-flags natives)) 0)
;; :static natives are cc-linked into the binary, so a C compiler must be on
;; PATH — the self-contained joltc bundles the Chez kernel (libkernel.a +
;; scheme.h) and relinks a custom stub (see build-self-contained), but still
;; needs the system cc for that link. Fail early (before the app's foreign-
;; procedure forms eval below) with an actionable message.
(unless (bld-have-cc?)
(error 'jolt-build
"static native linking needs a C compiler (cc) on PATH; install one, or pass --dynamic to load the library at runtime."))
;; Preload static archives' symbols into this process so step 1's foreign-
;; procedure evals resolve; the .build dir must exist first.
(bld-mkdir-p (string-append out-path ".build"))
(bld-preload-static-natives! natives (string-append out-path ".build")))
;; 1. record app namespaces in dependency order as they finish loading.
(let ((app-order '()))
(set-ns-loaded-hook!
(lambda (name file) (set! app-order (cons (cons name file) app-order))))
(load-namespace entry-ns)
(set-ns-loaded-hook! (lambda (name file) #f))
(let ((ordered (reverse app-order))) ; deps first, entry last
(when (null? ordered)
(error 'jolt-build (string-append "no source namespace loaded for " entry-ns
" — is it on the source roots?")))
;; 2. emit each app namespace. `optimized` turns on the inference + flatten
;; + scalar-replace passes; release/dev get const-fold only.
;; direct-link? (opt-in) commits to a closed world: app->app calls bind
;; directly, giving up runtime redefinition of those vars. Off by default in
;; every mode. The defined-set accumulates across the dependency-ordered
;; namespaces, so a dep's defs are direct-linkable by the time the entry that
;; calls them is emitted.
;; set-optimize!/set-direct-link! are process-global flags in the back end;
;; dynamic-wind guarantees they revert even if a strict form errors mid-emit
;; (a failing form errors the build by design), so the compiler isn't left in
;; optimize/direct-link mode for a later caller.
(let*-values
(((core-strs app-strs drop-compiler?)
(dynamic-wind
(lambda ()
(set-optimize! (string=? mode "optimized"))
(when direct-link?
((var-deref "jolt.backend-scheme" "set-direct-link!") #t)
((var-deref "jolt.backend-scheme" "direct-link-reset!")))
;; whole-program param-type fixpoint before per-form emit
(when (string=? mode "optimized") (bld-wp-infer! ordered)))
(lambda ()
;; A #tag data-reader literal must compile in the binary the same as
;; it loads interpreted — apply the reader rewrite to each emitted
;; form too (no-op unless the app registered data readers).
(parameterize ((ei-emit-form-hook
(lambda (form) (if data-readers-active (ldr-apply-readers form) form))))
(if tree-shake?
(dce-shake
(dce-blob-records "host/chez/seed/prelude.ss")
(apply append
(map (lambda (nf)
;; ns-prelude forms (always kept, no fqn/refs) set the
;; ns + register aliases before this ns's forms; dce
;; keeps original order.
(let ((src (ldr-read-source (cdr nf))))
(parameterize ((rdr-source-file (cdr nf)))
(append
(map (lambda (s) (dce-rec #t #f '() s))
(bld-ns-prelude (car nf) src))
(ei-emit-ns-records (car nf) src)))))
ordered))
(string-append entry-ns "/-main"))
(values #f
(apply append
(map (lambda (nf)
(let ((src (ldr-read-source (cdr nf))))
(parameterize ((rdr-source-file (cdr nf)))
(append (bld-ns-prelude (car nf) src)
(bld-emit-ns (car nf) src)))))
ordered))
#f))))
(lambda ()
(set-optimize! #f)
((var-deref "jolt.backend-scheme" "set-direct-link!") #f)))))
(when drop-compiler? (display "jolt build: dropping compiler image (no runtime eval)\n"))
(let* ((builddir (string-append out-path ".build"))
(flat-ss (string-append builddir "/flat.ss"))
(flat-so (string-append builddir "/flat.so"))
(boot (string-append builddir "/jolt.boot"))
(boot-h (string-append builddir "/boot_data.h"))
(main-c (string-append builddir "/main.c")))
(bld-mkdir-p builddir)
;; 3. flat source = runtime + app + launcher.
(let ((out (open-output-file flat-ss 'replace)))
(bld-emit-runtime out drop-compiler? core-strs)
;; Load native libs, bake embedded resources, and point source roots at
;; the build-time app roots — all BEFORE the app forms. The app's
;; top-level forms run at binary startup (Sbuild_heap), and they include
;; foreign-procedure evals (a library's defcfn) and (slurp (io/resource …))
;; reads. So the libraries must be loaded and resources resolvable by the
;; time those forms run, not later in the scheme-start launcher.
(put-string out "\n;; === native libraries (required) ===\n")
(bld-emit-natives out natives 'required)
(put-string out "\n;; === embedded resources ===\n")
(bld-emit-embeds out embed-dirs)
(put-string out (string-append
"(set-source-roots! (list "
(fold-left (lambda (s r) (string-append s (ei-str-lit r) " ")) ""
(get-source-roots))
"))\n"))
(put-string out "\n;; === app ===\n")
(for-each (lambda (s) (put-string out s) (put-string out "\n")) app-strs)
;; The launcher runs as Chez's scheme-start (so argv reaches -main —
;; top-level boot forms run during heap build, before args are set), and
;; suppresses the interactive greeting. It resets source roots to the
;; app's resource dirs resolved against JOLT_PWD (or cwd) so a runtime
;; io/resource that wasn't embedded still resolves next to the binary.
(put-string out "\n;; === launcher ===\n")
(put-string out "(suppress-greeting #t)\n")
(put-string out "(scheme-start\n (lambda args\n")
(bld-emit-natives out natives 'optional)
(put-string out (string-append
" (let ((base (or (getenv \"JOLT_PWD\") \".\")))\n"
" (set-source-roots!\n"
" (append (map (lambda (r) (string-append base \"/\" r)) (list "
(fold-left (lambda (s r) (string-append s (ei-str-lit r) " ")) "" (bld-strs ext-roots))
"))\n"
" (list \"jolt-core\" \"stdlib\"))))\n"))
(put-string out (string-append
;; Call -main only if the entry namespace defines one;
;; a script ns (top-level side effects, no -main) has
;; already run its forms at heap build, so invoking a nil
;; -main would crash ("nil cannot be cast to IFn") — just
;; exit cleanly instead.
" (let ((maincell (var-cell-lookup " (ei-str-lit entry-ns) " \"-main\")))\n"
;; render an uncaught throw (+ Clojure backtrace) instead
;; of Chez's opaque dump, then exit non-zero.
" (guard (v (#t (jolt-report-throwable v (current-error-port)) (exit 1)))\n"
;; Loading the app left the current ns at the entry ns; reset
;; it to `user` before -main, matching clojure.main (*ns* is
;; `user` when a `-m` -main runs, so a runtime resolve of an
;; aliased symbol behaves the same as on the JVM / interpreted
;; joltc, not off the entry ns's alias table).
" (set-chez-ns! \"user\")\n"
" (when (and maincell (var-cell-defined? maincell))\n"
" (apply jolt-invoke (var-cell-root maincell) args))))\n"
" (exit 0)))\n"))
(close-port out))
;; 4. compile -> boot -> link. Two paths, chosen by whether this process
;; carries the bundled Chez boots + launcher stub:
;; - SELF-CONTAINED (the distributed joltc, jolt-eaj): compile-file +
;; make-boot-file run IN PROCESS (the compiler is resident — joltc is
;; built from scheme.boot), then the boot is appended to a copy of the
;; embedded stub. No external Chez, no cc.
;; - LEGACY (dev bin/joltc): spawn a fresh Chez for compile-file/
;; make-boot-file, then xxd the boot into a C array and cc-link against
;; libkernel.a. Kept so `make buildsmoke` still exercises the cc path.
(if (jolt-embedded-bytes "stub/launcher")
(build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot
(bld-native-link-flags natives))
(build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c
(bld-native-link-flags natives)))))))))
;; --- self-contained link (in-process compile + append the boot to the stub) ---
;; compile-file runs against the DEFAULT interaction environment, so the boot's
;; top-level defines land in the real symbol cells — the runtime compiler's
;; eval'd code must resolve them (var-deref, jolt-invoke, the jolt-n* macros)
;; when the built binary dynamically requires a namespace. Compiling in a clean
;; copy-environment instead orphans every define in locations eval can't see,
;; and the binary dies with "variable var-deref is not bound" the moment a
;; runtime require compiles source.
;;
;; The default env has a wrinkle the legacy fresh-Chez path doesn't: THIS
;; process's cells hold jolt's redefinitions of some kernel names (`error`,
;; regex.ss), so references to them compile as cell reads — and a read that
;; runs before the redefining form would find the fresh binary's cell unbound.
;; The prologue closes that: it first binds each redefined kernel name's cell
;; to its kernel value, making the boot's earliest reads identical to the
;; legacy path's primitive references.
;; every top-level (define nm …)/(define (nm …) …) name in the flat file that
;; shadows a scheme-environment VARIABLE (syntax names don't eval; skip them).
(define (bld-kernel-prologue flat-ss)
(let ((seen (make-eq-hashtable))
(kenv (scheme-environment))
(names '()))
(let ((ip (open-input-file flat-ss)))
(let loop ()
(let ((f (read ip)))
(unless (eof-object? f)
(when (and (pair? f) (eq? (car f) 'define) (pair? (cdr f)))
(let* ((h (cadr f))
(nm (if (pair? h) (car h) h)))
(when (and (symbol? nm)
(not (hashtable-ref seen nm #f))
(guard (e (#t #f)) (begin (eval nm kenv) #t)))
(hashtable-set! seen nm #t)
(set! names (cons nm names)))))
(loop))))
(close-port ip))
(apply string-append
(map (lambda (nm)
(let ((s (symbol->string nm)))
(string-append "(define " s " (eval '" s " (scheme-environment)))\n")))
(reverse names)))))
;; prepend the prologue to the flat file in place.
(define (bld-prepend-prologue! flat-ss)
(let ((prologue (bld-kernel-prologue flat-ss))
(body (read-file-string flat-ss)))
(let ((out (open-output-file flat-ss 'replace)))
(put-string out ";; kernel-name cells pre-bound so early reads match the kernel primitives\n")
(put-string out prologue)
(put-string out body)
(close-port out))))
(define (build-self-contained entry-ns out-path mode builddir flat-ss flat-so boot native-link)
(let ((petite (string-append builddir "/petite.boot"))
(scheme (string-append builddir "/scheme.boot")))
(jolt-spill-embedded! "csv/petite.boot" petite)
(jolt-spill-embedded! "csv/scheme.boot" scheme)
(display (string-append "jolt build: compiling " entry-ns " (" mode " mode, self-contained)\n"))
(bld-prepend-prologue! flat-ss)
(compile-file flat-ss flat-so)
(make-boot-file boot '() petite scheme flat-so)
;; The stub is the native launcher the boot is appended to. With no :static
;; natives it's the prebuilt one bundled in joltc (no cc needed); with :static
;; natives it's re-linked here from the bundled kernel + launcher source so the
;; archives are baked in and their symbols resolve in the running binary.
(if (> (string-length native-link) 0)
(bld-relink-stub builddir native-link out-path)
(jolt-spill-embedded! "stub/launcher" out-path))
;; link: stub bytes ++ boot ++ frame, then make it executable.
(jolt-append-payload! out-path (read-file-bytes boot))
(jolt-chmod-755 out-path)
(display (string-append "jolt build: wrote " out-path "\n"))
(when bld-osx?
(display (string-append
"jolt build: note — on macOS this binary is unsigned; to share it,\n"
" `xattr -d com.apple.quarantine " out-path "` on the target, or sign it.\n")))))
;; Re-link the launcher stub with the app's static native archives baked in, to
;; OUT-PATH. The self-contained joltc bundles the Chez kernel (libkernel.a),
;; header, and launcher source; spill them and drive the system cc — the same link
;; build-joltc.ss ran once at joltc-build time, plus the force-load archive flags
;; (native-link) and, on Linux, -rdynamic so the baked-in symbols stay dlsym-
;; visible for (load-shared-object #f) + foreign-procedure at startup.
(define (bld-relink-stub builddir native-link out-path)
(let ((h (string-append builddir "/scheme.h"))
(lk (string-append builddir "/libkernel.a"))
(lc (string-append builddir "/launcher.c")))
(jolt-spill-embedded! "csv/scheme.h" h)
(jolt-spill-embedded! "csv/libkernel.a" lk)
(jolt-spill-embedded! "stub/launcher.c" lc)
(display "jolt build: relinking launcher stub with static native libraries\n")
(bld-system (string-append
"cc -O2 " (if bld-osx? "" "-rdynamic ")
"-I'" builddir "' '" lc "' '" lk "' -o '" out-path "' "
(bld-link-libs) native-link))))
;; --- legacy cc link (dev bin/joltc): fresh Chez compile + xxd + cc ------------
(define (build-with-cc entry-ns out-path mode builddir flat-ss flat-so boot boot-h main-c native-link)
(display (string-append "jolt build: compiling " entry-ns " (" mode " mode)\n"))
(let ((cs (string-append builddir "/compile.ss")))
(let ((p (open-output-file cs 'replace)))
(put-string p
(string-append
"(import (chezscheme))\n"
"(compile-file " (ei-str-lit flat-ss) " " (ei-str-lit flat-so) ")\n"
"(make-boot-file " (ei-str-lit boot) " '()\n "
(ei-str-lit (string-append bld-csv-dir "/petite.boot")) "\n "
(ei-str-lit (string-append bld-csv-dir "/scheme.boot")) "\n "
(ei-str-lit flat-so) ")\n"))
(close-port p))
(bld-system (string-append bld-chez " --script '" cs "'")))
(bld-system (string-append "xxd -i '" boot "' > '" boot-h "'"))
;; The xxd symbol is derived from the path; normalize to jolt_boot.
(bld-system (string-append
"sed -i.bak -E 's/unsigned char [A-Za-z0-9_]+\\[\\]/unsigned char jolt_boot[]/; "
"s/unsigned int [A-Za-z0-9_]+_len/unsigned int jolt_boot_len/' '" boot-h "'"))
(let ((mc (open-output-file main-c 'replace)))
(put-string mc
(string-append
"#include \"scheme.h\"\n#include \"boot_data.h\"\n"
"int main(int argc, char *argv[]) {\n"
" Sscheme_init(0);\n"
" Sregister_boot_file_bytes(\"jolt\", jolt_boot, jolt_boot_len);\n"
" Sbuild_heap(0, 0);\n"
" int status = Sscheme_start(argc, (const char **)argv);\n"
" Sscheme_deinit();\n return status;\n}\n"))
(close-port mc))
;; -rdynamic (Linux) exports the executable's symbols into the dynamic table so
;; a statically-linked native lib's symbols resolve via (load-shared-object #f)
;; at startup. macOS keeps unstripped executable symbols dlsym-visible already.
(bld-system (string-append
"cc -O2 " (if (and (not bld-osx?) (> (string-length native-link) 0)) "-rdynamic " "")
"-I'" bld-csv-dir "' '" main-c "' '" bld-csv-dir "/libkernel.a' "
"-o '" out-path "' " (bld-link-libs) native-link))
(display (string-append "jolt build: wrote " out-path "\n")))
(def-var! "jolt.host" "build-binary"
(lambda (entry out mode natives embed-dirs ext-roots direct-link? tree-shake?)
(build-binary (jolt-str-render-one entry)
(jolt-str-render-one out)
(jolt-str-render-one mode)
natives embed-dirs ext-roots (jolt-truthy? direct-link?) (jolt-truthy? tree-shake?))
jolt-nil))

90
host/chez/cli.ss Normal file
View file

@ -0,0 +1,90 @@
;; cli.ss — the jolt runtime.
;;
;; Loads the checked-in seed (host/chez/seed/{prelude,image}.ss — the bootstrap
;; compiler) and the spine, then either evaluates a -e expression or dispatches a
;; CLI command (run/-M/repl/path/task) through jolt.main. The loader
;; (loader.ss) turns `require` into real file loading off the source roots, so a
;; multi-file project with deps.edn dependencies runs end to end.
;;
;; Run from the repo root (bin/joltc cd's there); the project dir is JOLT_PWD.
(import (chezscheme))
(define cli-args (cdr (command-line))) ; drop the script name
;; Fail early and actionably when the vendored submodules aren't checked out —
;; a plain `git clone` or GitHub's auto-generated "Source code" release archive
;; lacks them, and the raw failure ("load failed for vendor/irregex/irregex.scm")
;; doesn't say how to fix it. (The self-contained joltc binary embeds these and
;; never runs this file.)
(unless (file-exists? "vendor/irregex/irregex.scm")
(display "jolt: vendor submodules are missing (vendor/irregex).
" (current-error-port))
(display "GitHub's 'Source code' release archives don't include submodules.
" (current-error-port))
(display "Clone the repo instead:
" (current-error-port))
(display " git clone --recurse-submodules https://github.com/jolt-lang/jolt.git
" (current-error-port))
(display "or, in an existing checkout:
" (current-error-port))
(display " git submodule update --init --recursive
" (current-error-port))
(exit 1))
(load "host/chez/rt.ss")
(set-chez-ns! "clojure.core")
(load "host/chez/seed/prelude.ss")
(load "host/chez/post-prelude.ss")
(set-chez-ns! "user")
(load "host/chez/host-contract.ss")
(load "host/chez/seed/image.ss")
(load "host/chez/compile-eval.ss")
(load "host/chez/png.ss") ; jolt.png — a baked namespace before the snapshot
(load "host/chez/loader.ss")
;; jolt.ffi host primitives (memory / library loading) load AFTER the loader's
;; baked-ns snapshot, so a library's (require '[jolt.ffi]) still loads jolt.ffi's
;; Clojure side (the foreign-fn / defcfn macros, stdlib/jolt/ffi.clj).
(load "host/chez/java/ffi.ss") ; jolt.ffi (FFI: a library binds native code)
;; jolt.main + jolt.deps live under jolt-core; keep them (and stdlib) on the
;; roots so the CLI's own namespaces — and any jolt.* an app pulls in — resolve.
;; A project's resolved deps roots are prepended to these by jolt.main.
(set-source-roots! (list "jolt-core" "stdlib"))
;; Render an uncaught jolt throw (any value, not just a Chez condition) to stderr
;; and exit non-zero, instead of Chez's opaque "non-condition value" dump. The
;; message/ex-data/cause + a mapped Clojure backtrace come from the shared
;; renderer (source-registry.ss); the cli adds the top-level source location.
(define (jolt-report-uncaught raw)
(let ((v (jolt-unwrap-throw raw))
(port (current-error-port)))
(jolt-render-throwable v port)
;; The top-level form that was evaluating when this propagated (file:line:col).
(let ((loc (jolt-current-source-string)))
(when loc (display " at " port) (display loc port) (newline port)))
(let ((bt (jolt-backtrace-string v)))
(when bt (display " trace:\n" port) (display bt port)))
(exit 1)))
;; JOLT_TRACE opt-in, at runtime (before any app ns compiles) so the app is traced.
(jolt-trace-init-from-env!)
(guard (v (#t (jolt-report-uncaught v)))
(cond
;; -e EXPR — evaluate one expression and print it (blank for nil). Wrapped in
;; (do …) so a multi-form string evaluates every form and returns the last.
((and (= (length cli-args) 2) (string=? (car cli-args) "-e"))
(let ((result (jolt-final-str
(jolt-compile-eval (string-append "(do " (cadr cli-args) ")") "user"))))
(unless (string=? result "")
(display result) (newline))))
;; otherwise dispatch the argv through jolt.main/-main
(else
;; `build` AOT-compiles an app to a standalone binary — load the build
;; driver (the cross-compiler emitter) on demand so a normal run never pays
;; for it. It defines jolt.host/build-binary, which jolt.main's build cmd calls.
(when (and (pair? cli-args) (string=? (car cli-args) "build"))
(load "host/chez/build.ss"))
(load-namespace "jolt.main")
(let ((mainv (var-deref "jolt.main" "-main")))
(apply jolt-invoke mainv cli-args)))))

606
host/chez/collections.ss Normal file
View file

@ -0,0 +1,606 @@
;; persistent collections on the Chez RT.
;;
;; The vector / map / set the emitted programs construct from literals and
;; operate on via the lowered leaf ops (conj/get/nth/count/assoc/...). Loaded by
;; rt.ss after values.ss; jolt=2 / jolt-hash (values.ss) call into the
;; jolt-coll? / jolt-coll=? / jolt-coll-hash hooks defined here (forward refs,
;; resolved at run time — nothing is CALLED during load).
;;
;; The persistent vector is a copy-on-write Scheme vector and the map/set are a
;; bitmap HAMT. They live in Scheme; correctness, not perf, is the gate.
;; ============================================================================
;; small immutable-vector helpers (manual; avoid stdlib arg-order ambiguity)
;; ============================================================================
(define (vec-copy-range v start end)
(let ((out (make-vector (fx- end start))))
(let loop ((i start))
(when (fx<? i end) (vector-set! out (fx- i start) (vector-ref v i)) (loop (fx+ i 1))))
out))
(define (vec-insert v i x) ; copy of v with x spliced in at index i
(let* ((n (vector-length v)) (out (make-vector (fx+ n 1))))
(let loop ((j 0)) (when (fx<? j i) (vector-set! out j (vector-ref v j)) (loop (fx+ j 1))))
(vector-set! out i x)
(let loop ((j i)) (when (fx<? j n) (vector-set! out (fx+ j 1) (vector-ref v j)) (loop (fx+ j 1))))
out))
(define (vec-set v i x) ; functional update at index i
(let ((out (vec-copy-range v 0 (vector-length v)))) (vector-set! out i x) out))
(define (vec-remove v i) ; copy of v with index i dropped
(let* ((n (vector-length v)) (out (make-vector (fx- n 1))))
(let loop ((j 0)) (when (fx<? j i) (vector-set! out j (vector-ref v j)) (loop (fx+ j 1))))
(let loop ((j (fx+ i 1))) (when (fx<? j n) (vector-set! out (fx- j 1) (vector-ref v j)) (loop (fx+ j 1))))
out))
;; ============================================================================
;; persistent vector — 32-way trie + tail (Clojure's PersistentVector)
;; ============================================================================
;; cnt elements live in a trie of 32-wide nodes (root, height = shift bits) plus a
;; trailing `tail` chunk of 1..32. conj appends to the tail and, when it fills,
;; pushes it into the trie by path-copy — so conj is O(1) amortized and a linear
;; build is O(n), not the O(n^2) of a flat copy-on-write array. nth/assoc/pop are
;; O(log32 n). Trie nodes are Scheme vectors holding only their live children
;; (grown left-to-right), so a node's length is its child count.
;;
;; `ent` #t marks a MAP ENTRY (the [k v] pair seq'd out of a map). An entry has 2
;; elements (all in the tail), equals its [k v] vector and walks like one, and is
;; both vector? (Clojure's MapEntry implements IPersistentVector) and map-entry?.
;; Modifying an entry (conj/assoc/pop) yields a plain vector (ent #f).
;;
;; make-pvec and pvec-v keep the old flat-vector API: make-pvec builds a trie from
;; a Scheme vector (every existing caller still passes one) and pvec-v materializes
;; it back, so only this file's internals change.
(define pv-bits 5)
(define pv-width 32)
(define pv-mask 31)
(define pv-empty-node (vector))
(define-record-type (pvec mk-pvec pvec?)
(fields cnt shift root tail ent) (nongenerative chez-pvec-v2))
;; trailing helpers over Scheme vectors used by the trie
(define (vec-snoc v x) ; copy v with x appended
(let* ((n (vector-length v)) (out (make-vector (fx+ n 1))))
(let loop ((i 0)) (when (fx<? i n) (vector-set! out i (vector-ref v i)) (loop (fx+ i 1))))
(vector-set! out n x) out))
(define (vec-drop-last v) (vec-copy-range v 0 (fx- (vector-length v) 1)))
(define (vec-take v n) (vec-copy-range v 0 n))
(define (vec-set-or-snoc v i x) ; replace index i, or append when i = length
(let ((n (vector-length v))) (if (fx<? i n) (vec-set v i x) (vec-snoc v x))))
(define (pv-tailoff cnt)
(if (fx<? cnt pv-width) 0 (fxsll (fxsra (fx- cnt 1) pv-bits) pv-bits)))
;; the 32-chunk Scheme vector holding index i (the tail or a trie leaf)
(define (pv-chunk-for p i)
(if (fx>=? i (pv-tailoff (pvec-cnt p)))
(pvec-tail p)
(let loop ((node (pvec-root p)) (level (pvec-shift p)))
(if (fx>? level 0)
(loop (vector-ref node (fxand (fxsra i level) pv-mask)) (fx- level pv-bits))
node))))
;; jolt models every number as a double, so vector indices arrive as flonums —
;; coerce an integer-valued index to a Scheme fixnum before bounds math.
(define (->idx i) (if (fixnum? i) i (if (flonum? i) (exact (floor i)) i)))
(define (pvec-count p) (pvec-cnt p))
(define (pvec-nth-d p i d)
(let ((i (->idx i)))
(if (and (fixnum? i) (fx>=? i 0) (fx<? i (pvec-cnt p)))
(vector-ref (pv-chunk-for p i) (fxand i pv-mask))
d)))
;; new-path: wrap a node in single-child nodes up `level` bits.
(define (pv-new-path level node)
(if (fx=? level 0) node (vector (pv-new-path (fx- level pv-bits) node))))
;; push a full tail chunk into the trie under `parent` at `level`.
(define (pv-push-tail cnt level parent tail-node)
(let ((subidx (fxand (fxsra (fx- cnt 1) level) pv-mask)))
(if (fx=? level pv-bits)
(vec-set-or-snoc parent subidx tail-node)
(let ((child (and (fx<? subidx (vector-length parent)) (vector-ref parent subidx))))
(vec-set-or-snoc parent subidx
(if child (pv-push-tail cnt (fx- level pv-bits) child tail-node)
(pv-new-path (fx- level pv-bits) tail-node)))))))
(define (pvec-conj p x)
(let ((cnt (pvec-cnt p)) (shift (pvec-shift p)))
(if (fx<? (fx- cnt (pv-tailoff cnt)) pv-width)
;; room in the tail
(mk-pvec (fx+ cnt 1) shift (pvec-root p) (vec-snoc (pvec-tail p) x) #f)
;; tail full: push it into the trie, start a fresh tail
(let ((tail-node (pvec-tail p)))
(if (fx>? (fxsra cnt pv-bits) (fxsll 1 shift))
;; root overflow: grow the trie a level
(mk-pvec (fx+ cnt 1) (fx+ shift pv-bits)
(vector (pvec-root p) (pv-new-path shift tail-node))
(vector x) #f)
(mk-pvec (fx+ cnt 1) shift
(pv-push-tail cnt shift (pvec-root p) tail-node)
(vector x) #f))))))
(define (pv-assoc-trie level node i x)
(if (fx=? level 0)
(vec-set node (fxand i pv-mask) x)
(let ((subidx (fxand (fxsra i level) pv-mask)))
(vec-set node subidx (pv-assoc-trie (fx- level pv-bits) (vector-ref node subidx) i x)))))
(define (pvec-assoc p i x) ; i in [0,count]; =count appends
(let ((i (->idx i)) (cnt (pvec-cnt p)))
(cond
((fx=? i cnt) (pvec-conj p x))
((and (fx>=? i 0) (fx<? i cnt))
(if (fx>=? i (pv-tailoff cnt))
(mk-pvec cnt (pvec-shift p) (pvec-root p)
(vec-set (pvec-tail p) (fxand i pv-mask) x) #f)
(mk-pvec cnt (pvec-shift p)
(pv-assoc-trie (pvec-shift p) (pvec-root p) i x) (pvec-tail p) #f)))
(else (jolt-throw (jolt-host-throwable "java.lang.IndexOutOfBoundsException" "vector index out of bounds"))))))
(define (pvec-peek p)
(let ((n (pvec-cnt p))) (if (fx=? n 0) jolt-nil (pvec-nth-d p (fx- n 1) jolt-nil))))
;; pop the last trie chunk back into the tail; #f means the subtree emptied.
(define (pv-pop-tail cnt level node)
(let ((subidx (fxand (fxsra (fx- cnt 2) level) pv-mask)))
(cond
((fx>? level pv-bits)
(let ((newchild (pv-pop-tail cnt (fx- level pv-bits) (vector-ref node subidx))))
(cond ((and (not newchild) (fx=? subidx 0)) #f)
(newchild (vec-set node subidx newchild))
(else (vec-take node subidx)))))
((fx=? subidx 0) #f)
(else (vec-take node subidx)))))
(define (pvec-pop p)
(let ((cnt (pvec-cnt p)) (shift (pvec-shift p)))
(cond
((fx=? cnt 0) (error 'pop "can't pop empty vector"))
((fx=? cnt 1) empty-pvec)
((fx>? (fx- cnt (pv-tailoff cnt)) 1)
(mk-pvec (fx- cnt 1) shift (pvec-root p) (vec-drop-last (pvec-tail p)) #f))
(else
(let* ((new-tail (pv-chunk-for p (fx- cnt 2)))
(popped (pv-pop-tail cnt shift (pvec-root p)))
(new-root (or popped pv-empty-node)))
(if (and (fx>? shift pv-bits) (fx<? (vector-length new-root) 2))
(mk-pvec (fx- cnt 1) (fx- shift pv-bits)
(if (fx=? 0 (vector-length new-root)) pv-empty-node (vector-ref new-root 0))
new-tail #f)
(mk-pvec (fx- cnt 1) shift new-root new-tail #f)))))))
(define empty-pvec (mk-pvec 0 pv-bits pv-empty-node (vector) #f))
;; build a trie pvec from a flat Scheme vector (the public constructor).
(define make-pvec
(case-lambda
((v) (make-pvec v #f))
((v ent)
(let ((n (vector-length v)))
(if (fx<=? n pv-width)
(mk-pvec n pv-bits pv-empty-node v ent) ; fits in the tail
(let loop ((p empty-pvec) (i 0))
(if (fx=? i n) p (loop (pvec-conj p (vector-ref v i)) (fx+ i 1)))))))))
;; materialize the trie back to a flat Scheme vector (compatibility for callers
;; that read the backing array — all one-shot conversions, not hot loops).
(define (pvec-v p)
(let* ((cnt (pvec-cnt p)) (out (make-vector cnt)))
(let loop ((i 0))
(if (fx<? i cnt)
(let* ((chunk (pv-chunk-for p i)) (clen (vector-length chunk)))
(let cloop ((j 0) (k i))
(if (and (fx<? j clen) (fx<? k cnt))
(begin (vector-set! out k (vector-ref chunk j)) (cloop (fx+ j 1) (fx+ k 1)))
(loop k))))
out))))
(define (jolt-vector . xs) (make-pvec (list->vector xs)))
(define (make-map-entry k v) (make-pvec (vector k v) #t))
(define (jolt-map-entry? x) (and (pvec? x) (pvec-ent x) #t))
;; ============================================================================
;; bitmap HAMT — keys hashed by jolt-hash, leaves compared by jolt=
;; arr slot is one of: leaf (cons k v) | hnode (branch) | hcoll (hash bucket)
;; ============================================================================
(define-record-type hnode (fields bm arr) (nongenerative chez-hnode-v1))
(define-record-type hcoll (fields hash alist) (nongenerative chez-hcoll-v1))
(define empty-hnode (make-hnode 0 (vector)))
(define hmask #x3FFFFFFFFFFFFFF) ; 58-bit non-negative hash window
(define max-shift 55)
;; bitwise-and (not fxand): jolt-hash is set!-decorated per type (records/inst/
;; sorted return their own hash) and Chez's equal-hash can yield a BIGNUM, so a
;; key's hash isn't guaranteed to be a fixnum. Masking with the 58-bit window via
;; the generic bitwise-and always lands in fixnum range for the HAMT's fx slicing.
(define (key-hash k) (bitwise-and (jolt-hash k) hmask))
(define (chunk h shift) (fxand (fxsra h shift) 31))
(define (bitpos h shift) (fxsll 1 (chunk h shift)))
(define (popcount n) (let loop ((n n) (c 0)) (if (fx=? n 0) c (loop (fxand n (fx- n 1)) (fx+ c 1)))))
(define (arr-index bm bit) (popcount (fxand bm (fx- bit 1))))
;; jolt= alist ops (for hash-collision buckets)
(define (assoc-jolt k al) (cond ((null? al) #f) ((jolt= (caar al) k) (car al)) (else (assoc-jolt k (cdr al)))))
(define (alist-replace k v al) (if (jolt= (caar al) k) (cons (cons k v) (cdr al)) (cons (car al) (alist-replace k v (cdr al)))))
(define (alist-remove k al) (cond ((null? al) '()) ((jolt= (caar al) k) (cdr al)) (else (cons (car al) (alist-remove k (cdr al))))))
;; split two leaves that collided at `shift` into a subtree (or hcoll if the
;; full hashes are equal / the hash is exhausted).
(define (split-leaf shift ek ev h k v)
(let ((eh (key-hash ek)))
(if (or (fx>? shift max-shift) (fx=? eh h))
(make-hcoll h (list (cons ek ev) (cons k v)))
(let ((ei (chunk eh shift)) (ni (chunk h shift)))
(if (fx=? ei ni)
(make-hnode (fxsll 1 ei) (vector (split-leaf (fx+ shift 5) ek ev h k v)))
(let ((eb (fxsll 1 ei)) (nb (fxsll 1 ni)))
(if (fx<? ei ni)
(make-hnode (fxior eb nb) (vector (cons ek ev) (cons k v)))
(make-hnode (fxior eb nb) (vector (cons k v) (cons ek ev))))))))))
(define (node-assoc node shift h k v added)
(let* ((bit (bitpos h shift)) (bm (hnode-bm node)) (arr (hnode-arr node)))
(if (fx=? 0 (fxand bm bit))
(begin (set-box! added #t)
(make-hnode (fxior bm bit) (vec-insert arr (arr-index bm bit) (cons k v))))
(let* ((i (arr-index bm bit)) (child (vector-ref arr i)))
(cond
((hnode? child) (make-hnode bm (vec-set arr i (node-assoc child (fx+ shift 5) h k v added))))
((hcoll? child)
(let ((al (hcoll-alist child)))
(if (assoc-jolt k al)
(make-hnode bm (vec-set arr i (make-hcoll (hcoll-hash child) (alist-replace k v al))))
(begin (set-box! added #t)
(make-hnode bm (vec-set arr i (make-hcoll (hcoll-hash child) (cons (cons k v) al))))))))
((jolt= (car child) k) (make-hnode bm (vec-set arr i (cons k v)))) ; replace
(else (set-box! added #t)
(make-hnode bm (vec-set arr i (split-leaf (fx+ shift 5) (car child) (cdr child) h k v)))))))))
(define (node-get node shift h k default)
(let* ((bit (bitpos h shift)) (bm (hnode-bm node)))
(if (fx=? 0 (fxand bm bit)) default
(let ((child (vector-ref (hnode-arr node) (arr-index bm bit))))
(cond ((hnode? child) (node-get child (fx+ shift 5) h k default))
((hcoll? child) (let ((p (assoc-jolt k (hcoll-alist child)))) (if p (cdr p) default)))
((jolt= (car child) k) (cdr child))
(else default))))))
(define (node-dissoc node shift h k removed)
(let* ((bit (bitpos h shift)) (bm (hnode-bm node)) (arr (hnode-arr node)))
(if (fx=? 0 (fxand bm bit)) node
(let* ((i (arr-index bm bit)) (child (vector-ref arr i)))
(cond
((hnode? child) (make-hnode bm (vec-set arr i (node-dissoc child (fx+ shift 5) h k removed))))
((hcoll? child)
(if (assoc-jolt k (hcoll-alist child))
(begin (set-box! removed #t)
(let ((nal (alist-remove k (hcoll-alist child))))
(cond ((null? nal) (make-hnode (fxand bm (fxnot bit)) (vec-remove arr i)))
((null? (cdr nal)) (make-hnode bm (vec-set arr i (car nal)))) ; collapse to leaf
(else (make-hnode bm (vec-set arr i (make-hcoll (hcoll-hash child) nal)))))))
node))
((jolt= (car child) k)
(set-box! removed #t) (make-hnode (fxand bm (fxnot bit)) (vec-remove arr i)))
(else node))))))
(define (node-fold node proc acc) ; (proc k v acc) over every leaf
(let ((arr (hnode-arr node)))
(let loop ((i 0) (acc acc))
(if (fx<? i (vector-length arr))
(let ((child (vector-ref arr i)))
(loop (fx+ i 1)
(cond ((hnode? child) (node-fold child proc acc))
((hcoll? child)
(let cl ((al (hcoll-alist child)) (a acc))
(if (null? al) a (cl (cdr al) (proc (caar al) (cdar al) a)))))
(else (proc (car child) (cdr child) acc)))))
acc))))
;; ============================================================================
;; persistent map / set over the HAMT
;; ============================================================================
;; A small map keeps its keys in INSERTION order (Clojure's PersistentArrayMap),
;; converting to hash order past a threshold (PersistentHashMap). The HAMT root
;; always backs the values; `order` is the auxiliary insertion-order key list when
;; the map is in array mode, or #f once it has grown into hash mode. Equality and
;; hashing fold over the entries order-independently, so this only affects
;; iteration order (seq/keys/vals/print), matching the JVM.
(define-record-type pmap (fields root cnt order) (nongenerative chez-pmap-v2))
(define empty-pmap (make-pmap empty-hnode 0 '())) ; {} = empty array map
(define empty-pmap-hash (make-pmap empty-hnode 0 #f)) ; hash-order backing (sets)
(define pmap-absent (list 'absent)) ; unique missing-key sentinel
;; PersistentArrayMap threshold: assoc of a new key promotes to hash mode once the
;; map already holds 8 entries (array.length >= 16 in the reference). Clojure 1.13
;; raised the limit to 64 for maps whose keys are ALL keywords (the common
;; keyword-map case); mixed-key maps still cap at 8.
(define array-map-limit 8)
(define array-map-limit-kw 64)
(define (all-keywords? ks)
(or (null? ks) (and (keyword? (car ks)) (all-keywords? (cdr ks)))))
;; Should a map of `cnt` entries with insertion order `ord` stay in array mode
;; when key `k` is added? Under 8 always; a keyword-only map (existing keys + the
;; new key all keywords) grows to 64; otherwise it caps at 8.
(define (pmap-array-keep? cnt ord k)
(cond ((fx<? cnt array-map-limit) #t)
((fx>=? cnt array-map-limit-kw) #f)
((and (keyword? k) (all-keywords? ord)) #t)
(else #f)))
(define (append-key ord k) (append ord (list k)))
(define (remove-key ord k) (let loop ((o ord)) (cond ((null? o) '()) ((jolt= (car o) k) (cdr o)) (else (cons (car o) (loop (cdr o)))))))
;; growth rule (PersistentArrayMap.assoc): a new key appends to the order while in
;; array mode under the limit; otherwise the result is hash-ordered. Replacing an
;; existing key (or assoc onto an already-hash map) keeps the current order.
(define (pmap-assoc m k v)
(let* ((added (box #f)) (r (node-assoc (pmap-root m) 0 (key-hash k) k v added))
(cnt (pmap-cnt m)) (ord (pmap-order m)))
(if (unbox added)
(if (and ord (pmap-array-keep? cnt ord k))
(make-pmap r (fx+ cnt 1) (append-key ord k))
(make-pmap r (fx+ cnt 1) #f))
(make-pmap r cnt ord))))
;; force-ordered / force-hash inserts for rebuilding a map whose final mode is
;; already decided (array-map ctor, transient persistent!).
(define (pmap-put-ordered m k v)
(let* ((added (box #f)) (r (node-assoc (pmap-root m) 0 (key-hash k) k v added)))
(if (unbox added)
(make-pmap r (fx+ (pmap-cnt m) 1) (append-key (or (pmap-order m) '()) k))
(make-pmap r (pmap-cnt m) (pmap-order m)))))
(define (pmap-put-hash m k v)
(let* ((added (box #f)) (r (node-assoc (pmap-root m) 0 (key-hash k) k v added)))
(make-pmap r (if (unbox added) (fx+ (pmap-cnt m) 1) (pmap-cnt m)) #f)))
(define (pmap->hash m) (if (pmap-order m) (make-pmap (pmap-root m) (pmap-cnt m) #f) m))
(define (pmap-dissoc m k)
(let* ((removed (box #f)) (r (node-dissoc (pmap-root m) 0 (key-hash k) k removed))
(ord (pmap-order m)))
(if (unbox removed)
(make-pmap r (fx- (pmap-cnt m) 1) (if ord (remove-key ord k) #f))
m)))
(define (pmap-get m k default) (node-get (pmap-root m) 0 (key-hash k) k default))
(define (pmap-contains? m k) (not (eq? pmap-absent (node-get (pmap-root m) 0 (key-hash k) k pmap-absent))))
;; The universal fold idiom across the runtime is `(pmap-fold m (lambda (k v a)
;; (cons ... a)) '())`, which accumulates in REVERSE visitation order. So that this
;; reconstructs the map's INSERTION order, pmap-fold visits an array-mode map's keys
;; in reverse insertion order; a hash-mode map visits HAMT order (its iteration
;; order is unspecified, so reverse-of-HAMT is equivalent and matches prior
;; behaviour). Use pmap-fold-fwd when building a value directly in iteration order.
(define (pmap-fold m proc acc)
(let ((ord (pmap-order m)))
(if ord
(fold-right (lambda (k a) (proc k (pmap-get m k jolt-nil) a)) acc ord) ; visits last->first
(node-fold (pmap-root m) proc acc))))
;; visit entries in iteration (insertion) order — for code that builds a new map /
;; ordered value directly rather than via cons-accumulation.
(define (pmap-fold-fwd m proc acc)
(let ((ord (pmap-order m)))
(if ord
(let loop ((ks ord) (a acc))
(if (null? ks) a (loop (cdr ks) (proc (car ks) (pmap-get m (car ks) jolt-nil) a))))
(node-fold (pmap-root m) proc acc))))
;; map LITERAL ({...}): array map up to 8 entries (64 if keyword-only, per 1.13),
;; hash map beyond (RT.map).
(define (jolt-hash-map . kvs)
(let loop ((m empty-pmap) (kvs kvs))
(cond ((null? kvs)
(let ((cnt (pmap-cnt m)) (ord (pmap-order m)))
(if (fx>? cnt (if (all-keywords? ord) array-map-limit-kw array-map-limit))
(pmap->hash m) m)))
((null? (cdr kvs)) (error 'hash-map "odd number of map literal entries"))
(else (loop (pmap-put-ordered m (car kvs) (cadr kvs)) (cddr kvs))))))
;; array-map ctor: insertion-ordered regardless of size (createAsIfByAssoc).
(define (jolt-array-map-build kvs)
(let loop ((m empty-pmap) (kvs kvs))
(cond ((null? kvs) m)
((null? (cdr kvs)) (error 'array-map "odd number of map entries"))
(else (loop (pmap-put-ordered m (car kvs) (cadr kvs)) (cddr kvs))))))
;; hash-map ctor: hash order (PersistentHashMap).
(define (jolt-hash-map-build kvs)
(let loop ((m empty-pmap-hash) (kvs kvs))
(cond ((null? kvs) m)
((null? (cdr kvs)) (error 'hash-map "odd number of map entries"))
(else (loop (pmap-put-hash m (car kvs) (cadr kvs)) (cddr kvs))))))
(define-record-type pset (fields m) (nongenerative chez-pset-v1))
(define empty-pset (make-pset empty-pmap-hash)) ; sets are hash-ordered
(define (pset-conj s e) (if (pmap-contains? (pset-m s) e) s (make-pset (pmap-assoc (pset-m s) e e))))
(define (pset-disj s e) (make-pset (pmap-dissoc (pset-m s) e)))
(define (pset-contains? s e) (pmap-contains? (pset-m s) e))
(define (pset-count s) (pmap-cnt (pset-m s)))
(define (pset-fold s proc acc) (pmap-fold (pset-m s) (lambda (k v a) (proc k a)) acc))
(define (jolt-hash-set . xs) (let loop ((s empty-pset) (xs xs)) (if (null? xs) s (loop (pset-conj s (car xs)) (cdr xs)))))
;; ============================================================================
;; leaf ops the emitter lowers core/clojure fns to (mirrors native-ops)
;; ============================================================================
(define (jolt-conj1 coll x)
(cond ((pvec? coll) (pvec-conj coll x)) ; nil is a valid vector/set element
((pset? coll) (pset-conj coll x))
;; a list/seq conjs by PREPENDING (seq.ss: cseq / empty-list). conj onto a
;; list stays a list, conj onto a lazy/realized seq yields a seq cell (a
;; Cons) — list?-preserving.
((cseq? coll) (if (cseq-list? coll) (cseq-list x coll) (cseq-realized x coll)))
((empty-list-t? coll) (cseq-list x jolt-nil))
((pmap? coll)
(cond ((jolt-nil? x) coll) ; (conj m nil) = m
((pmap? x) (pmap-fold-fwd x (lambda (k v m) (pmap-assoc m k v)) coll)) ; merge in x's order
((and (pvec? x) (fx=? 2 (pvec-count x)))
(pmap-assoc coll (pvec-nth-d x 0 jolt-nil) (pvec-nth-d x 1 jolt-nil)))
(else (error 'conj "conj on a map expects a [k v] pair or a map"))))
((rec-coll-method coll "cons") => (lambda (m) (jolt-invoke m coll x)))
(else (error 'conj "unsupported collection"))))
;; (conj) -> []; (conj nil a b ...) builds a list (conj prepending -> (b a)).
(define (jolt-conj . args)
(if (null? args)
(jolt-vector)
(let ((coll (car args)) (xs (cdr args)))
(cond
;; 1-arity returns the coll untouched — (conj nil) is nil
((null? xs) coll)
((jolt-nil? coll) (fold-left jolt-conj1 jolt-empty-list xs))
(else (meta-carry coll (fold-left jolt-conj1 coll xs)))))))
;; A host shim registers a type's get via register-get-arm! (handler: (coll k d) ->
;; value) instead of set!-wrapping jolt-get — disjoint coll types, checked before the
;; base map/set/vec/string cases (cf. register-hash-arm!).
(define jolt-get-arms '())
(define (register-get-arm! pred handler)
(set! jolt-get-arms (cons (cons pred handler) jolt-get-arms)))
(define (jolt-get-base coll k d)
(cond ((pmap? coll) (pmap-get coll k d))
((pset? coll) (if (pset-contains? coll k) k d))
((pvec? coll) (pvec-nth-d coll k d))
((string? coll) (let ((i (->idx k)))
(if (and (fixnum? i) (fx>=? i 0) (fx<? i (string-length coll))) (string-ref coll i) d)))
(else d)))
;; jrec? / jrec-ref live in records.ss (loaded later); these are forward references
;; resolved at call time. A record field read is the hottest get, so check it first
;; and skip the get-arm walk.
(define (jolt-get-dispatch coll k d)
(if (jrec? coll)
(jrec-ref coll k d)
(let loop ((as jolt-get-arms))
(cond ((null? as) (jolt-get-base coll k d))
(((caar as) coll) ((cdar as) coll k d))
(else (loop (cdr as)))))))
(define jolt-get
(case-lambda
((coll k) (jolt-get-dispatch coll k jolt-nil))
((coll k d) (jolt-get-dispatch coll k d))))
;; A deftype implementing a clojure.lang collection interface (Indexed/Counted/
;; Associative/ILookup/ISeq/IPersistentCollection) carries the interface method
;; as an inline impl; the core collection fns fall back to it. find-method-any-
;; protocol / jolt-invoke load later — resolved at call time.
(define (rec-coll-method coll name)
(and (jrec? coll) (find-method-any-protocol (jrec-tag coll) name)))
(define (jolt-nth-nil-idx! i)
(when (jolt-nil? i)
(jolt-throw (jolt-host-throwable "java.lang.NullPointerException" "nth index"))))
(define jolt-nth
(case-lambda
((coll i)
(jolt-nth-nil-idx! i)
(let ((i (->idx i)))
(cond ((jolt-nil? coll) jolt-nil) ; RT.nth(nil, i) is nil at any index
((pvec? coll) (let ((v (pvec-v coll)))
(if (and (fx>=? i 0) (fx<? i (vector-length v))) (vector-ref v i)
(jolt-throw (jolt-host-throwable "java.lang.IndexOutOfBoundsException" "index out of bounds")))))
((string? coll) (if (and (fx>=? i 0) (fx<? i (string-length coll))) (string-ref coll i)
(jolt-throw (jolt-host-throwable "java.lang.IndexOutOfBoundsException" "index out of bounds"))))
((or (cseq? coll) (empty-list-t? coll)) (seq-nth coll i #f jolt-nil))
((rec-coll-method coll "nth") => (lambda (m) (jolt-invoke m coll i)))
(else (error 'nth "unsupported collection")))))
((coll i d)
(jolt-nth-nil-idx! i)
(let ((i (->idx i)))
(cond ((jolt-nil? coll) d) ; RT.nth(nil, i, notFound) is notFound
((pvec? coll) (pvec-nth-d coll i d))
((string? coll) (if (and (fx>=? i 0) (fx<? i (string-length coll))) (string-ref coll i) d))
((or (cseq? coll) (empty-list-t? coll)) (seq-nth coll i #t d))
((rec-coll-method coll "nth") => (lambda (m) (jolt-invoke m coll i d)))
(else d))))))
;; a count is an exact integer (JVM parity: count returns a long). jolt= is
;; exactness-aware, so this must be exact to match an exact integer literal:
;; (= 2 (count m)) -> 2 vs exact 2 -> true.
(define (jolt-count coll)
(begin
(cond ((pvec? coll) (pvec-count coll))
((pmap? coll) (pmap-cnt coll))
((pset? coll) (pset-count coll))
((string? coll) (string-length coll))
((jolt-nil? coll) 0)
((empty-list-t? coll) 0)
((cseq? coll) (let loop ((s coll) (n 0)) ; walk (forces a finite seq)
(if (jolt-nil? s) n (loop (jolt-seq (seq-more s)) (fx+ n 1)))))
((rec-coll-method coll "count") => (lambda (m) (jolt-invoke m coll)))
(else (error 'count "uncountable")))))
(define (jolt-assoc1 coll k v)
(cond ((pmap? coll) (pmap-assoc coll k v))
((pvec? coll) (pvec-assoc coll k v))
((jolt-nil? coll) (pmap-assoc empty-pmap k v))
((rec-coll-method coll "assoc") => (lambda (m) (jolt-invoke m coll k v)))
(else (error 'assoc "unsupported collection"))))
(define (jolt-assoc coll . kvs)
(meta-carry coll
(let loop ((coll coll) (kvs kvs))
(cond ((null? kvs) coll)
((null? (cdr kvs)) (error 'assoc "assoc expects an even number of key/vals"))
(else (loop (jolt-assoc1 coll (car kvs) (cadr kvs)) (cddr kvs)))))))
(define (jolt-dissoc coll . ks)
(cond ((jolt-nil? coll) jolt-nil)
((pmap? coll) (meta-carry coll (fold-left pmap-dissoc coll ks)))
(else (error 'dissoc "unsupported collection"))))
(define (jolt-contains? coll k)
(cond ((pmap? coll) (pmap-contains? coll k))
((pset? coll) (pset-contains? coll k))
((pvec? coll) (let ((k (->idx k))) (and (fixnum? k) (fx>=? k 0) (fx<? k (pvec-count coll)))))
((jolt-nil? coll) #f)
;; a string supports contains? by INDEX only (RT.contains: CharSequence +
;; Number key); any other key — or any unsupported type — is the JVM's
;; IllegalArgumentException.
((string? coll)
(if (and (number? k) (exact? k) (integer? k))
(and (>= k 0) (< k (string-length coll)))
(jolt-throw (jolt-host-throwable
"java.lang.IllegalArgumentException"
"contains? not supported on type: java.lang.String"))))
((or (cseq? coll) (empty-list-t? coll) (number? coll) (boolean? coll)
(keyword? coll) (jolt-symbol? coll) (char? coll))
(jolt-throw (jolt-host-throwable
"java.lang.IllegalArgumentException"
(string-append "contains? not supported on type: "
(guard (e (#t "?")) (jolt-class-name coll))))))
(else #f)))
(define (jolt-empty? coll)
(cond ((jolt-nil? coll) #t)
((pvec? coll) (fx=? 0 (pvec-count coll)))
((pmap? coll) (fx=? 0 (pmap-cnt coll)))
((pset? coll) (fx=? 0 (pset-count coll)))
((string? coll) (fx=? 0 (string-length coll)))
((empty-list-t? coll) #t)
((cseq? coll) #f) ; a cseq is non-empty by construction
(else (error 'empty? "unsupported collection"))))
(define (jolt-stack-throw coll)
(jolt-throw (jolt-host-throwable
"java.lang.ClassCastException"
(string-append "class " (guard (e (#t "?")) (jolt-class-name coll))
" cannot be cast to class clojure.lang.IPersistentStack"))))
(define (jolt-peek coll)
(cond ((pvec? coll) (pvec-peek coll))
;; list peek = first; a non-list seq (range, a rest chain) is not an
;; IPersistentStack on the JVM
((and (cseq? coll) (cseq-list? coll)) (jolt-first coll))
((empty-list-t? coll) (jolt-first coll))
((jolt-nil? coll) jolt-nil)
(else (jolt-stack-throw coll))))
(define (jolt-pop coll)
(cond ((jolt-nil? coll) jolt-nil) ; RT.pop(nil) is nil
((pvec? coll) (meta-carry coll (pvec-pop coll)))
((and (cseq? coll) (cseq-list? coll)) (meta-carry coll (jolt-rest coll)))
((empty-list-t? coll) (error 'pop "can't pop empty list"))
(else (jolt-stack-throw coll))))
;; ============================================================================
;; equality / hash hooks called from values.ss (jolt=2 / jolt-hash)
;; ============================================================================
(define (jolt-coll? x) (or (pvec? x) (pmap? x) (pset? x)))
(define (jolt-coll=? a b)
(cond
((and (pvec? a) (pvec? b))
(let ((va (pvec-v a)) (vb (pvec-v b)))
(and (fx=? (vector-length va) (vector-length vb))
(let loop ((i 0))
(or (fx=? i (vector-length va))
(and (jolt= (vector-ref va i) (vector-ref vb i)) (loop (fx+ i 1))))))))
((and (pmap? a) (pmap? b))
(and (fx=? (pmap-cnt a) (pmap-cnt b))
(pmap-fold a (lambda (k v ok) (and ok (jolt= (pmap-get b k pmap-absent) v))) #t)))
((and (pset? a) (pset? b))
(and (fx=? (pset-count a) (pset-count b))
(pset-fold a (lambda (e ok) (and ok (pset-contains? b e))) #t)))
(else #f)))
(define (jolt-coll-hash x)
(cond
((pvec? x)
(let ((v (pvec-v x)))
(let loop ((i 0) (h 1))
(if (fx=? i (vector-length v)) (bitwise-and h hmask)
(loop (fx+ i 1) (bitwise-and (+ (* 31 h) (key-hash (vector-ref v i))) hmask))))))
;; maps/sets hash order-independently (sum), consistent with unordered =
((pmap? x) (bitwise-and (pmap-fold x (lambda (k v a) (+ a (fxxor (key-hash k) (key-hash v)))) 0) hmask))
((pset? x) (bitwise-and (pset-fold x (lambda (e a) (+ a (key-hash e))) 0) hmask))))

297
host/chez/compile-eval.ss Normal file
View file

@ -0,0 +1,297 @@
;; compile-eval.ss — the compile spine.
;;
;; Ties together the cross-compiled compiler image (jolt.ir + jolt.analyzer +
;; jolt.backend-scheme, loaded as def-var! forms) and the host contract
;; (host-contract.ss) into a runtime entry: a Clojure source string is read by the
;; Chez data reader, analyzed by the analyzer to IR, emitted to Scheme by the
;; emitter, and eval'd. This is the spine the stage2==stage3 bootstrap fixpoint
;; closes over.
;;
;; Loaded after host-contract.ss + the compiler image.
(define jolt-ce-analyze (var-deref "jolt.analyzer" "analyze"))
(define jolt-ce-emit (var-deref "jolt.backend-scheme" "emit"))
;; jolt.passes/run-passes: const-fold every analyzed form, plus inline + type
;; inference when the unit opted into direct-linking (jolt build --opt). Off that
;; path it is a pure const-fold. Loaded from the compiler image (jolt.passes).
(define jolt-ce-run-passes (var-deref "jolt.passes" "run-passes"))
;; The compiler reads source as FORMS (set literals stay {:jolt/type :jolt/set},
;; which the analyzer lowers) — the raw reader, not clojure.core/read-string,
;; whose data conversion would turn those into real sets.
(define jolt-ce-read jolt-read-form-raw)
;; --- current source location ------------------------------------------------
;; The position of the top-level form currently compiling/evaluating, so an
;; uncaught error can report where it came from (cli.ss jolt-report-uncaught).
;; Thread-local: a future/agent worker tracks its own form. Holds #f or a
;; {:line :column :file?} position map (jolt.host/form-position's shape).
;; Top-level granularity — one set per top-level form, nothing per call.
(define jolt-current-source (make-thread-parameter #f))
;; clojure.lang.Compiler/LINE and /COLUMN — derefable cells (Vars on the JVM)
;; holding the line/column of the form being compiled. Macros read @Compiler/LINE
;; as a fallback when &form carries no position (jolt's reader stamps :line on list
;; forms, so this is rarely hit). Updated per top-level form, like *current-source*.
(define compiler-line-cell (jolt-atom-new 0))
(define compiler-column-cell (jolt-atom-new 0))
;; clojure.lang.Compiler/specials — the JVM's special-form table (sym -> parser).
;; tools.macro reads (keys Compiler/specials) to know which heads NOT to expand.
;; Only the keys matter here; values are #t. The set matches Clojure 1.2/1.3.
(define compiler-specials
(let ((unq '("def" "loop*" "recur" "if" "case*" "let*" "letfn*" "do" "fn*"
"quote" "var" "." "set!" "try" "monitor-enter" "monitor-exit"
"throw" "new" "&" "catch" "finally" "reify*" "deftype*")))
(fold-left (lambda (m s) (jolt-assoc1 m (jolt-symbol #f s) #t))
(jolt-assoc1 (jolt-hash-map) (jolt-symbol "clojure.core" "import*") #t)
unq)))
;; clojure.lang.Compiler/demunge — reverse the name munging Clojure applies to
;; build JVM class/method names, so "clojure.core$odd_QMARK_" -> clojure.core/odd?.
;; clojure.spec.alpha's fn-sym uses it to recover a symbol from a fn's class name.
;; Longest tokens first; a standalone _ is a hyphen; $ separates ns from name.
(define demunge-token-map
'(("_DOUBLEQUOTE_" . "\"") ("_SINGLEQUOTE_" . "'") ("_AMPERSAND_" . "&") ("_PERCENT_" . "%")
("_LBRACE_" . "{") ("_RBRACE_" . "}") ("_LBRACK_" . "[") ("_RBRACK_" . "]")
("_BSLASH_" . "\\") ("_TILDE_" . "~") ("_CIRCA_" . "@") ("_SHARP_" . "#") ("_BANG_" . "!")
("_CARET_" . "^") ("_COLON_" . ":") ("_QMARK_" . "?") ("_SLASH_" . "/") ("_PLUS_" . "+")
("_STAR_" . "*") ("_BAR_" . "|") ("_GT_" . ">") ("_LT_" . "<") ("_EQ_" . "=") ("_DOT_" . ".")))
(define (compiler-demunge s)
(let* ((s (if (string? s) s (jolt-str-render-one s)))
(n (string-length s))
(out (open-output-string)))
(let loop ((i 0))
(if (>= i n) (get-output-string out)
(let ((tok (let scan ((ts demunge-token-map))
(cond ((null? ts) #f)
((let ((t (caar ts)))
(and (<= (+ i (string-length t)) n)
(string=? (substring s i (+ i (string-length t))) t)))
(car ts))
(else (scan (cdr ts)))))))
(cond
(tok (display (cdr tok) out) (loop (+ i (string-length (car tok)))))
((char=? (string-ref s i) #\_) (write-char #\- out) (loop (+ i 1)))
((char=? (string-ref s i) #\$) (write-char #\/ out) (loop (+ i 1)))
(else (write-char (string-ref s i) out) (loop (+ i 1)))))))))
(let ((members (list (cons "LINE" compiler-line-cell) (cons "COLUMN" compiler-column-cell)
(cons "specials" compiler-specials)
(cons "demunge" compiler-demunge))))
(register-class-statics! "Compiler" members)
(register-class-statics! "clojure.lang.Compiler" members))
(define (jolt-enter-form! form)
(let ((p (hc-form-position form)))
(when (pmap? p)
(jolt-current-source p)
(let ((line (jolt-get p hc-kw-line jolt-nil)) (col (jolt-get p hc-kw-column jolt-nil)))
(jolt-atom-val-set! compiler-line-cell (if (jolt-nil? line) 0 line))
(jolt-atom-val-set! compiler-column-cell (if (jolt-nil? col) 0 col))))))
;; "file:line:col" / "line:col" for the current form, or #f when none is set.
(define (jolt-current-source-string)
(let ((p (jolt-current-source)))
(and (pmap? p)
(let ((line (jolt-get p hc-kw-line jolt-nil))
(col (jolt-get p hc-kw-column jolt-nil))
(file (jolt-get p hc-kw-file jolt-nil)))
(string-append
(if (jolt-nil? file) "" (string-append file ":"))
(if (jolt-nil? line) "?" (number->string line)) ":"
(if (jolt-nil? col) "?" (number->string col)))))))
;; The spine ALWAYS runs with the full clojure.core prelude loaded, so a clojure.*
;; ref must lower to var-deref (resolved from the prelude), not trip the emitter's
;; "unsupported stdlib fn (no core on Chez yet)" out-of-subset guard — that guard
;; is only for the bare -e subset with no prelude. Turn prelude mode on once, here,
;; so every analyze->emit on this spine sees the full core.
((var-deref "jolt.backend-scheme" "set-prelude-mode!") #t)
;; Cache resolved var cells per reference site in runtime-compiled code (the big
;; win for libraries / REPL code). emit-image.ss turns this back off so the seed
;; mint and AOT build stay byte-deterministic. Guarded: the flag is absent in an
;; older seed during the first re-mint pass.
(let ((scv (var-deref "jolt.backend-scheme" "set-var-cache!")))
(when (procedure? scv) (scv #t)))
;; JOLT_TRACE is a falsey value (case-insensitive) — the single predicate both the
;; dev-mode enable and the whole-run enable consult, so "off" never accidentally
;; means "on". An empty / unset value is NOT falsey here — it carries no signal, so
;; dev mode still traces and a whole run still doesn't.
(define (jolt-trace-env-off? e)
(and (string? e)
(let ((s (string-downcase e)))
(or (string=? s "0") (string=? s "false") (string=? s "no")
(string=? s "off") (string=? s "n")))))
;; Tail-frame history. Turning it on makes the emitter add a per-fn history push to
;; every fn compiled AFTERWARD, and allocates this thread's ring. Suppressed when
;; JOLT_TRACE is a falsey value, so JOLT_TRACE=0 / off / no disables it in dev mode.
(define (jolt-enable-trace!)
(unless (jolt-trace-env-off? (getenv "JOLT_TRACE"))
(let ((stf (var-deref "jolt.backend-scheme" "set-trace-frames!")))
(when (procedure? stf) (stf #t)))
(jolt-trace-enable!)))
;; Exposed so the REPL / nREPL entrypoints (jolt.main, jolt.nrepl) can turn tracing
;; on for REPL-driven development without the user setting JOLT_TRACE. Because the
;; push is baked in at compile time, only code compiled after this call is traced —
;; which is exactly the code you eval / reload in a live session.
(def-var! "jolt.host" "enable-trace!" jolt-enable-trace!)
;; Explicit opt-in for a whole run (JOLT_TRACE=1): turn tracing on BEFORE any app
;; namespace is compiled, so a plain `-M:run` traces the app's own code too. Called
;; from the runtime entrypoints (cli.ss, and the built joltc launcher) — NOT at load
;; time: a built joltc runs top-level forms at heap-build time, where JOLT_TRACE is
;; always unset, so a load-time check would never see the user's runtime env. Only an
;; affirmative value (set, non-empty, not falsey) forces it on.
(define (jolt-trace-init-from-env!)
(let ((e (getenv "JOLT_TRACE")))
(when (and e (fx>? (string-length e) 0) (not (jolt-trace-env-off? e)))
(jolt-enable-trace!))))
;; (with-meta sym m) -> sym, else x — an (ns ^:no-doc name …) yields the name with
;; reader metadata as a with-meta form; strip it to read the bare ns symbol.
(define (ce-unwrap-meta x)
(if (and (cseq? x) (cseq-list? x))
(let ((items (seq->list x)))
(if (and (pair? items) (symbol-t? (car items))
(string=? (symbol-t-name (car items)) "with-meta") (pair? (cdr items)))
(cadr items) x))
x))
;; (quote X) -> X, else x — unwraps a quoted require spec.
(define (ce-unquote x)
(if (and (cseq? x) (cseq-list? x))
(let ((items (seq->list x)))
(if (and (pair? items) (symbol-t? (car items))
(string=? (symbol-t-name (car items)) "quote") (pair? (cdr items)))
(cadr items) x))
x))
;; Pre-register any (require ...)/(use ...) :as aliases under `ns` BEFORE analysis,
;; so a qualified s/foo resolves while compiling (analysis precedes the runtime
;; require). Walks the whole form (a require may be nested in a do/let).
(define (ce-clause-require? cl) ; (:require ...) / (:use ...) ns clause
(and (pair? cl) (keyword? (car cl))
(let ((kn (keyword-t-name (car cl)))) (or (string=? kn "require") (string=? kn "use")))))
(define (ce-scan-requires! form ns)
(when (and (cseq? form) (cseq-list? form))
(let ((items (seq->list form)))
(when (pair? items)
(let* ((h (car items)) (hn (and (symbol-t? h) (symbol-t-name h))))
(cond
;; (require spec...) / (use spec...) — specs are quoted
((and hn (or (string=? hn "require") (string=? hn "use")))
(for-each (lambda (a) (chez-register-spec! ns (ce-unquote a))) (cdr items)))
;; (ns name (:require [a :as x]) ...) — clause specs are literal. Register
;; the aliases under NAME (the ns being defined), not the passed `ns`:
;; when a file is loaded its ns form compiles while (chez-current-ns) is
;; still the requiring ns, so using `ns` would leak the loaded ns's
;; aliases into its requirer and clobber a same-named alias there
;; (rewrite-clj.zip.base's [node.protocols :as node] over the caller's node).
((and hn (string=? hn "ns"))
(let ((ns-name (if (and (pair? (cdr items)) (symbol-t? (ce-unwrap-meta (cadr items))))
(symbol-t-name (ce-unwrap-meta (cadr items)))
ns)))
(for-each (lambda (clause)
(when (and (cseq? clause) (cseq-list? clause))
(let ((cl (seq->list clause)))
(when (ce-clause-require? cl)
(for-each (lambda (spec) (chez-register-spec! ns-name spec)) (cdr cl))))))
(if (pair? (cdr items)) (cddr items) '()))))
(else (for-each (lambda (x) (ce-scan-requires! x ns)) items))))))))
;; Already-read FORM -> Scheme source string (analyze -> emit on Chez).
;; `ns` is the compile namespace unqualified symbols resolve against.
(define (jolt-analyze-emit-form form ns)
(ce-scan-requires! form ns)
(let* ((ctx (make-analyze-ctx ns))
(ir (jolt-ce-run-passes (jolt-ce-analyze ctx form) ctx)))
(jolt-ce-emit ir)))
;; --- runtime defmacro -------------------------------------------------------
;; Shared with emit-image.ss (loaded after this). A defmacro lowers to a def of
;; its expander fn + a macro flag, exactly as the prelude emits build-time macros.
;; Is `f` a (defmacro ...) / (definline ...) form?
(define (ce-macro-form? f)
(and (cseq? f) (cseq-list? f)
(let ((items (seq->list f)))
(and (pair? items) (symbol-t? (car items))
(let ((h (symbol-t-name (car items))))
(or (string=? h "defmacro") (string=? h "definline")))))))
;; (defmacro NAME [docstring] [attr-map] params body...) -> (values "NAME" (fn ...)).
;; Strips a leading docstring (native string) + attr-map (a non-symbol pmap), then
;; re-heads the rest with `fn` so a destructured macro arglist desugars. Emits the
;; BARE fn (the caller wraps it in def-var! + mark-macro!), never a (def NAME ...) —
;; interning NAME would make require skip the real macro.
(define (ce-defmacro->fn f)
(let* ((items (seq->list f))
(name-sym (cadr items))
(after-name (cddr items))
(a1 (if (and (pair? after-name) (string? (car after-name)))
(cdr after-name) after-name))
(after-meta (if (and (pair? a1) (pmap? (car a1)))
(cdr a1) a1))
(fn-sym (jolt-symbol #f "fn")))
(values (symbol-t-name name-sym)
(apply jolt-list (cons fn-sym after-meta)))))
;; A bare top-level (do ...) form — head is the unqualified `do` symbol.
(define (ce-top-do? form)
(and (cseq? form) (cseq-list? form)
(let ((h (seq-first form)))
(and (symbol-t? h) (jolt-nil? (hc-sym-ns h))
(string=? (symbol-t-name h) "do")))))
;; Compile + eval ONE already-read form in compile ns `ns`; returns the value.
;; A top-level (do ...) is UNROLLED — each subform compiled+eval'd in turn, like
;; Clojure's top-level do — so a runtime defmacro/def in an earlier subform is
;; visible (macro flag set, var interned) before a later subform is analyzed.
;; a non-form VALUE (a function object, a BigDecimal, a reference type)
;; self-evaluates, like eval on the JVM.
(define (jolt-compile-eval-form form ns)
(if (or (procedure? form) (jbigdec? form) (jolt-atom? form) (jolt-multifn? form))
form
(jolt-compile-eval-form* form ns)))
(define (jolt-compile-eval-form* form ns)
(cond
;; thread the current ns: an earlier subform may switch it (ns/in-ns call
;; set-chez-ns!), and the next subform must be ANALYZED in that ns so its defs
;; land there and its refs resolve (cross-ns def/require in one program).
((ce-top-do? form)
(let loop ((fs (cdr (seq->list form))) (result jolt-nil) (cur ns))
(if (null? fs)
result
(let ((r (jolt-compile-eval-form (car fs) cur)))
(loop (cdr fs) r (chez-current-ns))))))
;; defmacro is compiled like any other form — the analyzer lowers it to a def
;; of the expander fn + (mark-macro! …) so subsequent forms expand it. One
;; macro-expansion path (no separate spine interception).
(else
;; record this form's source location first, so a compile- or run-time error
;; in it reports the right place.
(jolt-enter-form! form)
;; drop tail-frame history from earlier top-level forms, so an error's trace
;; shows only this form's own call history (a no-op unless JOLT_TRACE is on).
(jolt-trace-reset!)
(eval (read (open-input-string (jolt-analyze-emit-form form ns)))
(interaction-environment)))))
;; Source string -> value (read one form, compile + eval on Chez, in the
;; top-level environment where rt.ss's runtime procedures live).
(define (jolt-compile-eval src ns)
(jolt-compile-eval-form (jolt-ce-read src) ns))
;; clojure.core/load-string: read every form from the source string and compile+
;; eval each in the current ns, returning the last value (nil for blank input).
(define (jolt-load-string s)
(let loop ((src s) (result jolt-nil))
(let ((pn (jolt-parse-next src)))
(if (jolt-nil? pn)
result
(loop (jolt-nth pn 1)
(jolt-compile-eval-form (jolt-nth pn 0) (chez-current-ns)))))))
;; eval / load-string are FUNCTIONS on the spine (the compiler image is resident
;; at runtime). eval takes an already-read FORM (e.g. from quote / list); it and
;; load-string compile+eval in the current ns. eval is removed from the analyzer's
;; special-symbol lists (host-contract.ss) so it resolves as an ordinary core var.
(def-var! "clojure.core" "eval"
(lambda (form) (jolt-compile-eval-form form (chez-current-ns))))
(def-var! "clojure.core" "load-string" jolt-load-string)

288
host/chez/converters.ss Normal file
View file

@ -0,0 +1,288 @@
;; converters + string ops — host-coupled natives def-var!'d into clojure.core,
;; resolved in prelude mode. Loaded last (after jolt-pr-str), since `str` reuses
;; the printer. int/long truncate toward zero to an exact integer; compare returns
;; an exact -1/0/1; double yields a flonum.
;; str rendering for the value types not handled by the fast arms below. A host
;; shim loaded later (records, host-table, inst-time, …) registers an arm with
;; register-str-render! instead of set!-wrapping jolt-str-render-one — the arms
;; are type-disjoint, so the full behavior is the base arms here plus the
;; registry, gathered in one place rather than scattered across a set! chain.
;; Newest registration is checked first (matches the old outermost-wins order).
(define str-render-registry '()) ; list of (pred . render), checked front-to-back
(define (register-str-render! pred render)
(set! str-render-registry (cons (cons pred render) str-render-registry)))
;; str: nil -> "", string raw, char bare (not \c), regex -> raw source, a
;; registered host type via its arm, else the printer (which renders collections
;; with readable elements).
(define (jolt-str-render-one v)
(cond
((jolt-nil? v) "")
((string? v) v)
((char? v) (string v))
((regex-t? v) (regex-t-source v))
;; str/print render the infinities and NaN long-form (Clojure .toString),
;; unlike the -e printer's inf/-inf/nan.
((and (flonum? v) (fl= v +inf.0)) "Infinity")
((and (flonum? v) (fl= v -inf.0)) "-Infinity")
((and (flonum? v) (not (fl= v v))) "NaN")
;; a symbol stringifies to its name (JVM Symbol.toString returns the interned
;; name), so (str sym) of a no-ns symbol is the SAME string object the symbol
;; holds — code that compares those by identity (core.logic's non-unique lvar
;; equality) depends on it.
((symbol-t? v)
(let ((ns (symbol-t-ns v)))
(if (or (not ns) (jolt-nil? ns))
(symbol-t-name v)
(string-append ns "/" (symbol-t-name v)))))
(else
(let loop ((rs str-render-registry))
(cond
((null? rs) (jolt-pr-str v))
(((caar rs) v) ((cdar rs) v))
(else (loop (cdr rs))))))))
;; print/println render non-readably: a nested string is raw. jolt-str-render-one
;; is exactly that (collections fall through to jolt-pr-str). The print family
;; uses this seam, NOT the str fn — which renders readably (below). A top-level nil
;; prints "nil" (str renders it ""), so the seam special-cases it.
(define (jolt-print-one v) (if (jolt-nil? v) "nil" (jolt-str-render-one v)))
(def-var! "clojure.core" "__print1" jolt-print-one)
;; str: a top-level string/scalar renders as jolt-str-render-one (raw string,
;; "Infinity"…), but a COLLECTION renders as its readable form — nested strings
;; are QUOTED ((str ["x"]) => "[\"x\"]"), matching the JVM (a collection's
;; toString is readable). jolt-pr-readable resolves at call time.
(define (jolt-str-one v)
(if (or (pvec? v) (pmap? v) (pset? v) (cseq? v) (empty-list-t? v) (jolt-lazyseq? v))
(jolt-pr-readable v)
(jolt-str-render-one v)))
(define (jolt-str . xs)
(cond
((null? xs) "")
;; single arg returns its rendering directly (no string-append copy), so
;; (str sym) hands back the symbol's own name string — JVM (str x) is
;; x.toString(), and core.logic's non-unique lvar equality compares those by
;; identity.
((null? (cdr xs)) (jolt-str-one (car xs)))
(else (let loop ((xs xs) (acc '()))
(if (null? xs)
(apply string-append (reverse acc))
(loop (cdr xs) (cons (jolt-str-one (car xs)) acc)))))))
;; jolt indices are flonums; substring etc. need exact ints.
(define (jolt->idx n) (exact (truncate n)))
(define (jolt-subs s start . end)
(substring s (jolt->idx start)
(if (null? end) (string-length s) (jolt->idx (car end)))))
;; vec: a pvec from any seqable (already-pvec returns itself).
(define (jolt-vec coll)
(cond
((jolt-nil? coll) (jolt-vector))
((pvec? coll) coll)
((string? coll) (apply jolt-vector (string->list coll)))
(else (apply jolt-vector (seq->list coll)))))
(define (jolt-keyword . args)
(cond
((= (length args) 1)
(let ((a (car args)))
(cond
((jolt-nil? a) jolt-nil)
((keyword? a) a)
;; a 1-arg string splits on the FIRST "/" into ns/name:
;; (keyword "x/y") => :x/y with ns "x" — destructure's {:keys [x/y]} builds
;; the key this way, so without the split the namespaced key never matches.
((string? a)
(let ((si (let loop ((i 0))
(cond ((>= i (string-length a)) #f)
((char=? (string-ref a i) #\/) i)
(else (loop (+ i 1)))))))
(if (and si (> si 0) (< si (- (string-length a) 1)))
(keyword (substring a 0 si) (substring a (+ si 1) (string-length a)))
(keyword #f a))))
((jolt-symbol? a)
(let ((ns (symbol-t-ns a)))
(keyword (if (or (jolt-nil? ns) (not ns) (eq? ns '())) #f ns) (symbol-t-name a))))
(else (error #f "keyword: requires string/symbol/keyword" a)))))
((= (length args) 2)
(keyword (let ((ns (car args))) (if (jolt-nil? ns) #f ns)) (cadr args)))
(else (error #f "keyword: wrong arity"))))
(define (jolt-symbol-new . args)
(cond
((= (length args) 1)
(let ((a (car args)))
(cond
((jolt-symbol? a) a)
;; (symbol "ns/name") splits the namespace at the FIRST "/" (JVM
;; Symbol.intern), so (namespace (symbol "foo/bar/baz")) => "foo" with
;; name "bar/baz". A lone "/" or a leading slash has no namespace. The
;; no-ns sentinel is #f — matches emit's quoted-symbol lowering
;; (jolt-symbol #f "x"), so (= 'x (symbol "x")) holds (jolt= compares
;; ns with strict equal?).
((string? a)
(let ((slen (string-length a)))
(if (string=? a "/")
(jolt-symbol #f "/")
(let loop ((i 1))
(cond ((>= i slen) (jolt-symbol #f a))
((char=? (string-ref a i) #\/)
(jolt-symbol (substring a 0 i) (substring a (+ i 1) slen)))
(else (loop (+ i 1))))))))
((keyword? a) (jolt-symbol (keyword-t-ns a) (keyword-t-name a)))
;; (symbol a-var) -> the var's qualified symbol (clojure.spec.alpha/->sym).
((var-cell? a) (jolt-symbol (var-cell-ns a) (var-cell-name a)))
(else (error #f "symbol: requires string/symbol" a)))))
;; (symbol ns name): a nil namespace is the no-ns sentinel #f (NOT jolt-nil),
;; so (symbol nil "x") equals (symbol "x") and the reader literal 'x — jolt=
;; compares ns with strict equal?, so a jolt-nil ns would differ from #f.
((= (length args) 2)
(let ((ns (car args)))
(jolt-symbol (if (jolt-nil? ns) #f ns) (cadr args))))
(else (error #f "symbol: wrong arity"))))
;; gensym: per-process counter.
(define jolt-gensym-counter 0)
(define (jolt-gensym . prefix)
(let ((p (if (null? prefix) "G__" (car prefix))))
(set! jolt-gensym-counter (+ jolt-gensym-counter 1))
(jolt-symbol #f
(string-append (if (string? p) p (jolt-str-render-one p))
(number->string jolt-gensym-counter)))))
;; int/long: truncate toward zero to an EXACT integer (= JVM long). char -> code
;; point (exact). double: always a flonum (= JVM double).
(define (jolt-int x) (if (char? x) (char->integer x) (exact (truncate x))))
;; a numeric type outside Chez's tower converts through this hook (bigdec).
(define (jolt-double-slow x) (jolt-num-cast-throw x))
(define (jolt-double x)
(cond ((char? x) (exact->inexact (char->integer x)))
((number? x) (exact->inexact x))
(else (jolt-double-slow x))))
;; compare: 3-way, returns an EXACT integer (= JVM compare -> int).
(define (jolt-cmp3 x y) (cond ((< x y) -1) ((> x y) 1) (else 0)))
(define (jolt-strcmp a b) (cond ((string<? a b) -1) ((string>? a b) 1) (else 0)))
(define (jolt-kw->string k)
(let ((ns (keyword-t-ns k))) (if ns (string-append ns "/" (keyword-t-name k)) (keyword-t-name k))))
(define (jolt-sym-ns-string s)
(let ((n (symbol-t-ns s))) (if (or (jolt-nil? n) (not n) (eq? n '())) "" n)))
;; compare returns an EXACT integer -1/0/1 (= JVM compare -> int).
(define (jolt-compare a b)
(cond
((and (jolt-nil? a) (jolt-nil? b)) 0)
((jolt-nil? a) -1)
((jolt-nil? b) 1)
((and (number? a) (number? b)) (jolt-cmp3 a b))
((and (string? a) (string? b)) (jolt-strcmp a b))
;; keywords order like symbols: a nil namespace sorts before any namespace,
;; then by namespace, then by name (Keyword.compareTo -> Symbol.compareTo)
((and (keyword? a) (keyword? b))
(let ((r (jolt-strcmp (or (keyword-t-ns a) "") (or (keyword-t-ns b) ""))))
(if (= r 0) (jolt-strcmp (keyword-t-name a) (keyword-t-name b)) r)))
((and (jolt-symbol? a) (jolt-symbol? b))
(let ((r (jolt-strcmp (jolt-sym-ns-string a) (jolt-sym-ns-string b))))
(if (= r 0) (jolt-strcmp (symbol-t-name a) (symbol-t-name b)) r)))
((and (boolean? a) (boolean? b)) (cond ((eq? a b) 0) ((eq? a #f) -1) (else 1)))
((and (char? a) (char? b)) (jolt-cmp3 (char->integer a) (char->integer b)))
((and (pvec? a) (pvec? b))
(let ((la (pvec-count a)) (lb (pvec-count b)))
(if (not (= la lb))
(jolt-cmp3 la lb)
(let loop ((i 0))
(if (>= i la)
0
(let ((r (jolt-compare (pvec-nth-d a i jolt-nil) (pvec-nth-d b i jolt-nil))))
(if (= r 0) (loop (+ i 1)) r)))))))
(else (error #f "compare: cannot compare these types" a b))))
(def-var! "clojure.core" "str" jolt-str)
(def-var! "clojure.core" "subs" jolt-subs)
(def-var! "clojure.core" "vec" jolt-vec)
(def-var! "clojure.core" "keyword" jolt-keyword)
(def-var! "clojure.core" "symbol" jolt-symbol-new)
(def-var! "clojure.core" "gensym" jolt-gensym)
;; --- checked narrow casts (RT.byteCast/shortCast/intCast/longCast/charCast) --
;; One helper carries the JVM ranges: truncate toward zero, then range-check.
;; NaN casts to 0 (Java (long)NaN); an out-of-range value (including a float
;; infinity) is IllegalArgumentException "Value out of range for <type>: x".
;; A non-numeric operand is the usual ClassCastException. Numeric types outside
;; Chez's tower truncate through a hook the shim extends (BigDecimal).
(define (jolt-cast-range-throw name x)
(jolt-throw (jolt-host-throwable
"java.lang.IllegalArgumentException"
(string-append "Value out of range for " name ": " (jolt-str x)))))
(define (jolt-cast-truncate-slow x) (jolt-num-cast-throw x))
(define (jolt-checked-cast name lo hi x)
(let ((n (cond ((char? x) (char->integer x))
((and (number? x) (exact? x)) (truncate x))
;; a double range-checks ITSELF (before truncation): (byte
;; 127.000001) throws, (byte 1.1) is 1; NaN casts to 0; an
;; infinity always fails the compare.
((flonum? x) (cond ((nan? x) 0)
((or (< x lo) (> x hi)) (+ hi 1))
(else (exact (truncate x)))))
(else (jolt-cast-truncate-slow x)))))
(if (and (>= n lo) (<= n hi)) n (jolt-cast-range-throw name x))))
(define (jolt-byte-cast x) (jolt-checked-cast "byte" -128 127 x))
(define (jolt-short-cast x) (jolt-checked-cast "short" -32768 32767 x))
(define (jolt-int-cast x) (jolt-checked-cast "int" -2147483648 2147483647 x))
(define (jolt-long-cast x) (jolt-checked-cast "long" -9223372036854775808 9223372036854775807 x))
(def-var! "clojure.core" "int" jolt-int-cast)
(def-var! "clojure.core" "long" jolt-long-cast)
(def-var! "clojure.core" "byte" jolt-byte-cast)
(def-var! "clojure.core" "short" jolt-short-cast)
;; char: pass a char through; a code point must be in [0, 0xFFFF] (charCast).
(define (jolt-char x)
(if (char? x) x (integer->char (jolt-checked-cast "char" 0 65535 x))))
(def-var! "clojure.core" "char" jolt-char)
;; unchecked-long: truncate + wrap to 64 bits (RT.uncheckedLongCast — a float
;; infinity saturates, NaN is 0). unchecked-int wraps and sign-folds to 32.
(define (jolt-cast-saturate n lo hi) (cond ((< n lo) lo) ((> n hi) hi) (else n)))
(define (jolt-unchecked-long x)
(cond ((char? x) (char->integer x))
;; an exact integer wraps (long narrowing); a double SATURATES (Java's
;; double->long conversion clamps at the bounds, NaN is 0).
((and (number? x) (exact? x)) (jolt-wrap64 (truncate x)))
((flonum? x) (if (nan? x) 0
(jolt-cast-saturate (if (infinite? x) (if (> x 0.0) unc-2^63 (- unc-2^63)) (exact (truncate x)))
-9223372036854775808 9223372036854775807)))
(else (jolt-wrap64 (jolt-cast-truncate-slow x)))))
(define (jolt-unchecked-int x)
(if (flonum? x)
;; double->int clamps like Java
(if (nan? x) 0
(jolt-cast-saturate (if (infinite? x) (if (> x 0.0) #x80000000 (- #x80000000)) (exact (truncate x)))
-2147483648 2147483647))
(let ((i (bitwise-and (jolt-unchecked-long x) #xffffffff)))
(if (>= i #x80000000) (- i #x100000000) i))))
(def-var! "clojure.core" "unchecked-long" jolt-unchecked-long)
(def-var! "clojure.core" "unchecked-int" jolt-unchecked-int)
(def-var! "clojure.core" "double" jolt-double)
;; float: Chez has no single-float type, so the value stays a flonum — but the
;; cast range-checks against Float/MAX_VALUE like RT.floatCast (an infinity is
;; out of range; NaN passes).
(define fl-float-max 3.4028234663852886e38)
(define (jolt-float x)
(let ((d (jolt-double x)))
(if (and (flonum? d) (not (nan? d))
(or (< d (- fl-float-max)) (> d fl-float-max)))
(jolt-cast-range-throw "float" x)
d)))
(def-var! "clojure.core" "float" jolt-float)
;; numerator/denominator: jolt ratios are Chez exact rationals; a non-ratio is
;; the JVM's Ratio cast failure.
(define (jolt-ratio-part name f)
(lambda (x)
(if (and (number? x) (exact? x) (rational? x) (not (integer? x)))
(f x)
(jolt-throw (jolt-host-throwable
"java.lang.ClassCastException"
(string-append "class " (guard (e (#t "?")) (jolt-class-name x))
" cannot be cast to class clojure.lang.Ratio"))))))
(def-var! "clojure.core" "numerator" (jolt-ratio-part "numerator" numerator))
(def-var! "clojure.core" "denominator" (jolt-ratio-part "denominator" denominator))
(def-var! "clojure.core" "compare" jolt-compare)

120
host/chez/cts.sh Executable file
View file

@ -0,0 +1,120 @@
#!/bin/bash
# clojure-test-suite gate: run the vendored jank-lang/clojure-test-suite
# (vendor/clojure-test-suite) against joltc, one process per test namespace (a
# hang or crash is contained), and compare per-namespace fail/error counts
# against the checked-in baseline test/chez/cts-known-failures.txt.
#
# The comparison is exact, like certify's allowlist: a namespace doing WORSE
# than the baseline fails the gate (regression), and one doing BETTER also
# fails (stale baseline — update the file in the same change that improved it).
#
# JOLT_CTS_JOBS=N parallel workers (default 4)
# JOLT_CTS_TIMEOUT=SECS per-namespace timeout (default 120)
# JOLT_CTS_WRITE_BASELINE=1 regenerate the baseline file instead of gating
# JOLT_CTS_NS=ns1,ns2 run only these namespaces, verbose, no gating
set -u
root="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)"
cd "$root"
suite="vendor/clojure-test-suite/test"
baseline="test/chez/cts-known-failures.txt"
app="$root/test/chez/cts-app"
jobs="${JOLT_CTS_JOBS:-4}"
tmo="${JOLT_CTS_TIMEOUT:-120}"
if [ ! -d "$suite/clojure" ]; then
echo "cts: skipped (git submodule update --init vendor/clojure-test-suite)"
exit 0
fi
work="$(mktemp -d)"
trap 'rm -rf "$work"' EXIT
# test namespaces from the .cljc files (portability is a helper, not a test ns)
find "$suite" -name '*.cljc' | sed "s|^$suite/||;s|\.cljc$||;s|/|.|g;s|_|-|g" \
| grep -v '\.portability$' | sort > "$work/nses"
if [ -n "${JOLT_CTS_NS:-}" ]; then
echo "${JOLT_CTS_NS}" | tr ',' '\n' > "$work/nses"
fi
# round-robin the namespaces over N sequential workers; each worker appends
# "ns pass fail error" lines (HUNG/CRASH in the pass column) to its own file.
awk -v j="$jobs" '{print > ("'"$work"'/chunk." (NR % j))}' "$work/nses"
run_chunk() {
chunk="$1"; out="$2"
while IFS= read -r ns; do
res=$(JOLT_PWD="$app" perl -e "alarm $tmo; exec @ARGV" -- "$root/bin/joltc" -M:cts "$ns" 2>&1 </dev/null)
rc=$?
line=$(echo "$res" | grep '^CTS-RESULT' | head -1)
if [ -n "$line" ]; then
echo "$line" | awk '{print $2, $3, $4, $5}' >> "$out"
if [ -n "${JOLT_CTS_NS:-}" ]; then
echo "$res" | grep -E 'FAIL:|ERROR:|LOAD:' | sed 's/^/ /' >> "$out"
fi
elif [ $rc -ge 128 ]; then
echo "$ns HUNG 0 0" >> "$out"
else
echo "$ns CRASH 0 0" >> "$out"
fi
done < "$chunk"
}
for c in "$work"/chunk.*; do
run_chunk "$c" "$c.res" &
done
wait
cat "$work"/chunk.*.res 2>/dev/null | sort > "$work/results"
if [ -n "${JOLT_CTS_NS:-}" ]; then
cat "$work/results"
exit 0
fi
summary=$(awk '$2!="HUNG" && $2!="CRASH" {p+=$2; f+=$3; e+=$4; c++}
$2=="HUNG" {h++} $2=="CRASH" {x++}
END {printf "%d namespaces: pass %d, fail %d, error %d, hung %d, crash %d",
c+h+x, p, f, e, h, x}' "$work/results")
if [ "${JOLT_CTS_WRITE_BASELINE:-0}" = "1" ]; then
{
echo "# clojure-test-suite known failures: <namespace> <fail> <error>"
echo "# The gate fails on any per-namespace change, worse OR better; regenerate"
echo "# with: JOLT_CTS_WRITE_BASELINE=1 host/chez/cts.sh"
awk '$2=="HUNG" || $2=="CRASH" {print $1, $2, $2; next}
$3 != 0 || $4 != 0 {print $1, $3, $4}' "$work/results"
} > "$baseline"
echo "cts: $summary"
echo "cts: baseline written to $baseline ($(grep -cv '^#' "$baseline") namespaces)"
exit 0
fi
if [ ! -f "$baseline" ]; then
echo "cts: FAIL — no baseline; run JOLT_CTS_WRITE_BASELINE=1 host/chez/cts.sh"
exit 1
fi
status=0
while read -r ns p f e; do
case "$p" in HUNG|CRASH) f="$p"; e="$p" ;; esac
bl=$(grep -v '^#' "$baseline" | awk -v n="$ns" '$1==n {print $2, $3; exit}')
if [ -n "$bl" ]; then bf="${bl%% *}"; be="${bl##* }"; else bf=0; be=0; fi
if [ "$f" = "$bf" ] && [ "$e" = "$be" ]; then
continue
elif [ "$f" = "HUNG" ] || [ "$f" = "CRASH" ] \
|| { [ "$bf" != "HUNG" ] && [ "$bf" != "CRASH" ] \
&& { [ "$f" -gt "$bf" ] || [ "$e" -gt "$be" ]; }; }; then
echo "cts: NEW regression in $ns — fail $f error $e (baseline $bf $be)"
status=1
else
echo "cts: STALE baseline for $ns — now fail $f error $e (baseline $bf $be); update $baseline"
status=1
fi
done < "$work/results"
# a baseline entry whose namespace no longer reports is stale too
while read -r ns bf be; do
grep -q "^$ns " "$work/results" || { echo "cts: STALE baseline entry $ns (namespace gone)"; status=1; }
done < <(grep -v '^#' "$baseline")
echo "cts: $summary"
if [ $status -eq 0 ]; then echo "cts: passed (matches baseline)"; else echo "cts: FAILED"; fi
exit $status

186
host/chez/dce.ss Normal file
View file

@ -0,0 +1,186 @@
;; dce.ss — tree-shaking (jolt build --tree-shake): whole-program reachability DCE.
;;
;; Build one call graph over the re-emitted app + libraries AND the clojure.core
;; prelude, keep -main + every side-effecting top-level form + everything reachable
;; from those, drop the rest. Bails (keeps everything) if reachable code resolves a
;; var by name at runtime (eval/resolve/...), which a static graph can't follow. Per
;; Stalin's rule, ANY reference — a call OR a value/#'x — keeps its target live, so a
;; fn passed to map or referenced as #'x is never dropped.
;;
;; Loaded by build.ss after the compiler image (needs jolt.ir/reduce-ir-children).
;; The records it consumes come from ei-emit-ns-records (app/libs) + dce-blob-records
;; (the prelude); both build the (dce-rec …) shape below.
;; --- the DCE record ---------------------------------------------------------
;; keep?: #t = a non-def form (side effect / registration) — always emitted, and its
;; refs are reachability roots. #f = a prunable def emitted only if fqn is reached.
;; fqn: "ns/name" of a prunable def, else #f. refs: "ns/name" strings it references.
;; str: the Scheme source to emit.
(define (dce-rec keep? fqn refs str) (vector keep? fqn refs str))
(define (dce-rec-keep? r) (vector-ref r 0))
(define (dce-rec-fqn r) (vector-ref r 1))
(define (dce-rec-refs r) (vector-ref r 2))
(define (dce-rec-str r) (vector-ref r 3))
;; --- reference extraction from IR -------------------------------------------
(define dce-kw-op (keyword #f "op"))
(define dce-kw-var (keyword #f "var"))
(define dce-kw-the-var (keyword #f "the-var"))
(define dce-kw-def (keyword #f "def"))
(define dce-kw-ns (keyword #f "ns"))
(define dce-kw-name (keyword #f "name"))
(define dce-reduce-children (var-deref "jolt.ir" "reduce-ir-children"))
;; "ns/name" of every var reference anywhere in an IR node, prepended to acc. Counts
;; a :var (call head or value) and a :the-var (#'x). Arg order (acc node) matches
;; reduce-ir-children's fold fn so it nests directly.
(define (dce-collect-refs acc node)
(let ((op (jolt-get node dce-kw-op)))
(if (or (eq? op dce-kw-var) (eq? op dce-kw-the-var))
(cons (string-append (jolt-get node dce-kw-ns) "/" (jolt-get node dce-kw-name)) acc)
(dce-reduce-children dce-collect-refs acc node))))
;; The fqn of a bare top-level def (the only prunable IR form), else #f.
(define (dce-def-fqn node)
(and (eq? (jolt-get node dce-kw-op) dce-kw-def)
(string-append (jolt-get node dce-kw-ns) "/" (jolt-get node dce-kw-name))))
;; --- reference sets that gate the analysis ----------------------------------
;; A reference whose presence in reachable code forces keep-everything (the static
;; graph can't follow runtime name resolution).
(define dce-bail-refs
'("clojure.core/eval" "clojure.core/resolve" "clojure.core/ns-resolve"
"clojure.core/requiring-resolve" "clojure.core/find-var" "clojure.core/intern"
"clojure.core/load-string" "clojure.core/load-file" "clojure.core/load-reader"
"clojure.core/load"))
;; A reference that needs the analyzer/back end at runtime (compile-from-source). If
;; reachable code uses none of these, the compiler image is dropped from the binary —
;; an AOT app is fully compiled. (resolve/require don't need it: resolve is a
;; var-table lookup; a require of a baked ns no-ops.)
(define dce-compile-refs
'("clojure.core/eval" "clojure.core/load-string" "clojure.core/load-file"
"clojure.core/load-reader" "clojure.core/load"))
;; clojure.core fns the runtime .ss shims reference by name (via var-deref) — they
;; aren't visible in the IR call graph, so seed them as roots. (Found by grepping the
;; runtime shims; the smoke harness catches a miss as a diff/crash.)
(define dce-runtime-core-roots
'("clojure.core/identity" "clojure.core/isa?" "clojure.core/line-seq"
"clojure.core/make-hierarchy" "clojure.core/read" "clojure.core/read-string"
"clojure.core/read+string" "clojure.core/realized?" "clojure.core/reset!"))
;; --- reading a minted blob (prelude.ss) into records ------------------------
;; The prelude is a flat list of (guard CLAUSE (def-var! "ns" "name" V)) forms (+ the
;; occasional side-effecting init). Read each with Chez `read` so it joins the graph
;; instead of being baked wholesale: a def-var! is a prunable node whose core->core
;; edges are the (var-deref/jolt-var "ns" "name") calls in V; any other form is
;; non-prunable (kept, refs are roots).
(define (dce-unwrap form)
(if (and (pair? form) (eq? (car form) 'guard) (pair? (cddr form))) (caddr form) form))
(define (dce-sexp-refs form acc)
(cond
((and (pair? form) (memq (car form) '(var-deref jolt-var))
(pair? (cdr form)) (string? (cadr form)) (pair? (cddr form)) (string? (caddr form)))
(cons (string-append (cadr form) "/" (caddr form)) acc))
((pair? form) (dce-sexp-refs (cdr form) (dce-sexp-refs (car form) acc)))
(else acc)))
;; str re-serializes the read form (compiled identically; comments/whitespace are
;; irrelevant).
(define (dce-blob-records path)
;; bld-source-string (build.ss) reads the embedded copy when running from a
;; self-contained joltc, else the file on disk — so tree-shake works with no
;; jolt checkout present. Forward ref: build.ss loads after this file.
(call-with-port (open-input-string (bld-source-string path))
(lambda (p)
(let loop ((acc '()))
(let ((form (read p)))
(if (eof-object? form)
(reverse acc)
(let ((b (dce-unwrap form))
(str (with-output-to-string (lambda () (write form))))
(refs (dce-sexp-refs form '())))
(loop (cons
(if (and (pair? b) (eq? (car b) 'def-var!) (pair? (cdr b)) (string? (cadr b))
(pair? (cddr b)) (string? (caddr b)))
(dce-rec #f (string-append (cadr b) "/" (caddr b)) refs str)
(dce-rec #t #f refs str))
acc)))))))))
;; --- the shake: graph -> reachable -> bail check -> partition ----------------
;; edges: fqn -> refs (prunable defs only). roots: -main + the runtime-core roots +
;; every non-def form's refs.
(define (dce-build-graph records entry-main)
(let ((edges (make-hashtable string-hash string=?))
(roots (cons entry-main dce-runtime-core-roots)))
(for-each (lambda (r)
(if (dce-rec-keep? r)
(set! roots (append (dce-rec-refs r) roots))
(hashtable-set! edges (dce-rec-fqn r) (dce-rec-refs r))))
records)
(values edges roots)))
;; Closure of roots over edges -> a reached set (hashtable fqn -> #t).
(define (dce-reachable edges roots)
(let ((reached (make-hashtable string-hash string=?)))
(let bfs ((work roots))
(unless (null? work)
(let ((fq (car work)))
(if (hashtable-ref reached fq #f)
(bfs (cdr work))
(begin (hashtable-set! reached fq #t)
(bfs (append (or (hashtable-ref edges fq #f) '()) (cdr work))))))))
reached))
(define (dce-rec-reached? r reached)
(or (dce-rec-keep? r) (hashtable-ref reached (dce-rec-fqn r) #f)))
;; Scan the KEPT records: does any resolve a var at runtime (bail), and does any need
;; the compiler? Returns (values bail? bail-why needs-compiler?). bail-why is up to 6
;; (def . bail-ref) pairs for the diagnostic.
(define (dce-bail-scan records reached)
(let ((bail #f) (why '()) (needs-compiler #f))
(for-each
(lambda (r)
(when (dce-rec-reached? r reached)
(for-each (lambda (b)
(when (member b (dce-rec-refs r))
(set! bail #t)
(when (< (length why) 6)
(set! why (cons (cons (or (dce-rec-fqn r) "<form>") b) why)))))
dce-bail-refs)
(when (ormap (lambda (c) (and (member c (dce-rec-refs r)) #t)) dce-compile-refs)
(set! needs-compiler #t))))
records)
(values bail (reverse why) needs-compiler)))
;; Kept records -> (values kept-strings n-defs n-kept-defs).
(define (dce-partition records reached)
(let loop ((rs records) (acc '()) (n 0) (k 0))
(if (null? rs)
(values (reverse acc) n k)
(let* ((r (car rs)) (isdef (and (dce-rec-fqn r) #t)))
(if (dce-rec-reached? r reached)
(loop (cdr rs) (cons (dce-rec-str r) acc) (if isdef (+ n 1) n) (if isdef (+ k 1) k))
(loop (cdr rs) acc (if isdef (+ n 1) n) k))))))
;; Returns (values core-strs app-strs drop-compiler?). core-strs is #f on a bail,
;; signalling "inline prelude.ss unshaken" + keep the compiler.
(define (dce-shake core-records app-records entry-main)
(let-values (((edges roots) (dce-build-graph (append core-records app-records) entry-main)))
(let* ((reached (dce-reachable edges roots)))
(let-values (((bail why needs-compiler) (dce-bail-scan (append core-records app-records) reached)))
(let ((drop-compiler? (and (not bail) (not needs-compiler))))
(if bail
(begin
(display "jolt build: tree-shake skipped (reachable code resolves vars at runtime):\n")
(for-each (lambda (w) (display (string-append " " (car w) " -> " (cdr w) "\n"))) why)
(values #f (map dce-rec-str app-records) drop-compiler?))
(let-values (((core-strs cn ck) (dce-partition core-records reached))
((app-strs an ak) (dce-partition app-records reached)))
(display (string-append "jolt build: tree-shake kept " (number->string (+ ck ak))
" of " (number->string (+ cn an)) " defs (core "
(number->string ck) "/" (number->string cn) ")\n"))
(values core-strs app-strs drop-compiler?))))))))

167
host/chez/dyn-binding.ss Normal file
View file

@ -0,0 +1,167 @@
;; dynamic var binding — binding / with-bindings* / var-set / thread-bound? /
;; with-local-vars / with-redefs / bound-fn* / get-thread-bindings.
;;
;; A per-thread dynamic-binding stack: a list of frames, innermost (most recently
;; pushed) at the HEAD. Each frame is an alist of (var-cell . value) MUTABLE pairs
;; — so var-set can update the innermost binding in place (set-cdr!), matching
;; Clojure where var-set targets the current binding, not the root.
;;
;; The binding macro builds a frame as a jolt map (array-map of (var x) -> value);
;; push-thread-bindings folds it into the alist. Lookups walk frames by cell
;; IDENTITY (eq?) — vars are interned, so (var x) always yields the same cell, and
;; this sidesteps a persistent-hash-map-can't-find-a-var-key quirk.
;;
;; var reads (var-deref in compiled code, jolt-var-get / deref on a cell) consult
;; the stack before falling back to the cell root. Loaded LAST (after vars.ss and
;; ns.ss) so it chains the fully-extended jolt-var-get and overrides rt.ss var-deref.
;; THREAD-LOCAL: a Chez thread parameter, so each OS thread (a future / go block)
;; has its own binding stack. Chez initializes a new thread's parameter
;; to the spawning thread's value at fork time, giving Clojure binding conveyance
;; for free (the future shim also installs an explicit snapshot, belt-and-suspenders).
(define dyn-binding-stack (make-thread-parameter '()))
;; find the innermost (cell . value) pair binding CELL, or #f.
(define (dyn-find-binding cell)
(let loop ((frames (dyn-binding-stack)))
(and (pair? frames)
(or (assq cell (car frames))
(loop (cdr frames))))))
;; a unique sentinel: distinguishes "no thread binding" from a binding whose
;; value happens to be jolt-nil.
(define dyn-no-binding (list 'no-binding))
(define (dyn-binding-value cell)
(if (pair? (dyn-binding-stack))
(let ((p (dyn-find-binding cell)))
(if p
(let ((val (cdr p)))
(if (var-cell? val) (jolt-var-get val) val)) ; nested var deref (Clojure)
dyn-no-binding))
dyn-no-binding))
;; push-thread-bindings: frame is a jolt map of var-cell -> value. Fold it into an
;; identity-keyed alist of mutable pairs and push.
(define (jolt-push-thread-bindings frame)
(dyn-binding-stack
(cons (pmap-fold frame (lambda (k v acc) (cons (cons k v) acc)) '())
(dyn-binding-stack)))
jolt-nil)
(define (jolt-pop-thread-bindings)
(when (pair? (dyn-binding-stack))
(dyn-binding-stack (cdr (dyn-binding-stack))))
jolt-nil)
;; get-thread-bindings: a jolt map of every currently-bound cell -> value,
;; innermost wins. Merge oldest-frame-first (the stack head is innermost). The
;; result can be re-pushed by with-bindings* / bound-fn*.
(define (jolt-get-thread-bindings)
(let loop ((frames (reverse (dyn-binding-stack))) (m (jolt-hash-map)))
(if (null? frames)
m
(loop (cdr frames)
(let frame-loop ((alist (car frames)) (m m))
(if (null? alist)
m
(frame-loop (cdr alist)
(pmap-assoc m (caar alist) (cdar alist)))))))))
;; __thread-bound? — single var; true iff it has a thread binding.
(define (jolt-thread-bound? v)
(and (var-cell? v) (dyn-find-binding v) #t))
;; var-set: update the innermost frame that binds v (in place); else set the root.
(define (jolt-var-set v val)
(if (var-cell? v)
(let ((p (dyn-find-binding v)))
(if p
(begin (set-cdr! p val) val)
;; a ROOT change is Var.bindRoot: validate, set, notify watches
;; (a thread-binding set does not notify, like the JVM).
(let ((old (var-cell-root v)))
(iref-validate v val)
(var-cell-root-set! v val) (var-cell-defined?-set! v #t)
(iref-notify v old val)
val)))
(error #f "var-set: not a var" v)))
;; alter-var-root: atomically apply f to the current root plus args.
(define (jolt-alter-var-root v f . args)
(let* ((old (var-cell-root v))
(new (apply jolt-invoke f old args)))
(iref-validate v new)
(var-cell-root-set! v new)
(var-cell-defined?-set! v #t)
(iref-notify v old new)
new))
;; __local-var: a fresh free-standing var cell (not interned). with-local-vars
;; binds these as lexical locals; var-get/var-set read/write the root. Each gets a
;; unique name so two locals never compare/hash equal as map keys.
(define local-var-counter 0)
(define (jolt-local-var . args)
(set! local-var-counter (fx+ local-var-counter 1))
(make-var-cell "" (string-append "local-" (number->string local-var-counter))
(if (pair? args) (car args) jolt-nil)
#t))
;; --- chain the var-read paths onto the binding stack -------------------------
;; var-deref (rt.ss): the compiled-code read path for every clojure.core var
;; reference. Consult the stack first; fall straight back to the root (NOT through
;; jolt-var-get's unbound-error path) so undefined-var reads keep prior behaviour.
;; The *ns* var cell — its reads are thread-local: with no thread-binding they
;; derive from chez-current-ns (a thread-parameter), so *ns* tracks in-ns per
;; thread and a (binding [*ns* ..]) drives resolution. Captured now that *ns* is
;; defined (ns.ss loaded earlier); chez-current-ns consults it too.
(set! star-ns-cell (jolt-var "clojure.core" "*ns*"))
(define %dyn-rt-var-deref var-deref)
(set! var-deref
(lambda (ns name)
(let ((cell (jolt-var ns name)))
(let ((bv (dyn-binding-value cell)))
(cond ((not (eq? bv dyn-no-binding)) bv)
((eq? cell star-ns-cell) (intern-ns! (chez-current-ns)))
(else (var-cell-root cell)))))))
;; var-deref's read on an ALREADY-RESOLVED cell — what compiled code emits when it
;; caches the cell at a reference site. Binding stack first, then *ns* thread-local,
;; else the raw root. Lenient on an unbound root (returns the sentinel), matching
;; var-deref — NOT the strict jolt-var-get, which throws "Unbound var".
(define (var-cell-deref cell)
(let ((bv (dyn-binding-value cell)))
(cond ((not (eq? bv dyn-no-binding)) bv)
((eq? cell star-ns-cell) (intern-ns! (chez-current-ns)))
(else (var-cell-root cell)))))
;; jolt-var-get (vars.ss): the var-get fn + deref/@ on a cell. Stack first, then
;; the original (which errors on an unbound root, matching Clojure).
(define %dyn-var-get jolt-var-get)
(set! jolt-var-get
(lambda (v)
(if (var-cell? v)
(let ((bv (dyn-binding-value v)))
(cond ((not (eq? bv dyn-no-binding)) bv)
((eq? v star-ns-cell) (intern-ns! (chez-current-ns)))
(else (%dyn-var-get v))))
(%dyn-var-get v))))
;; var-cell keys hash/compare by ns/name (jolt=2 in vars.ss already compares
;; ns/name) — stable under root mutation, so a var works as a map key (with-redefs
;; builds (hash-map (var f) v); get-thread-bindings returns a var-keyed map).
(register-hash-arm! var-cell? (lambda (x) (equal-hash (cons (var-cell-ns x) (var-cell-name x)))))
;; --- bind the host seams the overlay references -----------------------------
(def-var! "clojure.core" "push-thread-bindings" jolt-push-thread-bindings)
(def-var! "clojure.core" "pop-thread-bindings" jolt-pop-thread-bindings)
(def-var! "clojure.core" "get-thread-bindings" jolt-get-thread-bindings)
(def-var! "clojure.core" "__thread-bound?" jolt-thread-bound?)
(def-var! "clojure.core" "var-set" jolt-var-set)
(def-var! "clojure.core" "alter-var-root" jolt-alter-var-root)
(def-var! "clojure.core" "__local-var" jolt-local-var)
;; re-assert var-get / deref to the new (stack-aware) closures (vars.ss captured
;; the pre-chain values).
(def-var! "clojure.core" "var-get" jolt-var-get)
(def-var! "clojure.core" "deref" jolt-deref)

View file

@ -0,0 +1,70 @@
;; dynamic-var-defaults.ss — default values for the handful of clojure.core dynamic
;; vars that aren't emitted into the prelude (*clojure-version*, *assert*, …). Plain
;; constant def-var!s; *ns* (a namespace object) needs a value type with
;; get-see-through and map?=false and is tracked separately. The binding-stack
;; machinery (binding / var-set / thread-bound?) lives in dyn-binding.ss. Loaded
;; from rt.ss after the value model + def-var!.
;; *clojure-version* — a map {:major 1 :minor 11 :incremental 0 :qualifier nil}.
(def-var! "clojure.core" "*clojure-version*"
(jolt-hash-map (keyword #f "major") 1
(keyword #f "minor") 11
(keyword #f "incremental") 0
(keyword #f "qualifier") jolt-nil))
;; *unchecked-math* — jolt does no unchecked-math elision; the var reads false.
(def-var! "clojure.core" "*unchecked-math*" #f)
;; *warn-on-reflection* — jolt has no reflection, so the var reads false; (set!
;; *warn-on-reflection* …) resolves and updates it (a no-op effect).
(def-var! "clojure.core" "*warn-on-reflection*" #f)
;; *assert* — gates `assert`; settable/bindable (malli.assert toggles it). Default
;; true, like the JVM.
(def-var! "clojure.core" "*assert*" #t)
;; *print-readably* — bound by pr-family / with-out-str-style code; default true.
(def-var! "clojure.core" "*print-readably*" #t)
;; *print-meta* — when true, pr prints metadata with a ^ prefix; default false.
(def-var! "clojure.core" "*print-meta*" #f)
;; *print-length* / *print-level* — collection print limits, honored by both
;; printers (rt.ss jolt-pr-str + printing.ss jolt-pr-readable). nil = unlimited
;; (the default); a number truncates elements / collapses depth to "#".
;; *print-length* limits a lazy/infinite seq before realizing it.
(def-var! "clojure.core" "*print-length*" jolt-nil)
(def-var! "clojure.core" "*print-level*" jolt-nil)
;; *default-data-reader-fn* — a (fn [tag value]) the reader consults for an
;; unregistered #tag before raising; nil = no default handler.
(def-var! "clojure.core" "*default-data-reader-fn*" jolt-nil)
;; Portable clojure.core dynamic vars whose DEFAULT already matches jolt's
;; behaviour, so exposing them is sound (resolve/binding work, reads return the
;; right value) — not a silent divergence.
;;
;; *read-eval* — gates #=() read-eval. jolt's reader has no #=, so it reads true
;; (no eval-on-read happens regardless); a lib can (binding [*read-eval* false] …).
(def-var! "clojure.core" "*read-eval*" #t)
;; *print-dup* — gates print-dup (a multimethod that exists); default false.
(def-var! "clojure.core" "*print-dup*" #f)
;; *print-namespace-maps* — jolt never prints the #:ns{…} map shorthand, so the
;; var reads false (accurate); settable for code that toggles it.
(def-var! "clojure.core" "*print-namespace-maps*" #f)
;; *flush-on-newline* — jolt flushes line output; default true.
(def-var! "clojure.core" "*flush-on-newline*" #t)
;; *compile-files* — jolt has no separate compile phase that emits .class files.
(def-var! "clojure.core" "*compile-files*" #f)
;; *math-context* — BigDecimal rounding context; nil = unlimited, jolt's default.
(def-var! "clojure.core" "*math-context*" jolt-nil)
;; *command-line-args* — the args after the script/-main; nil outside a -m run.
(def-var! "clojure.core" "*command-line-args*" jolt-nil)
;; *file* — the source file being loaded; "NO_SOURCE_PATH" when none, like the JVM.
(def-var! "clojure.core" "*file*" "NO_SOURCE_PATH")
;; REPL result/exception history. Bound by the REPL after each evaluation; nil
;; outside a REPL, which is what reading them returns here.
(def-var! "clojure.core" "*1" jolt-nil)
(def-var! "clojure.core" "*2" jolt-nil)
(def-var! "clojure.core" "*3" jolt-nil)
(def-var! "clojure.core" "*e" jolt-nil)

191
host/chez/emit-image.ss Normal file
View file

@ -0,0 +1,191 @@
;; emit-image.ss — the on-Chez compiler-image emitter.
;;
;; This is the stage2/stage3 half of the self-hosting fixpoint. The
;; analyze->emit runs ON CHEZ (jolt-ce-analyze / jolt-ce-emit, loaded from a
;; previously-built image): feed it stage1's image and it produces stage2; feed it
;; stage2 and it produces stage3. stage2 == stage3 byte-for-byte proves the
;; on-Chez compiler reproduces itself (self-hosting-bootstrap-research §4).
;;
;; Loaded after compile-eval.ss (needs jolt-ce-analyze/jolt-ce-emit/ce-scan-requires!,
;; make-analyze-ctx) and rt.ss (read-file-string, the reader's rdr-read-form).
;; Read every top-level form from a source string (a Chez read-all).
;; Uses the same reader the spine reads single forms with.
(define (ei-read-all src)
(let ((end (string-length src)))
(let loop ((i 0) (acc '()))
(let-values (((form j) (rdr-read-form src i end)))
(if (rdr-eof? form)
(reverse acc)
(loop j (cons form acc)))))))
;; Is `f` an (ns ...) form? (Its only role in the image is alias registration; we
;; never emit it — the def-var!s carry explicit ns names.)
(define (ei-ns-form? f)
(and (cseq? f) (cseq-list? f)
(let ((items (seq->list f)))
(and (pair? items) (symbol-t? (car items))
(string=? (symbol-t-name (car items)) "ns")))))
;; ei-macro-form? / ei-defmacro->fn moved to compile-eval.ss (ce-macro-form? /
;; ce-defmacro->fn, loaded before this) — shared with the runtime defmacro spine.
;; Cross-compile one namespace's source to a list of guard-wrapped Scheme strings.
;; Each form is analyzed with a fresh ctx — resolution is via the runtime var-table
;; + alias tables, not ctx-accumulated state, so this matches the spine's per-form
;; analyze. A defmacro emits its expander fn as (def-var! ns name <fn>) +
;; (mark-macro! ns name) so the on-Chez analyzer can expand it.
;; Analyze -> (optionally run passes) -> emit one form. optimize? runs
;; jolt.passes/run-passes (build optimizes; the seed minter stays un-optimized so
;; the self-host fixpoint is independent of the passes). emit-top-form is the
;; top-level entry: in direct-link mode it binds jv$<fqn> for a top-level def; off
;; that mode (the minter, runtime eval) it is exactly emit, so output is unchanged.
(define jolt-ce-emit-top (var-deref "jolt.backend-scheme" "emit-top-form"))
;; Seed mint and AOT build must stay byte-deterministic, so emit the image with var
;; cell-caching OFF (compile-eval.ss turned it on for runtime eval; this file loads
;; after it). Guarded for the first re-mint pass off an older seed.
(let ((scv (var-deref "jolt.backend-scheme" "set-var-cache!")))
(when (procedure? scv) (scv #f)))
;; Tail-frame tracing off for the mint + `jolt build`: the seed must stay a
;; byte-fixpoint, and a built app should carry no per-call trace overhead.
(let ((stf (var-deref "jolt.backend-scheme" "set-trace-frames!")))
(when (procedure? stf) (stf #f)))
(define (ei-compile-form ctx f optimize?)
(let ((ir (jolt-ce-analyze ctx f)))
(jolt-ce-emit-top (if optimize? (jolt-ce-run-passes ir ctx) ir))))
;; The emitted `(def-var! …)(mark-macro! …)` pair for a defmacro, guard-wrapped
;; (tolerant) or bare (strict) to match guard?.
(define (ei-macro-string ns-name nm scm guard?)
(if guard?
(string-append "(guard (e (#t #f))\n (def-var! " (ei-str-lit ns-name) " " (ei-str-lit nm)
"\n " scm ")\n (mark-macro! " (ei-str-lit ns-name) " " (ei-str-lit nm) "))")
(string-append "(def-var! " (ei-str-lit ns-name) " " (ei-str-lit nm) "\n " scm
")\n(mark-macro! " (ei-str-lit ns-name) " " (ei-str-lit nm) ")")))
;; Cross-compile one namespace's source to a list of Scheme strings — shared by
;; the seed minter (ei-emit-ns: optimize? #f, guard? #t — tolerant, skips a form
;; that fails to emit) and `jolt build` (bld-emit-ns: optimize? #t, guard? #f —
;; strict, a failing form errors the build).
;; A per-form transform applied to each read form before emit — the build sets it
;; to the data-reader rewrite (loader.ss ldr-apply-readers) so a registered #tag
;; literal compiles in a `jolt build` the same as it does in an interpreted load.
;; #f (the default, and during the seed mint where loader.ss isn't loaded) is no
;; transform, so emit-image.ss carries no loader dependency.
(define ei-emit-form-hook (make-parameter #f))
(define (ei-emit-ns* ns-name src optimize? guard?)
;; set the ns before reading so ::kw auto-resolves against this ns (the runtime
;; loader reads form-by-form after the ns form sets it; the cross-compile reads
;; all forms up front, so set it here).
(set-chez-ns! ns-name)
(let ((hook (ei-emit-form-hook)))
(let loop ((forms (ei-read-all src)) (acc '()))
(if (null? forms)
(reverse acc)
(let ((f (let ((f0 (car forms))) (if hook (hook f0) f0))))
(ce-scan-requires! f ns-name)
(cond
((ei-ns-form? f) (loop (cdr forms) acc))
((ce-macro-form? f)
(let-values (((nm fn-form) (ce-defmacro->fn f)))
(let ((scm (if guard?
(guard (e (#t #f)) (ei-compile-form (make-analyze-ctx ns-name) fn-form optimize?))
(ei-compile-form (make-analyze-ctx ns-name) fn-form optimize?))))
(loop (cdr forms)
(if (and guard? (not scm)) acc
(cons (ei-macro-string ns-name nm scm guard?) acc))))))
(else
(let ((scm (if guard?
(guard (e (#t #f)) (ei-compile-form (make-analyze-ctx ns-name) f optimize?))
(ei-compile-form (make-analyze-ctx ns-name) f optimize?))))
(loop (cdr forms)
(if (and guard? (not scm)) acc
(cons (if guard? (string-append "(guard (e (#t #f))\n " scm ")") scm) acc)))))))))))
(define (ei-emit-ns ns-name src) (ei-emit-ns* ns-name src #f #t))
;; --- DCE record producer ----------------------------------------------------
;; Cross-compile a namespace's source to tree-shaking records — the app/library
;; counterpart to dce-blob-records (the prelude). The shake itself and all dce-*
;; helpers live in dce.ss; this stays here because it drives the ei-* compiler. A
;; top-level def becomes a prunable record; any other form a kept (side-effecting)
;; record whose refs are roots. A macro is prunable — its expander isn't called at
;; runtime in an AOT build.
(define (ei-emit-ns-records ns-name src)
(set-chez-ns! ns-name) ; ::kw resolves against this ns (see ei-emit-ns*)
(let loop ((forms (ei-read-all src)) (acc '()))
(if (null? forms)
(reverse acc)
(let ((f (car forms)))
(ce-scan-requires! f ns-name)
(cond
((ei-ns-form? f) (loop (cdr forms) acc))
((ce-macro-form? f)
(let-values (((nm fn-form) (ce-defmacro->fn f)))
(let* ((ctx (make-analyze-ctx ns-name))
(ir (jolt-ce-run-passes (jolt-ce-analyze ctx fn-form) ctx))
(str (ei-macro-string ns-name nm (jolt-ce-emit-top ir) #f))
(refs (dce-collect-refs '() ir)))
(loop (cdr forms) (cons (dce-rec #f (string-append ns-name "/" nm) refs str) acc)))))
(else
(let* ((ctx (make-analyze-ctx ns-name))
(ir (jolt-ce-run-passes (jolt-ce-analyze ctx f) ctx))
(str (jolt-ce-emit-top ir))
(fqn (dce-def-fqn ir))
(refs (dce-collect-refs '() ir)))
(loop (cdr forms)
(cons (if fqn (dce-rec #f fqn refs str) (dce-rec #t #f refs str)) acc)))))))))
;; Scheme string literal for a ns/name — uses the runtime's own writer
;; (printable ASCII identifiers only here).
(define (ei-str-lit s) (with-output-to-string (lambda () (write s))))
;; The compiler namespaces, in load order. The passes (fold/inline/types + the
;; jolt.passes façade) load after ir so run-passes is available to the back end;
;; fold/inline/types come before the façade that :refers them.
(define ei-compiler-ns-files
(list (cons "jolt.ir" "jolt-core/jolt/ir.clj")
(cons "jolt.analyzer" "jolt-core/jolt/analyzer.clj")
(cons "jolt.backend-scheme" "jolt-core/jolt/backend_scheme.clj")
(cons "jolt.passes.fold" "jolt-core/jolt/passes/fold.clj")
(cons "jolt.passes.numeric" "jolt-core/jolt/passes/numeric.clj")
(cons "jolt.passes.inline" "jolt-core/jolt/passes/inline.clj")
(cons "jolt.passes.types.lattice" "jolt-core/jolt/passes/types/lattice.clj")
(cons "jolt.passes.types.check" "jolt-core/jolt/passes/types/check.clj")
(cons "jolt.passes.types" "jolt-core/jolt/passes/types.clj")
(cons "jolt.passes" "jolt-core/jolt/passes.clj")))
;; The clojure.core tiers + stdlib namespaces, in load order.
;; Re-emitting these on Chez is the
;; prelude half of the fixpoint (the whole emitted system reproducing itself).
(define ei-prelude-ns-files
(append
(map (lambda (tf) (cons "clojure.core" (string-append "jolt-core/clojure/core/" tf ".clj")))
'("00-syntax" "00-kernel" "10-seq" "20-coll" "21-coll" "22-coll" "25-sorted" "30-macros" "40-lazy" "50-io"))
(list (cons "clojure.string" "stdlib/clojure/string.clj")
(cons "clojure.walk" "stdlib/clojure/walk.clj")
(cons "clojure.template" "stdlib/clojure/template.clj")
(cons "clojure.edn" "stdlib/clojure/edn.clj")
(cons "clojure.set" "stdlib/clojure/set.clj")
(cons "clojure.pprint" "stdlib/clojure/pprint.clj"))))
;; Join a list of form strings with "\n", no trailing newline.
(define (ei-join forms)
(let join ((fs forms) (out ""))
(cond
((null? fs) out)
((string=? out "") (join (cdr fs) (car fs)))
(else (join (cdr fs) (string-append out "\n" (car fs)))))))
;; Re-emit the whole list of (ns . file) pairs ON CHEZ as one Scheme string.
(define (ei-emit-ns-files nfs)
(ei-join (apply append
(map (lambda (nf) (ei-emit-ns (car nf) (read-file-string (cdr nf)))) nfs))))
;; Emit the compiler image (jolt.ir + jolt.analyzer + jolt.backend-scheme) on Chez.
(define (jolt-emit-image) (ei-emit-ns-files ei-compiler-ns-files))
;; Emit the clojure.core prelude (all tiers + stdlib) on Chez — the prelude half of
;; the self-hosting fixpoint.
(define (jolt-emit-prelude) (ei-emit-ns-files ei-prelude-ns-files))

528
host/chez/host-contract.ss Normal file
View file

@ -0,0 +1,528 @@
;; host-contract.ss — the jolt.host contract on Chez.
;;
;; The portable seam between jolt-core (analyzer/IR/emitter, cross-compiled to
;; Scheme) and the host. Every
;; contract fn is def-var!'d into the "jolt.host" namespace so the cross-compiled
;; jolt.analyzer / jolt.backend-scheme — whose unqualified form-*/resolve-global/
;; ... refs lower to (var-deref "jolt.host" ...) — resolve here at runtime.
;;
;; This is what puts analyze->IR->emit ON CHEZ. It runs
;; over the Chez data reader's forms (reader.ss): symbols are symbol-t, lists are
;; cseq (list?), () is empty-list-t, vectors/maps are pvec/pmap, sets and #tag/
;; regex/inst/uuid are pmaps tagged :jolt/type, chars are NATIVE Chez chars.
;;
;; Loaded after rt.ss + reader.ss + the core prelude; before the compiler image.
;; --- the analyze ctx --------------------------------------------------------
;; ctx is opaque to the analyzer (only ever threaded to these contract fns); we
;; make it a box carrying the compile namespace. The var/ns registry it consults
;; is the global var-table (rt.ss).
(define-record-type chez-actx (fields (mutable cns)) (nongenerative chez-actx-v1))
(define (make-analyze-ctx ns) (make-chez-actx ns))
;; Interned keywords reused for form tags + resolve-global's result map.
(define hc-kw-jolt-type (keyword "jolt" "type"))
(define hc-kw-jolt-set (keyword "jolt" "set"))
(define hc-kw-jolt-tagged (keyword "jolt" "tagged"))
(define hc-kw-value (keyword #f "value"))
(define hc-kw-tag (keyword #f "tag"))
(define hc-kw-form (keyword #f "form"))
(define hc-kw-kind (keyword #f "kind"))
(define hc-kw-ns (keyword #f "ns"))
(define hc-kw-name (keyword #f "name"))
(define hc-kw-var (keyword #f "var"))
(define hc-kw-unresolved (keyword #f "unresolved"))
(define hc-kw-class (keyword #f "class"))
(define hc-kw-num-ret (keyword #f "num-ret"))
(define hc-kw-double (keyword #f "double"))
(define hc-kw-long (keyword #f "long"))
(define hc-kw-regex (keyword #f "regex"))
(define hc-kw-inst (keyword #f "#inst"))
(define hc-kw-uuid (keyword #f "#uuid"))
(define hc-kw-bigdec (keyword #f "bigdec"))
;; --- form predicates --------------------------------------------------------
(define (hc-sym? x) (symbol-t? x))
;; ANY non-empty seq is a list form for analysis (a macro/eval form built via
;; concat/map/cons is a lazy cseq with list?=#f, but evaluating it still means
;; calling its head) — not just reader-built lists.
;; a lazy seq is a list form too: a macro that builds its expansion with map/for
;; (now a LazySeq, not an eager cseq) and splices it must still analyze.
(define (hc-list? x) (or (empty-list-t? x) (cseq? x) (jolt-lazyseq? x)))
(define (hc-vec? x) (pvec? x))
(define (hc-map? x) (and (pmap? x) (jolt-nil? (jolt-get x hc-kw-jolt-type))))
;; A set form is the reader's tagged map {:jolt/type :jolt/set :value <pvec>} OR a
;; real pset value — a macro template's #{...} expansion (syntax-quote.ss jolt-sqset)
;; produces a pset, which the analyzer must still read as a set literal.
(define (hc-set? x)
(or (pset? x)
(and (pmap? x) (eq? (jolt-get x hc-kw-jolt-type) hc-kw-jolt-set))))
(define (hc-char? x) (char? x))
(define (hc-keyword? x) (keyword? x))
(define (hc-literal? x)
(or (jolt-nil? x) (boolean? x) (number? x) (string? x) (keyword-t? x) (char? x)))
(define (hc-tagged-of x tag)
(and (pmap? x)
(eq? (jolt-get x hc-kw-jolt-type) hc-kw-jolt-tagged)
(eq? (jolt-get x hc-kw-tag) tag)))
(define (hc-regex? x) (regex-t? x)) ; #"..." reads as a regex VALUE now
(define (hc-inst? x) (hc-tagged-of x hc-kw-inst))
(define (hc-uuid? x) (hc-tagged-of x hc-kw-uuid))
(define (hc-bigdec? x) (hc-tagged-of x hc-kw-bigdec))
(define (hc-bigdec-source x) (jolt-get x hc-kw-form))
;; A live namespace value spliced into a form (e.g. `(str ~*ns*) in a macro):
;; the analyzer can't carry an opaque runtime value, so recognize a jns and
;; reconstruct it by name at the call site.
(define (hc-ns-value? x) (jns? x))
(define (hc-ns-value-name x) (jns-name x))
;; a live Var value spliced into a form (a macro that does `(~v …)` with v a
;; resolved var) — the analyzer turns it into a :the-var reference by ns+name.
(define (hc-var-value? x) (var-cell? x))
(define (hc-var-value-ns x) (var-cell-ns x))
(define (hc-var-value-name x) (var-cell-name x))
;; *unchecked-math* read at compile time: when truthy (a file's (set!
;; *unchecked-math* …)), the analyzer rewrites +/-/*/inc/dec to their wrapping
;; unchecked-* forms for the rest of that file, like the JVM.
(define (hc-unchecked-math?)
(jolt-truthy? (guard (e (#t #f)) (var-deref "clojure.core" "*unchecked-math*"))))
;; --- form accessors ---------------------------------------------------------
(define (hc-char-code x) (char->integer x)) ; native Chez char -> codepoint
(define (hc-sym-name x) (symbol-t-name x))
;; The reader stores an unqualified symbol's ns inconsistently (#f, '(), or
;; jolt-nil — see converters.ss). The contract is jolt-nil for unqualified (the
;; analyzer tests (nil? ns)), so normalize; a real ns string passes through.
(define (hc-sym-ns x)
(let ((ns (symbol-t-ns x)))
(if (and ns (not (jolt-nil? ns)) (not (null? ns))) ns jolt-nil)))
(define (hc-sym-meta x)
(let ((m (symbol-t-meta x)))
(if (and m (not (jolt-nil? m)) (not (null? m))) m jolt-nil)))
;; Metadata the reader attached to a collection literal (vec/map/set/list), or
;; jolt-nil. The analyzer re-emits a runtime (with-meta ..) for a meta-carrying
;; vector/map/set so the value keeps its metadata.
(define (hc-coll-meta x) (jolt-meta x))
;; list items -> jolt vector (pvec); the analyzer mapv's over the result.
(define (hc-elements x)
(cond ((empty-list-t? x) empty-pvec)
((or (cseq? x) (jolt-lazyseq? x)) (make-pvec (list->vector (seq->list x))))
(else empty-pvec)))
(define (hc-vec-items x) x) ; already a pvec
(define (hc-set-items x)
(if (pset? x)
(apply jolt-vector (pset-fold x cons '()))
(jolt-get x hc-kw-value)))
(define (hc-map-pairs x)
(let ((kv (hashtable-ref rdr-map-order x #f)))
(if kv
;; reader-built map literal: emit pairs in SOURCE order (kv = k1 v1 k2 v2 …)
;; so the analyzer evaluates the values left-to-right.
(let loop ((kv kv) (acc '()))
(if (null? kv) (apply jolt-vector (reverse acc))
(loop (cddr kv) (cons (jolt-vector (car kv) (cadr kv)) acc))))
;; a runtime/non-reader map: pmap iteration order
(let loop ((ks (if (jolt-nil? (jolt-seq (jolt-keys x))) '()
(seq->list (jolt-seq (jolt-keys x))))) (acc '()))
(if (null? ks) (apply jolt-vector (reverse acc))
(loop (cdr ks) (cons (jolt-vector (car ks) (jolt-get x (car ks))) acc)))))))
(define (hc-regex-source x) (regex-t-source x))
(define (hc-inst-source x) (jolt-get x hc-kw-form))
(define (hc-uuid-source x) (jolt-get x hc-kw-form))
;; Source position for a list form: the reader stamps :line/:column (+ :file when
;; compiling a file) into the form's metadata. Return a clean {:line :column
;; :file?} map, or nil for a synthetic/macro-built form that carries none.
(define hc-kw-line (keyword #f "line"))
(define hc-kw-column (keyword #f "column"))
(define hc-kw-file (keyword #f "file"))
(define (hc-form-position x)
(let ((m (jolt-meta x)))
(if (and (pmap? m) (not (jolt-nil? (jolt-get m hc-kw-line))))
(let ((line (jolt-get m hc-kw-line))
(col (jolt-get m hc-kw-column))
(file (jolt-get m hc-kw-file)))
(if (jolt-nil? file)
(jolt-hash-map hc-kw-line line hc-kw-column col)
(jolt-hash-map hc-kw-line line hc-kw-column col hc-kw-file file)))
jolt-nil)))
;; --- special forms ----------------------------------------------------------
;; Mirrors host_iface special-names + interop-head? — forms the analyzer marks
;; uncompilable (the handled specials are dispatched in analyze-list BEFORE this).
;; `eval` is NOT here: it is a clojure.core FUNCTION on the spine (compile-eval.ss
;; def-var!s it), so it must resolve as an ordinary var, not punt.
;; `defmacro` stays special — the spine intercepts it before analysis.
(define hc-special-names
'("quote" "syntax-quote" "unquote" "unquote-splicing" "do" "if" "def"
"defmacro" "fn*" "let*" "loop*" "recur" "throw" "try" "set!" "new"
"." "gen-class" "monitor-enter" "monitor-exit" "letfn"))
(define (hc-interop-head? name)
(let ((n (string-length name)))
(and (> n 1)
(not (string=? name "..")) ; the .. threading macro, not an interop form
(or (char=? (string-ref name 0) #\.)
(char=? (string-ref name (- n 1)) #\.)))))
(define (hc-special? name)
(if (or (member name hc-special-names) (hc-interop-head? name)) #t #f))
;; --- compile-time environment -----------------------------------------------
(define (hc-current-ns ctx) (chez-actx-cns ctx))
(define (hc-late-bind? ctx) #t) ; Chez has no interpreter to punt to
;; Resolve a global symbol to its var cell against the compile ns then clojure.core
;; (a qualified ns wins). Shared by resolve-global / form-macro? / form-expand-1.
;; Normalizes the reader's unqualified-ns sentinel (#f / '() / jolt-nil) like
;; hc-sym-ns, so an unqualified symbol never looks up a bogus "#f" namespace.
(define (hc-resolve-cell ctx sym)
(let* ((nm (symbol-t-name sym))
(sns (symbol-t-ns sym))
(qualified (and sns (not (jolt-nil? sns)) (not (null? sns)) sns)))
(if qualified
;; a qualified ns may be a require :as alias (s/split -> clojure.string/split)
(let ((target (or (chez-resolve-alias (chez-actx-cns ctx) qualified) qualified)))
(var-cell-lookup target nm))
(or (let ((c (var-cell-lookup (chez-actx-cns ctx) nm)))
;; an undefined forward-intern must not shadow a real referred
;; or clojure.core var — e.g. the compiler ns referencing `set`,
;; which late-binds (interns `jolt.backend-scheme/set` undefined)
;; and would otherwise hide clojure.core/set on the mint fixpoint.
(and c (var-cell-defined? c) c))
;; a :refer'd name resolves to its source ns
(let ((ref (chez-resolve-refer (chez-actx-cns ctx) nm)))
(and ref (var-cell-lookup ref nm)))
(var-cell-lookup "clojure.core" nm)))))
;; Runtime macros: a defmacro is emitted into the prelude as a
;; def-var! of its cross-compiled expander fn plus (mark-macro! ns name), so the
;; var cell is flagged a macro (rt.ss var-macro-table). form-macro? checks the
;; flag; form-expand-1 applies the expander to the unevaluated arg forms (the rest
;; of the list), and the analyzer re-analyzes the returned form.
(define (hc-macro? ctx sym)
(macro-var? (hc-resolve-cell ctx sym)))
;; Clojure parity: a macro expansion inherits the call form's source position, so
;; errors/traces in macro-generated code point at the macro call site. Carry it
;; onto the top of a LIST expansion (code) that has none of its own — merged under
;; any meta the macro set, leaving collection literals (runtime data) alone. The
;; recursion through analyze re-expands inner macros, so each level's top form
;; picks up the position the same way (as the reference compiler does).
(define (hc-propagate-pos src dst)
(if (and (cseq? dst) (cseq-list? dst))
(let ((sp (hc-form-position src))
(dm (jolt-meta dst)))
(if (and (pmap? sp)
(or (jolt-nil? dm) (jolt-nil? (jolt-get dm hc-kw-line))))
(jolt-with-meta dst
(if (pmap? dm)
(pmap-fold-fwd sp (lambda (k v acc) (jolt-assoc1 acc k v)) dm)
sp))
dst))
dst))
;; A set literal reads as the tagged set-form {:jolt/type :jolt/set :value [...]}
;; for the analyzer, but a macro must see a real set value (Clojure parity, so
;; (set? arg) / seq / conj work — hiccup's compiler does this). Convert a set-form
;; argument to a set; elements stay as read (a deeply-nested set literal inside
;; another form is rarer and left for the analyzer).
(define (hc-macro-arg x)
(if (rdr-set-form? x)
(let ((items (jolt-get x rdr-kw-value)))
(let loop ((i 0) (s empty-pset))
(if (fx>=? i (pvec-count items)) s
(loop (fx+ i 1) (pset-conj s (pvec-nth-d items i jolt-nil))))))
x))
;; &form and &env are bound (as dynamic vars) around the expander call, so a
;; macro body can read the call form / lexical env without changing the calling
;; convention. The analyzer passes amp-env (the in-scope locals); macroexpand-1
;; has none, so it defaults to {}.
(define hc-amp-form-cell (declare-var! "clojure.core" "&form"))
(define hc-amp-env-cell (declare-var! "clojure.core" "&env"))
(define (hc-expand-1 ctx form . maybe-env)
(let* ((items (seq->list form))
(head (car items))
(args (map hc-macro-arg (cdr items)))
(expander (var-cell-root (hc-resolve-cell ctx head)))
(amp-env (if (pair? maybe-env) (car maybe-env) (jolt-hash-map))))
(dynamic-wind
(lambda () (jolt-push-thread-bindings
(jolt-hash-map hc-amp-form-cell form hc-amp-env-cell amp-env)))
(lambda () (hc-propagate-pos form (apply jolt-invoke expander args)))
(lambda () (jolt-pop-thread-bindings)))))
;; Classify a global (non-local) symbol reference against the var registry:
;; {:kind :var :ns NS :name NAME} — a defined var (compile ns / clojure.core)
;; {:kind :unresolved :name NAME} — not found (late-bind -> var-ref @ compile ns;
;; a qualified one -> host-static in the analyzer)
;; No :host branch: there is no separate native-op env — the hot
;; clojure.core primitives (+,-,map,...) are declared in clojure.core below so
;; they classify as :var and the emitter's native-op path lowers them.
;; A var's declared numeric return (^double/^long on its name) -> :double/:long,
;; read from its meta. Lets jolt.passes.numeric type a call to it.
(define (hc-cell-num-ret cell)
(let ((m (and cell (hashtable-ref var-meta-table cell #f))))
(and m (let* ((t (jolt-get m hc-kw-tag)) ; ^double/^long is a symbol; ^"double" a string
(s (cond ((symbol-t? t) (symbol-t-name t)) ((string? t) t) (else #f))))
(cond ((equal? s "double") hc-kw-double)
((equal? s "long") hc-kw-long)
(else #f))))))
;; A slash-free dotted symbol whose final segment is Capitalized is a class
;; reference (java.util.Map, clojure.lang.Named) — Clojure has no such vars. With
;; no JVM classes, jolt models a class as its name string, so the symbol
;; self-evaluates to that string (the analyzer emits a :const). This lets a lib
;; extend a protocol to / instance?-check a host class jolt has no shim for.
(define (hc-fq-class-name? nm)
(let ((n (string-length nm)))
(let loop ((i (fx- n 1)))
(cond ((fx<? i 0) #f)
((char=? (string-ref nm i) #\.)
(and (fx<? (fx+ i 1) n) (char-upper-case? (string-ref nm (fx+ i 1)))))
(else (loop (fx- i 1)))))))
(define (hc-resolve-global ctx sym)
(let* ((nm (symbol-t-name sym))
(cell (hc-resolve-cell ctx sym)))
(if (and cell (var-cell-defined? cell))
(let ((base (jolt-hash-map hc-kw-kind hc-kw-var
hc-kw-ns (var-cell-ns cell)
hc-kw-name (var-cell-name cell)))
(nr (hc-cell-num-ret cell)))
(if nr (jolt-assoc base hc-kw-num-ret nr) base))
(cond
;; java.util.Map / clojure.lang.Named — a dotted class name.
((hc-fq-class-name? nm) (jolt-hash-map hc-kw-kind hc-kw-class hc-kw-name nm))
;; a bare Capitalized name that names a registered host class — an
;; imported short name (`(:import [java.time ZonedDateTime])` then
;; `(. ZonedDateTime parse s)`). Only when otherwise unresolved, so a
;; same-named var still wins.
((and (fx>? (string-length nm) 0) (char-upper-case? (string-ref nm 0))
(hashtable-ref class-statics-tbl nm #f))
(jolt-hash-map hc-kw-kind hc-kw-class hc-kw-name nm))
(else (jolt-hash-map hc-kw-kind hc-kw-unresolved hc-kw-name nm))))))
(define (hc-intern! ctx ns-name nm) (declare-var! ns-name nm) jolt-nil)
;; --- syntax-quote lowering ---------------------------------------------------
;; Lowers a `form
;; to CONSTRUCTION CODE — Chez reader forms calling __sqcat/__sqvec/__sqmap/
;; __sqset/__sq1 + quote — that the analyzer re-analyzes, so a backtick compiles
;; with zero runtime cost (read -> macroexpand -> compile). Symbols resolve to
;; clojure.core / the compile ns; a foo# auto-gensym is stable within one `.
(define hc-special-symbols
'("quote" "syntax-quote" "unquote" "unquote-splicing" "do" "if" "def"
"defmacro" "fn*" "let*" "loop*" "recur" "throw" "try" "set!" "var"
"new" "."))
(define (hc-special-symbol? nm) (and (member nm hc-special-symbols) #t))
(define hc-sq-gensym-counter 0)
(define (hc-sq-gensym base)
(set! hc-sq-gensym-counter (+ hc-sq-gensym-counter 1))
(jolt-symbol #f (string-append base "__" (number->string hc-sq-gensym-counter) "__auto")))
(define (hc-sym nm) (jolt-symbol #f nm))
;; is `x` a non-empty list FORM whose head is the unqualified symbol `nm`?
;; Detect a (unquote …) / (unquote-splicing …) form in a syntax-quote template.
;; Any seq counts, not just a proper list: a macro that builds the template with
;; map/for (e.g. deftype's rewrite-set) yields a LAZY seq, and its ~unquotes must
;; still be recognized.
;; head symbol matches name nm, bare or clojure.core-qualified — the reader
;; produces clojure.core/unquote(-splicing) for ~/~@ (JVM parity), and this is
;; only used to spot those heads in syntax-quote templates.
(define (hc-head-is? x nm)
(and (cseq? x)
(let ((h (seq-first x)))
(and (symbol-t? h) (string=? (symbol-t-name h) nm)
(let ((ns (hc-sym-ns h)))
(or (jolt-nil? ns) (and (string? ns) (string=? ns "clojure.core"))))))))
(define (hc-second x) (seq-first (jolt-seq (seq-more x))))
(define (hc-sq-symbol ctx form gsmap)
(let ((sns (hc-sym-ns form)) (nm (symbol-t-name form)))
(if (jolt-nil? sns)
(cond
;; foo# -> a stable per-` auto-gensym
((and (> (string-length nm) 0)
(char=? (string-ref nm (- (string-length nm) 1)) #\#))
(or (hashtable-ref gsmap nm #f)
(let ((g (hc-sq-gensym (substring nm 0 (- (string-length nm) 1)))))
(hashtable-set! gsmap nm g) g)))
((hc-special-symbol? nm) form) ; special form: leave bare
((hc-interop-head? nm) form) ; interop (.method / Class. / .-field): bare
;; a fully-qualified class name (java.util.Map, clojure.lang.ILookup) is
;; a class token, not a var to namespace-qualify — leave it bare, as
;; Clojure's syntax-quote resolves it to the class.
((hc-fq-class-name? nm) form)
;; the compile ns's OWN def shadows clojure.core — a name the ns
;; excluded and redefined (e.g. core.logic's `==` after
;; (:refer-clojure :exclude [==])), or any ns-local redefinition.
;; Referred names live in a separate table, so this only hits a real
;; local intern, matching how the analyzer resolves the bare symbol.
((var-cell-lookup (chez-actx-cns ctx) nm) (jolt-symbol (chez-actx-cns ctx) nm))
;; a name the compile ns excluded from clojure.core (:refer-clojure
;; :exclude) is not clojure.core/nm even before the ns defines its own —
;; qualify to the compile ns, like Clojure (core.logic.fd's `==`).
((chez-core-excluded? (chez-actx-cns ctx) nm) (jolt-symbol (chez-actx-cns ctx) nm))
((var-cell-lookup "clojure.core" nm) (jolt-symbol "clojure.core" nm))
;; a name referred into the compile ns (:require :refer / :use :only)
;; qualifies to its SOURCE ns, not the compile ns — so a macro that
;; syntax-quotes a referred var (e.g. clojure.tools.logging/spy using
;; clojure.pprint's pprint) expands to the real var.
((chez-resolve-refer (chez-actx-cns ctx) nm)
=> (lambda (target) (jolt-symbol target nm)))
(else (jolt-symbol (chez-actx-cns ctx) nm))) ; else: qualify to compile ns
;; qualified: if the ns part is an :as alias in the compile ns, resolve it
;; to the target namespace — Clojure resolves the alias part of a qualified
;; symbol in syntax-quote, so a macro's `impl/foo` expands to its real
;; (clojure.tools.logging.impl/foo) name and stays unambiguous even when
;; another loaded ns shares the alias's short name. Otherwise
;; leave it as written (a real ns or an interop class token).
(let ((target (chez-resolve-alias (chez-actx-cns ctx) sns)))
(if target (jolt-symbol target nm) form)))))
(define (hc-sq-lower ctx form gsmap)
(cond
((hc-head-is? form "unquote") (hc-second form))
((hc-head-is? form "unquote-splicing")
(jolt-throw (jolt-ex-info "~@ used outside of a list or vector in syntax-quote"
(jolt-hash-map))))
((hc-literal? form) form)
((symbol-t? form) (jolt-list (hc-sym "quote") (hc-sq-symbol ctx form gsmap)))
((hc-list? form)
(apply jolt-list (hc-sym "__sqcat")
(map (lambda (it) (hc-sq-lower-part ctx it gsmap)) (seq->list form))))
((hc-vec? form)
(apply jolt-list (hc-sym "__sqvec")
(map (lambda (it) (hc-sq-lower-part ctx it gsmap)) (seq->list form))))
((hc-set? form)
(apply jolt-list (hc-sym "__sqset")
(map (lambda (it) (hc-sq-lower-part ctx it gsmap)) (seq->list (hc-set-items form)))))
((hc-map? form)
(apply jolt-list (hc-sym "__sqmap")
(let loop ((pairs (seq->list (hc-map-pairs form))) (acc '()))
(if (null? pairs) (reverse acc)
(let ((p (seq->list (car pairs))))
(loop (cdr pairs)
(cons (hc-sq-lower ctx (cadr p) gsmap)
(cons (hc-sq-lower ctx (car p) gsmap) acc))))))))
(else (jolt-list (hc-sym "quote") form)))) ; tagged (char/regex/...) etc.
;; a list/vector/set element: a ~@ splice passes through (its seq is spliced by
;; __sqcat), any other item is wrapped (__sq1 <lowered>) so __sqcat flattens it.
(define (hc-sq-lower-part ctx item gsmap)
(if (hc-head-is? item "unquote-splicing")
(hc-second item)
(jolt-list (hc-sym "__sq1") (hc-sq-lower ctx item gsmap))))
(define (hc-syntax-quote-lower ctx inner)
(hc-sq-lower ctx inner (make-hashtable string-hash string=?)))
;; a ^Type param hint: name is the tag (a symbol, sometimes a string). Resolve it
;; against the record registry (records.ss) so the inference seeds the param as
;; that record — the open-world / cross-ns path where no caller type is inferred.
(define (hc-record-tag-name name)
(cond ((symbol-t? name) (symbol-t-name name))
((string? name) name)
(else #f)))
(define (hc-record-type? ctx name)
(let ((nm (hc-record-tag-name name)))
(if (and nm (chez-find-ctor-key nm (chez-current-ns))) #t #f)))
(define (hc-record-ctor-key ctx name)
(let ((nm (hc-record-tag-name name)))
(or (and nm (chez-find-ctor-key nm (chez-current-ns))) jolt-nil)))
;; The fully-qualified deftype tag ("ns.Name") IFF `class` names a deftype DEFINED
;; in the ctx's compile ns — the analyzer qualifies a bare (Name. …) to it, so a
;; deftype doesn't shadow a same-named built-in host class in an unrelated ns
;; (rewrite-clj imports java.io.PushbackReader; tools.reader defines its own). Strict:
;; only this ns's own def (the preferred shape key) counts, not the global
;; simple-name fallback, so a ns that merely uses the built-in resolves nil.
(define (hc-deftype-ctor-class ctx class)
(let* ((nm (jolt-str-render-one class))
(cns (hc-current-ns ctx))
(key (string-append cns "/->" nm)))
(if (hashtable-ref chez-record-shapes-tbl key #f)
(string-append cns "." nm)
jolt-nil)))
;; record + protocol-method shapes for the inference, from the runtime registries
;; (records.ss) populated as deftype/defprotocol forms load.
(define (hc-record-shapes ctx) (chez-record-shapes-map))
(define (hc-protocol-methods ctx) (chez-protocol-methods-map))
;; Optimization gate. Off for ordinary runs (open world, redefinition); `jolt
;; build` flips it on during app emission for release/optimized modes (closed
;; world), turning on the inference + flatten + scalar-replace passes.
(define hc-optimize? #f)
(define (set-optimize! on) (set! hc-optimize? on))
(define (hc-inline-enabled? ctx) hc-optimize?)
;; Inline-body registry: jolt.passes stashes an inline-eligible defn's
;; {:params :body :nhints :ret} here (keyed ns/name) as its form is optimized;
;; jolt.passes.inline fetches it to splice the body at a call site. The stash is an
;; opaque jolt value to the host — IR maps round-tripping through the table.
(define inline-stash-table (make-hashtable string-hash string=?))
(define (hc-stash-inline! ctx ns-name nm m)
(hashtable-set! inline-stash-table (string-append ns-name "/" nm) m) jolt-nil)
(define (hc-inline-ir ctx ns-name nm)
(or (hashtable-ref inline-stash-table (string-append ns-name "/" nm) #f) jolt-nil))
;; --- declare the hot clojure.core primitives so resolve-global sees them ------
;; (mirrors backend_scheme.clj native-ops keys — the emitter lowers these inline,
;; so the declared cell's unbound root is never deref'd.)
(for-each (lambda (nm) (declare-var! "clojure.core" nm))
'("+" "-" "*" "/" "<" ">" "<=" ">=" "=" "inc" "dec" "not" "min" "max"
"mod" "rem" "quot" "vector" "hash-map" "hash-set" "conj" "get" "nth" "count"
"assoc" "dissoc" "contains?" "empty?" "peek" "pop" "first" "rest" "next" "seq"
"cons" "list" "reverse" "last" "map" "filter" "remove" "reduce" "into" "concat"
"apply" "range" "take" "drop" "keys" "vals" "even?" "odd?" "pos?" "neg?"
"zero?" "identity" "ex-info"))
;; --- install: bind the contract into the jolt.host namespace -----------------
(define (hc-install!)
(def-var! "jolt.host" "form-sym?" hc-sym?)
(def-var! "jolt.host" "form-sym-name" hc-sym-name)
(def-var! "jolt.host" "form-sym-ns" hc-sym-ns)
(def-var! "jolt.host" "form-sym-meta" hc-sym-meta)
(def-var! "jolt.host" "form-coll-meta" hc-coll-meta)
(def-var! "jolt.host" "form-list?" hc-list?)
(def-var! "jolt.host" "form-vec?" hc-vec?)
(def-var! "jolt.host" "form-map?" hc-map?)
(def-var! "jolt.host" "form-set?" hc-set?)
(def-var! "jolt.host" "form-char?" hc-char?)
(def-var! "jolt.host" "form-char-code" hc-char-code)
(def-var! "jolt.host" "form-literal?" hc-literal?)
(def-var! "jolt.host" "form-keyword?" hc-keyword?)
(def-var! "jolt.host" "form-regex?" hc-regex?)
(def-var! "jolt.host" "form-inst?" hc-inst?)
(def-var! "jolt.host" "form-uuid?" hc-uuid?)
(def-var! "jolt.host" "form-ns-value?" hc-ns-value?)
(def-var! "jolt.host" "form-ns-value-name" hc-ns-value-name)
(def-var! "jolt.host" "form-var-value?" hc-var-value?)
(def-var! "jolt.host" "form-var-value-ns" hc-var-value-ns)
(def-var! "jolt.host" "form-var-value-name" hc-var-value-name)
(def-var! "jolt.host" "unchecked-math?" hc-unchecked-math?)
(def-var! "jolt.host" "form-bigdec?" hc-bigdec?)
(def-var! "jolt.host" "form-bigdec-source" hc-bigdec-source)
(def-var! "jolt.host" "form-elements" hc-elements)
(def-var! "jolt.host" "form-vec-items" hc-vec-items)
(def-var! "jolt.host" "form-set-items" hc-set-items)
(def-var! "jolt.host" "form-map-pairs" hc-map-pairs)
(def-var! "jolt.host" "form-regex-source" hc-regex-source)
(def-var! "jolt.host" "form-inst-source" hc-inst-source)
(def-var! "jolt.host" "form-uuid-source" hc-uuid-source)
(def-var! "jolt.host" "form-position" hc-form-position)
(def-var! "jolt.host" "form-special?" hc-special?)
(def-var! "jolt.host" "compile-ns" hc-current-ns)
(def-var! "jolt.host" "late-bind?" hc-late-bind?)
(def-var! "jolt.host" "form-macro?" hc-macro?)
(def-var! "jolt.host" "form-expand-1" hc-expand-1)
(def-var! "jolt.host" "resolve-global" hc-resolve-global)
(def-var! "jolt.host" "host-intern!" hc-intern!)
(def-var! "jolt.host" "form-syntax-quote-lower" hc-syntax-quote-lower)
(def-var! "jolt.host" "record-type?" hc-record-type?)
(def-var! "jolt.host" "record-ctor-key" hc-record-ctor-key)
(def-var! "jolt.host" "deftype-ctor-class" hc-deftype-ctor-class)
(def-var! "jolt.host" "record-shapes" hc-record-shapes)
(def-var! "jolt.host" "protocol-methods" hc-protocol-methods)
(def-var! "jolt.host" "inline-enabled?" hc-inline-enabled?)
(def-var! "jolt.host" "inline-ir" hc-inline-ir)
(def-var! "jolt.host" "stash-inline!" hc-stash-inline!))
(hc-install!)

178
host/chez/host-table.ss Normal file
View file

@ -0,0 +1,178 @@
;; host tables + sorted collections — the jolt.host value primitives and the
;; 25-sorted tier's runtime.
;;
;; jolt.host/tagged-table + ref-put! + ref-get back the whole sorted tier
;; (sorted-map/sorted-set/subseq/rsubseq) AND every overlay fn that calls
;; (sorted? x) — empty, ifn?, reversible?, map?, set?, coll?. This provides:
;; 1. tagged-table / ref-put! / ref-get over a Chez mutable tagged-table type
;; (a string-keyed hashtable wrapped in an `htable` record), def-var!'d into
;; the jolt.host ns. The sorted tier (25-sorted.clj) mints its wrapper with
;; these — a red-black tree + :ops table travel inside the htable.
;; 2. a sorted-coll arm on the collection dispatchers, set!-extended the same
;; way records.ss extends them for jrec: each op routes through the value's
;; own :ops table (the dispatch pattern). first/rest/
;; next/last fall out free once jolt-seq has a sorted arm (they seq first).
;;
;; Loaded LAST (after records.ss / transients.ss / natives-meta.ss): it wraps the
;; jrec-extended dispatchers + value-host-tags, delegating to the captured prior.
;; --- jolt.host primitives ----------------------------------------------------
;; A tagged-table: a string-keyed hashtable (keyword field -> value). Keyword
;; keys collapse to their ns/name string so interning isn't relied on.
(define-record-type htable (fields (immutable h)) (nongenerative chez-htable-v1))
(define (kw->key k)
(let ((ns (keyword-t-ns k)))
(if (and ns (not (jolt-nil? ns))) (string-append ns "/" (keyword-t-name k)) (keyword-t-name k))))
(define (jolt-tagged-table tag)
(let ((h (make-hashtable string-hash string=?)))
(hashtable-set! h "jolt/type" tag)
(make-htable h)))
;; ref-put! threads the table back; a nil value REMOVES the key. Errors on a
;; non-htable so the atom-watch / volatile uses (which pass a different ref type
;; and have no table yet) stay a crash rather than silently diverging.
(define (jolt-ref-put! t k v)
(unless (htable? t) (error #f "ref-put!: not a host table" t))
(if (jolt-nil? v)
(hashtable-delete! (htable-h t) (kw->key k))
(hashtable-set! (htable-h t) (kw->key k) v))
t)
(define (jolt-ref-get t k)
(if (htable? t) (hashtable-ref (htable-h t) (kw->key k) jolt-nil) jolt-nil))
(def-var! "jolt.host" "tagged-table" jolt-tagged-table)
(def-var! "jolt.host" "ref-put!" jolt-ref-put!)
(def-var! "jolt.host" "ref-get" jolt-ref-get)
;; map-entry constructor: a 2-elem entry-flagged pvec (map-entry? true, vector?
;; false), so sorted-map seq/first produce real map entries that key/val accept.
(def-var! "jolt.host" "map-entry" make-map-entry)
;; --- sorted-coll recognition + ops access ------------------------------------
(define kw-jtype (keyword "jolt" "type"))
(define kw-sorted-map (keyword "jolt" "sorted-map"))
(define kw-sorted-set (keyword "jolt" "sorted-set"))
(define kw-ops (keyword #f "ops"))
(define kw-op-count (keyword #f "count"))
(define kw-op-seq (keyword #f "seq"))
(define kw-op-get (keyword #f "get"))
(define kw-op-contains (keyword #f "contains"))
(define kw-op-assoc (keyword #f "assoc"))
(define kw-op-dissoc (keyword #f "dissoc"))
(define kw-op-conj (keyword #f "conj"))
(define kw-op-disj (keyword #f "disj"))
(define (htable-sorted-map? x) (and (htable? x) (jolt=2 (jolt-ref-get x kw-jtype) kw-sorted-map)))
(define (htable-sorted-set? x) (and (htable? x) (jolt=2 (jolt-ref-get x kw-jtype) kw-sorted-set)))
(define (htable-sorted? x) (or (htable-sorted-map? x) (htable-sorted-set? x)))
;; the op fn for `op-kw` from the value's attached :ops map, then invoke it on sc.
(define (sc-op sc op-kw) (jolt-get (jolt-ref-get sc kw-ops) op-kw jolt-nil))
(define (sc-call sc op-kw . args) (apply jolt-invoke (sc-op sc op-kw) sc args))
;; --- extend the collection dispatchers with a sorted arm ---------------------
(define %h-seq jolt-seq)
(set! jolt-seq (lambda (x) (if (htable-sorted? x) (sc-call x kw-op-seq) (%h-seq x))))
(define %h-count jolt-count)
(set! jolt-count (lambda (coll) (if (htable-sorted? coll) (sc-call coll kw-op-count) (%h-count coll))))
(register-get-arm! htable-sorted? (lambda (coll k d) (sc-call coll kw-op-get k d)))
(define %h-contains? jolt-contains?)
(set! jolt-contains? (lambda (coll k)
(if (htable-sorted? coll) (if (jolt-truthy? (sc-call coll kw-op-contains k)) #t #f) (%h-contains? coll k))))
(define %h-assoc1 jolt-assoc1)
(set! jolt-assoc1 (lambda (coll k v)
(if (htable-sorted-map? coll) (sc-call coll kw-op-assoc (jolt-vector k v)) (%h-assoc1 coll k v))))
(define %h-dissoc jolt-dissoc)
(set! jolt-dissoc (lambda (coll . ks)
(if (htable-sorted-map? coll) (sc-call coll kw-op-dissoc (apply jolt-vector ks)) (apply %h-dissoc coll ks))))
(define %h-conj1 jolt-conj1)
(set! jolt-conj1 (lambda (coll x)
(if (htable-sorted? coll) (sc-call coll kw-op-conj (jolt-vector x)) (%h-conj1 coll x))))
(define %h-disj jolt-disj)
(set! jolt-disj (lambda (s . xs)
(if (htable-sorted-set? s) (sc-call s kw-op-disj (apply jolt-vector xs)) (apply %h-disj s xs))))
(def-var! "clojure.core" "disj" jolt-disj)
(define %h-empty? jolt-empty?)
(set! jolt-empty? (lambda (coll) (if (htable-sorted? coll) (zero? (sc-call coll kw-op-count)) (%h-empty? coll))))
(define %h-keys jolt-keys)
(set! jolt-keys (lambda (m)
(if (htable-sorted-map? m)
(list->cseq (map (lambda (e) (jolt-nth e 0)) (seq->list (sc-call m kw-op-seq))))
(%h-keys m))))
(define %h-vals jolt-vals)
(set! jolt-vals (lambda (m)
(if (htable-sorted-map? m)
(list->cseq (map (lambda (e) (jolt-nth e 1)) (seq->list (sc-call m kw-op-seq))))
(%h-vals m))))
;; sorted colls are collections (callable as fns via jolt-invoke, conj-able).
(define %h-coll? jolt-coll?)
(set! jolt-coll? (lambda (x) (or (htable-sorted? x) (%h-coll? x))))
;; public predicates: a sorted-map is map?, a sorted-set is set?, both coll?.
;; predicates.ss/records.ss def-var!'d a snapshot, so re-def-var! after set!.
(define %h-map? jolt-map?)
(set! jolt-map? (lambda (x) (or (htable-sorted-map? x) (%h-map? x))))
(def-var! "clojure.core" "map?" jolt-map?)
(define %h-set? jolt-set?)
(set! jolt-set? (lambda (x) (or (htable-sorted-set? x) (%h-set? x))))
(def-var! "clojure.core" "set?" jolt-set?)
(def-var! "clojure.core" "coll?" (lambda (x) (or (htable-sorted? x) (jrec-collection? x) (jolt-coll-pred? x))))
;; --- equality / hash ---------------------------------------------------------
;; A sorted coll canonicalizes like its unordered counterpart:
;; a sorted-map equals ANY map (hash or sorted) with the same entries, a
;; sorted-set ANY set with the same elements — the comparator is irrelevant to =.
;; Convert to the plain persistent coll and delegate to the prior jolt=2 / hash.
;; (htable-sorted? short-circuits on a non-htable BEFORE any jolt=2, so extending
;; jolt=2 here doesn't recurse: the inner tag compare gets two keywords.)
(define (sorted-map->pmap sc)
(fold-left (lambda (m e) (pmap-assoc m (jolt-nth e 0) (jolt-nth e 1)))
empty-pmap (seq->list (sc-call sc kw-op-seq))))
(define (sorted-set->pset sc)
(fold-left (lambda (s x) (pset-conj s x)) empty-pset (seq->list (sc-call sc kw-op-seq))))
(define (sorted->plain x) (if (htable-sorted-map? x) (sorted-map->pmap x) (sorted-set->pset x)))
;; a sorted coll compares as its plain equivalent: normalize and re-dispatch (the
;; normalized values aren't sorted, so this arm won't re-match — the base compares).
(register-eq-arm! (lambda (a b) (or (htable-sorted? a) (htable-sorted? b)))
(lambda (a b) (jolt=2 (if (htable-sorted? a) (sorted->plain a) a)
(if (htable-sorted? b) (sorted->plain b) b))))
;; a sorted coll hashes as its plain equivalent (jolt-hash recurses through the base).
(register-hash-arm! htable-sorted? (lambda (x) (jolt-hash (sorted->plain x))))
;; --- printing ----------------------------------------------------------------
;; sorted colls render in SORTED order (the value's :seq), not HAMT order — and
;; a sorted-map prints "{k v, k v}" (", " between pairs),
;; NOT the space-only form the unordered pmap arm uses.
(define (sorted-map-render sc render)
(string-append "{"
(let loop ((es (seq->list (sc-call sc kw-op-seq))) (first #t) (acc ""))
(if (null? es) acc
(loop (cdr es) #f
(string-append acc (if first "" ", ")
(render (jolt-nth (car es) 0)) " " (render (jolt-nth (car es) 1))))))
"}"))
(define (sorted-set-render sc render)
(string-append "#{" (jolt-str-join (map render (seq->list (sc-call sc kw-op-seq)))) "}"))
(define (sorted-render x render)
(if (htable-sorted-map? x) (sorted-map-render x render) (sorted-set-render x render)))
;; sorted colls render in :seq order via the calling printer (str vs readable).
(register-pr-readable-arm! htable-sorted? (lambda (x) (sorted-render x jolt-pr-readable)))
(register-pr-str-arm! htable-sorted? (lambda (x) (sorted-render x jolt-pr-str)))
(register-str-render! htable-sorted? (lambda (x) (sorted-render x jolt-str-render-one)))
;; --- protocol dispatch over builtins (extend-protocol Map/Set on sorted) ------
;; value-host-tags (records.ss) drives extend-protocol on host values; a
;; sorted-map must answer to "Map", a sorted-set to "Set"/"Collection".
(define %h-value-host-tags value-host-tags)
(set! value-host-tags (lambda (obj)
(cond
((htable-sorted-map? obj) '("PersistentTreeMap" "Sorted" "IPersistentMap" "Associative"
"Map" "java.util.Map" "IPersistentCollection" "Object"))
((htable-sorted-set? obj) '("PersistentTreeSet" "Sorted" "IPersistentSet"
"Set" "java.util.Set" "Collection" "IPersistentCollection" "Object"))
(else (%h-value-host-tags obj)))))
;; (class e) on a throwable tagged-table (a library's ex-info envelope carrying a
;; JVM :class, e.g. jolt-lang/http-client's UnknownHostException) reads that
;; class name, so clojure.test's (thrown? Class …) / (= Class (class e)) match.
;; an htable carrying a string "class" entry reports it (a host-object class mirror).
(register-class-arm! (lambda (x) (and (htable? x) (string? (hashtable-ref (htable-h x) "class" #f))))
(lambda (x) (hashtable-ref (htable-h x) "class" #f)))

315
host/chez/java/async.ss Normal file
View file

@ -0,0 +1,315 @@
;; async.ss — clojure.core.async channel primitives on real OS threads.
;;
;; A `go` block is an OS thread and a channel is a Chez mutex+condition blocking
;; queue: <! / >! are the blocking <!! / >!! (they "park" by blocking the thread),
;; and work ANYWHERE — no CPS transform, no go-only restriction. Real parallelism,
;; shared heap. This is a superset of the JVM model: it has no fixed go-block
;; thread pool, no MAX-QUEUE-SIZE on pending ops, and parking ops are legal outside
;; a go block. One OS thread per go block (fine for typical use).
;;
;; Channel: an unbuffered channel is a rendezvous (the putter blocks until its
;; value is taken); a buffered (chan n) put blocks only when full; dropping/sliding
;; buffers never block the putter. A transducer is applied on the put side; an
;; optional ex-handler catches a throw from the transducer step.
;;
;; This file provides the primitives; the higher-level dataflow API (mult, mix,
;; pub/sub, pipeline, map, merge, reduce, …) is a Clojure overlay over them.
;; go/go-loop/thread are macros (mark-macro!) expanding to go-spawn. Loaded after
;; concurrency.ss (reuses ms->duration). Requires a threaded Chez build.
;; --- buffers ----------------------------------------------------------------
(define-record-type async-buffer (fields n kind) (nongenerative async-buffer-v1))
(define (jolt-async-buffer n) (make-async-buffer n 'fixed))
(define (jolt-async-dropping-buffer n) (make-async-buffer n 'dropping))
(define (jolt-async-sliding-buffer n) (make-async-buffer n 'sliding))
(define (jolt-async-unblocking-buffer? b)
(if (and (async-buffer? b) (memq (async-buffer-kind b) '(dropping sliding promise))) #t #f))
;; --- channels ---------------------------------------------------------------
;; items: an amortized-O(1) FIFO held as a mutable #(out in len) — `out` is the
;; front (pop from its head), `in` holds pushed entries reversed onto it, `len` is
;; the count (an append-to-a-list FIFO is O(n) per push and O(n) to measure).
;; Each entry is (value . box); box is #f for a buffered value or a 1-slot vector
;; for an unbuffered rendezvous put (set #t when taken, waking the putter).
;; cap 0 + kind 'unbuffered = rendezvous; cap>0 with kind fixed/dropping/sliding.
;; takew counts threads parked in a blocking take (so a non-blocking offer! to an
;; unbuffered channel can tell a taker is waiting). xrf is the transducer reducing
;; fn (or #f); exh the ex-handler (or #f).
(define-record-type async-chan
(fields mu cv (mutable items) cap kind (mutable closed?) (mutable xrf) (mutable takew) exh)
(nongenerative async-chan-v2))
(define (ac-qnew) (vector '() '() 0))
(define (ac-qlen ch) (vector-ref (async-chan-items ch) 2))
(define (ac-qempty? ch) (fx=? 0 (vector-ref (async-chan-items ch) 2)))
(define (ac-qpush! ch entry)
(let ((q (async-chan-items ch)))
(vector-set! q 1 (cons entry (vector-ref q 1)))
(vector-set! q 2 (fx+ 1 (vector-ref q 2)))))
(define (ac-qfront! q) ; ensure `out` is non-empty: out := reverse in
(when (null? (vector-ref q 0))
(vector-set! q 0 (reverse (vector-ref q 1)))
(vector-set! q 1 '())))
(define (ac-qpop! ch)
(let ((q (async-chan-items ch)))
(ac-qfront! q)
(let ((out (vector-ref q 0)))
(vector-set! q 0 (cdr out))
(vector-set! q 2 (fx- (vector-ref q 2) 1))
(car out))))
(define (ac-qdrop-oldest! ch)
(let ((q (async-chan-items ch)))
(ac-qfront! q)
(vector-set! q 0 (cdr (vector-ref q 0)))
(vector-set! q 2 (fx- (vector-ref q 2) 1))))
;; enqueue honoring the buffer kind (used by the transducer step + buffered puts).
(define (ac-buf-give! ch v)
(case (async-chan-kind ch)
((dropping) (when (< (ac-qlen ch) (async-chan-cap ch)) (ac-qpush! ch (cons v #f))))
((sliding) (when (>= (ac-qlen ch) (async-chan-cap ch)) (ac-qdrop-oldest! ch))
(ac-qpush! ch (cons v #f)))
(else (ac-qpush! ch (cons v #f)))) ; fixed: caller ensured room
(condition-broadcast (async-chan-cv ch)))
;; A transducer is a jolt fn (xform); (xform add-rf) yields the channel's reducing
;; fn. add-rf: 0-arg init, 1-arg completion, 2-arg step (enqueue the output). A
;; `reduced` step result closes the channel.
(define (ac-make-add-rf ch)
(lambda args
(cond ((null? args) ch) ; init
((null? (cdr args)) (car args)) ; completion
(else (ac-buf-give! ch (cadr args)) (car args))))) ; step
;; run the transducer step (or completion) guarded by the channel's ex-handler:
;; if the xform throws and exh returns non-nil, that value is added to the buffer.
(define (ac-xrf-apply ch . v)
(let ((xrf (async-chan-xrf ch)) (exh (async-chan-exh ch)))
(guard (e (#t (if exh
(let ((else (jolt-invoke exh e)))
(unless (jolt-nil? else) (ac-buf-give! ch else))
(async-chan-xrf ch)) ; treat as non-reduced
(raise e))))
(apply jolt-invoke xrf ch v))))
(define (ac-make cap kind xrf) (make-async-chan (make-mutex) (make-condition) (ac-qnew) cap kind #f xrf 0 #f))
(define (ac-make/exh cap kind exh) (make-async-chan (make-mutex) (make-condition) (ac-qnew) cap kind #f #f 0 exh))
;; (chan) | (chan n) | (chan buf) | (chan n|buf xform) | (chan n|buf xform exh)
(define (jolt-async-chan . args)
(let ((buf (if (pair? args) (car args) jolt-nil))
(xform (if (and (pair? args) (pair? (cdr args))) (cadr args) jolt-nil))
(exh (if (and (pair? args) (pair? (cdr args)) (pair? (cddr args))) (caddr args) jolt-nil)))
(let-values (((cap kind)
(cond ((async-buffer? buf) (values (async-buffer-n buf) (async-buffer-kind buf)))
((and (number? buf) (> buf 0)) (values buf 'fixed))
(else (values 0 'unbuffered)))))
(let ((ch (ac-make/exh cap kind (if (jolt-nil? exh) #f exh))))
(unless (jolt-nil? xform)
(async-chan-xrf-set! ch (jolt-invoke xform (ac-make-add-rf ch))))
ch))))
;; close! (idempotent): mark closed, flush a stateful transducer's completion, and
;; wake everyone. ac-close! assumes the lock is held; the public form takes it.
(define (ac-close! ch)
(unless (async-chan-closed? ch)
(async-chan-closed?-set! ch #t)
(when (async-chan-xrf ch) (guard (e (#t #f)) (ac-xrf-apply ch)))
(condition-broadcast (async-chan-cv ch)))
jolt-nil)
(define (jolt-async-close! ch) (with-mutex (async-chan-mu ch) (ac-close! ch)))
;; >! / >!! — put, blocking. false if closed; nil may not be put. With a
;; transducer the value is run through it (one put -> zero or more channel values);
;; a `reduced` result closes the channel.
(define (jolt-async-give ch v)
(when (jolt-nil? v) (jolt-throw (jolt-host-throwable "java.lang.IllegalArgumentException" "Can't put nil on a channel")))
(with-mutex (async-chan-mu ch)
(cond
((async-chan-closed? ch) #f)
((async-chan-xrf ch)
(let ((r (ac-xrf-apply ch v)))
(when (jolt-reduced? r) (ac-close! ch))
#t))
(else
(case (async-chan-kind ch)
((dropping sliding) (ac-buf-give! ch v) #t)
;; a promise channel takes ONE value, delivered to every taker; further
;; puts are dropped. Never blocks.
((promise) (when (ac-qempty? ch)
(ac-qpush! ch (cons v #f)) (condition-broadcast (async-chan-cv ch)))
#t)
(else
(if (> (async-chan-cap ch) 0)
(let loop () ; buffered fixed: wait for room
(cond ((async-chan-closed? ch) #f)
((< (ac-qlen ch) (async-chan-cap ch))
(ac-qpush! ch (cons v #f)) (condition-broadcast (async-chan-cv ch)) #t)
(else (condition-wait (async-chan-cv ch) (async-chan-mu ch)) (loop))))
(let ((box (vector #f))) ; unbuffered: rendezvous
(ac-qpush! ch (cons v box))
(condition-broadcast (async-chan-cv ch))
(let loop ()
(cond ((vector-ref box 0) #t)
((async-chan-closed? ch) #f)
(else (condition-wait (async-chan-cv ch) (async-chan-mu ch)) (loop))))))))))))
;; remove + return the head value, waking a parked rendezvous putter.
(define (ac-take-head! ch)
(let* ((entry (ac-qpop! ch)) (v (car entry)) (box (cdr entry)))
(when box (vector-set! box 0 #t))
(condition-broadcast (async-chan-cv ch))
v))
;; peek the front value without removing it (promise channels keep their value).
(define (ac-peek ch)
(let ((q (async-chan-items ch)))
(ac-qfront! q)
(car (car (vector-ref q 0)))))
;; <! / <!! — take, blocking. Drains buffered values, then nil once closed + empty.
;; A promise channel PEEKS — its one value stays for every taker.
(define (jolt-async-take ch)
(with-mutex (async-chan-mu ch)
(let loop ()
(cond ((eq? (async-chan-kind ch) 'promise)
(cond ((not (ac-qempty? ch)) (ac-peek ch))
((async-chan-closed? ch) jolt-nil)
(else (ac-take-wait ch) (loop))))
((not (ac-qempty? ch)) (ac-take-head! ch))
((async-chan-closed? ch) jolt-nil)
(else (ac-take-wait ch) (loop))))))
;; park in a take, tracking the waiter count so a concurrent offer! to an
;; unbuffered channel can see that a taker is ready.
(define (ac-take-wait ch)
(async-chan-takew-set! ch (fx+ 1 (async-chan-takew ch)))
(condition-wait (async-chan-cv ch) (async-chan-mu ch))
(async-chan-takew-set! ch (fx- (async-chan-takew ch) 1)))
;; non-blocking take for alts!/poll!: a value, jolt-nil (closed+empty), or ac-poll-empty.
(define ac-poll-empty (list 'empty))
(define (ac-poll! ch)
(with-mutex (async-chan-mu ch)
(cond ((and (eq? (async-chan-kind ch) 'promise) (not (ac-qempty? ch))) (ac-peek ch))
((not (ac-qempty? ch)) (ac-take-head! ch))
((async-chan-closed? ch) jolt-nil)
(else ac-poll-empty))))
;; non-blocking give: 'ok (accepted), 'full (would block), or 'closed.
(define (ac-try-give! ch v)
(when (jolt-nil? v) (jolt-throw (jolt-host-throwable "java.lang.IllegalArgumentException" "Can't put nil on a channel")))
(with-mutex (async-chan-mu ch)
(cond
((async-chan-closed? ch) 'closed)
((async-chan-xrf ch) (let ((r (ac-xrf-apply ch v)))
(when (jolt-reduced? r) (ac-close! ch)) 'ok))
(else
(case (async-chan-kind ch)
((dropping sliding) (ac-buf-give! ch v) 'ok)
((promise) (when (ac-qempty? ch) (ac-qpush! ch (cons v #f))
(condition-broadcast (async-chan-cv ch))) 'ok)
(else
(cond
((> (async-chan-cap ch) 0)
(if (< (ac-qlen ch) (async-chan-cap ch))
(begin (ac-qpush! ch (cons v #f)) (condition-broadcast (async-chan-cv ch)) 'ok)
'full))
;; unbuffered: only immediate if a taker is parked to receive it.
((> (async-chan-takew ch) 0)
(let ((box (vector #f)))
(ac-qpush! ch (cons v box))
(condition-broadcast (async-chan-cv ch))
'ok))
(else 'full))))))))
;; offer! / poll! — never block. offer! returns #t/#f(closed) on completion, nil if
;; it would block; poll! returns a value, nil (closed+empty), or the ::none sentinel.
(define cca-none (keyword "clojure.core.async" "none"))
(define (jolt-async-offer! ch v)
(case (ac-try-give! ch v) ((ok) #t) ((closed) #f) (else jolt-nil)))
(define (jolt-async-poll! ch)
(let ((r (ac-poll! ch))) (if (eq? r ac-poll-empty) cca-none r)))
;; (timeout ms) — a channel that closes after ms milliseconds.
(define (jolt-async-timeout ms)
(let ((w (ac-make 0 'unbuffered #f)))
(fork-thread (lambda () (sleep (ms->duration ms)) (jolt-async-close! w)))
w))
;; (put! ch v [cb [on-caller?]]) — async put, optional completion callback. If the
;; put completes immediately and on-caller? (default #t), the callback runs on the
;; calling thread; otherwise on another thread. Returns true unless already closed.
(define (jolt-async-put! ch v . rest)
(let* ((cb (if (pair? rest) (car rest) jolt-nil))
(on-caller? (if (and (pair? rest) (pair? (cdr rest))) (jolt-truthy? (cadr rest)) #t))
(call-cb (lambda (ok) (unless (jolt-nil? cb) (jolt-invoke cb ok)))))
(case (ac-try-give! ch v)
((ok) (if on-caller? (call-cb #t) (fork-thread (lambda () (call-cb #t)))) #t)
((closed) (if on-caller? (call-cb #f) (fork-thread (lambda () (call-cb #f)))) #f)
(else (fork-thread (lambda () (call-cb (jolt-async-give ch v)))) #t))))
;; (take! ch cb [on-caller?]) — async take. Same on-caller? rule as put!.
(define (jolt-async-take! ch cb . rest)
(let* ((on-caller? (if (pair? rest) (jolt-truthy? (car rest)) #t))
(call-cb (lambda (v) (unless (jolt-nil? cb) (jolt-invoke cb v))))
(r (ac-poll! ch)))
(cond
((eq? r ac-poll-empty) (fork-thread (lambda () (call-cb (jolt-async-take ch)))))
(on-caller? (call-cb r))
(else (fork-thread (lambda () (call-cb r)))))
jolt-nil))
;; (go-spawn thunk) — run thunk on a thread; return a buffered(1) channel that
;; conveys its value once then closes (a nil result just closes). Dynamic bindings
;; are conveyed (Chez inherits the thread-parameter at fork; we install explicitly).
(define (async-go-spawn thunk)
(let ((w (ac-make 1 'fixed #f)) (snap (dyn-binding-stack)))
(fork-thread
(lambda ()
(dyn-binding-stack snap)
(let ((r (guard (e (#t (cons #f e))) (cons #t (jolt-invoke thunk)))))
(when (and (car r) (not (jolt-nil? (cdr r)))) (jolt-async-give w (cdr r)))
(jolt-async-close! w))))
w))
;; --- macros (expander fns over the reader forms) ----------------------------
(define cca-go-spawn-sym (jolt-symbol "clojure.core.async" "go-spawn"))
(define cca-go-sym (jolt-symbol "clojure.core.async" "go"))
(define cca-fn*-sym (jolt-symbol #f "fn*"))
(define cca-loop-sym (jolt-symbol #f "loop"))
;; (go body...) -> (clojure.core.async/go-spawn (fn* [] body...))
(define (cca-go-macro . body)
(jolt-list cca-go-spawn-sym (apply jolt-list cca-fn*-sym empty-pvec body)))
;; (go-loop bindings body...) -> (go (loop bindings body...))
(define (cca-go-loop-macro bindings . body)
(jolt-list cca-go-sym (apply jolt-list cca-loop-sym bindings body)))
;; (thread body...) — a real OS thread (same shape as go here).
(define (cca-thread-macro . body)
(jolt-list cca-go-spawn-sym (apply jolt-list cca-fn*-sym empty-pvec body)))
;; --- install clojure.core.async ---------------------------------------------
(define (cca-def! name v) (def-var! "clojure.core.async" name v))
(cca-def! "chan" jolt-async-chan)
(cca-def! "promise-chan" (lambda args (ac-make 1 'promise #f)))
(cca-def! "chan?" async-chan?)
(cca-def! "buffer" jolt-async-buffer)
(cca-def! "dropping-buffer" jolt-async-dropping-buffer)
(cca-def! "sliding-buffer" jolt-async-sliding-buffer)
(cca-def! "__promise-buffer" (lambda () (make-async-buffer 1 'promise)))
(cca-def! "unblocking-buffer?" jolt-async-unblocking-buffer?)
(cca-def! "close!" jolt-async-close!)
(cca-def! "<!" jolt-async-take) (cca-def! "<!!" jolt-async-take)
(cca-def! ">!" jolt-async-give) (cca-def! ">!!" jolt-async-give)
(cca-def! "timeout" jolt-async-timeout)
(cca-def! "put!" jolt-async-put!)
(cca-def! "take!" jolt-async-take!)
(cca-def! "offer!" jolt-async-offer!)
(cca-def! "go-spawn" async-go-spawn)
;; non-blocking primitives the Clojure overlay's do-alts polls over.
(cca-def! "__poll!" jolt-async-poll!)
(cca-def! "__offer!" jolt-async-offer!)
(cca-def! "go" cca-go-macro) (mark-macro! "clojure.core.async" "go")
(cca-def! "go-loop" cca-go-loop-macro) (mark-macro! "clojure.core.async" "go-loop")
(cca-def! "thread" cca-thread-macro) (mark-macro! "clojure.core.async" "thread")

402
host/chez/java/bigdec.ss Normal file
View file

@ -0,0 +1,402 @@
;; BigDecimal. A jbigdec is {unscaled, scale} over Chez arbitrary-precision exact
;; integers; its value is unscaled * 10^-scale (1.5M = {15,1}, 1.00M = {100,2},
;; 3M = {3,0}). M-suffix literals read to a :bigdec form that the back end lowers
;; to jolt-bigdec-from-string; bigdec coerces a number/string. Equality is by
;; value (1.0M = 1.00M), str drops the M, pr keeps it, class is
;; java.math.BigDecimal.
;;
;; Arithmetic follows java.math.BigDecimal's scale rules: add/sub align to the
;; larger scale; multiply adds scales; divide gives the exact quotient at minimal
;; scale or throws ArithmeticException on a non-terminating expansion (a bound
;; *math-context* rounds instead). Clojure contagion: a bigdec mixed with an
;; integer or ratio stays a bigdec; a flonum operand wins (the result is a
;; double). jbd-add/-sub/-mul/-div, jbd-min/-max, the jbd-lt?/…/zero? helpers,
;; and jbd-quot/-rem are the shared engine. Two paths reach it, both leaving the
;; inlined fast path untouched:
;; - the seq.ss binary dispatch: every generic op (any position — (+ (bigdec x)
;; 1), (reduce + bigs), (quot 10.0 3M)) whose operand is outside Chez's tower
;; falls to the jolt-*-slow hooks extended below.
;; - static call position ((+ 1.5M 2.5M), (< a b), (zero? b)): jolt.passes.numeric
;; tags the invoke :num-kind :bigdec when every operand is statically a bigdec
;; (M literal or a let-bound copy, integer literals allowed), and the back end
;; lowers it directly to the jbd op.
(define-record-type jbigdec (fields unscaled scale) (nongenerative chez-jbigdec-v1))
(define (bd-index-char s ch)
(let loop ((i 0))
(cond ((>= i (string-length s)) #f)
((char=? (string-ref s i) ch) i)
(else (loop (+ i 1))))))
;; "1.50" -> {150,2}; "3" -> {3,0}; "-0.0" -> {0,1}; ".5" -> {5,1}.
(define (jolt-bigdec-from-string s)
(let* ((neg (and (> (string-length s) 0) (char=? (string-ref s 0) #\-)))
(sgn (and (> (string-length s) 0) (or neg (char=? (string-ref s 0) #\+))))
(s1 (if sgn (substring s 1 (string-length s)) s))
(sign (if neg -1 1))
(dot (bd-index-char s1 #\.)))
(if dot
(let* ((intp (substring s1 0 dot))
(fracp (substring s1 (+ dot 1) (string-length s1)))
(digs (string-append intp fracp))
(unscaled (if (= 0 (string-length digs)) 0 (string->number digs))))
(make-jbigdec (* sign unscaled) (string-length fracp)))
(make-jbigdec (* sign (string->number s1)) 0))))
;; bigdec coercion: a bigdec is itself; an exact integer keeps scale 0; a string
;; or any other number routes through its decimal text.
(define (jolt-bigdec x)
(cond
((jbigdec? x) x)
((and (number? x) (exact? x) (integer? x)) (make-jbigdec x 0))
((string? x) (jolt-bigdec-from-string x))
((number? x) (jolt-bigdec-from-string (jolt-num->string x)))
(else (error #f "bigdec: cannot coerce" x))))
;; value equality: unscaled_a * 10^scale_b == unscaled_b * 10^scale_a.
(define (jbigdec=? a b)
(= (* (jbigdec-unscaled a) (expt 10 (jbigdec-scale b)))
(* (jbigdec-unscaled b) (expt 10 (jbigdec-scale a)))))
;; render the decimal text (no M): insert the point `scale` digits from the right.
(define (jbigdec->string bd)
(let* ((u (jbigdec-unscaled bd)) (sc (jbigdec-scale bd))
(neg (< u 0)) (digs (number->string (abs u))))
(string-append
(if neg "-" "")
(if (<= sc 0)
digs
(let* ((padded (if (<= (string-length digs) sc)
(string-append (make-string (- (+ sc 1) (string-length digs)) #\0) digs)
digs))
(pl (string-length padded)))
(string-append (substring padded 0 (- pl sc)) "." (substring padded (- pl sc) pl)))))))
;; value as a Chez flonum (for double contagion: a flonum operand wins).
(define (jbigdec->flonum b)
(exact->inexact (/ (jbigdec-unscaled b) (expt 10 (jbigdec-scale b)))))
;; coerce an exact operand to a bigdec; pass a bigdec through. Used on the
;; non-flonum mixed path (bigdec + long -> bigdec). A Ratio converts like
;; Numbers.toBigDecimal — exact decimal expansion or throw on non-terminating.
(define (jbd-coerce x)
(cond ((jbigdec? x) x)
((and (number? x) (exact? x) (integer? x)) (make-jbigdec x 0))
((and (number? x) (exact? x) (rational? x)) (jbd-rational->bigdec x))
(else (error #f "bigdec arithmetic: cannot coerce operand" x))))
;; --- core arithmetic on the {unscaled, scale} pair --------------------------
;; align two bigdecs to a common scale, returning (unscaled-a unscaled-b scale).
(define (jbd-align a b)
(let ((sa (jbigdec-scale a)) (sb (jbigdec-scale b)))
(cond
((= sa sb) (values (jbigdec-unscaled a) (jbigdec-unscaled b) sa))
((> sa sb) (values (jbigdec-unscaled a)
(* (jbigdec-unscaled b) (expt 10 (- sa sb))) sa))
(else (values (* (jbigdec-unscaled a) (expt 10 (- sb sa)))
(jbigdec-unscaled b) sb)))))
(define (jbd2+ a b) (let-values (((ua ub s) (jbd-align a b))) (make-jbigdec (+ ua ub) s)))
(define (jbd2- a b) (let-values (((ua ub s) (jbd-align a b))) (make-jbigdec (- ua ub) s)))
(define (jbd2* a b) (make-jbigdec (* (jbigdec-unscaled a) (jbigdec-unscaled b))
(+ (jbigdec-scale a) (jbigdec-scale b))))
(define (jbd-negate a) (make-jbigdec (- (jbigdec-unscaled a)) (jbigdec-scale a)))
;; exact rational -> bigdec at minimal scale, or throw if non-terminating. den must
;; factor into 2s and 5s; scale = max(count2, count5).
(define (jbd-rational->bigdec r)
(let ((p (numerator r)) (q (denominator r)))
(let loop ((d q) (c2 0) (c5 0))
(cond
((= d 1) (let ((sc (max c2 c5)))
(make-jbigdec (* p (quotient (expt 10 sc) q)) sc)))
((= 0 (modulo d 2)) (loop (quotient d 2) (+ c2 1) c5))
((= 0 (modulo d 5)) (loop (quotient d 5) c2 (+ c5 1)))
(else (jolt-throw (jolt-host-throwable
"java.lang.ArithmeticException"
"Non-terminating decimal expansion; no exact representable decimal result.")))))))
;; floor(log10 |r|) for a nonzero exact rational.
(define (jbd-exp10 r)
(let ((n (abs (numerator r))) (d (denominator r)))
(if (>= n d)
(- (jbd-digits (quotient n d)) 1)
(let loop ((x (* n 10)) (e -1))
(if (>= x d) e (loop (* x 10) (- e 1)))))))
;; round an exact rational to `prec` significant digits (the MathContext divide).
(define (jbd-rational-prec r prec mode)
(if (= r 0)
(make-jbigdec 0 0)
(let* ((neg (< r 0)) (ar (abs r))
(s (- prec 1 (jbd-exp10 ar)))
(scaled (* ar (expt 10 s)))
(q (floor scaled)) (frac (- scaled q))
(q2 (if (jbd-round-inc? q frac 1 mode neg) (+ q 1) q))
(res (make-jbigdec (if neg (- q2) q2) s)))
;; a carry can add a digit (9.99 -> 10.0); re-normalizing drops an exact
;; trailing zero, never re-rounds.
(if (> (jbd-digits q2) prec) (jbd-round-prec res prec mode) res))))
(define (jbd2-div a b)
(when (= 0 (jbigdec-unscaled b))
(jolt-throw (jolt-host-throwable "java.lang.ArithmeticException" "Divide by zero")))
;; a/b = (ua * 10^sb) / (ub * 10^sa) as an exact rational. Unlimited context:
;; exact result at minimal scale or throw on a non-terminating expansion. A
;; bound *math-context* instead rounds to its precision.
(let ((r (/ (* (jbigdec-unscaled a) (expt 10 (jbigdec-scale b)))
(* (jbigdec-unscaled b) (expt 10 (jbigdec-scale a)))))
(mc (jbd-math-context)))
(if mc
(jbd-rational-prec r (jbd-mc-precision mc) (jbd-mc-mode mc))
(jbd-rational->bigdec r))))
;; integer-division semantics (quot/rem): truncate toward zero, scale 0.
(define (jbd-int-quot a b)
(when (= 0 (jbigdec-unscaled b))
(jolt-throw (jolt-host-throwable "java.lang.ArithmeticException" "Divide by zero")))
(let-values (((ua ub s) (jbd-align a b))) (make-jbigdec (quotient ua ub) 0)))
(define (jbd-int-rem a b)
(when (= 0 (jbigdec-unscaled b))
(jolt-throw (jolt-host-throwable "java.lang.ArithmeticException" "Divide by zero")))
(let-values (((ua ub s) (jbd-align a b)))
(make-jbigdec (remainder ua ub) (max (jbigdec-scale a) (jbigdec-scale b)))))
;; scale-independent ordering: compare unscaled values aligned to a common scale.
(define (jbd-compare2 a b)
(let-values (((ua ub s) (jbd-align a b))) (cond ((< ua ub) -1) ((> ua ub) 1) (else 0))))
;; --- *math-context* (with-precision) -----------------------------------------
;; with-precision binds clojure.core/*math-context* to {:precision N :rounding
;; MODE}; every exact bigdec result rounds through it (java.math.MathContext).
(define jbd-kw-precision (keyword #f "precision"))
(define jbd-kw-rounding (keyword #f "rounding"))
(define (jbd-math-context)
(let ((mc (var-deref "clojure.core" "*math-context*")))
(if (jolt-nil? mc) #f mc)))
(define (jbd-mc-precision mc) (jolt-get mc jbd-kw-precision))
(define (jbd-mc-mode mc)
(let ((r (jolt-get mc jbd-kw-rounding)))
(cond ((symbol-t? r) (symbol-t-name r))
((string? r) r)
(else "HALF_UP"))))
;; should |value| = q + r/div (0 <= r < div) round up in magnitude? neg is the
;; value's sign; r/div may be exact rationals (the division path).
(define (jbd-round-inc? q r div mode neg)
(cond ((= r 0) #f)
((string=? mode "UP") #t)
((string=? mode "DOWN") #f)
((string=? mode "CEILING") (not neg))
((string=? mode "FLOOR") neg)
((string=? mode "HALF_DOWN") (> (* 2 r) div))
((string=? mode "HALF_EVEN")
(let ((c (- (* 2 r) div)))
(cond ((> c 0) #t) ((< c 0) #f) (else (odd? q)))))
((string=? mode "UNNECESSARY")
(jolt-throw (jolt-host-throwable "java.lang.ArithmeticException" "Rounding necessary")))
(else (>= (* 2 r) div)))) ; HALF_UP, the MathContext default
(define (jbd-digits n) (string-length (number->string (abs n))))
;; round a bigdec to `prec` significant digits with `mode` (a RoundingMode name).
(define (jbd-round-prec bd prec mode)
(let ((u (jbigdec-unscaled bd)) (s (jbigdec-scale bd)))
(if (= u 0)
bd
(let ((digs (jbd-digits u)))
(if (<= digs prec)
bd
(let* ((drop (- digs prec)) (div (expt 10 drop))
(neg (< u 0)) (au (abs u))
(q (quotient au div)) (r (remainder au div))
(q2 (if (jbd-round-inc? q r div mode neg) (+ q 1) q))
(res (make-jbigdec (if neg (- q2) q2) (- s drop))))
;; a carry can add a digit back (99 -> 100 at precision 2)
(if (> (jbd-digits q2) prec) (jbd-round-prec res prec mode) res)))))))
(define (jbd-mc-round x)
(let ((mc (and (jbigdec? x) (jbd-math-context))))
(if mc (jbd-round-prec x (jbd-mc-precision mc) (jbd-mc-mode mc)) x)))
;; A binary op over operands that may mix bigdec / integer / flonum. flonum-op is
;; the native fallback for the double-contagion path; bd-op is the exact bigdec op
;; (its result rounds through a bound *math-context*).
(define (jbd-binop flonum-op bd-op a b)
(if (or (flonum? a) (flonum? b))
(flonum-op (if (jbigdec? a) (jbigdec->flonum a) a)
(if (jbigdec? b) (jbigdec->flonum b) b))
(jbd-mc-round (bd-op (jbd-coerce a) (jbd-coerce b)))))
;; --- variadic engine ops (Phase-2 emit targets + value-position folds) -------
(define (jbd-fold flonum-op bd-op init xs)
(let loop ((acc init) (rest xs))
(if (null? rest) acc (loop (jbd-binop flonum-op bd-op acc (car rest)) (cdr rest)))))
(define (jbd-add . xs)
(cond ((null? xs) (make-jbigdec 0 0))
((null? (cdr xs)) (car xs))
(else (jbd-fold + jbd2+ (car xs) (cdr xs)))))
(define (jbd-sub . xs)
(cond ((null? xs) (error #f "- needs at least 1 arg"))
((null? (cdr xs)) (if (jbigdec? (car xs)) (jbd-negate (car xs)) (- (car xs))))
(else (jbd-fold - jbd2- (car xs) (cdr xs)))))
(define (jbd-mul . xs)
(cond ((null? xs) (make-jbigdec 1 0))
((null? (cdr xs)) (car xs))
(else (jbd-fold * jbd2* (car xs) (cdr xs)))))
(define (jbd-div . xs)
(cond ((null? xs) (error #f "/ needs at least 1 arg"))
((null? (cdr xs)) (jbd-binop / jbd2-div (make-jbigdec 1 0) (car xs)))
(else (jbd-fold / jbd2-div (car xs) (cdr xs)))))
;; comparison / predicate helpers (Phase-2 emit targets). A flonum operand demotes
;; to the native comparison on the flonum values.
(define (jbd-cmp-num op flop a b)
(if (or (flonum? a) (flonum? b))
(flop (if (jbigdec? a) (jbigdec->flonum a) a) (if (jbigdec? b) (jbigdec->flonum b) b))
(op (jbd-compare2 (jbd-coerce a) (jbd-coerce b)) 0)))
(define (jbd-lt? a b) (jbd-cmp-num < < a b))
(define (jbd-gt? a b) (jbd-cmp-num > > a b))
(define (jbd-le? a b) (jbd-cmp-num <= <= a b))
(define (jbd-ge? a b) (jbd-cmp-num >= >= a b))
(define (jbd-zero? a) (= 0 (jbigdec-unscaled a)))
(define (jbd-pos? a) (> (jbigdec-unscaled a) 0))
(define (jbd-neg? a) (< (jbigdec-unscaled a) 0))
(define (jbd-quot a b) (jbd-int-quot (jbd-coerce a) (jbd-coerce b)))
(define (jbd-rem a b) (jbd-int-rem (jbd-coerce a) (jbd-coerce b)))
;; min/max compare by value but return the ORIGINAL operand (its type and scale
;; unchanged), matching java/Clojure: (min 1M 2.0) -> 1M, (max 1M 2.0) -> 2.0,
;; (min 1.50M 2M) -> 1.50M. Comparison handles a bigdec mixed with an int / flonum.
(define (jbd-value-compare a b)
(if (or (flonum? a) (flonum? b))
(let ((fa (if (jbigdec? a) (jbigdec->flonum a) a)) (fb (if (jbigdec? b) (jbigdec->flonum b) b)))
(cond ((< fa fb) -1) ((> fa fb) 1) (else 0)))
(jbd-compare2 (jbd-coerce a) (jbd-coerce b))))
;; strict comparison so a tie keeps the second operand, like Clojure's
;; (if (< x y) x y) / (if (> x y) x y): (max 1.5M 1.50M) -> 1.50M.
(define (jbd-min2 a b) (if (< (jbd-value-compare a b) 0) a b))
(define (jbd-max2 a b) (if (> (jbd-value-compare a b) 0) a b))
(define (jbd-min x . xs) (fold-left jbd-min2 x xs))
(define (jbd-max x . xs) (fold-left jbd-max2 x xs))
;; --- wire into the value model ----------------------------------------------
(def-var! "clojure.core" "bigdec" jolt-bigdec)
;; The seq.ss binary numeric dispatch (jolt-add2/… and the jolt-n* macros) routes
;; any op whose operand is outside Chez's tower to the *-slow hooks; extend each
;; with a bigdec arm. Every arithmetic position (call, value, higher-order)
;; funnels through these, so contagion and *math-context* rounding apply
;; uniformly. min/max need no arm: the generic jolt-min2 compares through
;; jolt-num-cmp-slow and returns the original operand.
(set! jolt-num-slow?
(let ((prev jolt-num-slow?)) (lambda (x) (or (jbigdec? x) (prev x)))))
(define (jbd-extend-hook prev bd-op)
(lambda (a b)
(if (or (jbigdec? a) (jbigdec? b)) (bd-op a b) (prev a b))))
(set! jolt-add-slow (jbd-extend-hook jolt-add-slow (lambda (a b) (jbd-binop + jbd2+ a b))))
(set! jolt-sub-slow (jbd-extend-hook jolt-sub-slow (lambda (a b) (jbd-binop - jbd2- a b))))
(set! jolt-mul-slow (jbd-extend-hook jolt-mul-slow (lambda (a b) (jbd-binop * jbd2* a b))))
(set! jolt-div-slow (jbd-extend-hook jolt-div-slow (lambda (a b) (jbd-binop / jbd2-div a b))))
(set! jolt-num-cmp-slow
(let ((prev jolt-num-cmp-slow))
(lambda (a b)
(if (and (or (jbigdec? a) (jbigdec? b)) (jbd-numberish? a) (jbd-numberish? b))
(jbd-value-compare a b)
(prev a b)))))
;; quot/rem/mod: a double operand demotes to the double path; exact operands use
;; the integer-division bigdec ops (mod = rem, floor-adjusted to the divisor's sign).
(define (jbd->num x) (if (jbigdec? x) (jbigdec->flonum x) x))
(set! jolt-quot-slow
(jbd-extend-hook jolt-quot-slow
(lambda (a b) (if (or (flonum? a) (flonum? b))
(jolt-quot (jbd->num a) (jbd->num b))
(jbd-int-quot (jbd-coerce a) (jbd-coerce b))))))
(set! jolt-rem-slow
(jbd-extend-hook jolt-rem-slow
(lambda (a b) (if (or (flonum? a) (flonum? b))
(jolt-rem (jbd->num a) (jbd->num b))
(jbd-int-rem (jbd-coerce a) (jbd-coerce b))))))
(set! jolt-mod-slow
(jbd-extend-hook jolt-mod-slow
(lambda (a b)
(if (or (flonum? a) (flonum? b))
(jolt-mod (jbd->num a) (jbd->num b))
(let* ((bb (jbd-coerce b))
(m (jbd-int-rem (jbd-coerce a) bb)))
(if (or (jbd-zero? m) (eq? (jbd-neg? m) (jbd-neg? bb))) m (jbd2+ m bb)))))))
;; unary shims: inc/dec and the sign predicates take a bigdec arm. set! updates
;; call-position references; the re-def-var! updates the var cell AND claims the
;; wrapped proc's class name before the prelude's inc'/dec' aliases are defined
;; ((type inc) stays clojure.core$inc — first def wins in the class registry).
(define jbd-one (make-jbigdec 1 0))
(set! jolt-inc (let ((prev jolt-inc)) (lambda (x) (if (jbigdec? x) (jbd-mc-round (jbd2+ x jbd-one)) (prev x)))))
(set! jolt-dec (let ((prev jolt-dec)) (lambda (x) (if (jbigdec? x) (jbd-mc-round (jbd2- x jbd-one)) (prev x)))))
(set! jolt-zero? (let ((prev jolt-zero?)) (lambda (x) (if (jbigdec? x) (jbd-zero? x) (prev x)))))
(set! jolt-pos? (let ((prev jolt-pos?)) (lambda (x) (if (jbigdec? x) (jbd-pos? x) (prev x)))))
(set! jolt-neg? (let ((prev jolt-neg?)) (lambda (x) (if (jbigdec? x) (jbd-neg? x) (prev x)))))
;; a BigDecimal IS a number (java.lang.Number): extend the number? native so the
;; predicate — and everything defined over it (num, =='s guard) — accepts it.
;; The compiled fast paths test Chez number? directly and are unaffected.
(set! jolt-number? (let ((prev jolt-number?)) (lambda (x) (if (jbigdec? x) #t (prev x)))))
(def-var! "clojure.core" "number?" jolt-number?)
(def-var! "clojure.core" "inc" jolt-inc)
(def-var! "clojure.core" "dec" jolt-dec)
(def-var! "clojure.core" "zero?" jolt-zero?)
(def-var! "clojure.core" "pos?" jolt-pos?)
(def-var! "clojure.core" "neg?" jolt-neg?)
;; rationalize: reference Clojure goes through BigDecimal.valueOf(double) — the
;; SHORTEST decimal print of the double, not its exact binary value — so
;; (rationalize 1.1) is 11/10. A bigdec is exact already; other exacts pass through.
(define (jolt-rationalize x)
(cond ((jbigdec? x) (/ (jbigdec-unscaled x) (expt 10 (jbigdec-scale x))))
((flonum? x)
(if (or (nan? x) (infinite? x))
(jolt-throw (jolt-host-throwable "java.lang.NumberFormatException"
(string-append "Invalid input: " (number->string x))))
(let ((bd (jolt-bigdec-from-string (jolt-num->string x))))
(/ (jbigdec-unscaled bd) (expt 10 (jbigdec-scale bd))))))
((number? x) x)
(else (jolt-num-cast-throw x))))
(def-var! "clojure.core" "rationalize" jolt-rationalize)
;; double/float of a bigdec is its flonum value.
(set! jolt-double-slow
(let ((prev jolt-double-slow))
(lambda (x) (if (jbigdec? x) (jbigdec->flonum x) (prev x)))))
;; narrow casts truncate a bigdec like Number.longValue.
(set! jolt-cast-truncate-slow
(let ((prev jolt-cast-truncate-slow))
(lambda (x)
(if (jbigdec? x)
(truncate (/ (jbigdec-unscaled x) (expt 10 (jbigdec-scale x))))
(prev x)))))
;; compare: add a bigdec arm (enables compare / sort / sorted collections). A
;; bigdec vs a plain number compares by value; bigdec vs bigdec is scale-independent.
(define jbd-prev-compare jolt-compare)
(define (jbd-numberish? x) (or (jbigdec? x) (number? x)))
(set! jolt-compare
(lambda (a b)
(if (and (or (jbigdec? a) (jbigdec? b)) (jbd-numberish? a) (jbd-numberish? b))
(if (or (flonum? a) (flonum? b))
(let ((fa (if (jbigdec? a) (jbigdec->flonum a) a))
(fb (if (jbigdec? b) (jbigdec->flonum b) b)))
(cond ((< fa fb) -1) ((> fa fb) 1) (else 0)))
(jbd-compare2 (jbd-coerce a) (jbd-coerce b)))
(jbd-prev-compare a b))))
(def-var! "clojure.core" "compare" jolt-compare)
;; equality: a bigdec equals only another bigdec, by value (matching (= 3M 3) = false).
(register-eq-arm! (lambda (a b) (or (jbigdec? a) (jbigdec? b)))
(lambda (a b) (and (jbigdec? a) (jbigdec? b) (jbigdec=? a b))))
;; str drops the M; pr/pr-str keep it.
(register-str-render! jbigdec? jbigdec->string)
(register-pr-arm! jbigdec? (lambda (x) (string-append (jbigdec->string x) "M")))
;; class / decimal?
(register-class-arm! jbigdec? (lambda (x) "java.math.BigDecimal"))
(set! jolt-decimal? (lambda (x) (jbigdec? x)))
(def-var! "clojure.core" "decimal?" jolt-decimal?)

View file

@ -0,0 +1,85 @@
;; byte-buffer.ss — java.nio.ByteBuffer over a jolt byte-array. A buffer is a
;; jhost tagged "byte-buffer" with mutable #(backing-array position limit); the
;; backing is a jolt byte-array (vector of 0..255). Covers the slice of the API
;; portable code reaches for — wrap / get(byte[]) / array / remaining / position /
;; limit / duplicate / flip / rewind — e.g. cognitect aws-api wrapping blob bytes.
(define (make-byte-buffer backing pos limit) (make-jhost "byte-buffer" (vector backing pos limit)))
(define (bb? x) (and (jhost? x) (string=? (jhost-tag x) "byte-buffer")))
(define (bb-backing b) (vector-ref (jhost-state b) 0))
(define (bb-pos b) (vector-ref (jhost-state b) 1))
(define (bb-limit b) (vector-ref (jhost-state b) 2))
(define (bb-pos! b n) (vector-set! (jhost-state b) 1 n))
(define (bb-limit! b n) (vector-set! (jhost-state b) 2 n))
(define (bb-capacity b) (vector-length (jolt-array-vec (bb-backing b))))
;; (ByteBuffer/wrap ba) | (ByteBuffer/wrap ba off len) | (ByteBuffer/allocate n)
(register-class-statics! "ByteBuffer"
(list
(cons "wrap" (lambda (ba . rest)
(let ((cap (vector-length (jolt-array-vec ba))))
(if (pair? rest)
(let ((off (jnum->exact (car rest))) (len (jnum->exact (cadr rest))))
(make-byte-buffer ba off (+ off len)))
(make-byte-buffer ba 0 cap)))))
(cons "allocate" (lambda (n)
(let ((cap (jnum->exact n)))
(make-byte-buffer (make-jolt-array (make-vector cap 0) 'byte) 0 cap))))
;; jolt has one heap; a direct buffer is just a buffer here.
(cons "allocateDirect" (lambda (n)
(let ((cap (jnum->exact n)))
(make-byte-buffer (make-jolt-array (make-vector cap 0) 'byte) 0 cap))))))
(register-host-methods! "byte-buffer"
(list
(cons "remaining" (lambda (self) (->num (- (bb-limit self) (bb-pos self)))))
(cons "hasRemaining" (lambda (self) (> (bb-limit self) (bb-pos self))))
;; position / limit are getters with no arg, setters (returning the buffer) with one
(cons "position" (lambda (self . a)
(if (pair? a) (begin (bb-pos! self (jnum->exact (car a))) self) (->num (bb-pos self)))))
(cons "limit" (lambda (self . a)
(if (pair? a) (begin (bb-limit! self (jnum->exact (car a))) self) (->num (bb-limit self)))))
(cons "capacity" (lambda (self) (->num (bb-capacity self))))
(cons "hasArray" (lambda (self) #t))
(cons "array" (lambda (self) (bb-backing self)))
(cons "duplicate" (lambda (self) (make-byte-buffer (bb-backing self) (bb-pos self) (bb-limit self))))
(cons "rewind" (lambda (self) (bb-pos! self 0) self))
(cons "flip" (lambda (self) (bb-limit! self (bb-pos self)) (bb-pos! self 0) self))
(cons "clear" (lambda (self) (bb-pos! self 0) (bb-limit! self (bb-capacity self)) self))
;; (.get dst) | (.get dst off len): bulk copy from position into a byte-array,
;; advancing position. Returns the buffer like the JVM.
;; (.put src): copy bytes into the buffer at position, advancing it. src is
;; another ByteBuffer (its remaining bytes), a byte-array, or a single byte.
(cons "put" (lambda (self src . rest)
(let ((dv (jolt-array-vec (bb-backing self))) (dp (bb-pos self)))
(cond
((bb? src)
(let* ((sv (jolt-array-vec (bb-backing src))) (sp (bb-pos src))
(n (- (bb-limit src) sp)))
(do ((i 0 (fx+ i 1))) ((fx=? i n))
(vector-set! dv (+ dp i) (vector-ref sv (+ sp i))))
(bb-pos! src (bb-limit src)) (bb-pos! self (+ dp n))))
((jolt-array? src)
(let* ((sv (jolt-array-vec src)) (n (vector-length sv)))
(do ((i 0 (fx+ i 1))) ((fx=? i n))
(vector-set! dv (+ dp i) (vector-ref sv i)))
(bb-pos! self (+ dp n))))
(else (vector-set! dv dp (jnum->exact src)) (bb-pos! self (+ dp 1))))
self)))
(cons "get" (lambda (self dst . rest)
(let* ((src (jolt-array-vec (bb-backing self)))
(dv (jolt-array-vec dst))
(off (if (pair? rest) (jnum->exact (car rest)) 0))
(len (if (and (pair? rest) (pair? (cdr rest))) (jnum->exact (cadr rest)) (vector-length dv)))
(p (bb-pos self)))
(do ((i 0 (+ i 1))) ((= i len))
(vector-set! dv (+ off i) (vector-ref src (+ p i))))
(bb-pos! self (+ p len))
self)))))
(register-class-arm! bb? (lambda (x) "java.nio.ByteBuffer"))
(register-instance-check-arm!
(lambda (type-sym val)
(if (and (symbol-t? type-sym) (bb? val)
(member (last-dot (symbol-t-name type-sym)) '("ByteBuffer")))
#t 'pass)))

View file

@ -0,0 +1,261 @@
;; class-hierarchy.ss — one JVM class/interface graph, the single source of truth
;; for every "what classes does this satisfy" question. value-host-tags (protocol
;; dispatch), instance?, isa?/supers/ancestors, and the exception hierarchy all
;; derive from the ONE table here instead of maintaining parallel hand-kept lists
;; that drift apart.
;;
;; The graph is keyed by canonical (FQN) class name -> its DIRECT super
;; interfaces/classes (also FQN). Transitivity is computed (jch-closure), so a row
;; lists only what a class directly extends/implements, matching the JVM source.
;;
;; It is OPEN: a library registers a class and its supers with
;; jolt.host/register-class-supers! (plus a class-arm in host-class.ss to map its
;; values to that class name), and every derived view picks the class up with no
;; core change. Loaded before records.ss so value-host-tags can derive from it.
;; canonical-name -> list of direct super canonical-names. Mutable + extensible.
(define jvm-class-parents (make-hashtable string-hash string=?))
;; closure cache, invalidated whenever the graph is extended.
(define jch-closure-cache (make-hashtable string-hash string=?))
(define jch-tags-cache (make-hashtable string-hash string=?))
;; Merge direct supers for a class (union with any already registered). Public so
;; libraries can graft their own classes onto the modeled hierarchy.
(define (jch-register-supers! name supers)
(let ((cur (hashtable-ref jvm-class-parents name '())))
(hashtable-set! jvm-class-parents name
(let add ((ss supers) (acc cur))
(cond ((null? ss) acc)
((member (car ss) acc) (add (cdr ss) acc))
(else (add (cdr ss) (append acc (list (car ss)))))))))
(hashtable-clear! jch-closure-cache)
(hashtable-clear! jch-tags-cache))
(define (jch-direct-supers name) (hashtable-ref jvm-class-parents name '()))
;; Replace a class's direct supers outright (defrecord re-declares the row its
;; deftype half registered). Same cache invalidation as a register.
(define (jch-set-supers! name supers)
(hashtable-set! jvm-class-parents name supers)
(hashtable-clear! jch-closure-cache)
(hashtable-clear! jch-tags-cache)
(set! jch-known-cache #f)
(set! jch-simple->fqn-cache #f))
;; transitive supers of NAME (canonical), excluding NAME and Object; Object is the
;; universal root supplied by callers. Breadth-first, deduped, stable order.
(define (jch-closure name)
(or (hashtable-ref jch-closure-cache name #f)
(let ((result
(let loop ((pending (jch-direct-supers name)) (seen '()))
(cond ((null? pending) (reverse seen))
((member (car pending) seen) (loop (cdr pending) seen))
(else (loop (append (jch-direct-supers (car pending)) (cdr pending))
(cons (car pending) seen)))))))
(hashtable-set! jch-closure-cache name result)
result)))
;; ns segment munging for a JVM-spelled class name: dashes become underscores
;; (clojure.core-test.x -> clojure.core_test.x).
(define (jch-munge-segments s)
(list->string (map (lambda (c) (if (char=? c #\-) #\_ c)) (string->list s))))
(define (jch-last-segment s)
(let loop ((i (- (string-length s) 1)))
(cond ((< i 0) s)
((char=? (string-ref s i) #\.) (substring s (+ i 1) (string-length s)))
((char=? (string-ref s i) #\$) (substring s (+ i 1) (string-length s)))
(else (loop (- i 1))))))
;; The protocol-dispatch / instance? tag list for a value of class NAME: the class
;; and its whole ancestry, each in BOTH canonical and simple spelling (extend-protocol
;; and instance? accept either "Associative" or "clojure.lang.Associative"), plus
;; "Object". Memoized — this is on the hot protocol-dispatch path.
(define (jch-tags name)
(or (hashtable-ref jch-tags-cache name #f)
(let* ((chain (cons name (jch-closure name)))
(result
(let build ((cs chain) (acc '()))
(if (null? cs)
(reverse (cons "Object" acc))
(let* ((fqn (car cs))
(simple (jch-last-segment fqn))
(acc1 (if (member fqn acc) acc (cons fqn acc)))
(acc2 (if (or (string=? simple fqn) (member simple acc1))
acc1 (cons simple acc1))))
(build (cdr cs) acc2))))))
(hashtable-set! jch-tags-cache name result)
result)))
;; Is WANTED (canonical or simple) the class CHILD (canonical) or one of its
;; ancestors? Object is every class's root. Matched by full name or last segment so
;; "IOException" and "java.io.IOException" both hit.
(define (jch-isa? child wanted)
(let ((wseg (jch-last-segment wanted)))
(or (string=? wanted "java.lang.Object") (string=? wanted "Object")
(let loop ((names (cons child (jch-closure child))))
(cond ((null? names) #f)
((or (string=? wanted (car names))
(string=? wseg (jch-last-segment (car names)))) #t)
(else (loop (cdr names))))))))
;; Does the graph model WANTED at all (as a class or as any class's ancestor)? Used
;; by instance? to decide between a definitive #f and 'pass (defer to other arms).
(define jch-known-cache #f)
(define (jch-known? wanted)
(when (not jch-known-cache)
(set! jch-known-cache (make-hashtable string-hash string=?))
(let-values (((keys vals) (hashtable-entries jvm-class-parents)))
(vector-for-each
(lambda (k supers)
(hashtable-set! jch-known-cache k #t)
(hashtable-set! jch-known-cache (jch-last-segment k) #t)
(for-each (lambda (s)
(hashtable-set! jch-known-cache s #t)
(hashtable-set! jch-known-cache (jch-last-segment s) #t))
supers))
keys vals)))
(or (hashtable-ref jch-known-cache wanted #f)
(hashtable-ref jch-known-cache (jch-last-segment wanted) #f)))
;; simple last-segment -> canonical FQN for a modeled class (first registered
;; wins). Lets a simple exception name (from chez-condition-exc-class) resolve to
;; its graph key so the exception hierarchy answers through the one graph.
(define jch-simple->fqn-cache #f)
(define (jch-fqn-of-simple name)
(when (not jch-simple->fqn-cache)
(set! jch-simple->fqn-cache (make-hashtable string-hash string=?))
(let-values (((keys vals) (hashtable-entries jvm-class-parents)))
(vector-for-each
(lambda (k supers)
(for-each (lambda (n)
(let ((seg (jch-last-segment n)))
(when (not (hashtable-ref jch-simple->fqn-cache seg #f))
(hashtable-set! jch-simple->fqn-cache seg n))))
(cons k supers)))
keys vals)))
(or (hashtable-ref jch-simple->fqn-cache name #f) name))
;; A register also invalidates the derived caches.
(define jch-register-supers!-inner jch-register-supers!)
(set! jch-register-supers!
(lambda (name supers)
(set! jch-known-cache #f)
(set! jch-simple->fqn-cache #f)
(jch-register-supers!-inner name supers)))
;; ---- interface marking ---------------------------------------------------------
;; The JVM distinguishes a concrete class (whose bases/supers chain roots at
;; Object) from an interface (whose don't). The graph marks the modeled
;; interfaces; anything unmarked is treated as a concrete class.
(define jch-interface-set (make-hashtable string-hash string=?))
(define (jch-mark-interface! name) (hashtable-set! jch-interface-set name #t))
(define (jch-interface? name) (hashtable-ref jch-interface-set name #f))
(for-each jch-mark-interface!
'("clojure.lang.Seqable" "clojure.lang.Sequential" "clojure.lang.Sorted"
"clojure.lang.Reversible" "clojure.lang.Indexed" "clojure.lang.Counted"
"clojure.lang.Named" "clojure.lang.Fn" "clojure.lang.IFn"
"clojure.lang.IPersistentCollection" "clojure.lang.ISeq"
"clojure.lang.Associative" "clojure.lang.ILookup"
"clojure.lang.IPersistentStack" "clojure.lang.IPersistentVector"
"clojure.lang.IPersistentMap" "clojure.lang.IPersistentSet"
"clojure.lang.IPersistentList" "clojure.lang.IObj" "clojure.lang.IMeta"
"clojure.lang.IDeref" "clojure.lang.IRecord" "clojure.lang.IType"
"clojure.lang.IHashEq" "clojure.lang.IEditableCollection"
"clojure.lang.IExceptionInfo" "clojure.lang.IReduceInit"
"java.util.List" "java.util.Set" "java.util.Collection" "java.util.Map"
"java.util.Iterator" "java.lang.Iterable" "java.lang.CharSequence"
"java.lang.Comparable" "java.lang.Runnable"
"java.util.concurrent.Callable" "java.io.Serializable"))
;; ---- seed the built-in graph: direct supers only, faithful to the JVM ---------
;; core clojure.lang interfaces
(jch-register-supers! "clojure.lang.IPersistentCollection" '("clojure.lang.Seqable"))
(jch-register-supers! "clojure.lang.ISeq" '("clojure.lang.IPersistentCollection"))
(jch-register-supers! "clojure.lang.Associative" '("clojure.lang.IPersistentCollection" "clojure.lang.ILookup"))
(jch-register-supers! "clojure.lang.IPersistentStack" '("clojure.lang.IPersistentCollection"))
(jch-register-supers! "clojure.lang.IPersistentVector" '("clojure.lang.Associative" "clojure.lang.Sequential"
"clojure.lang.IPersistentStack" "clojure.lang.Reversible"
"clojure.lang.Indexed"))
(jch-register-supers! "clojure.lang.IPersistentMap" '("java.lang.Iterable" "clojure.lang.Associative" "clojure.lang.Counted"))
(jch-register-supers! "clojure.lang.IPersistentSet" '("clojure.lang.IPersistentCollection" "clojure.lang.Counted"))
(jch-register-supers! "clojure.lang.IPersistentList" '("clojure.lang.Sequential" "clojure.lang.IPersistentStack"))
(jch-register-supers! "clojure.lang.IObj" '("clojure.lang.IMeta"))
(jch-register-supers! "clojure.lang.IFn" '("java.lang.Runnable" "java.util.concurrent.Callable"))
(jch-register-supers! "clojure.lang.Fn" '("clojure.lang.IFn"))
(jch-register-supers! "clojure.lang.AFn" '("clojure.lang.IFn"))
(jch-register-supers! "clojure.lang.AFunction" '("clojure.lang.AFn" "clojure.lang.Fn"))
;; java.util collection interfaces
(jch-register-supers! "java.util.List" '("java.util.Collection"))
(jch-register-supers! "java.util.Set" '("java.util.Collection"))
(jch-register-supers! "java.util.Collection" '("java.lang.Iterable"))
;; concrete collection classes
(jch-register-supers! "clojure.lang.APersistentVector" '("clojure.lang.IPersistentVector" "java.util.List"))
(jch-register-supers! "clojure.lang.PersistentVector" '("clojure.lang.APersistentVector" "clojure.lang.IObj"
"java.util.List" "java.lang.Comparable"))
(jch-register-supers! "clojure.lang.APersistentMap" '("clojure.lang.IPersistentMap" "java.util.Map"))
(jch-register-supers! "clojure.lang.PersistentArrayMap" '("clojure.lang.APersistentMap" "clojure.lang.IObj"))
(jch-register-supers! "clojure.lang.PersistentHashMap" '("clojure.lang.APersistentMap" "clojure.lang.IObj"))
(jch-register-supers! "clojure.lang.PersistentTreeMap" '("clojure.lang.APersistentMap" "clojure.lang.IObj" "clojure.lang.Sorted" "clojure.lang.Reversible"))
(jch-register-supers! "clojure.lang.APersistentSet" '("clojure.lang.IPersistentSet" "java.util.Set"))
(jch-register-supers! "clojure.lang.PersistentHashSet" '("clojure.lang.APersistentSet" "clojure.lang.IObj"))
(jch-register-supers! "clojure.lang.PersistentTreeSet" '("clojure.lang.APersistentSet" "clojure.lang.IObj" "clojure.lang.Sorted" "clojure.lang.Reversible"))
(jch-register-supers! "clojure.lang.ASeq" '("clojure.lang.ISeq" "clojure.lang.Sequential" "java.util.List"))
(jch-register-supers! "clojure.lang.PersistentList" '("clojure.lang.ASeq" "clojure.lang.IPersistentList" "clojure.lang.Counted"))
(jch-register-supers! "clojure.lang.PersistentList$EmptyList" '("clojure.lang.PersistentList"))
(jch-register-supers! "clojure.lang.LazySeq" '("clojure.lang.ISeq" "clojure.lang.Sequential" "java.util.List" "clojure.lang.IObj"))
(jch-register-supers! "clojure.lang.Cons" '("clojure.lang.ASeq"))
(jch-register-supers! "clojure.lang.PersistentQueue" '("clojure.lang.IPersistentList" "clojure.lang.IPersistentCollection" "java.util.Collection"))
;; scalars / named / callable
(jch-register-supers! "clojure.lang.Keyword" '("clojure.lang.IFn" "clojure.lang.Named" "java.lang.Comparable"))
(jch-register-supers! "clojure.lang.Symbol" '("clojure.lang.IObj" "clojure.lang.IFn" "clojure.lang.Named" "java.lang.Comparable"))
(jch-register-supers! "clojure.lang.Var" '("clojure.lang.IDeref" "clojure.lang.IFn"))
(jch-register-supers! "clojure.lang.Atom" '("clojure.lang.IDeref"))
(jch-register-supers! "clojure.lang.Ratio" '("java.lang.Number" "java.lang.Comparable"))
(jch-register-supers! "clojure.lang.BigInt" '("java.lang.Number"))
(jch-register-supers! "java.lang.String" '("java.lang.CharSequence" "java.lang.Comparable"))
(jch-register-supers! "java.lang.Long" '("java.lang.Number" "java.lang.Comparable"))
(jch-register-supers! "java.lang.Integer" '("java.lang.Number" "java.lang.Comparable"))
(jch-register-supers! "java.lang.Double" '("java.lang.Number" "java.lang.Comparable"))
(jch-register-supers! "java.lang.Float" '("java.lang.Number" "java.lang.Comparable"))
(jch-register-supers! "java.math.BigDecimal" '("java.lang.Number" "java.lang.Comparable"))
(jch-register-supers! "java.math.BigInteger" '("java.lang.Number" "java.lang.Comparable"))
(jch-register-supers! "java.lang.Boolean" '("java.lang.Comparable"))
(jch-register-supers! "java.lang.Character" '("java.lang.Comparable"))
(jch-register-supers! "java.util.UUID" '("java.lang.Comparable"))
;; exception hierarchy (folds in the former exception-parent table)
(jch-register-supers! "java.lang.Exception" '("java.lang.Throwable"))
(jch-register-supers! "java.lang.RuntimeException" '("java.lang.Exception"))
(jch-register-supers! "clojure.lang.ExceptionInfo" '("java.lang.RuntimeException" "clojure.lang.IExceptionInfo"))
(jch-register-supers! "java.lang.IllegalArgumentException" '("java.lang.RuntimeException"))
(jch-register-supers! "clojure.lang.ArityException" '("java.lang.IllegalArgumentException"))
(jch-register-supers! "java.lang.NumberFormatException" '("java.lang.IllegalArgumentException"))
(jch-register-supers! "java.lang.IllegalStateException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.lang.UnsupportedOperationException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.lang.ArithmeticException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.lang.NullPointerException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.lang.ClassCastException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.lang.IndexOutOfBoundsException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.util.ConcurrentModificationException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.util.NoSuchElementException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.io.UncheckedIOException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.time.DateTimeException" '("java.lang.RuntimeException"))
(jch-register-supers! "java.time.format.DateTimeParseException" '("java.time.DateTimeException"))
(jch-register-supers! "java.lang.InterruptedException" '("java.lang.Exception"))
(jch-register-supers! "java.io.IOException" '("java.lang.Exception"))
(jch-register-supers! "java.io.InterruptedIOException" '("java.io.IOException"))
(jch-register-supers! "java.io.FileNotFoundException" '("java.io.IOException"))
(jch-register-supers! "java.io.UnsupportedEncodingException" '("java.io.IOException"))
(jch-register-supers! "java.net.UnknownHostException" '("java.io.IOException"))
(jch-register-supers! "java.net.SocketException" '("java.io.IOException"))
(jch-register-supers! "java.net.ConnectException" '("java.net.SocketException"))
(jch-register-supers! "java.net.SocketTimeoutException" '("java.io.InterruptedIOException"))
(jch-register-supers! "java.net.MalformedURLException" '("java.io.IOException"))
(jch-register-supers! "javax.net.ssl.SSLException" '("java.io.IOException"))
(jch-register-supers! "java.lang.Error" '("java.lang.Throwable"))
(jch-register-supers! "java.lang.AssertionError" '("java.lang.Error"))
;; Throwable's only super is Object (universal), so no row needed for it.
;; Public seam: libraries extend the modeled hierarchy.
(def-var! "jolt.host" "register-class-supers!"
(lambda (name supers) (jch-register-supers! name (seq->list supers)) jolt-nil))

View file

@ -0,0 +1,611 @@
;; concurrency.ss — real OS-thread futures + promises for the Chez host.
;;
;; SHARED-HEAP semantics, like JVM Clojure: a future body runs on a native thread
;; (fork-thread) over the SAME heap, so a captured atom is shared and the body's
;; mutations are visible to the parent. deref blocks on a mutex+condition latch.
;;
;; future / future-call / future-cancel / future? / future-done? / future-cancelled?
;; promise / deliver, and the deref extension for both, are bound here (some
;; re-asserted in post-prelude.ss over the overlay's versions).
;;
;; pmap / pcalls / pvalues live in the clojure.core overlay (40-lazy) expressed
;; over `future`, so they light up for free once future-call exists.
;;
;; Loaded near the end of rt.ss — after atoms.ss (jolt-deref, the atom lock) and
;; dyn-binding.ss (the thread-local binding stack we convey into the worker).
;; Requires a threaded Chez build (fork-thread / make-mutex / make-condition).
;; --- time helpers -----------------------------------------------------------
;; A relative duration / absolute deadline from a millisecond count (a jolt number).
(define (ms->duration ms)
(let* ((ms* (exact (floor ms)))
(secs (quotient ms* 1000))
(nanos (* (remainder ms* 1000) 1000000)))
(make-time 'time-duration nanos secs)))
(define (ms->deadline ms) (add-duration (current-time 'time-utc) (ms->duration ms)))
;; --- futures ----------------------------------------------------------------
;; A future is a mutable cell guarded by `mu`; workers/derefs coordinate on `cv`.
;; done? — result (or cancellation) is final; derefs may proceed
;; cancelled? — future-cancel won before the body finished
;; ok? — payload is a value (else payload is a raised condition/value)
;; payload — the result value, or the captured throw
(define-record-type jolt-future
(fields (mutable done?) (mutable cancelled?) (mutable ok?) (mutable payload) mu cv)
(nongenerative jolt-future-v1))
;; (future-call thunk): spawn a thread running (thunk). The dynamic bindings in
;; effect now are conveyed into the worker (Chez inherits thread-parameters at
;; fork; we also install an explicit snapshot for certainty). The result — value
;; or thrown condition — is latched and broadcast; a cancel that already finalized
;; the future makes the late result a no-op.
(define (jolt-future-call thunk)
(let ((f (make-jolt-future #f #f #f jolt-nil (make-mutex) (make-condition)))
(snap (dyn-binding-stack)))
(fork-thread
(lambda ()
(dyn-binding-stack snap)
(let ((r (guard (e (#t (cons #f e))) (cons #t (jolt-invoke thunk)))))
(with-mutex (jolt-future-mu f)
(unless (jolt-future-done? f) ; not already cancelled
(jolt-future-ok?-set! f (car r))
(jolt-future-payload-set! f (cdr r))
(jolt-future-done?-set! f #t))
(condition-broadcast (jolt-future-cv f))))))
f))
;; Final value of a settled future (called OUTSIDE the lock): re-raise a captured
;; throw, signal a cancellation, else the value.
(define (jolt-future-finish f)
(cond
((jolt-future-cancelled? f)
(jolt-throw (jolt-ex-info "Future cancelled" (jolt-hash-map))))
((jolt-future-ok? f) (jolt-future-payload f))
(else (raise (jolt-future-payload f)))))
(define (jolt-future-deref f)
(with-mutex (jolt-future-mu f)
(let loop ()
(unless (jolt-future-done? f)
(condition-wait (jolt-future-cv f) (jolt-future-mu f))
(loop))))
(jolt-future-finish f))
;; (deref f timeout-ms timeout-val): wait up to timeout-ms; return timeout-val if
;; it has not settled by the absolute deadline.
(define (jolt-future-deref-timed f ms timeout-val)
(let* ((deadline (ms->deadline ms))
(settled (with-mutex (jolt-future-mu f)
(let loop ()
(cond ((jolt-future-done? f) #t)
((condition-wait (jolt-future-cv f) (jolt-future-mu f) deadline)
(loop)) ; woken — recheck
(else (jolt-future-done? f))))))) ; timed out: final check
(if settled (jolt-future-finish f) timeout-val)))
;; future-cancel: the running thread can't be interrupted, but the future object
;; reflects the cancellation — if not already settled, mark it cancelled+done so
;; derefs raise and the predicates flip. Returns true iff this call cancelled it.
(define (jolt-future-cancel f)
(let ((cancelled (with-mutex (jolt-future-mu f)
(if (jolt-future-done? f)
#f
(begin (jolt-future-cancelled?-set! f #t)
(jolt-future-done?-set! f #t)
(condition-broadcast (jolt-future-cv f))
#t)))))
cancelled))
(define (jolt-native-future-done? x)
(if (jolt-future? x) (jolt-future-done? x)
(jolt-throw (jolt-ex-info "future-done? requires a future" (jolt-hash-map)))))
(define (jolt-native-future-cancelled? x)
(and (jolt-future? x) (jolt-future-cancelled? x)))
;; --- promises ---------------------------------------------------------------
;; A blocking promise (like the JVM): deref parks until deliver, then caches the
;; value. deliver wins once; later delivers return nil.
(define-record-type jolt-promise
(fields (mutable delivered?) (mutable value) mu cv)
(nongenerative jolt-promise-v1))
(define (jolt-promise-new) (make-jolt-promise #f jolt-nil (make-mutex) (make-condition)))
(define (jolt-deliver p v)
(if (jolt-promise? p)
(let ((won (with-mutex (jolt-promise-mu p)
(if (jolt-promise-delivered? p)
#f
(begin (jolt-promise-value-set! p v)
(jolt-promise-delivered?-set! p #t)
(condition-broadcast (jolt-promise-cv p))
#t)))))
(if won p jolt-nil))
(jolt-throw (jolt-ex-info "deliver requires a promise" (jolt-hash-map)))))
(define (jolt-promise-deref p)
(with-mutex (jolt-promise-mu p)
(let loop ()
(unless (jolt-promise-delivered? p)
(condition-wait (jolt-promise-cv p) (jolt-promise-mu p))
(loop))))
(jolt-promise-value p))
(define (jolt-promise-deref-timed p ms timeout-val)
(let* ((deadline (ms->deadline ms))
(got (with-mutex (jolt-promise-mu p)
(let loop ()
(cond ((jolt-promise-delivered? p) #t)
((condition-wait (jolt-promise-cv p) (jolt-promise-mu p) deadline)
(loop))
(else (jolt-promise-delivered? p)))))))
(if got (jolt-promise-value p) timeout-val)))
;; --- agents (async, per-agent serialized dispatch) --------------------------
;; JVM semantics: send/send-off enqueue an action and a single worker thread
;; applies them to the state IN ORDER; deref reads the
;; (possibly not-yet-updated) state without blocking; await blocks until the queue
;; drains. An action error is captured (agent-error) and stops the queue.
(define-record-type jolt-agent
(fields (mutable state) (mutable err) (mutable validator)
(mutable queue) (mutable running?) mu cv)
(nongenerative jolt-agent-v1))
;; (agent state :meta m :validator f :error-mode e): the ARef ctor contract like
;; atom's — the validator runs against the initial state, :meta must be a map.
;; :error-mode is accepted/ignored (jolt agents are always :fail).
(define (jolt-agent-new state . opts)
(let loop ((o opts) (validator jolt-nil) (m #f))
(cond
((or (null? o) (null? (cdr o)))
(let ((a (make-jolt-agent state jolt-nil validator (vector '() '()) #f (make-mutex) (make-condition))))
(when (and (not (jolt-nil? validator)) (jolt-not (jolt-invoke validator state)))
(jolt-iref-state-throw))
(when (and m (not (jolt-nil? m)))
(unless (jolt-map? m)
(jolt-throw (jolt-host-throwable
"java.lang.ClassCastException"
(string-append "class " (jolt-class-name m)
" cannot be cast to class clojure.lang.IPersistentMap"))))
(hashtable-set! meta-table a m))
a))
((and (keyword-t? (car o)) (string=? (keyword-t-name (car o)) "validator"))
(loop (cddr o) (cadr o) m))
((and (keyword-t? (car o)) (string=? (keyword-t-name (car o)) "meta"))
(loop (cddr o) validator (cadr o)))
(else (loop (cddr o) validator m)))))
;; agents are watchable IRefs; the worker notifies on each state change.
(register-iref-arm! jolt-agent?)
;; The action queue is an amortized-O(1) FIFO held as a mutable #(out in): `out` is
;; the front, `in` holds sends reversed onto it (an append-to-a-list send was O(n)).
;; All three helpers run under the agent mutex.
(define (jagent-q-empty? a)
(let ((q (jolt-agent-queue a))) (and (null? (vector-ref q 0)) (null? (vector-ref q 1)))))
(define (jagent-q-push! a entry)
(let ((q (jolt-agent-queue a))) (vector-set! q 1 (cons entry (vector-ref q 1)))))
(define (jagent-q-pop! a)
(let ((q (jolt-agent-queue a)))
(when (null? (vector-ref q 0))
(vector-set! q 0 (reverse (vector-ref q 1))) (vector-set! q 1 '()))
(let ((out (vector-ref q 0))) (vector-set! q 0 (cdr out)) (car out))))
;; Drain the queue, applying each action (f state arg*) outside the lock (an action
;; may send/deref the same agent). A validator rejection or a thrown action puts the
;; agent in an error state and halts the queue (JVM :fail mode).
(define (jolt-agent-worker a)
(let loop ()
(let ((act (with-mutex (jolt-agent-mu a)
(if (or (not (jolt-nil? (jolt-agent-err a))) (jagent-q-empty? a))
(begin (jolt-agent-running?-set! a #f)
(condition-broadcast (jolt-agent-cv a)) #f)
(jagent-q-pop! a)))))
(when act
(guard (e (#t (with-mutex (jolt-agent-mu a)
(jolt-agent-err-set! a e)
(condition-broadcast (jolt-agent-cv a)))))
(let* ((old (jolt-agent-state a))
(nv (apply jolt-invoke (car act) old (cdr act))))
(let ((vf (jolt-agent-validator a)))
(when (and (not (jolt-nil? vf)) (jolt-not (jolt-invoke vf nv)))
(jolt-iref-state-throw)))
(jolt-agent-state-set! a nv)
(iref-notify a old nv)))
(loop)))))
;; send / send-off: enqueue the action, start the worker if idle. (jolt treats them
;; identically — one serialized worker per agent — which is observably a superset of
;; the JVM's fixed/cached pool split.)
(define (jolt-agent-send a f . args)
(with-mutex (jolt-agent-mu a)
(jagent-q-push! a (cons f args))
(unless (jolt-agent-running? a)
(jolt-agent-running?-set! a #t)
(fork-thread (lambda () (jolt-agent-worker a)))))
a)
;; (await & agents): block until each agent's queue has drained.
(define (jolt-agent-await . agents)
(for-each
(lambda (a)
(with-mutex (jolt-agent-mu a)
(let loop ()
(when (or (jolt-agent-running? a) (not (jagent-q-empty? a)))
(condition-wait (jolt-agent-cv a) (jolt-agent-mu a)) (loop)))))
agents)
jolt-nil)
(define (jolt-agent-error a) (jolt-agent-err a))
(define (jolt-agent-restart a new-state . _opts)
(jolt-agent-err-set! a jolt-nil)
(jolt-agent-state-set! a new-state)
a)
;; --- delay (lazy once-forced computation) -----------------------------------
;; (delay body) -> (make-delay (fn [] body)) (overlay macro); force/deref run the
;; thunk once under a lock and cache the value (JVM delays are thread-safe). force
;; (overlay) is (if (delay? x) (deref x) x), so it works once delay?/deref do.
(define-record-type jolt-delay (fields thunk (mutable realized?) (mutable value) (mutable exn) mu)
(nongenerative jolt-delay-v1))
(define (jolt-make-delay thunk) (make-jolt-delay thunk #f jolt-nil #f (make-mutex)))
;; run the thunk once, like Clojure's Delay: if it throws, cache the exception
;; (the delay IS realized) and re-throw it on every deref — do NOT re-run the
;; body (so value-fns memoize and there is no cache-stampede / retried side
;; effect). Store the exception inside the lock, re-raise outside it so the mutex
;; is always released.
(define (jolt-delay-force d)
(with-mutex (jolt-delay-mu d)
(unless (jolt-delay-realized? d)
(guard (e (#t (jolt-delay-exn-set! d e) (jolt-delay-realized?-set! d #t)))
(jolt-delay-value-set! d (jolt-invoke (jolt-delay-thunk d)))
(jolt-delay-realized?-set! d #t))))
(if (jolt-delay-exn d) (raise (jolt-delay-exn d)) (jolt-delay-value d)))
;; --- deref extension --------------------------------------------------------
;; Chain the fully-built jolt-deref (atoms/vars/volatiles/reduced) with futures,
;; promises, agents, and delays; accept the timed (deref ref ms val) arity for the
;; blocking ref types.
(define %pre-conc-deref jolt-deref)
(set! jolt-deref
(lambda (x . opts)
(cond
((jolt-future? x)
(if (null? opts) (jolt-future-deref x)
(jolt-future-deref-timed x (car opts) (cadr opts))))
((jolt-promise? x)
(if (null? opts) (jolt-promise-deref x)
(jolt-promise-deref-timed x (car opts) (cadr opts))))
((jolt-agent? x) (jolt-agent-state x))
((jolt-delay? x) (jolt-delay-force x))
;; a record/reify implementing clojure.lang.IDeref: @x calls its `deref`
;; method with the value itself as the leading `this`.
((and (jrec? x) (find-method-any-protocol (jrec-tag x) "deref"))
=> (lambda (m) (jolt-invoke m x)))
((and (reified-methods x) (hashtable-ref (reified-methods x) "deref" #f))
=> (lambda (m) (jolt-invoke m x)))
(else (apply %pre-conc-deref x opts)))))
;; realized? for a future/promise/delay. Wrapped over the overlay version in
;; post-prelude.ss.
(define (jolt-conc-realized? x)
(cond ((jolt-future? x) (jolt-future-done? x))
((jolt-promise? x) (jolt-promise-delivered? x))
((jolt-delay? x) (jolt-delay-realized? x))
(else #f)))
;; --- bind into clojure.core -------------------------------------------------
(def-var! "clojure.core" "future-call" jolt-future-call)
(def-var! "clojure.core" "future-cancel" jolt-future-cancel)
(def-var! "clojure.core" "future?" jolt-future?)
(def-var! "clojure.core" "future-done?" jolt-native-future-done?)
(def-var! "clojure.core" "future-cancelled?" jolt-native-future-cancelled?)
(def-var! "clojure.core" "promise" jolt-promise-new)
(def-var! "clojure.core" "deliver" jolt-deliver)
;; a promise is an IFn on the JVM: (p val) delivers. Registered as a cold
;; invoke arm; callable-host? feeds the ifn? overlay (multimethods included).
(register-invoke-arm! jolt-promise?
(lambda (p args)
(if (and (pair? args) (null? (cdr args)))
(jolt-deliver p (car args))
(jolt-throw (jolt-host-throwable "clojure.lang.ArityException"
"Wrong number of args passed to a promise")))))
(def-var! "jolt.host" "callable-host?"
(lambda (x) (if (or (jolt-multifn? x) (jolt-promise? x)) #t jolt-nil)))
(def-var! "clojure.core" "agent" jolt-agent-new)
(def-var! "clojure.core" "agent?" jolt-agent?)
(def-var! "clojure.core" "send" jolt-agent-send)
(def-var! "clojure.core" "send-off" jolt-agent-send)
(def-var! "clojure.core" "await" jolt-agent-await)
(def-var! "clojure.core" "agent-error" jolt-agent-error)
(def-var! "clojure.core" "restart-agent" jolt-agent-restart)
(def-var! "clojure.core" "make-delay" jolt-make-delay)
(def-var! "clojure.core" "delay?" jolt-delay?)
(def-var! "clojure.core" "deref" jolt-deref)
;; --- object monitors (locking) ----------------------------------------------
;; (locking obj body…) takes obj's monitor for the body — a real per-object lock
;; now that futures/agents/threads share one heap. Each object gets a recursive
;; Chez mutex (a thread may re-enter a monitor it already holds, like the JVM),
;; held in an identity-keyed weak table so monitors are reclaimed with their
;; objects. dynamic-wind releases on normal, exceptional, and continuation exit.
(define monitor-table (make-weak-eq-hashtable))
(define monitor-table-lock (make-mutex))
(define (object-monitor obj)
(with-mutex monitor-table-lock
(or (hashtable-ref monitor-table obj #f)
(let ((m (make-mutex))) (hashtable-set! monitor-table obj m) m))))
(define (jolt-with-monitor obj thunk)
(let ((m (object-monitor obj)))
(dynamic-wind
(lambda () (mutex-acquire m))
thunk
(lambda () (mutex-release m)))))
(def-var! "jolt.host" "with-monitor" jolt-with-monitor)
;; --- cooperative thread interrupt -------------------------------------------
;; Chez has no force-kill, but its engine timer (set-timer + timer-interrupt-
;; handler, thread-local) is polled at procedure-call / loop back-edges — so a
;; running computation, even a tight Scheme loop, can be aborted from another
;; thread. An interrupt TOKEN is a shared box; run-interruptible arms a periodic
;; timer in the eval thread whose handler escapes (via call/cc) when the token is
;; set; interrupt! sets the token from any thread. The aborted eval throws a jolt
;; ex-info {:jolt/interrupted true}, so the thread is REUSED, not abandoned.
;;
;; Caveat: a thread blocked in a __collect_safe foreign call (socket recv/accept,
;; sleep) only sees the interrupt when it returns to Scheme — like the JVM not
;; killing native code.
(define interrupt-check-ticks 100000) ; ~poll interval; responsive + low overhead
(define interrupt-sentinel (cons 'jolt 'interrupted))
(define jolt-kw-interrupted (keyword "jolt" "interrupted"))
(define (jolt-make-interrupt) (box #f))
(define (jolt-interrupt! token) (when (box? token) (set-box! token #t)) jolt-nil)
(define (jolt-interrupted? token) (and (box? token) (unbox token) #t))
(define (jolt-run-interruptible token thunk)
(let ((prev-handler (timer-interrupt-handler)))
(let ((r (call/cc
(lambda (k)
(timer-interrupt-handler
(lambda ()
(if (and (box? token) (unbox token))
(k interrupt-sentinel)
(begin (set-timer interrupt-check-ticks) (void)))))
(set-timer interrupt-check-ticks)
(let ((v (thunk))) (set-timer 0) v)))))
;; restore the prior timer state regardless of outcome.
(set-timer 0)
(timer-interrupt-handler prev-handler)
(if (eq? r interrupt-sentinel)
(jolt-throw (jolt-ex-info "Evaluation interrupted" (jolt-hash-map jolt-kw-interrupted #t)))
r))))
(def-var! "jolt.host" "make-interrupt" jolt-make-interrupt)
(def-var! "jolt.host" "interrupt!" jolt-interrupt!)
(def-var! "jolt.host" "interrupted?" jolt-interrupted?)
(def-var! "jolt.host" "run-interruptible" jolt-run-interruptible)
;; --- java.lang.Thread / java.util.concurrent.CountDownLatch -----------------
;; Real OS threads over Chez fork-thread (shared heap — a captured atom/var is
;; shared). A Thread runs its Runnable thunk; start forks, join waits on a
;; condition latched at completion. CountDownLatch is a counting barrier.
(define (make-jthread thunk) (make-jhost "user-thread" (vector thunk #f (make-mutex) (make-condition))))
(for-each (lambda (nm) (register-class-ctor! nm (lambda (thunk . _) (make-jthread thunk))))
'("Thread" "java.lang.Thread"))
(register-host-methods! "user-thread"
(list (cons "start" (lambda (self)
(let ((st (jhost-state self)) (snap (dyn-binding-stack)))
(fork-thread (lambda ()
(dyn-binding-stack snap)
(guard (e (#t #f)) (jolt-invoke (vector-ref st 0)))
(with-mutex (vector-ref st 2)
(vector-set! st 1 #t)
(condition-broadcast (vector-ref st 3)))))
jolt-nil)))
(cons "run" (lambda (self) (jolt-invoke (vector-ref (jhost-state self) 0)) jolt-nil))
(cons "join" (lambda (self . _)
(let ((st (jhost-state self)))
(with-mutex (vector-ref st 2)
(let loop () (unless (vector-ref st 1) (condition-wait (vector-ref st 3) (vector-ref st 2)) (loop)))))
jolt-nil))
(cons "isAlive" (lambda (self) (not (vector-ref (jhost-state self) 1))))
(cons "interrupt" (lambda (self . _) jolt-nil))
(cons "setDaemon" (lambda (self . _) jolt-nil))))
(define (make-jlatch n) (make-jhost "count-down-latch" (vector n (make-mutex) (make-condition))))
(for-each (lambda (nm) (register-class-ctor! nm (lambda (n . _) (make-jlatch (jnum->exact n)))))
'("CountDownLatch" "java.util.concurrent.CountDownLatch"))
(register-host-methods! "count-down-latch"
(list (cons "countDown" (lambda (self)
(let ((st (jhost-state self)))
(with-mutex (vector-ref st 1)
(when (> (vector-ref st 0) 0) (vector-set! st 0 (- (vector-ref st 0) 1)))
(when (= (vector-ref st 0) 0) (condition-broadcast (vector-ref st 2)))))
jolt-nil))
(cons "await" (lambda (self . _)
(let ((st (jhost-state self)))
(with-mutex (vector-ref st 1)
(let loop () (when (> (vector-ref st 0) 0) (condition-wait (vector-ref st 2) (vector-ref st 1)) (loop)))))
jolt-nil))
(cons "getCount" (lambda (self) (vector-ref (jhost-state self) 0)))))
;; --- main-thread executor ---------------------------------------------------
;; Lets a worker thread (e.g. an nREPL eval future) run a thunk on the thread
;; that owns the GUI main loop. On macOS GTK quartz, g_application_run must run
;; on the process main thread or AppKit aborts (setMainMenu off-main → SIGABRT).
;; Under `joltc nrepl` the accept loop is backgrounded in a future and the
;; primordial thread enters jolt-run-main-pump; glimmer's run marshals its
;; startup through jolt-call-on-main-thread.
;;
;; - With no pump running (`joltc -M:run` calls run directly on the main thread),
;; call-on-main-thread runs the thunk INLINE — unchanged behaviour.
;; - A call from a thunk already executing on the pump runs inline too, so the
;; pump can't deadlock on itself.
;; - Otherwise the thunk is enqueued; the caller blocks until the pump runs it,
;; then receives the value, or the thrown condition is re-raised.
;;
;; stop-main-pump is the graceful-shutdown / external API: it tells the pump to
;; drain whatever is queued and return. The pump-active flag is flipped to #f
;; under jolt-main-queue-mu in the same critical section that decides to exit, and
;; call-on-main-thread reads that flag and enqueues under the SAME mutex, so a job
;; can never slip in after the pump has decided to leave — a call that loses the
;; race simply runs inline instead of blocking forever on a pump that is gone.
(define jolt-main-queue-mu (make-mutex))
(define jolt-main-queue-cv (make-condition))
(define jolt-main-queue '()) ; FIFO of jolt-main-job, guarded by mu
(define jolt-main-pump-active (box #f)) ; #t while run-main-pump owns this thread
(define jolt-main-pump-stop (box #f)) ; set by stop-main-pump to drain + exit
;; thread-local: this thread is the pump, mid-thunk → nested calls run inline.
(define jolt-in-main-pump? (make-thread-parameter #f))
(define-record-type jolt-main-job
(fields thunk (mutable done?) (mutable ok?) (mutable val) mu cv)
(nongenerative jolt-main-job-v1))
(define (jolt-call-on-main-thread thunk)
(if (jolt-in-main-pump?) ; reentrant — already on the pump
(jolt-invoke thunk)
;; Decide-and-enqueue atomically: read pump-active and (if active) push the
;; job under jolt-main-queue-mu, the same lock the pump holds when it flips
;; active to #f on exit. So we either get queued before the pump leaves, or
;; we see #f and fall through to inline — never enqueue onto a dead pump.
(let ((job (with-mutex jolt-main-queue-mu
(and (unbox jolt-main-pump-active)
(let ((j (make-jolt-main-job thunk #f #f jolt-nil
(make-mutex) (make-condition))))
(set! jolt-main-queue (append jolt-main-queue (list j)))
(condition-signal jolt-main-queue-cv)
j)))))
(if (not job)
(jolt-invoke thunk) ; no pump (or stopped) — inline, like -M:run
(begin
(with-mutex (jolt-main-job-mu job)
(let wait ()
(unless (jolt-main-job-done? job)
(condition-wait (jolt-main-job-cv job) (jolt-main-job-mu job))
(wait))))
(if (jolt-main-job-ok? job)
(jolt-main-job-val job)
(raise (jolt-main-job-val job))))))))
(define jolt-pump-kih
(lambda ()
(for-each (lambda (th) (guard (e (#t #f)) (th)))
(reverse (unbox jolt-shutdown-hooks)))
(exit 0)))
;; Park the calling thread until a keyboard interrupt (^C), then run the shutdown
;; hooks and exit. Unlike run-main-pump (whose tight recursive condition-wait
;; loop elides Chez's interrupt poll points, so the handler never fires), this
;; uses a single condition-wait — the form Chez reliably interrupts. The nREPL
;; server parks here; SIGINT is unblocked in this thread first (it was masked by
;; jolt-block-sigint so the accept loop inherited a blocked mask and couldn't
;; absorb ^C in its foreign accept() call).
(define jolt-park-mu (make-mutex))
(define jolt-park-cv (make-condition))
(define (jolt-park-until-interrupt)
(keyboard-interrupt-handler jolt-pump-kih)
(jolt-set-sigint-blocked #f)
(with-mutex jolt-park-mu (condition-wait jolt-park-cv jolt-park-mu))
jolt-nil)
(define (jolt-run-main-pump)
(with-mutex jolt-main-queue-mu
(set-box! jolt-main-pump-stop #f)
(set-box! jolt-main-pump-active #t))
;; dynamic-wind guarantees active is cleared even if the pump escapes abnormally,
;; so a later run-main-pump starts clean and call-on-main-thread never sees a
;; stale #t. The clean-exit path below also clears it under the mutex (the flip
;; that races call-on-main-thread); this is the belt-and-suspenders for escapes.
(dynamic-wind
(lambda () #f)
(lambda ()
(let loop ()
(let ((job (with-mutex jolt-main-queue-mu
(let wait ()
(cond
((not (null? jolt-main-queue))
(let ((j (car jolt-main-queue)))
(set! jolt-main-queue (cdr jolt-main-queue))
j))
((unbox jolt-main-pump-stop)
;; drain done, told to exit — clear active in the same
;; critical section so no job can be enqueued after.
(set-box! jolt-main-pump-active #f)
#f)
(else (condition-wait jolt-main-queue-cv jolt-main-queue-mu)
(wait)))))))
(when job
(let ((r (dynamic-wind
(lambda () (jolt-in-main-pump? #t))
(lambda ()
(guard (e (#t (cons #f e)))
(cons #t (jolt-invoke (jolt-main-job-thunk job)))))
(lambda () (jolt-in-main-pump? #f)))))
(with-mutex (jolt-main-job-mu job)
(jolt-main-job-ok?-set! job (car r))
(jolt-main-job-val-set! job (cdr r))
(jolt-main-job-done?-set! job #t)
(condition-broadcast (jolt-main-job-cv job))))
(loop)))))
(lambda ()
(with-mutex jolt-main-queue-mu (set-box! jolt-main-pump-active #f))))
jolt-nil)
(define (jolt-stop-main-pump)
(with-mutex jolt-main-queue-mu
(set-box! jolt-main-pump-stop #t)
(condition-broadcast jolt-main-queue-cv))
jolt-nil)
;; Shutdown hooks run by jolt-pump-kih (the keyboard-interrupt-handler installed by
;; park-until-interrupt) before (exit 0), so a foreground server (nREPL) can close
;; its socket and drop .nrepl-port on ^C instead of Chez's default mutex-corrupting
;; abort. Newest-first; each hook is isolated so one failing hook can't block the exit.
(define jolt-shutdown-hooks (box '()))
(define (jolt-add-shutdown-hook thunk)
(set-box! jolt-shutdown-hooks (cons thunk (unbox jolt-shutdown-hooks)))
jolt-nil)
;; Per-thread SIGINT mask. A worker thread parked in a foreign call (the nREPL
;; accept loop in c-accept, or a conn handler) can't run Chez's keyboard-interrupt
;; handler on ^C, so if SIGINT is delivered there the process hangs. Block SIGINT
;; in the primordial thread BEFORE forking such workers (they inherit the mask),
;; then park-until-interrupt unblocks it in the primordial once its handler is
;; installed, so ^C is always delivered to the parked thread. pthread_sigmask/
;; sigaddset are libc/libpthread symbols, resolvable once the process object is
;; loaded (as the socket fns already are). 128 bytes covers Linux's 1024-bit
;; sigset_t and is larger than macOS's 4-byte one.
;; foreign-procedure resolves its symbol eagerly, and these POSIX signal fns don't
;; exist on Windows — resolving them unguarded aborted startup ("no entry for
;; pthread_sigmask"). Guard so a non-POSIX host yields #f; jolt-set-sigint-blocked
;; then no-ops (Windows delivers ^C through the console, not a per-thread mask).
(define c-pthread-sigmask
(jolt-foreign-proc-safe "pthread_sigmask" '(int u8* u8*) 'int))
(define c-sigemptyset (jolt-foreign-proc-safe "sigemptyset" '(u8*) 'int))
(define c-sigaddset (jolt-foreign-proc-safe "sigaddset" '(u8* int) 'int))
;; POSIX SIG_BLOCK/SIG_UNBLOCK numerics differ by platform: Linux/glibc 0/1,
;; Darwin/macOS 1/2 (SIG_UNBLOCK is SIG_BLOCK+1 on both). Resolve SIG_BLOCK for
;; this host from the machine-type symbol — macOS builds contain "osx".
(define jolt-sig-block-how
(let* ((s (symbol->string (machine-type)))
(n (string-length s)))
(let loop ((i 0))
(cond
((> (+ i 3) n) 0) ; default: Linux/glibc
((string=? (substring s i (+ i 3)) "osx") 1) ; Darwin/macOS
(else (loop (+ i 1)))))))
(define (jolt-set-sigint-blocked block?)
(when (and c-pthread-sigmask c-sigemptyset c-sigaddset)
(let ((set (make-bytevector 128 0))
(old (make-bytevector 128 0)))
(c-sigemptyset set)
(c-sigaddset set 2) ; SIGINT = 2
(c-pthread-sigmask (if block? jolt-sig-block-how (+ jolt-sig-block-how 1)) set old)))
jolt-nil)
(def-var! "jolt.host" "call-on-main-thread" jolt-call-on-main-thread)
(def-var! "jolt.host" "run-main-pump" jolt-run-main-pump)
(def-var! "jolt.host" "stop-main-pump" jolt-stop-main-pump)
(def-var! "jolt.host" "add-shutdown-hook" jolt-add-shutdown-hook)
(def-var! "jolt.host" "block-sigint" (lambda () (jolt-set-sigint-blocked #t)))
(def-var! "jolt.host" "park-until-interrupt" jolt-park-until-interrupt)
(def-var! "jolt.host" "delete-file" delete-file)

149
host/chez/java/dot-forms.ss Normal file
View file

@ -0,0 +1,149 @@
;; dot-forms.ss — generic dispatch for the `.` special-form / `.-field` desugar.
;; The analyzer lowers (. target member arg*) and (.-field target)
;; to a :host-call; the Chez emit routes a non-shimmed :host-call through
;; record-method-dispatch. This file extends that dispatcher with the collection
;; arms the interpreter's dispatch-member covers but the record/string base does
;; not, with this precedence:
;;
;; * collection interop wins first — count/seq/nth/get/valAt/containsKey on a
;; vector/map/set/seq/record (so (. {:count 9} count) is the entry count, 1,
;; NOT the :count field).
;; * field access — a "-name" member reads the field (records and maps).
;; * map member — a stored fn is a method (called with self + args); any
;; other value is returned as a field.
;;
;; Anything not recognized falls through to the previous dispatcher (jhost /
;; number / regex / jrec protocol / string). Loaded LAST (after host-static.ss).
;; A record (jrec) is jolt-map? here (records.ss makes it so) and a collection,
;; so its protocol method (no dash, not a coll method) lands in the base.
;; Vectors / maps / sets only (records are jolt-map? here). Raw seqs are excluded:
;; coll-interop accepts some seq representations and not others (a
;; plain (seq v) returns nil from .count, a lazy-seq returns the count), an
;; inconsistency Chez's normalized cseq can't mirror — so a raw seq target falls
;; through to the base dispatcher rather than risk a divergence the corpus would
;; never exercise but a future case might.
(define (dot-coll? obj)
(or (jolt-vector? obj) (jolt-map? obj) (pset? obj)))
;; Mirror coll-interop: return a one-element list boxing the result (so a jolt-nil
;; result is still distinguishable from "not a collection method"), or #f.
(define (dot-coll-method obj name args)
(cond
((string=? name "count") (list (jolt-count obj)))
((string=? name "seq") (list (jolt-seq obj)))
((string=? name "nth") (list (apply jolt-nth obj args)))
((or (string=? name "get") (string=? name "valAt"))
(list (apply jolt-get obj args)))
((string=? name "containsKey") (list (jolt-contains? obj (car args))))
;; java.util.Collection.contains(o): VALUE membership (a set is O(1) via
;; contains?; a list/vector/seq is a linear scan — contains? on a vector tests
;; an index, so it is wrong here).
((string=? name "contains")
(list (if (pset? obj)
(jolt-contains? obj (car args))
(let ((x (car args)))
(let loop ((s (jolt-seq obj)))
(cond ((jolt-nil? s) #f)
((jolt=2 (seq-first s) x) #t)
(else (loop (jolt-seq (seq-more s))))))))))
((string=? name "size") (list (jolt-count obj)))
((string=? name "isEmpty") (list (jolt-empty? obj)))
;; java.util.Map views: keySet (a Set), values (a Collection), entrySet.
((and (jolt-map? obj) (string=? name "keySet"))
(list (apply jolt-hash-set (seq->list (jolt-keys obj)))))
((and (jolt-map? obj) (string=? name "values"))
(list (apply jolt-vector (seq->list (jolt-vals obj)))))
((and (jolt-map? obj) (string=? name "entrySet")) (list (jolt-seq obj)))
;; (.iterator coll): a java.util.Iterator over the seq — for a map this is the
;; entry iterator. Without this a map's .iterator falls into the map-as-object
;; branch and is mis-read as a missing :iterator key (nil). Some libraries
;; (e.g. malli's -vmap) iterate a map this way.
((string=? name "iterator") (list (make-jiterator (jolt-seq obj))))
;; (.reduce coll f) / (.reduce coll f init): clojure.lang.IReduce — every
;; persistent collection reduces itself on the JVM.
((string=? name "reduce")
(list (if (pair? (cdr args))
(jolt-reduce (car args) (cadr args) obj)
(jolt-reduce (car args) obj))))
(else #f)))
;; Universal object-methods: on a
;; non-record map these win OVER a field lookup, like dispatch-member. getMessage
;; on an ex-info reads its :message (the one the corpus exercises); getCause reads
;; :cause; toString/hashCode/equals round out the set. Returns a boxed result or
;; #f. Strings/numbers/records/jhost keep the base dispatcher (it shims them).
(define (dot-object-method obj name args)
(cond
((string=? name "getMessage")
(list (if (jolt=2 (jolt-get obj jolt-kw-ex-type jolt-nil) jolt-kw-ex-info)
(jolt-get obj jolt-kw-message jolt-nil)
(jolt-str-render-one obj))))
((string=? name "getCause") (list (jolt-get obj jolt-kw-cause jolt-nil)))
;; java.sql.SQLException chaining — ex-info / host throwables don't chain.
((string=? name "getNextException") (list jolt-nil))
((string=? name "getStackTrace") (list (jolt-vector)))
((string=? name "toString") (list (jolt-str-render-one obj)))
((string=? name "hashCode") (list (jolt-hash obj)))
((string=? name "equals") (list (if (jolt= obj (car args)) #t #f)))
(else #f)))
(register-method-arm! 30
(lambda (obj method-name rest-args)
(let* ((rest (if (jolt-nil? rest-args) '() (seq->list rest-args)))
(field? (and (> (string-length method-name) 0)
(char=? (string-ref method-name 0) #\-)))
(mname (if field?
(substring method-name 1 (string-length method-name))
method-name)))
(cond
;; clojure.lang.MultiFn .dispatchFn / .getMethod — clojure.spec.alpha's
;; multi-spec walks a multimethod through these.
((jolt-multifn? obj)
(cond
((string=? mname "dispatchFn") (jolt-multifn-dispatch-fn obj))
((string=? mname "getMethod")
(let ((methods (jolt-multifn-methods obj)) (dv (car rest)))
(or (hashtable-ref methods dv #f)
(mm-find-isa obj dv)
(hashtable-ref methods (jolt-multifn-default obj) #f)
jolt-nil)))
(else 'pass)))
;; (.applyTo f args): apply a fn to a seq of args (clojure.spec instrument).
((and (procedure? obj) (string=? mname "applyTo"))
(apply jolt-invoke obj (seq->list (jolt-seq (car rest)))))
;; a transient (ITransientCollection/Set/Map): .contains / .valAt / .count —
;; test.check's distinct-collection gen uses (.contains transient-set k).
((jolt-transient? obj)
(cond
((string=? mname "contains") (if (jolt-truthy? (t-contains? obj (car rest))) #t #f))
((or (string=? mname "valAt") (string=? mname "get"))
(t-get obj (car rest) (if (null? (cdr rest)) jolt-nil (cadr rest))))
((string=? mname "count") (t-count obj))
(else 'pass)))
;; a deftype/record's OWN declared method (matched by name AND arity) wins
;; over the generic collection interop below — e.g. data.priority-map
;; declares both seq[this] (Seqable) and seq[this ascending] (Sorted), and
;; (.seq pm false) must reach the 2-arg one, not dot-coll's plain seq.
((and (not field?) (jrec? obj)
(find-method-any-protocol-arity (jrec-tag obj) mname (+ 1 (length rest))))
=> (lambda (f) (apply jolt-invoke f obj rest)))
;; collection interop first (entry count / seq / nth / get / containsKey).
((and (dot-coll? obj) (dot-coll-method obj mname rest))
=> (lambda (box) (car box)))
;; clojure.lang.Sorted (comparator / entryKey / seqFrom) on a sorted
;; map/set, before the map arm below reads the method name as a key.
;; data.priority-map's subseq/rsubseq reach for these.
((and (not field?) (htable-sorted? obj) (sorted-iface-method? mname))
(sorted-iface-dispatch obj mname rest))
;; (.-field obj) / (. obj -field): field read on a record or map.
(field? (jolt-get obj (keyword #f mname) jolt-nil))
;; non-record map: a universal object-method (getMessage/...) wins first,
;; then a stored procedure is a method (call with self), else the field.
((and (jolt-map? obj) (not (jrec? obj)))
(cond
((dot-object-method obj mname rest) => car)
(else
(let ((v (jolt-get obj (keyword #f mname) jolt-nil)))
(if (procedure? v) (apply jolt-invoke v obj rest) v)))))
(else 'pass)))))

167
host/chez/java/ffi.ss Normal file
View file

@ -0,0 +1,167 @@
;; ffi.ss — the runtime side of jolt's foreign-function interface (jolt.ffi).
;;
;; A jolt LIBRARY binds native code itself: it loads a shared object and declares
;; typed foreign functions, then exposes a Clojure API. The TYPED CALL is lowered
;; at compile time to a Chez `foreign-procedure` by the backend (the
;; `jolt.ffi/foreign-fn` special form) — this file provides everything that does
;; NOT need compile-time types: loading libraries, allocating/reading/writing
;; foreign memory, and string/pointer marshaling. All exposed under `jolt.ffi`.
;;
;; A foreign pointer is a Chez machine address (an exact integer / uptr), the same
;; representation `void*` arguments and results use, so pointers flow between
;; foreign-fn calls and these helpers transparently.
;; --- loading shared objects --------------------------------------------------
;; (jolt.ffi/load-library name) loads a .so/.dylib by name (resolved by the OS
;; loader against the standard search paths). A library typically calls this once
;; at load with a platform-specific name. (load-library) with no name (or #f)
;; loads the running process's own symbols (libc, sockets).
(define (ffi-load-library . args)
(if (or (null? args) (jolt-nil? (car args)))
(begin (load-shared-object #f) jolt-nil)
(begin (load-shared-object (jolt-str-render-one (car args))) jolt-nil)))
(define (ffi-loaded? name)
(guard (e (#t #f)) (load-shared-object (jolt-str-render-one name)) #t))
;; --- foreign type keywords ---------------------------------------------------
;; The keyword type names jolt.ffi accepts (in foreign-fn signatures and the
;; memory accessors) map to Chez foreign types. Kept in one place so the backend
;; (compile-time, for foreign-procedure) and these accessors (runtime, for
;; foreign-ref/set!) agree.
(define (ffi-type->chez kw)
(let ((n (if (keyword-t? kw) (keyword-t-name kw) (jolt-str-render-one kw))))
(cond
((string=? n "int") 'int)
((string=? n "uint") 'unsigned-int)
((string=? n "long") 'long)
((string=? n "ulong") 'unsigned-long)
((string=? n "int64") 'integer-64)
((string=? n "uint64") 'unsigned-64)
((string=? n "size_t") 'size_t)
((string=? n "ssize_t") 'ssize_t)
((string=? n "iptr") 'iptr)
((string=? n "uptr") 'uptr)
((string=? n "double") 'double)
((string=? n "float") 'float)
((or (string=? n "pointer") (string=? n "void*")) 'void*)
((string=? n "string") 'string)
((string=? n "void") 'void)
((or (string=? n "uint8") (string=? n "u8") (string=? n "byte")) 'unsigned-8)
((string=? n "char") 'char)
(else (error #f (string-append "jolt.ffi: unknown foreign type :" n))))))
;; --- foreign memory ----------------------------------------------------------
;; alloc returns a pointer (integer address). The caller frees it. read/write take
;; a type keyword and an optional byte offset.
(define (ffi-alloc nbytes) (foreign-alloc (jnum->exact nbytes)))
(define (ffi-free ptr) (foreign-free (jnum->exact ptr)) jolt-nil)
(define (ffi-read ptr ty . off)
(foreign-ref (ffi-type->chez ty) (jnum->exact ptr) (if (pair? off) (jnum->exact (car off)) 0)))
(define (ffi-write ptr ty off val)
(foreign-set! (ffi-type->chez ty) (jnum->exact ptr) (jnum->exact off) val) jolt-nil)
;; sizeof a foreign type (for laying out structs / arrays).
(define (ffi-sizeof ty) (foreign-sizeof (ffi-type->chez ty)))
(define (ffi-null? ptr) (and (number? ptr) (= (jnum->exact ptr) 0)))
(define ffi-null 0)
;; --- buffer I/O (known length) ----------------------------------------------
;; read n bytes at ptr as a string (UTF-8, falling back to latin1 for invalid
;; sequences) — for a socket recv buffer and similar fixed-length reads.
(define (ffi-read-bytes ptr n)
(let* ((n (jnum->exact n)) (p (jnum->exact ptr)) (bv (make-bytevector n)))
(do ((i 0 (+ i 1))) ((= i n)) (bytevector-u8-set! bv i (foreign-ref 'unsigned-8 p i)))
(guard (e (#t (list->string (map integer->char (bytevector->u8-list bv))))) (utf8->string bv))))
;; write a string's UTF-8 bytes into ptr (no NUL terminator); return the count.
(define (ffi-write-bytes ptr s)
(let* ((bv (string->utf8 (jolt-str-render-one s))) (n (bytevector-length bv)) (p (jnum->exact ptr)))
(do ((i 0 (+ i 1))) ((= i n)) (foreign-set! 'unsigned-8 p i (bytevector-u8-ref bv i)))
n))
(def-var! "jolt.ffi" "read-bytes" ffi-read-bytes)
(def-var! "jolt.ffi" "write-bytes" ffi-write-bytes)
;; --- byte-array buffer I/O (binary-faithful) --------------------------------
;; Move raw bytes between a jolt byte-array (jolt-array kind 'byte) and foreign
;; memory, byte-exact (no UTF-8 / latin1 decode) — for socket recv/send and the
;; zlib / OpenSSL buffers an HTTP client passes through. read-array returns a
;; fresh byte-array of n bytes; write-array copies a byte-array's bytes into ptr
;; and returns the count.
(define (ffi-read-array ptr n)
(let* ((n (jnum->exact n)) (p (jnum->exact ptr)) (v (make-vector n 0)))
(do ((i 0 (+ i 1))) ((= i n)) (vector-set! v i (foreign-ref 'unsigned-8 p i)))
(make-jolt-array v 'byte)))
(define (ffi-write-array ptr arr)
(let* ((v (jolt-array-vec arr)) (n (vector-length v)) (p (jnum->exact ptr)))
(do ((i 0 (+ i 1))) ((= i n)) (foreign-set! 'unsigned-8 p i (bitwise-and (exact (vector-ref v i)) #xff)))
n))
(def-var! "jolt.ffi" "read-array" ffi-read-array)
(def-var! "jolt.ffi" "write-array" ffi-write-array)
;; --- string / bytevector marshaling ------------------------------------------
;; A C string result already comes back as a jolt string (the `string` foreign
;; type). For a `void*` that points at a NUL-terminated C string, read it here.
(define (ffi-ptr->string ptr)
(if (ffi-null? ptr) jolt-nil
(let ((p (jnum->exact ptr)))
(let loop ((i 0) (acc '()))
(let ((b (foreign-ref 'unsigned-8 p i)))
(if (= b 0) (utf8->string (u8-list->bytevector (reverse acc)))
(loop (+ i 1) (cons b acc))))))))
;; Copy a jolt string's UTF-8 bytes into a freshly alloc'd NUL-terminated buffer;
;; the caller frees it. Returns the pointer.
(define (ffi-string->ptr s)
(let* ((bv (string->utf8 (jolt-str-render-one s))) (n (bytevector-length bv))
(p (foreign-alloc (+ n 1))))
(do ((i 0 (+ i 1))) ((= i n)) (foreign-set! 'unsigned-8 p i (bytevector-u8-ref bv i)))
(foreign-set! 'unsigned-8 p n 0)
p))
;; --- callbacks: receive calls FROM C ----------------------------------------
;; jolt.ffi/foreign-callable lowers to (jolt-ffi-register-callable! (foreign-callable …)).
;; A foreign-callable code object must be LOCKED (so the collector neither moves
;; nor reclaims it) and RETAINED while C may still call through its entry point.
;; Register it keyed by that entry-point address (a jolt pointer integer) — which
;; is what the caller hands to C; free-callable unlocks and drops it. A callback
;; left registered lives for the process (the GTK-signal-handler common case).
(define ffi-callable-table (make-eqv-hashtable)) ; entry-point addr -> code object
(define (jolt-ffi-register-callable! co)
(lock-object co)
(let ((addr (foreign-callable-entry-point co)))
(hashtable-set! ffi-callable-table addr co)
addr))
(define (ffi-free-callable addr)
(let* ((a (jnum->exact addr)) (co (hashtable-ref ffi-callable-table a #f)))
(when co (unlock-object co) (hashtable-delete! ffi-callable-table a))
jolt-nil))
;; --- native libraries for a standalone binary -------------------------------
;; `jolt build` bakes a project's deps.edn :jolt/native declarations into the
;; launcher, which loads them at startup (load-shared-object isn't part of the
;; saved heap, so it must run in the built process, not at heap build). process?
;; loads the running binary's own symbols (libc sockets); otherwise try each
;; platform candidate in turn and fail unless the spec is optional.
(define (jolt-build-load-native cands optional? process?)
(if process?
(begin (load-shared-object #f) #t)
(let loop ((cs cands))
(cond
((null? cs)
(unless optional?
(error 'jolt-build "required native library not found" cands))
#f)
((guard (e (#t #f)) (load-shared-object (car cs)) #t) #t)
(else (loop (cdr cs)))))))
;; --- expose under jolt.ffi ---------------------------------------------------
(def-var! "jolt.ffi" "free-callable" ffi-free-callable)
(def-var! "jolt.ffi" "load-library" ffi-load-library)
(def-var! "jolt.ffi" "loaded?" (lambda (n) (if (ffi-loaded? n) #t #f)))
(def-var! "jolt.ffi" "alloc" ffi-alloc)
(def-var! "jolt.ffi" "free" ffi-free)
(def-var! "jolt.ffi" "read" ffi-read)
(def-var! "jolt.ffi" "write" ffi-write)
(def-var! "jolt.ffi" "sizeof" ffi-sizeof)
(def-var! "jolt.ffi" "null?" (lambda (p) (if (ffi-null? p) #t #f)))
(def-var! "jolt.ffi" "null" ffi-null)
(def-var! "jolt.ffi" "ptr->string" ffi-ptr->string)
(def-var! "jolt.ffi" "string->ptr" ffi-string->ptr)

View file

@ -0,0 +1,205 @@
;; host class tokens — a bare class name (String, Keyword, File...)
;; evaluates to its JVM canonical-name STRING, the same value (class instance)
;; returns, so (= String (class "x")) holds and a (defmethod m String ...) keys
;; against a (class …) dispatch (ring.util.request does this).
;; The analyzer resolves these names to clojure.core vars, so the back end emits
;; (var-deref "clojure.core" "String") — def-var!'ing the canonical strings here is
;; all that's needed at runtime.
;;
;; Loaded after natives-meta.ss (jolt-type) + the printer (jolt-str-render-one).
;; (class x) — Clojure's class of a value. Scalars map to their JVM class name,
;; matching core-class. Collections/seqs have no JVM class on this host;
;; (str (type x)) is the clean host taxonomy and
;; is never compared against a class token in the corpus. Records yield their
;; ns-qualified class name (= (str (type x))). Total — never crashes.
;; A host shim (bigdec, queue, host-table) registers its type's class name via
;; register-class-arm! instead of set!-wrapping jolt-class (cf. register-hash-arm!).
;; The entry is stable, so the var cell bound below stays current as arms register.
(define jolt-class-arms '())
(define (register-class-arm! pred handler)
(set! jolt-class-arms (cons (cons pred handler) jolt-class-arms)))
(define (jolt-class-base x)
(cond
((jolt-nil? x) jolt-nil)
((boolean? x) "java.lang.Boolean")
;; per-type number classes, like the JVM: integer -> Long, flonum -> Double,
;; exact non-integer -> Ratio.
((and (number? x) (flonum? x)) "java.lang.Double")
((and (number? x) (exact? x) (integer? x)) "java.lang.Long")
((and (number? x) (exact? x) (rational? x)) "clojure.lang.Ratio")
((number? x) "java.lang.Number")
((string? x) "java.lang.String")
((keyword? x) "clojure.lang.Keyword")
((symbol-t? x) "clojure.lang.Symbol")
((jolt-atom? x) "clojure.lang.Atom")
((char? x) "java.lang.Character")
((regex-t? x) "java.util.regex.Pattern")
;; an anonymous / unregistered fn — like the JVM, where (class #(..)) is a
;; concrete ns$fn__N subclass. The $fn marker lets clojure.spec.alpha's fn-sym
;; recognize it as anonymous and return ::s/unknown. A named fn is registered
;; (proc-name-tbl) and handled by a class-arm with its real ns$name.
((procedure? x) "clojure.lang.AFunction$fn")
;; an exception value (ex-info / host-constructed throwable) reports its JVM
;; class, so (= clojure.lang.ExceptionInfo (class e)) and clojure.test's
;; (thrown? Class …) match (records.ss ex-info-map?/ex-info-class).
((ex-info-map? x) (ex-info-class x))
;; persistent collections + namespace report their JVM class names (not jolt's
;; internal :vector/:set/… type keyword), so class-based dispatch — e.g. a
;; defmulti on [(class a) (class b)] — sees a real clojure.lang.* class.
((jns? x) "clojure.lang.Namespace")
((pvec? x) "clojure.lang.PersistentVector")
((pset? x) "clojure.lang.PersistentHashSet")
((pmap? x) "clojure.lang.PersistentArrayMap")
((jolt-lazyseq? x) "clojure.lang.LazySeq")
((empty-list-t? x) "clojure.lang.PersistentList$EmptyList")
((cseq? x) "clojure.lang.PersistentList")
(else (jolt-str-render-one (jolt-type x)))))
;; the class NAME of x (string), or nil for nil. (class x) wraps it in a Class
;; value (make-class-obj, host-static-classes.ss) so it renders like a JVM Class
;; while staying = its name string.
;; a raw Chez condition Clojure raises a specific class for (records-interop.ss
;; chez-condition-exc-class) reports that JVM class, so (class e) and a
;; (thrown? ArityException …) test match — not the opaque :object fallback.
(register-class-arm!
(lambda (x) (and (chez-condition-exc-class x) #t))
(lambda (x) (let ((p (assoc (chez-condition-exc-class x) class-token-alist)))
(if p (cdr p) "java.lang.IllegalArgumentException"))))
;; A fn def'd into a var reports a JVM-style class name "ns$munged-name" (the
;; forward CHAR_MAP), so clojure.spec.alpha's fn-sym (which splits on $ and
;; demunges) recovers the predicate's symbol. Anonymous / unregistered fns stay
;; clojure.lang.IFn (fn-sym yields :unknown, as on the JVM).
(define class-munge-map
'((#\? . "_QMARK_") (#\! . "_BANG_") (#\* . "_STAR_") (#\+ . "_PLUS_")
(#\> . "_GT_") (#\< . "_LT_") (#\= . "_EQ_") (#\/ . "_SLASH_") (#\- . "_")
(#\& . "_AMPERSAND_") (#\% . "_PERCENT_") (#\~ . "_TILDE_") (#\^ . "_CARET_")
(#\| . "_BAR_") (#\: . "_COLON_")))
(define (class-munge-name s)
(let ((out (open-output-string)))
(string-for-each
(lambda (c) (let ((t (assv c class-munge-map))) (if t (display (cdr t) out) (write-char c out))))
s)
(get-output-string out)))
(register-class-arm!
(lambda (x) (and (procedure? x) (hashtable-ref proc-name-tbl x #f)))
(lambda (x) (let ((p (hashtable-ref proc-name-tbl x #f)))
(string-append (car p) "$" (class-munge-name (cdr p))))))
(define (jolt-class-name x)
(let loop ((as jolt-class-arms))
(cond ((null? as) (jolt-class-base x))
(((caar as) x) ((cdar as) x))
(else (loop (cdr as))))))
(define (jolt-class x)
(let ((n (jolt-class-name x)))
(if (jolt-nil? n) jolt-nil (make-class-obj n))))
(def-var! "clojure.core" "class" jolt-class)
;; The PUBLIC clojure.core/type — Clojure's (or (:type meta) (class x)). This is the
;; java host layer's job: the core taxonomy (natives-meta.ss jolt-type, kept under
;; __type-tag for print-method) is JVM-free, and the JVM class mapping lives HERE,
;; next to (class …). The inst/array/byte-buffer host files extend `class` (a
;; class-arm or jolt-type fallthrough) and re-point `type` at this same fn, so the
;; remap of every value — :jolt/inst -> java.util.Date etc. — happens in one place.
(define ty-meta-key (keyword #f "type"))
(define (jolt-type-pub x)
(let* ((m (jolt-meta x))
(override (if (jolt-nil? m) jolt-nil (jolt-get m ty-meta-key jolt-nil))))
(if (not (jolt-nil? override)) override (jolt-class x))))
(def-var! "clojure.core" "type" jolt-type-pub)
;; bare class-name tokens -> canonical JVM class-name strings.
(define class-token-alist
'(("String" . "java.lang.String") ("Number" . "java.lang.Number")
("Boolean" . "java.lang.Boolean") ("Long" . "java.lang.Long")
("Integer" . "java.lang.Integer") ("Double" . "java.lang.Double")
("Float" . "java.lang.Float") ("Byte" . "java.lang.Byte") ("Short" . "java.lang.Short")
("Object" . "java.lang.Object") ("Character" . "java.lang.Character")
("InputStream" . "java.io.InputStream") ("OutputStream" . "java.io.OutputStream")
("File" . "java.io.File") ("Reader" . "java.io.Reader") ("Writer" . "java.io.Writer")
("ISeq" . "clojure.lang.ISeq") ("Keyword" . "clojure.lang.Keyword")
("Symbol" . "clojure.lang.Symbol") ("MapEntry" . "clojure.lang.MapEntry")
("StringReader" . "java.io.StringReader") ("StringWriter" . "java.io.StringWriter")
("StringBuilder" . "java.lang.StringBuilder")
("StringTokenizer" . "java.util.StringTokenizer")
("Charset" . "java.nio.charset.Charset") ("Base64" . "java.util.Base64")
("Exception" . "java.lang.Exception")
("IllegalArgumentException" . "java.lang.IllegalArgumentException")
("ArityException" . "clojure.lang.ArityException")
("IllegalStateException" . "java.lang.IllegalStateException")
("RuntimeException" . "java.lang.RuntimeException")
("UnsupportedOperationException" . "java.lang.UnsupportedOperationException")
("InterruptedException" . "java.lang.InterruptedException")
("IOException" . "java.io.IOException")
("UnknownHostException" . "java.net.UnknownHostException")
("ConnectException" . "java.net.ConnectException")
("SocketTimeoutException" . "java.net.SocketTimeoutException")
("MalformedURLException" . "java.net.MalformedURLException")
("SSLException" . "javax.net.ssl.SSLException")
("ExceptionInfo" . "clojure.lang.ExceptionInfo")
("IExceptionInfo" . "clojure.lang.IExceptionInfo")
("Pattern" . "java.util.regex.Pattern")
("URI" . "java.net.URI") ("UUID" . "java.util.UUID")
("ArrayList" . "java.util.ArrayList") ("PersistentQueue" . "clojure.lang.PersistentQueue")
("NumberFormatException" . "java.lang.NumberFormatException")
("ArithmeticException" . "java.lang.ArithmeticException")
("NullPointerException" . "java.lang.NullPointerException")
("ClassCastException" . "java.lang.ClassCastException")
("IndexOutOfBoundsException" . "java.lang.IndexOutOfBoundsException")
("UnsupportedEncodingException" . "java.io.UnsupportedEncodingException")
("FileNotFoundException" . "java.io.FileNotFoundException")
("Throwable" . "java.lang.Throwable")
;; clojure.lang / java.util types that class-based multimethods dispatch on.
("Fn" . "clojure.lang.Fn") ("IFn" . "clojure.lang.IFn")
("Namespace" . "clojure.lang.Namespace") ("Named" . "clojure.lang.Named")
("Set" . "java.util.Set") ("List" . "java.util.List") ("Map" . "java.util.Map")
("Collection" . "java.util.Collection") ("Iterable" . "java.lang.Iterable")
("CharSequence" . "java.lang.CharSequence") ("Comparable" . "java.lang.Comparable")
("Runnable" . "java.lang.Runnable") ("Callable" . "java.util.concurrent.Callable")
("IPersistentSet" . "clojure.lang.IPersistentSet")
("IPersistentVector" . "clojure.lang.IPersistentVector")
("IPersistentMap" . "clojure.lang.IPersistentMap")
("IPersistentCollection" . "clojure.lang.IPersistentCollection")
("Sequential" . "clojure.lang.Sequential") ("Seqable" . "clojure.lang.Seqable")
("Associative" . "clojure.lang.Associative")))
(for-each
(lambda (pair) (def-var! "clojure.core" (car pair) (cdr pair)))
class-token-alist)
;; resolve a ^Type hint symbol-name to its canonical class name at def time:
;; "String" -> "java.lang.String", matching the JVM compiler. An
;; already-canonical name maps to itself; an unknown name yields #f (left as-is).
(define class-hint-table (make-hashtable string-hash string=?))
(for-each (lambda (p) (hashtable-set! class-hint-table (car p) (cdr p))) class-token-alist)
(for-each (lambda (p) (hashtable-set! class-hint-table (cdr p) (cdr p))) class-token-alist)
(define (resolve-class-hint name) (hashtable-ref class-hint-table name #f))
(def-var! "jolt.host" "resolve-class-hint" resolve-class-hint)
;; fully-qualified canonical class names self-evaluate to their own name string,
;; so (= (class 1) java.lang.Long) and (instance? clojure.lang.Atom x) resolve the
;; class token (= what jolt-class / instance-check key on).
;; Value classes only — NOT the collection interfaces (ISeq/IPersistentMap/...),
;; which downstream code (e.g. SCI) references as protocols/interfaces.
(for-each
(lambda (nm) (def-var! "clojure.core" nm nm))
'("java.lang.Long" "java.lang.Integer" "java.lang.Double" "java.lang.Float"
"java.lang.Byte" "java.lang.Short"
"java.lang.Number" "java.lang.String" "java.lang.Boolean" "java.lang.Character"
"java.lang.Object"
;; exception classes compared against (class e): (= java.net.SocketTimeoutException (class e))
"java.lang.Exception" "java.lang.Throwable" "java.lang.RuntimeException"
"java.lang.IllegalArgumentException" "java.lang.IllegalStateException"
"java.lang.UnsupportedOperationException" "java.io.IOException"
"java.net.UnknownHostException" "java.net.ConnectException"
"java.net.SocketTimeoutException" "java.net.MalformedURLException"
"javax.net.ssl.SSLException"
"java.lang.NumberFormatException" "java.lang.ArithmeticException"
"java.lang.NullPointerException" "java.lang.ClassCastException"
"java.lang.IndexOutOfBoundsException" "java.io.FileNotFoundException"
"java.io.UnsupportedEncodingException"
;; clojure.lang.ExceptionInfo / IExceptionInfo compared against (class e)
"clojure.lang.ExceptionInfo" "clojure.lang.IExceptionInfo" "clojure.lang.ArityException"
"java.util.regex.Pattern" "java.net.URI" "java.util.UUID"
"clojure.lang.PersistentQueue"
"clojure.lang.Keyword" "clojure.lang.Symbol" "clojure.lang.Ratio" "clojure.lang.Atom"))

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,443 @@
;; host-static-methods.ss — the `Class/member` static surface: java.lang.Math,
;; System (properties/env), Thread, the Long/Integer/Double/Character/String static
;; methods, java.text.NumberFormat, and the Class registry. Registers into
;; host-static.ss's class-statics table (loaded just before this); instantiable host
;; object classes (ArrayList, StringBuilder, …) live in host-static-classes.ss.
;; ---- java.lang statics ------------------------------------------------------
;; java.lang.Math: sqrt/pow/floor/ceil/trig/log/exp always return a DOUBLE on the
;; JVM (Chez's sqrt/expt return EXACT for exact args, e.g. (sqrt 9) -> 3), so coerce
;; to flonum. round -> long (exact); abs/max/min preserve the argument's type.
(define (->dbl x) (exact->inexact x))
(register-class-statics! "Math"
(list (cons "sqrt" (lambda (x) (->dbl (sqrt x))))
(cons "pow" (lambda (a b) (->dbl (expt a b))))
(cons "floor" (lambda (x) (->dbl (floor x))))
(cons "ceil" (lambda (x) (->dbl (ceiling x))))
(cons "round" (lambda (x) (exact (floor (+ x 1/2))))) ; JVM round-half-up -> long
(cons "abs" (lambda (x) (abs x)))
(cons "sin" (lambda (x) (->dbl (sin x)))) (cons "cos" (lambda (x) (->dbl (cos x))))
(cons "tan" (lambda (x) (->dbl (tan x)))) (cons "asin" (lambda (x) (->dbl (asin x))))
(cons "acos" (lambda (x) (->dbl (acos x)))) (cons "atan" (lambda (x) (->dbl (atan x))))
(cons "log" (lambda (x) (->dbl (log x)))) (cons "log10" (lambda (x) (->dbl (/ (log x) (log 10)))))
(cons "exp" (lambda (x) (->dbl (exp x))))
;; getExponent: the unbiased binary exponent of a double (floor(log2|x|));
;; scalb: x * 2^n. test.check's double generator uses both.
(cons "getExponent" (lambda (x) (if (= x 0.0) -1023
(exact (floor (/ (log (abs (exact->inexact x))) (log 2.0)))))))
(cons "scalb" (lambda (x n) (->dbl (* (exact->inexact x) (expt 2.0 (jnum->exact n))))))
(cons "max" (lambda (a b) (if (> a b) a b))) (cons "min" (lambda (a b) (if (< a b) a b)))
(cons "signum" (lambda (x) (cond ((< x 0) -1.0) ((> x 0) 1.0) (else 0.0))))
(cons "PI" (->dbl (* 4 (atan 1)))) (cons "E" (->dbl (exp 1)))
(cons "random" (lambda args (random 1.0)))))
;; Thread: real OS threads back futures/promises.
;; - sleep parks the calling thread for `ms` ms (a worker sleeping doesn't block
;; the parent).
;; - yield hands the CPU to another runnable thread (libc sched_yield).
;; - each thread carries an interrupt flag; interrupted (static) reads AND clears
;; the current thread's flag, matching the JVM. currentThread / .interrupt /
;; .isInterrupted are wired in io.ss, where the thread handle is built.
;; Per-thread interrupt flag, lazily allocated so each OS thread gets its own box.
;; A thread handle (from currentThread) captures this box, so .interrupt from
;; another thread sets the target thread's flag.
(define thread-interrupt-box (make-thread-parameter #f))
(define (current-interrupt-box)
(or (thread-interrupt-box)
(let ((b (box #f))) (thread-interrupt-box b) b)))
(define (clear-thread-interrupt!) (set-box! (current-interrupt-box) #f))
;; libc sched_yield, resolved once; fall back to a zero-length park if the symbol
;; isn't available.
(define thread-yield!
(let ((fp #f) (tried? #f))
(lambda ()
(unless tried?
(set! tried? #t)
(set! fp (jolt-foreign-proc-safe "sched_yield" '() 'int)))
(if fp (fp) (sleep (make-time 'time-duration 0 0)))
jolt-nil)))
(define thread-statics
(list (cons "sleep" (lambda (ms . _)
(let* ((ms* (exact (floor ms)))
(secs (quotient ms* 1000))
(nanos (* (remainder ms* 1000) 1000000)))
(sleep (make-time 'time-duration nanos secs)))
jolt-nil))
(cons "yield" (lambda _ (thread-yield!)))
(cons "interrupted" (lambda _ (let* ((b (current-interrupt-box)) (v (unbox b)))
(set-box! b #f) (and v #t))))))
(register-class-statics! "Thread" thread-statics)
(register-class-statics! "java.lang.Thread" thread-statics)
;; clojure.lang.LockingTransaction: jolt has no STM (no refs/dosync), so a
;; transaction is never running. isRunning -> false.
(register-class-statics! "LockingTransaction" (list (cons "isRunning" (lambda () #f))))
(register-class-statics! "clojure.lang.LockingTransaction" (list (cons "isRunning" (lambda () #f))))
;; clojure.lang.LazilyPersistentVector/createOwning: build a vector from an array
;; (malli's -vmap fills an object-array then hands it over). jolt has no array
;; ownership transfer, so copy the array's elements into a persistent vector.
(define (lpv-create-owning arr) (apply jolt-vector (seq->list (jolt-seq arr))))
(register-class-statics! "LazilyPersistentVector" (list (cons "createOwning" lpv-create-owning)))
(register-class-statics! "clojure.lang.LazilyPersistentVector" (list (cons "createOwning" lpv-create-owning)))
;; clojure.lang.PersistentArrayMap/createWithCheck: build a map from a [k v k v…]
;; array, throwing on a duplicate key. malli's eager entry parser relies on the
;; throw to report ::duplicate-keys, so a missing class would mis-fire on every
;; map. Build the map and signal if a key collapsed (count*2 < array length).
(define (pam-create-with-check arr)
(let ((items (seq->list (jolt-seq arr))))
(let loop ((xs items) (m (jolt-hash-map)))
(if (null? xs) m
(if (null? (cdr xs)) (error #f "PersistentArrayMap: odd key/value count")
(let ((k (car xs)))
(if (jolt-contains? m k) (error #f "Duplicate key")
(loop (cddr xs) (jolt-assoc m k (cadr xs))))))))))
(register-class-statics! "PersistentArrayMap" (list (cons "createWithCheck" pam-create-with-check)))
(register-class-statics! "clojure.lang.PersistentArrayMap" (list (cons "createWithCheck" pam-create-with-check)))
;; clojure.lang.RT/map: build a map from a [k v k v…] array/seq (RT.map). Small
;; maps keep insertion order (PersistentArrayMap). tools.reader builds map and
;; namespaced-map literals this way.
(define (rt-map arr)
(let loop ((xs (if (jolt-nil? arr) '() (seq->list (jolt-seq arr)))) (m (jolt-hash-map)))
(cond ((null? xs) m)
((null? (cdr xs)) (error #f "RT/map: odd key/value count"))
(else (loop (cddr xs) (jolt-assoc m (car xs) (cadr xs)))))))
(register-class-statics! "RT" (list (cons "map" rt-map)))
(register-class-statics! "clojure.lang.RT" (list (cons "map" rt-map)))
;; clojure.lang.PersistentList/create: a list (in order) from a seq; empty -> ().
(define (plist-create x)
(let ((items (seq->list (jolt-seq x))))
(if (null? items) jolt-empty-list (list->cseq items))))
(register-class-statics! "PersistentList" (list (cons "create" plist-create)))
(register-class-statics! "clojure.lang.PersistentList" (list (cons "create" plist-create)))
;; clojure.lang.PersistentHashSet/createWithCheck: a set from a seq, throwing on a
;; duplicate element (tools.reader's #{…} reader reports the dup).
(define (phs-create-with-check x)
(let loop ((xs (seq->list (jolt-seq x))) (s (jolt-hash-set)))
(if (null? xs) s
(let ((e (car xs)))
(if (jolt-truthy? (jolt-contains? s e))
(jolt-throw (jolt-ex-info (string-append "Duplicate key: " (jolt-str-render-one e)) (jolt-hash-map)))
(loop (cdr xs) (jolt-conj1 s e)))))))
(register-class-statics! "PersistentHashSet" (list (cons "createWithCheck" phs-create-with-check)))
(register-class-statics! "clojure.lang.PersistentHashSet" (list (cons "createWithCheck" phs-create-with-check)))
;; java.lang.Character statics. digit(ch, radix) -> the digit value or -1; ch may
;; be a char or an int codepoint (tools.reader passes (int c)). isDigit/
;; isWhitespace take a char; valueOf boxes a char (identity on jolt).
(define (char->cp x) (if (char? x) (char->integer x) (jnum->exact x)))
(define (char-digit-value cp radix)
(let ((d (cond ((and (fx>=? cp 48) (fx<=? cp 57)) (fx- cp 48)) ; 0-9
((and (fx>=? cp 97) (fx<=? cp 122)) (fx+ 10 (fx- cp 97))) ; a-z
((and (fx>=? cp 65) (fx<=? cp 90)) (fx+ 10 (fx- cp 65))) ; A-Z
(else 99))))
(if (fx<? d radix) d -1)))
(define character-statics
(list (cons "digit" (lambda (ch radix) (->num (char-digit-value (char->cp ch) (jnum->exact radix)))))
(cons "isDigit" (lambda (ch) (let ((cp (char->cp ch))) (and (fx>=? cp 48) (fx<=? cp 57)))))
(cons "isWhitespace" (lambda (ch) (char-whitespace? (integer->char (char->cp ch)))))
(cons "valueOf" (lambda (ch) (if (char? ch) ch (integer->char (char->cp ch)))))))
(register-class-statics! "Character" character-statics)
(register-class-statics! "java.lang.Character" character-statics)
;; java.util.regex.Pattern/compile: a regex value from a string pattern.
(define pattern-statics (list (cons "compile" (lambda (s) (jolt-regex (jolt-str-render-one s))))))
(register-class-statics! "Pattern" pattern-statics)
(register-class-statics! "java.util.regex.Pattern" pattern-statics)
;; clojure.lang.BigInt / clojure.lang.Numbers: jolt has one exact-integer type
;; (Chez bignums auto-reduce), so BigInt.fromBigInteger and Numbers.reduceBigInt
;; are identity. tools.reader's number parser threads integers through these.
(define identity-num-statics (list (cons "fromBigInteger" (lambda (x) x))))
(register-class-statics! "BigInt" identity-num-statics)
(register-class-statics! "clojure.lang.BigInt" identity-num-statics)
(register-class-statics! "Numbers"
(list (cons "reduceBigInt" (lambda (x) x)) (cons "toRatio" (lambda (x) x))))
(register-class-statics! "clojure.lang.Numbers"
(list (cons "reduceBigInt" (lambda (x) x)) (cons "toRatio" (lambda (x) x))))
(define (now-millis)
(let ((t (current-time 'time-utc)))
(+ (* 1000 (time-second t)) (quotient (time-nanosecond t) 1000000))))
;; clojure.core/current-time-ms — epoch milliseconds; backs the `time` macro.
(def-var! "clojure.core" "current-time-ms" (lambda () (->num (now-millis))))
(register-class-statics! "System"
(list (cons "currentTimeMillis" (lambda () (->num (now-millis))))
(cons "nanoTime" (lambda () (->num (* 1000000 (now-millis)))))
(cons "exit" (lambda args (exit (if (null? args) 0 (jnum->exact (car args))))))
;; System/gc -> a full Chez collection (so weak references clear and their
;; guardians fire); Runtime.gc() routes here too.
(cons "gc" (lambda _ (collect (collect-maximum-generation)) jolt-nil))
;; wrapped in lambdas: the helpers are defined below, resolved at call time.
(cons "getProperty" (lambda (k . d) (apply sys-get-property k d)))
(cons "setProperty" (lambda (k v) (sys-set-property k v)))
(cons "clearProperty" (lambda (k) (sys-clear-property k)))
(cons "getProperties" (lambda () (sys-properties-map)))
(cons "getenv" (lambda k (apply sys-getenv k)))))
;; java.lang.Long.bitCount: the population count of the value's 64-bit two's-
;; complement (mask to 64 bits so a negative long counts like the JVM, e.g.
;; bitCount(-1) = 64). test.check's splittable PRNG uses it.
(define long-mask64 #xFFFFFFFFFFFFFFFF)
(define long-2^63 (expt 2 63))
(define long-2^64 (expt 2 64))
;; interpret a 64-bit value as a signed long (top bit = sign), like the JVM.
(define (as-signed64 v) (if (>= v long-2^63) (- v long-2^64) v))
(define (long-nlz n) (- 64 (integer-length (bitwise-and (jnum->exact n) long-mask64))))
(define (long-reverse n)
(let ((v (bitwise-and (jnum->exact n) long-mask64)))
(let loop ((i 0) (r 0))
(if (fx=? i 64) (as-signed64 r)
(loop (fx+ i 1)
(bitwise-ior (bitwise-arithmetic-shift-left r 1)
(bitwise-and (bitwise-arithmetic-shift-right v i) 1)))))))
(register-class-statics! "Long"
(list (cons "TYPE" "long")
(cons "MAX_VALUE" (->num 9223372036854775807))
(cons "MIN_VALUE" (->num -9223372036854775808))
(cons "bitCount" (lambda (n) (->num (bitwise-bit-count (bitwise-and (jnum->exact n) long-mask64)))))
(cons "numberOfLeadingZeros" (lambda (n) (->num (long-nlz n))))
(cons "reverse" (lambda (n) (->num (long-reverse n))))
(cons "parseLong" (lambda (s . r) (parse-int-or-throw s (if (null? r) 10 (jnum->exact (car r))) "parseLong")))
(cons "valueOf" (lambda (s . r) (parse-int-or-throw s (if (null? r) 10 (jnum->exact (car r))) "valueOf")))))
;; JVM Integer.toHexString/etc. treat the int as 32-bit unsigned.
(define (int->u32 n) (if (< n 0) (+ n 4294967296) n))
(register-class-statics! "Integer"
(list (cons "MAX_VALUE" (->num 2147483647)) (cons "MIN_VALUE" (->num -2147483648))
;; the primitive class token (int.class); jolt models a class as its name
(cons "TYPE" "int")
(cons "valueOf" (lambda (x . r)
(if (number? x) (->num x)
(parse-int-or-throw x (if (null? r) 10 (jnum->exact (car r))) "valueOf"))))
(cons "parseInt" (lambda (x . r) (parse-int-or-throw x (if (null? r) 10 (jnum->exact (car r))) "parseInt")))
;; lowercase, like the JVM; a negative int is the 32-bit unsigned form.
(cons "toHexString" (lambda (x) (string-downcase (number->string (int->u32 (jnum->exact x)) 16))))
(cons "toOctalString" (lambda (x) (number->string (int->u32 (jnum->exact x)) 8)))
(cons "toBinaryString" (lambda (x) (number->string (int->u32 (jnum->exact x)) 2)))
(cons "toString" (lambda (x . r) (number->string (jnum->exact x) (if (null? r) 10 (jnum->exact (car r))))))))
;; Byte / Short bounds (their values are plain integers on jolt; the statics let
;; libraries reference the JVM ranges — clojure.test.check generates over them).
(register-class-statics! "Byte"
(list (cons "TYPE" "byte")
(cons "MAX_VALUE" (->num 127)) (cons "MIN_VALUE" (->num -128))
(cons "valueOf" (lambda (x . r) (->num (if (number? x) x (parse-int-or-throw x 10 "valueOf")))))
(cons "parseByte" (lambda (x . r) (parse-int-or-throw x (if (null? r) 10 (jnum->exact (car r))) "parseByte")))
(cons "toString" (lambda (x . r) (number->string (jnum->exact x))))))
(register-class-statics! "Short"
(list (cons "TYPE" "short")
(cons "MAX_VALUE" (->num 32767)) (cons "MIN_VALUE" (->num -32768))
(cons "valueOf" (lambda (x . r) (->num (if (number? x) x (parse-int-or-throw x 10 "valueOf")))))
(cons "parseShort" (lambda (x . r) (parse-int-or-throw x (if (null? r) 10 (jnum->exact (car r))) "parseShort")))
(cons "toString" (lambda (x . r) (number->string (jnum->exact x))))))
;; java.util.Locale — jolt's case ops are codepoint-based (locale-independent), so
;; the default locale is a no-op token. Libraries set/restore it around formatting
;; to prove output is locale-stable (honeysql's Turkish-İ regression guard).
(register-class-statics! "Locale"
(list (cons "getDefault" (lambda () "und"))
(cons "setDefault" (lambda (x) jolt-nil))
(cons "forLanguageTag" (lambda (tag) (if (string? tag) tag (jolt-str-render-one tag))))
(cons "ROOT" "und") (cons "US" "en-US") (cons "ENGLISH" "en")))
(register-class-statics! "Boolean"
(list (cons "TYPE" "boolean")
(cons "parseBoolean" (lambda (s) (string=? "true" (ascii-string-down (if (string? s) s (jolt-str-render-one s))))))
(cons "TRUE" #t) (cons "FALSE" #f)))
(register-class-ctor! "Double" ->double)
(register-class-ctor! "Float" ->double)
(register-class-statics! "Double"
(list (cons "TYPE" "double")
(cons "parseDouble" parse-double-or-throw)
(cons "valueOf" ->double)
(cons "toString" (lambda (x) (jolt-str-render-one (->double x))))
(cons "isNaN" (lambda (x) (and (flonum? x) (nan? x))))
(cons "isInfinite" (lambda (x) (and (flonum? x) (infinite? x))))
(cons "MAX_VALUE" 1.7976931348623157e308) (cons "MIN_VALUE" 4.9e-324)
(cons "POSITIVE_INFINITY" +inf.0) (cons "NEGATIVE_INFINITY" -inf.0) (cons "NaN" +nan.0)))
(register-class-statics! "Float"
(list (cons "TYPE" "float")
(cons "parseFloat" parse-double-or-throw) (cons "valueOf" ->double)))
;; Character: ASCII predicates (the engine is byte/ASCII oriented).
(register-class-statics! "Character"
(list (cons "TYPE" "char")
(cons "isUpperCase" (lambda (c) (let ((n (char-code c))) (and (>= n 65) (<= n 90)))))
(cons "isLowerCase" (lambda (c) (let ((n (char-code c))) (and (>= n 97) (<= n 122)))))
(cons "isDigit" (lambda (c) (let ((n (char-code c))) (and (>= n 48) (<= n 57)))))
;; JVM Character.isWhitespace: Unicode whitespace (so U+2028 line separator
;; counts, like the JVM) MINUS the no-break spaces the JVM excludes
;; (U+00A0/U+2007/U+202F). char<=?space missed everything above ASCII.
(cons "isWhitespace" (lambda (c) (let ((cp (char-code c)))
(and (char-whitespace? (integer->char cp))
(not (fx=? cp #xA0)) (not (fx=? cp #x2007)) (not (fx=? cp #x202F))))))))
;; String/valueOf(Object): "null" for nil, else jolt's str semantics.
;; String/format(fmt args…) / (locale fmt args…) -> the clojure.core format engine.
(register-class-statics! "String"
(list (cons "valueOf" (lambda (x . _) (if (jolt-nil? x) "null" (jolt-str-render-one x))))
(cons "format" (lambda (a . rest)
(if (and (jhost? a) (string=? (jhost-tag a) "locale"))
(apply jolt-format (car rest) (cdr rest))
(apply jolt-format a rest))))))
;; ---- java.text.NumberFormat -------------------------------------------------
;; A grouping decimal formatter (selmer number-format / cuerdas). state:
;; #(grouping? min-frac max-frac). .format groups the integer part with commas.
(define (nf-make grouping? minf maxf) (make-jhost "numberformat" (vector grouping? minf maxf)))
(define (group-int-str s) ; "1234567" -> "1,234,567"
(let* ((neg (and (> (string-length s) 0) (char=? (string-ref s 0) #\-)))
(digs (if neg (substring s 1 (string-length s)) s))
(n (string-length digs)) (out '()))
(let loop ((i 0))
(when (< i n)
(when (and (> i 0) (= 0 (modulo (- n i) 3))) (set! out (cons #\, out)))
(set! out (cons (string-ref digs i) out)) (loop (+ i 1))))
(string-append (if neg "-" "") (list->string (reverse out)))))
(define (nf-format self x)
(let* ((grouping? (vector-ref (jhost-state self) 0))
(minf (vector-ref (jhost-state self) 1)) (maxf (vector-ref (jhost-state self) 2))
(neg (< x 0)) (ax (abs (exact->inexact x)))
(scale (expt 10 maxf))
(scaled (exact (round (* ax scale))))
(ipart (quotient scaled scale)) (fpart (remainder scaled scale))
(istr (number->string ipart))
(fstr0 (if (> maxf 0) (let ((s (number->string fpart)))
(string-append (make-string (max 0 (- maxf (string-length s))) #\0) s)) ""))
;; trim trailing zeros down to minf
(fstr (let loop ((s fstr0)) (if (and (> (string-length s) minf)
(char=? (string-ref s (- (string-length s) 1)) #\0))
(loop (substring s 0 (- (string-length s) 1))) s))))
(string-append (if neg "-" "") (if grouping? (group-int-str istr) istr)
(if (> (string-length fstr) 0) (string-append "." fstr) ""))))
(register-host-methods! "numberformat"
(list (cons "format" (lambda (self n) (nf-format self n)))
(cons "setMaximumFractionDigits" (lambda (self d) (vector-set! (jhost-state self) 2 (jnum->exact d)) jolt-nil))
(cons "setMinimumFractionDigits" (lambda (self d) (vector-set! (jhost-state self) 1 (jnum->exact d)) jolt-nil))
(cons "setGroupingUsed" (lambda (self b) (vector-set! (jhost-state self) 0 (jolt-truthy? b)) jolt-nil))))
(let ((nf-statics (list (cons "getInstance" (lambda _ (nf-make #t 0 3)))
(cons "getNumberInstance" (lambda _ (nf-make #t 0 3)))
(cons "getIntegerInstance" (lambda _ (nf-make #t 0 0))))))
(register-class-statics! "NumberFormat" nf-statics)
(register-class-statics! "java.text.NumberFormat" nf-statics))
;; Class.forName: an array descriptor ("[C") is its own class token; a class Jolt
;; can back (registered statics/ctor, or a java.*/clojure.* core class) yields a
;; class object; anything else throws a catchable ClassNotFoundException, like the
;; JVM — so the common `(try (Class/forName "optional.Dep") (catch …))` probe a
;; library uses to detect an absent dependency works (e.g. ring's joda-time check).
;; java.* / clojure.* packages jolt does NOT back, even though the broad prefix
;; below would otherwise claim them — optional backends a library feature-probes
;; with (Class/forName …) (e.g. tools.logging's java.util.logging / log4j). Listing
;; them here keeps class-found? honest so the probe sees them absent and skips the
;; backend (jolt has its own logging) instead of trying to use it and crashing.
(define forname-absent-prefixes
'("java.util.logging." "javax.management." "java.lang.management."))
(define (forname-known? nm)
;; exact lookups only — lookup-class would fall back to the short class name, so
;; any "x.y.Class" would spuriously match the registered java.lang.Class.
(or (hashtable-ref class-statics-tbl nm #f)
(hashtable-ref class-ctors-tbl nm #f)
(let ((pre? (lambda (p) (and (>= (string-length nm) (string-length p))
(string=? (substring nm 0 (string-length p)) p)))))
(and (or (pre? "java.") (pre? "clojure.") (pre? "jolt."))
(not (exists pre? forname-absent-prefixes))))))
(register-class-statics! "Class"
(list (cons "forName"
(lambda (nm . _)
(cond
((and (> (string-length nm) 0) (char=? (string-ref nm 0) #\[)) nm)
((forname-known? nm) (make-class-obj nm))
(else (jolt-throw (jolt-host-throwable "java.lang.ClassNotFoundException" nm))))))))
;; ---- System helpers (defined before use above via top-level order) ----------
;; os.name reflects the actual platform (Chez's machine-type names it): a *osx
;; machine is macOS, otherwise Linux. Code that branches on the OS (socket struct
;; layout, path handling) needs the truth, not a fixed value.
(define (substring-index needle hay)
(let ((nl (string-length needle)) (hl (string-length hay)))
(let loop ((i 0)) (cond ((> (+ i nl) hl) #f)
((string=? (substring hay i (+ i nl)) needle) i)
(else (loop (+ i 1)))))))
(define sys-os-name
(let ((m (symbol->string (machine-type))))
(cond ((or (substring-index "osx" m) (substring-index "macos" m)) "Mac OS X")
((or (substring-index "nt" m) (substring-index "windows" m)) "Windows")
(else "Linux"))))
;; runtime-settable system properties (System/setProperty). A set value wins over
;; the built-in defaults below; clearProperty removes it.
(define sys-prop-table (make-hashtable string-hash string=?))
(define (sys-set-property k v)
(let ((prev (hashtable-ref sys-prop-table k jolt-nil)))
(hashtable-set! sys-prop-table k (if (string? v) v (jolt-str-render-one v)))
prev))
(define (sys-clear-property k)
(let ((prev (hashtable-ref sys-prop-table k jolt-nil)))
(hashtable-delete! sys-prop-table k) prev))
(define (sys-get-property k . dflt)
(let ((set-val (hashtable-ref sys-prop-table k #f)))
(cond (set-val set-val)
((string=? k "os.name") sys-os-name)
((string=? k "line.separator") "\n")
((string=? k "file.separator") "/")
((string=? k "path.separator") ":")
((string=? k "user.dir") (or (getenv "PWD") "."))
((string=? k "user.home") (or (getenv "HOME") ""))
((string=? k "java.io.tmpdir") (or (getenv "TMPDIR") "/tmp"))
((pair? dflt) (car dflt))
(else jolt-nil))))
(define (sys-properties-map)
(jolt-hash-map "os.name" sys-os-name "line.separator" "\n" "file.separator" "/"
"user.dir" (or (getenv "PWD") ".") "user.home" (or (getenv "HOME") "")
"java.io.tmpdir" (or (getenv "TMPDIR") "/tmp")))
;; full environment as an alist of (name . value), via spawning `env`.
(define (all-env-pairs)
(call-with-values
(lambda () (open-process-ports "env" (buffer-mode block) (native-transcoder)))
(lambda (stdin stdout stderr pid)
(let loop ((acc '()))
(let ((l (get-line stdout)))
(if (eof-object? l) (reverse acc)
(let ((eq (let scan ((i 0)) (cond ((= i (string-length l)) #f)
((char=? (string-ref l i) #\=) i)
(else (scan (+ i 1)))))))
(loop (if eq (cons (cons (substring l 0 eq) (substring l (+ eq 1) (string-length l))) acc) acc)))))))))
;; JOLT_BAKE_ENV_ALLOWLIST: when set, only the listed comma-separated
;; names are served; unset (the normal case) reads are live and unfiltered.
(define (env-allowlist)
(let ((a (getenv "JOLT_BAKE_ENV_ALLOWLIST")))
(and a (map str-trim (str-literal-split a ",")))))
(define (sys-getenv . k)
(let ((allow (env-allowlist)))
(if (null? k)
(apply jolt-hash-map
(let loop ((ps (all-env-pairs)) (acc '()))
(cond ((null? ps) (reverse acc))
((and allow (not (member (caar ps) allow))) (loop (cdr ps) acc))
(else (loop (cdr ps) (cons (cdar ps) (cons (caar ps) acc)))))))
(let ((name (car k)))
(if (and allow (not (member name allow))) jolt-nil
(let ((v (getenv name))) (if v v jolt-nil)))))))
;; ---- StringBuilder ----------------------------------------------------------
;; state: a box (1-vector) holding the accumulated string.
(define (sb-str self) (vector-ref (jhost-state self) 0))
(define (sb-set! self s) (vector-set! (jhost-state self) 0 s))
(define (render-piece x)
(cond ((jolt-nil? x) "null") ((char? x) (string x)) ((string? x) x)
(else (jolt-str-render-one x))))
;; (Object.) — a fresh value with distinct identity (libraries use it as a lock
;; or a unique sentinel). Each call returns a new jhost so identical?/= separate.
(register-class-ctor! "Object" (lambda _ (make-jhost "object" (vector))))

View file

@ -0,0 +1,217 @@
;; host-static.ss — the host-interop registry core: the class-statics / class-ctors
;; / tagged-methods tables, the jhost record, and the coercion helpers. The actual
;; entries are registered by host-static-methods.ss (Class/member statics) and
;; host-static-classes.ss (instantiable object classes), loaded after this.
;;
;; The analyzer lowers `Class/member` to a :host-static node and `(Class. ...)` /
;; `(new Class ...)` to a :host-new node (jolt-core/jolt/analyzer.clj); the Chez
;; emit lowers a value ref to (host-static-ref "Class" "member"), a
;; call head to (host-static-call "Class" "member" args...), and a constructor to
;; (host-new "Class" args...). This file is the runtime registry those three
;; resolve against — the class-statics / class-ctors /
;; tagged-methods registries,
;; restricted to the java.lang/util/net/io surface portable cljc code calls.
;; (java.time formatting is a separate increment.)
;;
;; Constructed host objects are `jhost` records (a tag + mutable state); their
;; (.method ...) calls reach record-method-dispatch (records.ss), extended below
;; with a jhost arm that dispatches through host-tagged-methods.
;;
;; Loaded from rt.ss LAST (after natives-str.ss / records.ss): it extends
;; record-method-dispatch and reuses jolt-str-render-one / jolt-re-pattern.
;; ---- registries -------------------------------------------------------------
(define class-statics-tbl (make-hashtable string-hash string=?)) ; "Class" -> (member-ht)
(define class-ctors-tbl (make-hashtable string-hash string=?)) ; "Class" -> ctor proc
(define host-methods-tbl (make-hashtable string-hash string=?)) ; tag -> (method-ht)
;; A class token may arrive fully qualified (java.io.StringReader) or short
;; (StringReader). Register both; resolve by exact then by last dotted segment.
(define (short-class-name s)
(let loop ((i (- (string-length s) 1)))
(cond ((< i 0) s)
((char=? (string-ref s i) #\.) (substring s (+ i 1) (string-length s)))
(else (loop (- i 1))))))
(define (register-class-statics! name members) ; members: list of (str . val/proc)
(let ((h (or (hashtable-ref class-statics-tbl name #f)
(let ((h (make-hashtable string-hash string=?)))
(hashtable-set! class-statics-tbl name h) h))))
(for-each (lambda (p) (hashtable-set! h (car p) (cdr p))) members)))
(define (register-class-ctor! name proc) (hashtable-set! class-ctors-tbl name proc))
(define (register-host-methods! tag members)
(let ((h (or (hashtable-ref host-methods-tbl tag #f)
(let ((h (make-hashtable string-hash string=?)))
(hashtable-set! host-methods-tbl tag h) h))))
(for-each (lambda (p) (hashtable-set! h (car p) (cdr p))) members)))
(define (lookup-class h-tbl name)
(or (hashtable-ref h-tbl name #f)
(hashtable-ref h-tbl (short-class-name name) #f)))
;; ---- host object ------------------------------------------------------------
(define-record-type jhost (fields tag (mutable state)) (nongenerative chez-jhost-v1))
;; record-method-dispatch (records.ss) gets a jhost arm: dispatch (.method obj a*)
;; through the tag's method table.
;; clojure.lang.Sorted on jolt's sorted-map / sorted-set: comparator / entryKey /
;; seqFrom / seq. data.priority-map's subseq/rsubseq reach for these (its
;; PersistentPriorityMap delegates .comparator to the backing sorted-map). The
;; comparator is returned as a small Comparator object whose .compare runs the
;; map's 3-way fn, since (.. sc comparator (compare a b)) is the calling form.
(define sorted-cmp-kw (keyword #f "cmp"))
(register-host-methods! "jolt-comparator"
(list (cons "compare" (lambda (self a b) (jolt-invoke (jhost-state self) a b)))))
(define (sorted-comparator-of sc)
(let ((c (jolt-ref-get sc sorted-cmp-kw)))
(make-jhost "jolt-comparator" (if (jolt-nil? c) jolt-compare c))))
(define (sorted-iface-method? m)
(or (string=? m "comparator") (string=? m "entryKey")
(string=? m "seqFrom") (string=? m "seq")))
(define (sorted-iface-dispatch obj method rest)
(cond
((string=? method "comparator") (sorted-comparator-of obj))
((string=? method "entryKey") (jolt-first (car rest))) ; map entry -> its key
((string=? method "seq") ; (.seq sc) or (.seq sc ascending?)
(if (or (null? rest) (jolt-truthy? (car rest))) (jolt-seq obj) (jolt-rseq obj)))
;; (.seqFrom sc k ascending?) — the entries from k onward, in order. Done with a
;; comparator filter over the seq (jolt has no tree cursor), like subseq.
((string=? method "seqFrom")
(let* ((k (car rest)) (asc (jolt-truthy? (cadr rest)))
(cmp (jolt-ref-get obj sorted-cmp-kw))
(cmpf (if (jolt-nil? cmp) jolt-compare cmp))
(es (seq->list (jolt-seq obj)))
(keep (filter (lambda (e)
(let ((c (jnum->exact (jolt-invoke cmpf (jolt-first e) k))))
(if asc (>= c 0) (<= c 0))))
es)))
(list->cseq (if asc keep (reverse keep)))))
(else (error #f (string-append "No method " method " on sorted collection")))))
(register-method-arm! 44
(lambda (obj method-name rest-args)
(cond
((jhost? obj)
(let ((mh (hashtable-ref host-methods-tbl (jhost-tag obj) #f)))
(let ((f (and mh (hashtable-ref mh method-name #f))))
(if f
(apply f obj (if (jolt-nil? rest-args) '() (seq->list rest-args)))
(error #f (string-append "No method " method-name " on host " (jhost-tag obj)))))))
((number? obj) (apply number-method method-name obj (if (jolt-nil? rest-args) '() (seq->list rest-args))))
(else 'pass))))
;; java.lang.Number method surface (the boxed-number methods cljc code calls). The
;; integer projections wrap modulo their width (ring-codec relies on byteValue
;; overflow: (.byteValue 255) => -1); the float projections are identity flonums.
(define (number-method method n . args)
(cond
((string=? method "byteValue") (let ((b (modulo (jnum->exact n) 256))) (->num (if (>= b 128) (- b 256) b))))
((string=? method "shortValue") (let ((b (modulo (jnum->exact n) 65536))) (->num (if (>= b 32768) (- b 65536) b))))
((string=? method "intValue") (->num (jnum->exact n)))
((string=? method "longValue") (->num (jnum->exact n)))
((string=? method "doubleValue") (->num n))
((string=? method "floatValue") (->num n))
;; .toString(radix) — BigInteger/Integer render in a base, lowercase like the
;; JVM (rewrite-clj's integer node reconstructs 0xff / 0377 / 2r1001 this way).
((string=? method "toString")
(if (pair? args)
(string-downcase (number->string (jnum->exact n) (jnum->exact (car args))))
(jolt-num->string n)))
((string=? method "hashCode") (->num (jnum->exact n)))
;; Double/Float .isNaN / .isInfinite (a non-flonum is neither).
((string=? method "isNaN") (and (flonum? n) (not (= n n))))
((string=? method "isInfinite") (and (flonum? n) (infinite? n)))
;; BigInteger interop: .negate / .bitLength / .signum / .abs. A jolt integer is
;; a Chez exact integer, so these are native (integer-length = JVM bitLength,
;; matching for negative values too). tools.reader's number parser uses them.
((string=? method "negate") (->num (- (jnum->exact n))))
((string=? method "abs") (->num (abs (jnum->exact n))))
((string=? method "bitLength") (->num (integer-length (jnum->exact n))))
((string=? method "signum") (->num (let ((e (jnum->exact n))) (cond ((> e 0) 1) ((< e 0) -1) (else 0)))))
;; BigInteger.shiftLeft/shiftRight (test.check's size-bounded-bigint): arbitrary
;; precision, so an arithmetic shift by the (positive) amount.
((string=? method "shiftLeft") (->num (bitwise-arithmetic-shift-left (jnum->exact n) (jnum->exact (car args)))))
((string=? method "shiftRight") (->num (bitwise-arithmetic-shift-right (jnum->exact n) (jnum->exact (car args)))))
(else (error #f (string-append "No method " method " for number")))))
;; Mutable static fields: "Class" -> (member -> 1-vector cell). A library that
;; writes a static field — clojure.spec.alpha's (set! (. clojure.lang.RT
;; checkSpecAsserts) flag) — lands here; the analyzer lowers the set! to a
;; set-static-field! call and a plain Class/member read consults the cell first.
(define mutable-statics-tbl (make-hashtable string-hash string=?))
(define (mutable-static-cell class member create?)
(let ((h (or (hashtable-ref mutable-statics-tbl class #f)
(and create? (let ((nh (make-hashtable string-hash string=?)))
(hashtable-set! mutable-statics-tbl class nh) nh)))))
(and h (or (hashtable-ref h member #f)
(and create? (let ((c (vector jolt-nil))) (hashtable-set! h member c) c))))))
(def-var! "jolt.host" "set-static-field!"
(lambda (class member val)
(vector-set! (mutable-static-cell class member #t) 0 val)
val))
;; clojure.lang.RT.checkSpecAsserts — a JVM-internal flag clojure.spec.alpha reads
;; and writes; default false. Pre-seed the cell so a read before any write works.
(vector-set! (mutable-static-cell "clojure.lang.RT" "checkSpecAsserts" #t) 0 #f)
;; ---- emit entry points ------------------------------------------------------
(define (host-static-ref class member)
(let ((cell (mutable-static-cell class member #f)))
(if cell
(vector-ref cell 0)
(let ((h (lookup-class class-statics-tbl class)))
(if h
(let ((v (hashtable-ref h member #f)))
(if v v (error #f (string-append "No static " class "/" member))))
(error #f (string-append "Unknown class " class)))))))
(define (host-static-call class member . args)
(apply (host-static-ref class member) args))
(define (host-new class . args)
(let ((ctor (lookup-class class-ctors-tbl class)))
(cond
(ctor (apply ctor args))
;; deftype/defrecord: the type name is bound as a VAR (the
;; make-deftype-ctor closure) in its defining ns, not a registered host class.
;; Resolve it in the current ns / clojure.core and invoke it — so (P. args)
;; works the same as the ->P factory.
(else
(let ((cell (or (var-cell-lookup (chez-current-ns) class)
(var-cell-lookup "clojure.core" class))))
(if (and cell (var-cell-defined? cell) (procedure? (var-cell-root cell)))
(apply (var-cell-root cell) args)
(error #f (string-append "No constructor for class " class))))))))
;; ---- coercion helpers -------------------------------------------------------
;; numeric tower: currentTimeMillis/nanoTime are exact longs (JVM).
(define (->num x) x)
(define (jnum->exact n) (exact (truncate n)))
;; parse an integer string in radix; #f on failure
(define (parse-int-str s radix)
(let ((n (string->number (str-trim (if (string? s) s (jolt-str-render-one s))) radix)))
(and n (integer? n) (->num n))))
(define (parse-int-or-throw s radix what)
(or (parse-int-str s radix)
(jolt-throw (jolt-host-throwable "java.lang.NumberFormatException"
(string-append "For input string: \""
(if (string? s) s (jolt-str-render-one s)) "\"")))))
(define (char-code c) (if (char? c) (char->integer c) (jnum->exact c)))
;; parse a double string (Double/parseDouble, (Double. s)); JVM accepts NaN /
;; Infinity / decimal / scientific. #f on failure.
(define (parse-double-str s)
(let ((t (str-trim (if (string? s) s (jolt-str-render-one s)))))
(cond
((or (string=? t "NaN") (string=? t "+NaN") (string=? t "-NaN")) +nan.0)
((or (string=? t "Infinity") (string=? t "+Infinity")) +inf.0)
((string=? t "-Infinity") -inf.0)
(else (let ((n (string->number t))) (and n (real? n) (exact->inexact n)))))))
(define (parse-double-or-throw s)
(or (parse-double-str s)
(jolt-throw (jolt-host-throwable "java.lang.NumberFormatException"
(string-append "For input string: \""
(if (string? s) s (jolt-str-render-one s)) "\"")))))
(define (->double x) (if (number? x) (exact->inexact x) (parse-double-or-throw x)))

596
host/chez/java/inst-time.ss Normal file
View file

@ -0,0 +1,596 @@
;; #inst values + a java.time formatting shim.
;;
;; A #inst literal lowers (analyzer :inst node -> emit) to (jolt-inst-from-string
;; "…"); this file parses the RFC3339 string to epoch-ms and models the value as a
;; `jinst` record (one flonum field, ms). Equality / map-key hashing are by the
;; INSTANT (offset-normalized). The overlay inst?/inst-ms read (get x :jolt/type)/(get x :ms),
;; so jolt-get answers those off a jinst — the overlay fns then work unchanged.
;;
;; The java.time surface (DateTimeFormatter/Instant/ZoneId/LocalDateTime/
;; FormatStyle/Locale + the .format/.atZone/.toInstant/… methods) is
;; registered through host-static.ss's class-statics / host-
;; methods registries — so this loads LAST in rt.ss, after host-static.ss and io.ss.
;; --- civil <-> days since the Unix epoch (Howard Hinnant's algorithms) -------
;; No portable UTC mktime on Chez, so compute epoch days directly from y/m/d.
(define (days-from-civil y m d)
(let* ((y2 (if (<= m 2) (- y 1) y))
(era (quotient (if (>= y2 0) y2 (- y2 399)) 400))
(yoe (- y2 (* era 400)))
(doy (+ (quotient (+ (* 153 (+ m (if (> m 2) -3 9))) 2) 5) (- d 1)))
(doe (+ (* yoe 365) (quotient yoe 4) (- (quotient yoe 100)) doy)))
(+ (* era 146097) doe -719468)))
(define (civil-from-days z) ; -> (values year month day)
(let* ((z2 (+ z 719468))
(era (quotient (if (>= z2 0) z2 (- z2 146096)) 146097))
(doe (- z2 (* era 146097)))
(yoe (quotient (+ doe (- (quotient doe 1460)) (quotient doe 36524) (- (quotient doe 146096))) 365))
(y (+ yoe (* era 400)))
(doy (- doe (+ (* 365 yoe) (quotient yoe 4) (- (quotient yoe 100)))))
(mp (quotient (+ (* 5 doy) 2) 153))
(d (+ (- doy (quotient (+ (* 153 mp) 2) 5)) 1))
(m (+ mp (if (< mp 10) 3 -9))))
(values (if (<= m 2) (+ y 1) y) m d)))
;; --- RFC3339 parse: yyyy[-MM[-dd[Thh[:mm[:ss[.fff]]]]]][Z|±hh:mm] -> ms -------
(define-record-type jinst (fields ms) (nongenerative chez-jinst-v1))
(define (digit? c) (and (char>=? c #\0) (char<=? c #\9)))
(define (digits-at s i n) ; n digits from i -> integer, or #f
(and (<= (+ i n) (string-length s))
(let loop ((j i) (acc 0))
(if (= j (+ i n))
acc
(and (digit? (string-ref s j))
(loop (+ j 1) (+ (* acc 10) (- (char->integer (string-ref s j)) 48))))))))
(define (jolt-inst-from-string ts0)
;; a leading '-' marks a negative (proleptic) year; the rest of the field may be
;; more than 4 digits (java.time prints -999999999-…). Read the year up to the
;; first '-' that separates it from the month.
(define neg-year (and (> (string-length ts0) 0) (char=? (string-ref ts0 0) #\-)))
(define ts (if neg-year (substring ts0 1 (string-length ts0)) ts0))
(define len (string-length ts))
(define (fail) (error #f (string-append "Unrecognized #inst timestamp: " ts0)))
(define (read-year)
;; >=4 digits up to a non-digit; java.time uses min-4 but allows more.
(let loop ((j 0) (acc 0) (n 0))
(if (and (< j len) (digit? (string-ref ts j)))
(loop (+ j 1) (+ (* acc 10) (- (char->integer (string-ref ts j)) 48)) (+ n 1))
(if (>= n 4) (cons acc j) #f))))
(let* ((yr (or (read-year) (fail)))
(year (if neg-year (- (car yr)) (car yr)))
(i (cdr yr)) (month 1) (day 1) (hh 0) (mm 0) (ss 0) (frac-ms 0) (off-s 0))
;; -MM
(when (and (< i len) (char=? (string-ref ts i) #\-) (digits-at ts (+ i 1) 2))
(set! month (digits-at ts (+ i 1) 2)) (set! i (+ i 3)))
;; -dd
(when (and (< i len) (char=? (string-ref ts i) #\-) (digits-at ts (+ i 1) 2))
(set! day (digits-at ts (+ i 1) 2)) (set! i (+ i 3)))
;; Thh
(when (and (< i len) (or (char=? (string-ref ts i) #\T) (char=? (string-ref ts i) #\t))
(digits-at ts (+ i 1) 2))
(set! hh (digits-at ts (+ i 1) 2)) (set! i (+ i 3))
;; :mm
(when (and (< i len) (char=? (string-ref ts i) #\:) (digits-at ts (+ i 1) 2))
(set! mm (digits-at ts (+ i 1) 2)) (set! i (+ i 3))
;; :ss
(when (and (< i len) (char=? (string-ref ts i) #\:) (digits-at ts (+ i 1) 2))
(set! ss (digits-at ts (+ i 1) 2)) (set! i (+ i 3))
;; .fff (truncate beyond 3)
(when (and (< i len) (char=? (string-ref ts i) #\.))
(let loop ((j (+ i 1)) (k 0) (acc 0))
(if (and (< j len) (digit? (string-ref ts j)))
(loop (+ j 1) (+ k 1) (if (< k 3) (+ (* acc 10) (- (char->integer (string-ref ts j)) 48)) acc))
(begin
(set! frac-ms (* acc (expt 10 (max 0 (- 3 k)))))
(set! i j))))))))
;; offset Z | ±hh:mm
(when (< i len)
(let ((c (string-ref ts i)))
(cond
((or (char=? c #\Z) (char=? c #\z)) (set! i (+ i 1)))
((or (char=? c #\+) (char=? c #\-))
(let ((oh (digits-at ts (+ i 1) 2)) (om (digits-at ts (+ i 4) 2)))
(unless (and oh om (char=? (string-ref ts (+ i 3)) #\:)) (fail))
(set! off-s (* (if (char=? c #\-) -1 1) (+ (* oh 3600) (* om 60))))
(set! i (+ i 6))))
(else (fail)))))
(unless (= i len) (fail))
(let ((base-s (+ (* (days-from-civil year month day) 86400) (* hh 3600) (* mm 60) ss)))
(make-jinst (- (+ (* base-s 1000) frac-ms) (* off-s 1000))))))
;; --- canonical print form: yyyy-MM-ddThh:mm:ss.fff-00:00 (UTC) ---------------
(define (pad2 n) (if (< n 10) (string-append "0" (number->string n)) (number->string n)))
(define (pad4 n) (let ((s (number->string n))) (string-append (make-string (max 0 (- 4 (string-length s))) #\0) s)))
(define (pad3 n) (let ((s (number->string n))) (string-append (make-string (max 0 (- 3 (string-length s))) #\0) s)))
(define (inst-floor-div a b) (let ((q (quotient a b)) (r (remainder a b))) (if (and (not (= r 0)) (< (* a b) 0)) (- q 1) q)))
(define (inst-floor-mod a b) (- a (* (inst-floor-div a b) b)))
(define (inst-fields ms) ; -> list (y mo d hh mm ss frac dow)
(let* ((total-s (inst-floor-div (exact (truncate ms)) 1000))
(frac (- (exact (truncate ms)) (* total-s 1000)))
(days (inst-floor-div total-s 86400))
(sod (inst-floor-mod total-s 86400))
(hh (quotient sod 3600)) (mm (quotient (remainder sod 3600) 60)) (ss (remainder sod 60))
(dow (inst-floor-mod (+ days 4) 7))) ; 1970-01-01 = Thursday; 0=Sunday
(call-with-values (lambda () (civil-from-days days))
(lambda (y mo d) (list y mo d hh mm ss frac dow)))))
(define (inst-rfc3339 inst)
(let ((f (inst-fields (jinst-ms inst))))
(string-append (pad4 (list-ref f 0)) "-" (pad2 (list-ref f 1)) "-" (pad2 (list-ref f 2))
"T" (pad2 (list-ref f 3)) ":" (pad2 (list-ref f 4)) ":" (pad2 (list-ref f 5))
"." (pad3 (list-ref f 6)) "-00:00")))
;; --- DateTimeFormatter pattern engine -----
(define month-names (vector "January" "February" "March" "April" "May" "June" "July"
"August" "September" "October" "November" "December"))
(define day-names (vector "Sunday" "Monday" "Tuesday" "Wednesday" "Thursday" "Friday" "Saturday"))
(define (format-ms pattern ms)
(let ((f (inst-fields ms)) (n (string-length pattern)) (out (open-output-string)))
(let ((y (list-ref f 0)) (mo (list-ref f 1)) (d (list-ref f 2))
(hh (list-ref f 3)) (mi (list-ref f 4)) (se (list-ref f 5)) (dow (list-ref f 7)))
(define (run-len i c) (let loop ((j i)) (if (and (< j n) (char=? (string-ref pattern j) c)) (loop (+ j 1)) (- j i))))
(let loop ((i 0))
(when (< i n)
(let* ((c (string-ref pattern i)) (k (run-len i c)))
(cond
((char=? c #\')
(if (and (< (+ i 1) n) (char=? (string-ref pattern (+ i 1)) #\'))
(begin (write-char #\' out) (loop (+ i 2)))
(let close ((j (+ i 1)))
(cond ((>= j n) (loop j))
((char=? (string-ref pattern j) #\') (loop (+ j 1)))
(else (write-char (string-ref pattern j) out) (close (+ j 1)))))))
((char=? c #\y) (display (if (>= k 4) (number->string y) (pad2 (modulo y 100))) out) (loop (+ i k)))
((char=? c #\M)
(display (cond ((= k 1) (number->string mo)) ((= k 2) (pad2 mo))
((= k 3) (substring (vector-ref month-names (- mo 1)) 0 3))
(else (vector-ref month-names (- mo 1)))) out)
(loop (+ i k)))
((char=? c #\d) (display (if (= k 1) (number->string d) (pad2 d)) out) (loop (+ i k)))
((char=? c #\E)
(display (if (>= k 4) (vector-ref day-names dow) (substring (vector-ref day-names dow) 0 3)) out)
(loop (+ i k)))
((char=? c #\H) (display (if (= k 1) (number->string hh) (pad2 hh)) out) (loop (+ i k)))
((char=? c #\h)
(let ((h12 (let ((h (modulo hh 12))) (if (= h 0) 12 h))))
(display (if (= k 1) (number->string h12) (pad2 h12)) out)) (loop (+ i k)))
((char=? c #\m) (display (if (= k 1) (number->string mi) (pad2 mi)) out) (loop (+ i k)))
((char=? c #\s) (display (if (= k 1) (number->string se) (pad2 se)) out) (loop (+ i k)))
((char=? c #\a) (display (if (< hh 12) "AM" "PM") out) (loop (+ i k)))
;; timezone — format-ms renders UTC, the HTTP zone is GMT: z/zzz -> GMT,
;; Z (RFC822) -> +0000, X (ISO) -> Z.
((char=? c #\z) (display "GMT" out) (loop (+ i k)))
((char=? c #\Z) (display "+0000" out) (loop (+ i k)))
((char=? c #\X) (display "Z" out) (loop (+ i k)))
(else (write-char c out) (loop (+ i 1)))))))
(get-output-string out))))
;; --- SimpleDateFormat .parse: pattern-driven parse to epoch-ms (UTC/GMT) ------
(define (month-from-name s)
(let ((m3 (ascii-string-down (substring s 0 (min 3 (string-length s))))))
(let loop ((i 0))
(cond ((= i 12) #f)
((string=? (ascii-string-down (substring (vector-ref month-names i) 0 3)) m3) (+ i 1))
(else (loop (+ i 1)))))))
(define (parse-ms pattern input)
(let ((pn (string-length pattern)) (inn (string-length input))
(y 1970) (mo 1) (d 1) (hh 0) (mi 0) (ss 0) (frac-ms 0) (pm 'none))
;; a parse failure is a java.time.format.DateTimeParseException (typed, so a
;; (catch DateTimeParseException …) over a bad date matches), like the JVM.
(define (pfail)
(jolt-throw (jolt-host-throwable "java.time.format.DateTimeParseException"
(string-append "unparseable date \"" input "\"") jolt-nil)))
(define (run-len i c) (let loop ((j i)) (if (and (< j pn) (char=? (string-ref pattern j) c)) (loop (+ j 1)) (- j i))))
;; read up to `maxw` digits (#f = unbounded). A fixed-width field (k>=2, e.g.
;; HHmm) caps the read at its run length so adjacent numeric fields split.
(define (read-digits-w ii maxw) ; -> (val . next), pfail if none
(let loop ((j ii) (acc 0) (n 0) (any #f))
(if (and (< j inn) (digit? (string-ref input j)) (or (not maxw) (< n maxw)))
(loop (+ j 1) (+ (* acc 10) (- (char->integer (string-ref input j)) 48)) (+ n 1) #t)
(if any (cons acc j) (pfail)))))
(define (read-digits ii) (read-digits-w ii #f))
(define (read-alpha ii) ; -> (str . next)
(let loop ((j ii)) (if (and (< j inn) (char-alphabetic? (string-ref input j))) (loop (+ j 1))
(cons (substring input ii j) j))))
(define (read-tz ii) ; consume GMT/UTC/Z or ±hhmm; -> next
(cond ((>= ii inn) ii)
((char-alphabetic? (string-ref input ii)) (cdr (read-alpha ii)))
((or (char=? (string-ref input ii) #\+) (char=? (string-ref input ii) #\-))
(let loop ((j (+ ii 1))) (if (and (< j inn) (or (digit? (string-ref input j)) (char=? (string-ref input j) #\:))) (loop (+ j 1)) j)))
(else ii)))
(let loop ((pi 0) (ii 0))
(if (>= pi pn)
(begin
(when (eq? pm 'pm) (when (< hh 12) (set! hh (+ hh 12))))
(when (eq? pm 'am) (when (= hh 12) (set! hh 0)))
(make-jinst (+ (* 1000 (+ (* (days-from-civil y mo d) 86400) (* hh 3600) (* mi 60) ss)) frac-ms)))
(let ((c (string-ref pattern pi)))
(cond
((char-alphabetic? c)
(let ((k (run-len pi c)))
(cond
((char=? c #\y) (let ((r (read-digits-w ii (if (>= k 3) #f k))))
;; 2-digit year (value < 100): JVM sliding window — 00-68 -> 20xx,
;; 69-99 -> 19xx (rfc1036 HTTP dates). A full year stays as-is.
(set! y (let ((v (car r))) (if (and (= k 2) (< v 100)) (if (< v 69) (+ 2000 v) (+ 1900 v)) v)))
(loop (+ pi k) (cdr r))))
((char=? c #\M) (if (>= k 3)
(let ((r (read-alpha ii))) (set! mo (or (month-from-name (car r)) (pfail))) (loop (+ pi k) (cdr r)))
(let ((r (read-digits-w ii (if (>= k 2) k #f)))) (set! mo (car r)) (loop (+ pi k) (cdr r)))))
((char=? c #\d) (let ((r (read-digits-w ii (if (>= k 2) k #f)))) (set! d (car r)) (loop (+ pi k) (cdr r))))
((or (char=? c #\H) (char=? c #\h)) (let ((r (read-digits-w ii (if (>= k 2) k #f)))) (set! hh (car r)) (loop (+ pi k) (cdr r))))
((char=? c #\m) (let ((r (read-digits-w ii (if (>= k 2) k #f)))) (set! mi (car r)) (loop (+ pi k) (cdr r))))
((char=? c #\s) (let ((r (read-digits-w ii (if (>= k 2) k #f))))
(set! ss (car r))
;; an ISO formatter (modeled here as an ss-pattern with no S
;; field) still accepts an optional fractional second; consume
;; .fff -> millis from the input. Skip when the pattern carries
;; the fraction itself (a following '.'/S handles it).
(let ((j (cdr r)) (pnext (if (< (+ pi k) pn) (string-ref pattern (+ pi k)) #\nul)))
(if (and (not (char=? pnext #\.)) (not (char=? pnext #\S))
(< j inn) (char=? (string-ref input j) #\.)
(< (+ j 1) inn) (digit? (string-ref input (+ j 1))))
(let frac ((p (+ j 1)) (kk 0) (acc 0))
(if (and (< p inn) (digit? (string-ref input p)))
(frac (+ p 1) (+ kk 1) (if (< kk 3) (+ (* acc 10) (- (char->integer (string-ref input p)) 48)) acc))
(begin (set! frac-ms (* acc (expt 10 (max 0 (- 3 kk))))) (loop (+ pi k) p))))
(loop (+ pi k) j)))))
((char=? c #\S) (let frac ((p ii) (kk 0) (acc 0))
(if (and (< p inn) (< kk k) (digit? (string-ref input p)))
(frac (+ p 1) (+ kk 1) (+ (* acc 10) (- (char->integer (string-ref input p)) 48)))
(begin (set! frac-ms (* acc (expt 10 (max 0 (- 3 kk))))) (loop (+ pi k) p)))))
((char=? c #\E) (loop (+ pi k) (cdr (read-alpha ii))))
((char=? c #\a) (let ((r (read-alpha ii)))
(set! pm (if (string=? (ascii-string-down (car r)) "pm") 'pm 'am))
(loop (+ pi k) (cdr r))))
((or (char=? c #\z) (char=? c #\Z) (char=? c #\X) (char=? c #\x) (char=? c #\V) (char=? c #\v)) (loop (+ pi k) (read-tz ii)))
(else (loop (+ pi k) ii)))))
((char=? c #\')
(if (and (< (+ pi 1) pn) (char=? (string-ref pattern (+ pi 1)) #\'))
(loop (+ pi 2) (if (and (< ii inn) (char=? (string-ref input ii) #\')) (+ ii 1) ii))
(let lit ((pj (+ pi 1)) (ij ii))
(cond ((>= pj pn) (loop pj ij))
((char=? (string-ref pattern pj) #\') (loop (+ pj 1) ij))
((and (< ij inn) (char=? (string-ref input ij) (string-ref pattern pj))) (lit (+ pj 1) (+ ij 1)))
(else (pfail))))))
;; literal: match it; a pattern space tolerates missing/extra spaces.
((char=? c #\space)
(let skip ((ij ii)) (if (and (< ij inn) (char=? (string-ref input ij) #\space)) (skip (+ ij 1)) (loop (+ pi 1) ij))))
((and (< ii inn) (char=? (string-ref input ii) c)) (loop (+ pi 1) (+ ii 1)))
(else (pfail))))))))
;; --- value integration: get / = / hash / pr / type / instance? --------------
(define kw-jolt-type (keyword "jolt" "type"))
(define kw-ms (keyword #f "ms"))
(define inst-type-kw (keyword "jolt" "inst"))
(register-get-arm! jinst?
(lambda (coll k d)
(cond ((jolt=2 k kw-jolt-type) inst-type-kw)
((jolt=2 k kw-ms) (jinst-ms coll))
(else d))))
(register-eq-arm! (lambda (a b) (or (jinst? a) (jinst? b)))
(lambda (a b) (and (jinst? a) (jinst? b) (= (jinst-ms a) (jinst-ms b)))))
(register-hash-arm! jinst? (lambda (x) (jolt-hash (jinst-ms x))))
;; #inst is a java.util.Date — (class x) / (type x) report that, not the internal
;; :jolt/inst tag (which print-method still dispatches on via __type-tag).
(register-class-arm! jinst? (lambda (x) "java.util.Date"))
;; java.time.Instant is nano-precise: two Instants are = when their epoch-nanos
;; match (so an Instant and one shifted by a single nanosecond differ).
(define (jt-instant-tag? x) (and (jhost? x) (string=? (jhost-tag x) "instant")))
(register-eq-arm! (lambda (a b) (or (jt-instant-tag? a) (jt-instant-tag? b)))
(lambda (a b) (and (jt-instant-tag? a) (jt-instant-tag? b)
(= (inst-nanos a) (inst-nanos b)))))
(register-hash-arm! jt-instant-tag? (lambda (x) (jolt-hash (inst-nanos x))))
;; ZonedDateTime / java.sql.Date shim values (mk-zoned/mk-sql-date jhosts) are
;; equal when same kind + same epoch-ms.
(define (time-jhost? x) (and (jhost? x) (member (jhost-tag x) '("zoned-dt" "sql-date")) #t))
(register-eq-arm! (lambda (a b) (or (time-jhost? a) (time-jhost? b)))
(lambda (a b) (and (time-jhost? a) (time-jhost? b)
(string=? (jhost-tag a) (jhost-tag b))
(= (ms-of a) (ms-of b)))))
(register-hash-arm! time-jhost? (lambda (x) (jolt-hash (ms-of x))))
(define (inst-pr i) (string-append "#inst \"" (inst-rfc3339 i) "\""))
(register-pr-arm! jinst? inst-pr)
(register-str-render! jinst? inst-rfc3339)
(define %it-type jolt-type)
(set! jolt-type (lambda (x) (if (jinst? x) inst-type-kw (%it-type x))))
;; instance? java.util.Date -> a jinst; java.time.Instant/LocalDateTime -> the
;; matching jhost tag. The instance? macro passes the class-name symbol.
(define (class-short tn) (let loop ((i (- (string-length tn) 1)))
(cond ((< i 0) tn) ((char=? (string-ref tn i) #\.) (substring tn (+ i 1) (string-length tn))) (else (loop (- i 1))))))
(register-instance-check-arm!
(lambda (type-sym val)
(let ((tn (class-short (symbol-t-name type-sym))))
(cond
;; a #inst / (Date.) is a java.util.Date; it is NOT a java.sql.Timestamp
;; (on the JVM a Date is not a Timestamp), so answer Timestamp explicitly #f.
((jinst? val) (cond ((string=? tn "Date") #t)
((string=? tn "Timestamp") #f)
(else 'pass)))
((and (jhost? val) (string=? (jhost-tag val) "instant")) (if (string=? tn "Instant") #t 'pass))
;; java.sql.Date is a java.util.Date subclass (but not a Timestamp).
((and (jhost? val) (string=? (jhost-tag val) "sql-date"))
(cond ((or (string=? tn "Date")) #t) ((string=? tn "Timestamp") #f) (else 'pass)))
(else 'pass)))))
;; inst-ms* is a seed native (the overlay inst-ms reads (get x :ms), now answered).
(def-var! "clojure.core" "inst-ms*" (lambda (i) (jinst-ms i)))
;; --- java.time shim values (jhost objects over host-static.ss registries) -----
;; "local-date" stores an epoch-day (java-time.ss owns the type); ms-of projects it
;; to UTC midnight so existing date math keeps working. "local-dt" stores epoch-day +
;; nano-of-day; the others store epoch-ms.
(define (ms-of d)
(cond ((number? d) d)
((jinst? d) (jinst-ms d))
((and (jhost? d) (string=? (jhost-tag d) "local-date"))
(* (vector-ref (jhost-state d) 0) 86400000))
((and (jhost? d) (string=? (jhost-tag d) "local-date-time"))
(+ (* (vector-ref (jhost-state d) 0) 86400000)
(quotient (vector-ref (jhost-state d) 1) 1000000)))
;; "instant" stores epoch-nanos; project to ms (floor) for ms-based callers.
((and (jhost? d) (string=? (jhost-tag d) "instant"))
(inst-floor-div (vector-ref (jhost-state d) 0) 1000000))
((and (jhost? d) (member (jhost-tag d) '("zoned-dt" "calendar" "sql-date")))
(vector-ref (jhost-state d) 0))
(else (error #f "not a date value" d))))
;; A java.time.Instant stores epoch-nanos (exact integer). mk-instant takes ms,
;; for the many ms-based call sites; mk-instant-nanos is the nano-precise ctor and
;; inst-nanos the nano accessor (java-time.ss owns the nano-aware arithmetic).
(define (mk-instant-nanos n) (make-jhost "instant" (vector (exact (truncate n)))))
(define (inst-nanos x) (vector-ref (jhost-state x) 0))
(define (mk-instant ms) (mk-instant-nanos (* (ms->exact ms) 1000000)))
(define (mk-zoned ms) (make-jhost "zoned-dt" (vector ms)))
;; LocalDateTime from epoch-ms (UTC): the java-time.ss "local-date-time" jhost,
;; state [epoch-day nano-of-day].
(define (mk-local ms)
(let* ((ems (exact (truncate ms)))
(ed (inst-floor-div ems 86400000))
(mod (inst-floor-mod ems 86400000)))
(make-jhost "local-date-time" (vector ed (* mod 1000000)))))
;; local-date from epoch-ms: the epoch-day of the UTC day containing ms.
(define (mk-local-date ms) (make-jhost "local-date" (vector (inst-floor-div (exact (truncate ms)) 86400000))))
;; start of the UTC day containing ms.
(define (start-of-utc-day ms)
(* (inst-floor-div (exact (truncate ms)) 86400000) 86400000))
;; a formatter carries its pattern and a locale id (default "en"); the locale
;; selects month/day names in the java-time.ss format engine.
(define (mk-formatter pat . loc) (make-jhost "dt-formatter" (vector pat (if (null? loc) "en" (car loc)))))
(define (fmt-pat f) (vector-ref (jhost-state f) 0))
(define (fmt-locale f) (let ((s (jhost-state f))) (if (> (vector-length s) 1) (vector-ref s 1) "en")))
(define (locale-id l) (if (and (jhost? l) (string=? (jhost-tag l) "locale")) (vector-ref (jhost-state l) 0) "en"))
(define (now-ms) (now-millis)) ; exact ms (= JVM long); now-millis from host-static.ss
;; coerce a user-supplied ms (exact or flonum) to an exact integer for storage.
(define (ms->exact ms) (exact (round ms)))
(register-host-methods! "instant"
(list (cons "atZone" (lambda (self zone) (mk-zoned (ms-of self))))
(cons "toEpochMilli" (lambda (self) (ms-of self)))
(cons "toString" (lambda (self) (inst-rfc3339 (make-jinst (ms-of self)))))))
(register-host-methods! "zoned-dt"
(list (cons "toLocalDateTime" (lambda (self) (mk-local (ms-of self))))
(cons "toInstant" (lambda (self) (mk-instant (ms-of self))))))
;; LocalDate.atZone(zone): the UTC layer treats it as a zoned value at midnight.
;; (java-time.ss registers atStartOfDay and the rest of the local-date surface.)
(register-host-methods! "local-date"
(list (cons "atZone" (lambda (self zone) (mk-zoned (ms-of self))))))
(register-host-methods! "dt-formatter"
(list (cons "withLocale" (lambda (self locale) (mk-formatter (fmt-pat self) (locale-id locale))))
(cons "withZone" (lambda (self zone) (mk-formatter (fmt-pat self) (fmt-locale self))))
(cons "format" (lambda (self d) (format-ms (fmt-pat self) (ms-of d))))
;; parse a string per the pattern -> an instant value; Instant/from / the
;; LocalDateTime/parse static read its ms back out.
(cons "parse" (lambda (self s) (mk-instant (jinst-ms (parse-ms (fmt-pat self) (jolt-str-render-one s))))))))
;; FormatStyle approximations (no locale DB on this host).
(define style-patterns
'((date . ((short . "M/d/yy") (medium . "MMM d, yyyy") (long . "MMMM d, yyyy") (full . "EEEE, MMMM d, yyyy")))
(time . ((short . "h:mm a") (medium . "h:mm:ss a") (long . "h:mm:ss a") (full . "h:mm:ss a")))
(datetime . ((short . "M/d/yy, h:mm a") (medium . "MMM d, yyyy, h:mm:ss a")
(long . "MMMM d, yyyy, h:mm:ss a") (full . "EEEE, MMMM d, yyyy, h:mm:ss a")))))
(define (style-of fs) (vector-ref (jhost-state fs) 0)) ; a symbol: short/medium/long/full
(define (style-fmt kind fs)
(mk-formatter (or (let ((row (assq kind style-patterns))) (and row (let ((e (assq (style-of fs) (cdr row)))) (and e (cdr e)))))
"yyyy-MM-dd HH:mm:ss")))
(register-class-statics! "FormatStyle"
(list (cons "SHORT" (make-jhost "format-style" (vector 'short)))
(cons "MEDIUM" (make-jhost "format-style" (vector 'medium)))
(cons "LONG" (make-jhost "format-style" (vector 'long)))
(cons "FULL" (make-jhost "format-style" (vector 'full)))))
(register-class-statics! "DateTimeFormatter"
(list (cons "ofPattern" (lambda (p . _) (mk-formatter p)))
(cons "ISO_LOCAL_DATE" (mk-formatter "yyyy-MM-dd"))
(cons "ISO_LOCAL_DATE_TIME" (mk-formatter "yyyy-MM-dd'T'HH:mm:ss"))
;; ISO_INSTANT always renders in UTC with a trailing Z (format-ms is UTC; X -> "Z").
(cons "ISO_INSTANT" (mk-formatter "yyyy-MM-dd'T'HH:mm:ssX"))
;; ISO_ZONED_DATE_TIME: the UTC layer renders/parses it like ISO_INSTANT.
(cons "ISO_ZONED_DATE_TIME" (mk-formatter "yyyy-MM-dd'T'HH:mm:ssX"))
(cons "ofLocalizedDate" (lambda (fs) (style-fmt 'date fs)))
(cons "ofLocalizedTime" (lambda (fs) (style-fmt 'time fs)))
(cons "ofLocalizedDateTime" (lambda (fs) (style-fmt 'datetime fs)))))
(register-class-statics! "Instant"
(list (cons "ofEpochMilli" (lambda (ms) (mk-instant (ms->exact ms))))
(cons "now" (lambda () (mk-instant (now-ms))))
;; Instant/parse an ISO-8601 instant ("…T…Z") -> an instant value.
(cons "parse" (lambda (s) (mk-instant (jinst-ms (jolt-inst-from-string
(if (string? s) s (jolt-str-render-one s)))))))
;; Instant/from a temporal accessor -> an instant at the same epoch-ms.
(cons "from" (lambda (t) (mk-instant (ms-of t))))))
(register-class-statics! "ZoneId"
(list (cons "systemDefault" (lambda () (make-jhost "zone-id" (vector "system"))))
(cons "of" (lambda (id) (make-jhost "zone-id" (vector id))))))
(register-class-statics! "LocalDateTime"
(list (cons "ofInstant" (lambda (inst zone) (mk-local (ms-of inst))))
(cons "now" (lambda () (mk-local (now-ms))))
;; LocalDateTime/parse text, or text + a formatter (the UTC layer ignores
;; the parsed offset) -> a local-dt at the parsed instant.
(cons "parse" (lambda (s . fmt)
(let ((str (if (string? s) s (jolt-str-render-one s))))
(mk-local (jinst-ms (if (null? fmt)
(jolt-inst-from-string str)
(parse-ms (fmt-pat (car fmt)) str)))))))))
(let ((locale-ctor (lambda (id . _) (make-jhost "locale" (vector (if (string? id) id (jolt-str-render-one id)))))))
(register-class-ctor! "Locale" locale-ctor)
(register-class-ctor! "java.util.Locale" locale-ctor))
(register-class-statics! "Locale"
(list (cons "getDefault" (lambda () (make-jhost "locale" (vector "default"))))
(cons "ENGLISH" (make-jhost "locale" (vector "en")))
(cons "US" (make-jhost "locale" (vector "en-US")))
(cons "FRENCH" (make-jhost "locale" (vector "fr")))
(cons "FRANCE" (make-jhost "locale" (vector "fr-FR")))
(cons "GERMAN" (make-jhost "locale" (vector "de")))
(cons "ROOT" (make-jhost "locale" (vector "root")))))
;; java.util.Date / java.sql.Timestamp: #inst's classes. (Date.) = now, (Date. ms)
;; or (Date. another-date) -> a jinst (ms-of accepts a number / jinst / instant), so
;; .getTime / inst? / instance? Date|Timestamp work.
(define (date-ctor . args)
(cond
((null? args) (make-jinst (now-ms)))
((null? (cdr args)) (make-jinst (ms->exact (ms-of (car args)))))
;; deprecated (Date. year-1900 month0 date [hrs min sec]) — civil fields in UTC.
(else
(let* ((y (+ 1900 (jnum->exact (list-ref args 0))))
(mo (+ 1 (jnum->exact (list-ref args 1))))
(d (jnum->exact (list-ref args 2)))
(hh (if (> (length args) 3) (jnum->exact (list-ref args 3)) 0))
(mm (if (> (length args) 4) (jnum->exact (list-ref args 4)) 0))
(ss (if (> (length args) 5) (jnum->exact (list-ref args 5)) 0)))
(make-jinst (* 1000 (+ (* (days-from-civil y mo d) 86400) (* hh 3600) (* mm 60) ss)))))))
(register-class-ctor! "Date" date-ctor)
(register-class-ctor! "java.util.Date" date-ctor)
(register-class-ctor! "Timestamp" date-ctor)
(register-class-ctor! "java.sql.Timestamp" date-ctor)
;; Date/from(Instant) -> a java.util.Date at the instant's epoch-ms.
(let ((date-statics (list (cons "from" (lambda (inst) (make-jinst (ms->exact (ms-of inst))))))))
(register-class-statics! "Date" date-statics)
(register-class-statics! "java.util.Date" date-statics))
;; java.sql.Date: a distinct class from java.util.Date (a "sql-date" jhost over
;; epoch-ms) so a protocol extended to both routes a sql.Date to its own impl.
;; (Date. year-1900 month0 day) builds UTC midnight of that civil date; valueOf
;; parses "yyyy-MM-dd" to the same instant (so the two agree).
(define (mk-sql-date ms) (make-jhost "sql-date" (vector (ms->exact ms))))
(define (sql-date-midnight y mo d) (mk-sql-date (* 1000 (* (days-from-civil y mo d) 86400))))
(register-class-ctor! "java.sql.Date"
(case-lambda
((ms) (mk-sql-date (ms-of ms))) ; (Date. epoch-ms)
((y m d) (sql-date-midnight (+ 1900 (jnum->exact y)) (+ 1 (jnum->exact m)) (jnum->exact d)))))
(register-class-statics! "java.sql.Date"
(list (cons "valueOf" (lambda (s) (mk-sql-date (jinst-ms (parse-ms "yyyy-MM-dd" (if (string? s) s (jolt-str-render-one s)))))))))
(register-host-methods! "sql-date"
(list (cons "getTime" (lambda (self) (ms-of self)))
(cons "toInstant" (lambda (self) (mk-instant (ms-of self))))
(cons "toLocalDate" (lambda (self) (mk-local-date (ms-of self))))
(cons "toString" (lambda (self) (inst-rfc3339 (make-jinst (ms-of self)))))))
;; java.util.Calendar: a mutable broken-down UTC time over an epoch-ms. setTime/
;; getTime read/write it; set(field,value) recomputes ms from the field projection.
;; Field constants are Java's int values so .set/.get dispatch on the right field.
(define cal-YEAR 1) (define cal-MONTH 2) (define cal-DAY_OF_MONTH 5)
(define cal-HOUR_OF_DAY 11) (define cal-MINUTE 12) (define cal-SECOND 13)
(define cal-MILLISECOND 14)
(define (cal-ms->fields ms) ; -> vector [y mo0 d hh mi ss frac] (MONTH 0-based, JVM)
(let ((f (inst-fields ms)))
(vector (list-ref f 0) (- (list-ref f 1) 1) (list-ref f 2)
(list-ref f 3) (list-ref f 4) (list-ref f 5) (list-ref f 6))))
(define (cal-fields->ms v)
(+ (* 1000 (+ (* (days-from-civil (vector-ref v 0) (+ 1 (vector-ref v 1)) (vector-ref v 2)) 86400)
(* (vector-ref v 3) 3600) (* (vector-ref v 4) 60) (vector-ref v 5)))
(vector-ref v 6)))
(define (cal-field-index fld)
(cond ((= fld cal-YEAR) 0) ((= fld cal-MONTH) 1) ((= fld cal-DAY_OF_MONTH) 2)
((= fld cal-HOUR_OF_DAY) 3) ((= fld cal-MINUTE) 4) ((= fld cal-SECOND) 5)
((= fld cal-MILLISECOND) 6) (else #f)))
(register-host-methods! "calendar"
(list (cons "setTime" (lambda (self d) (vector-set! (jhost-state self) 0 (ms->exact (ms-of d))) jolt-nil))
(cons "getTime" (lambda (self) (make-jinst (vector-ref (jhost-state self) 0))))
(cons "getTimeInMillis" (lambda (self) (vector-ref (jhost-state self) 0)))
(cons "setTimeInMillis" (lambda (self ms) (vector-set! (jhost-state self) 0 (ms->exact ms)) jolt-nil))
(cons "set" (lambda (self field val)
(let ((v (cal-ms->fields (vector-ref (jhost-state self) 0)))
(idx (cal-field-index (jnum->exact field))))
(when idx (vector-set! v idx (jnum->exact val))
(vector-set! (jhost-state self) 0 (cal-fields->ms v)))
jolt-nil)))
(cons "get" (lambda (self field)
(let ((v (cal-ms->fields (vector-ref (jhost-state self) 0)))
(idx (cal-field-index (jnum->exact field))))
(if idx (vector-ref v idx) 0))))))
(define calendar-statics
(list (cons "getInstance" (lambda _ (make-jhost "calendar" (vector (now-ms)))))
(cons "YEAR" cal-YEAR) (cons "MONTH" cal-MONTH) (cons "DAY_OF_MONTH" cal-DAY_OF_MONTH)
(cons "HOUR_OF_DAY" cal-HOUR_OF_DAY) (cons "MINUTE" cal-MINUTE)
(cons "SECOND" cal-SECOND) (cons "MILLISECOND" cal-MILLISECOND)))
(register-class-statics! "Calendar" calendar-statics)
(register-class-statics! "java.util.Calendar" calendar-statics)
;; java.util.TimeZone: an opaque id holder (format-ms is UTC, so a non-UTC zone is
;; not honored — only the UTC case the corpus uses is exercised).
(define (timezone-of id) (make-jhost "timezone" (vector (if (string? id) id (jolt-str-render-one id)))))
(define timezone-statics
(list (cons "getTimeZone" timezone-of)
(cons "getDefault" (lambda () (timezone-of "default")))))
(register-class-statics! "TimeZone" timezone-statics)
(register-class-statics! "java.util.TimeZone" timezone-statics)
;; java.text.SimpleDateFormat: holds a pattern; .setTimeZone is accepted (format-ms
;; is UTC); .format(date) renders the date per the pattern via the format-ms engine.
(define (sdf-ctor pat . _) (make-jhost "sdf" (vector (if (string? pat) pat (jolt-str-render-one pat)))))
(register-class-ctor! "SimpleDateFormat" sdf-ctor)
(register-class-ctor! "java.text.SimpleDateFormat" sdf-ctor)
(register-host-methods! "sdf"
(list (cons "setTimeZone" (lambda (self tz) jolt-nil))
(cons "setLenient" (lambda (self b) jolt-nil))
(cons "applyPattern" (lambda (self p) (vector-set! (jhost-state self) 0 (jolt-str-render-one p)) jolt-nil))
(cons "toPattern" (lambda (self) (vector-ref (jhost-state self) 0)))
(cons "parse" (lambda (self s) (parse-ms (vector-ref (jhost-state self) 0) (jolt-str-render-one s))))
(cons "format" (lambda (self d) (format-ms (vector-ref (jhost-state self) 0) (ms-of d))))))
;; a jinst's java.util.Date method surface (record-method-dispatch arm).
(register-method-arm! 40
(lambda (obj method-name rest-args)
(cond
((jinst? obj)
(cond ((string=? method-name "getTime") (jinst-ms obj))
;; deprecated java.util.Date accessors (UTC civil fields).
((string=? method-name "getYear") (- (list-ref (inst-fields (jinst-ms obj)) 0) 1900))
((string=? method-name "getMonth") (- (list-ref (inst-fields (jinst-ms obj)) 1) 1))
((string=? method-name "getDate") (list-ref (inst-fields (jinst-ms obj)) 2))
((string=? method-name "getHours") (list-ref (inst-fields (jinst-ms obj)) 3))
((string=? method-name "getMinutes") (list-ref (inst-fields (jinst-ms obj)) 4))
((string=? method-name "getSeconds") (list-ref (inst-fields (jinst-ms obj)) 5))
((string=? method-name "getDay") (list-ref (inst-fields (jinst-ms obj)) 7))
((string=? method-name "toInstant") (mk-instant (jinst-ms obj)))
((string=? method-name "toLocalDate") (mk-local-date (jinst-ms obj)))
((string=? method-name "toLocalDateTime") (mk-local (jinst-ms obj)))
((string=? method-name "toString") (inst-rfc3339 obj))
((string=? method-name "equals") (and (pair? (if (jolt-nil? rest-args) '() (seq->list rest-args)))
(jinst? (car (seq->list rest-args)))
(= (jinst-ms obj) (jinst-ms (car (seq->list rest-args))))))
((string=? method-name "before") (< (jinst-ms obj) (ms-of (car (seq->list rest-args)))))
((string=? method-name "after") (> (jinst-ms obj) (ms-of (car (seq->list rest-args)))))
(else (error #f (string-append "No method " method-name " on Date")))))
(else 'pass))))
;; Clojure's built-in data readers, so a library that merges default-data-readers
;; or binds *data-readers* (e.g. aero's reader opts) resolves #inst / #uuid.
;; Keyed by symbol, like Clojure. *data-readers* is the bindable user table.
(def-var! "clojure.core" "default-data-readers"
(jolt-hash-map (jolt-symbol #f "inst") jolt-inst-from-string
(jolt-symbol #f "uuid") jolt-uuid-from-string))
(def-var! "clojure.core" "*data-readers*" empty-pmap)

View file

@ -0,0 +1,333 @@
;; java.io byte/char streams over Chez ports. Each stream is a jhost wrapping a
;; Chez port, so buffering, EOF and binary<->char transcoding come from Chez
;; rather than a hand-rolled buffer.
;;
;; in-stream #(binary-input-port) FileInputStream / ByteArrayInputStream
;; out-stream #(binary-output-port extract acc) FileOutputStream / ByteArrayOutputStream
;; char-reader #(textual-input-port) FileReader / InputStreamReader
;; char-writer #(textual-output-port) FileWriter / OutputStreamWriter
;;
;; Buffered{Reader,Writer,Input,Output}Stream are buffering wrappers; Chez ports
;; are already buffered, so their constructors return the wrapped stream.
;;
;; Loaded after io.ss + natives-array.ss (uses make-jfile/slurp helpers + the
;; byte-array <-> bytevector bridge), and extends io.ss's reader-jhost? / slurp /
;; __close so the new readers/streams flow through slurp / line-seq / with-open.
;; --- byte input stream ------------------------------------------------------
(define (in-stream-port self) (vector-ref (jhost-state self) 0))
(define (make-in-stream port) (make-jhost "in-stream" (vector port)))
(define (in-stream? x) (and (jhost? x) (string=? (jhost-tag x) "in-stream")))
(register-host-methods! "in-stream"
(list
(cons "read"
(lambda (self . rest)
(let ((port (in-stream-port self)))
(if (null? rest)
(let ((b (get-u8 port))) (if (eof-object? b) -1 (->num b)))
(let* ((buf (car rest))
(vec (jolt-array-vec buf))
(off (if (>= (length rest) 3) (jnum->exact (cadr rest)) 0))
(len (if (>= (length rest) 3) (jnum->exact (caddr rest)) (vector-length vec))))
(let loop ((i 0))
(if (>= i len) (->num i)
(let ((b (get-u8 port)))
(if (eof-object? b)
(if (= i 0) -1 (->num i))
(begin (vector-set! vec (+ off i) b) (loop (+ i 1))))))))))))
(cons "readAllBytes" (lambda (self) (let ((bv (get-bytevector-all (in-stream-port self))))
(na-byte-array (if (eof-object? bv) (make-bytevector 0) bv)))))
(cons "skip" (lambda (self n) (let ((bv (get-bytevector-n (in-stream-port self) (jnum->exact n))))
(->num (if (eof-object? bv) 0 (bytevector-length bv))))))
(cons "available" (lambda (self) (->num 0)))
(cons "close" (lambda (self) (close-port (in-stream-port self)) jolt-nil))
(cons "mark" (lambda (self . _) jolt-nil))
(cons "reset" (lambda (self) (guard (e (#t jolt-nil)) (set-port-position! (in-stream-port self) 0) jolt-nil)))
(cons "markSupported" (lambda (self) #f))
(cons "toString" (lambda (self) "#<InputStream>"))))
;; --- byte output stream -----------------------------------------------------
;; state #(port extract acc): extract/acc are #f for a file/passthrough stream;
;; a ByteArrayOutputStream carries the R6RS extraction proc + an accumulator
;; bytevector (Chez's extract resets the port, so snapshot on demand, not per write).
(define (out-stream-port self) (vector-ref (jhost-state self) 0))
(define (out-stream? x) (and (jhost? x) (string=? (jhost-tag x) "out-stream")))
(define (make-out-stream port) (make-jhost "out-stream" (vector port #f #f)))
(define (bv-concat a b)
(if (= 0 (bytevector-length b)) a
(let ((m (make-bytevector (+ (bytevector-length a) (bytevector-length b)))))
(bytevector-copy! a 0 m 0 (bytevector-length a))
(bytevector-copy! b 0 m (bytevector-length a) (bytevector-length b))
m)))
;; all bytes written to a ByteArrayOutputStream so far (folds the latest extract
;; into the accumulator).
(define (baos-bytes self)
(let* ((st (jhost-state self)) (port (vector-ref st 0)) (extract (vector-ref st 1)) (acc (vector-ref st 2)))
(flush-output-port port)
(let ((merged (bv-concat acc (extract))))
(vector-set! st 2 merged) merged)))
(register-host-methods! "out-stream"
(list
(cons "write"
(lambda (self x . rest)
(let ((port (out-stream-port self)))
(cond
((number? x) (put-u8 port (bitwise-and (jnum->exact x) #xff)))
((and (jolt-array? x) (eq? (jolt-array-kind x) 'byte))
(let ((bv (na-bytearray->bv x)))
(if (pair? rest)
(put-bytevector port bv (jnum->exact (car rest)) (jnum->exact (cadr rest)))
(put-bytevector port bv))))
((bytevector? x) (put-bytevector port x))
(else (error #f "OutputStream/write: unsupported" x)))
jolt-nil)))
(cons "flush" (lambda (self) (flush-output-port (out-stream-port self)) jolt-nil))
(cons "close" (lambda (self) (flush-output-port (out-stream-port self))
;; a ByteArrayOutputStream's close is a no-op (toByteArray stays valid);
;; a file stream's port is closed.
(unless (vector-ref (jhost-state self) 1) (close-port (out-stream-port self))) jolt-nil))
(cons "toByteArray" (lambda (self) (na-byte-array (bytevector-copy (baos-bytes self)))))
(cons "size" (lambda (self) (->num (bytevector-length (baos-bytes self)))))
(cons "reset" (lambda (self) (baos-bytes self) (vector-set! (jhost-state self) 2 (make-bytevector 0)) jolt-nil))
(cons "toString" (lambda (self . cs) (decode-bytevector (baos-bytes self)
(if (pair? cs) (list (jolt-str-render-one (car cs))) '()))))))
;; --- char input (Reader) ----------------------------------------------------
(define (char-reader-port self) (vector-ref (jhost-state self) 0))
(define (char-reader? x) (and (jhost? x) (string=? (jhost-tag x) "char-reader")))
(define (make-char-reader port) (make-jhost "char-reader" (vector port)))
(register-host-methods! "char-reader"
(list
(cons "read"
(lambda (self . rest)
(let ((port (char-reader-port self)))
(if (null? rest)
(let ((c (get-char port))) (if (eof-object? c) -1 (->num (char->integer c))))
(let* ((buf (car rest))
(vec (jolt-array-vec buf))
(off (if (>= (length rest) 3) (jnum->exact (cadr rest)) 0))
(len (if (>= (length rest) 3) (jnum->exact (caddr rest)) (vector-length vec))))
(let loop ((i 0))
(if (>= i len) (->num i)
(let ((c (get-char port)))
(if (eof-object? c)
(if (= i 0) -1 (->num i))
(begin (vector-set! vec (+ off i) c) (loop (+ i 1))))))))))))
(cons "readLine" (lambda (self) (let ((l (get-line (char-reader-port self)))) (if (eof-object? l) jolt-nil l))))
(cons "lines" (lambda (self)
(let loop ((acc '()))
(let ((l (get-line (char-reader-port self))))
(if (eof-object? l) (list->cseq (reverse acc)) (loop (cons l acc)))))))
(cons "ready" (lambda (self) #t))
(cons "skip" (lambda (self n) (let loop ((i 0) (k (jnum->exact n)))
(if (or (>= i k) (eof-object? (get-char (char-reader-port self)))) (->num i)
(loop (+ i 1) k)))))
(cons "close" (lambda (self) (close-port (char-reader-port self)) jolt-nil))
(cons "mark" (lambda (self . _) jolt-nil))
(cons "reset" (lambda (self) (guard (e (#t jolt-nil)) (set-port-position! (char-reader-port self) 0) jolt-nil)))
(cons "toString" (lambda (self) "#<Reader>"))))
;; --- char output (Writer) ---------------------------------------------------
(define (char-writer-port self) (vector-ref (jhost-state self) 0))
(define (char-writer? x) (and (jhost? x) (string=? (jhost-tag x) "char-writer")))
(define (make-char-writer port) (make-jhost "char-writer" (vector port)))
(define (cw-text x) (if (number? x) (string (integer->char (jnum->exact x))) (jolt-str-render-one x)))
(register-host-methods! "char-writer"
(list
(cons "write" (lambda (self x . rest)
;; (write str) | (write int) | (write str off len)
(let ((s (cw-text x)))
(put-string (char-writer-port self)
(if (>= (length rest) 2) (substring s (jnum->exact (car rest))
(+ (jnum->exact (car rest)) (jnum->exact (cadr rest)))) s)))
jolt-nil))
(cons "append" (lambda (self x . rest) (put-string (char-writer-port self) (cw-text x)) self))
(cons "newLine" (lambda (self) (put-char (char-writer-port self) #\newline) jolt-nil))
(cons "flush" (lambda (self) (flush-output-port (char-writer-port self)) jolt-nil))
(cons "close" (lambda (self) (close-port (char-writer-port self)) jolt-nil))
(cons "toString" (lambda (self) "#<Writer>"))))
;; --- constructors -----------------------------------------------------------
(define utf8-tx (make-transcoder (utf-8-codec)))
(define (path-of x) (project-relative (file-path-of x)))
(define (src-bytevector x) ; a byte[] or Chez bytevector -> bytevector
(cond ((bytevector? x) x)
((and (jolt-array? x) (eq? (jolt-array-kind x) 'byte)) (na-bytearray->bv x))
(else (error #f "expected a byte array" x))))
(define (reg-ctor! names ctor) (for-each (lambda (n) (register-class-ctor! n ctor)) names))
(reg-ctor! '("FileInputStream" "java.io.FileInputStream")
(lambda (src . _) (make-in-stream (open-file-input-port (path-of src) (file-options) (buffer-mode block)))))
(reg-ctor! '("FileOutputStream" "java.io.FileOutputStream")
(lambda (src . rest)
(let ((append? (and (pair? rest) (jolt-truthy? (car rest)))))
(make-out-stream (open-file-output-port (path-of src)
(if append? (file-options no-fail no-truncate append) (file-options no-fail))
(buffer-mode block))))))
(reg-ctor! '("ByteArrayInputStream" "java.io.ByteArrayInputStream")
(lambda (bytes . rest)
(let ((bv (src-bytevector bytes)))
(make-in-stream (open-bytevector-input-port
(if (>= (length rest) 2)
(let ((off (jnum->exact (car rest))) (len (jnum->exact (cadr rest))))
(let ((sub (make-bytevector len))) (bytevector-copy! bv off sub 0 len) sub))
bv))))))
(reg-ctor! '("ByteArrayOutputStream" "java.io.ByteArrayOutputStream")
(lambda _
(call-with-values open-bytevector-output-port
(lambda (port extract) (make-jhost "out-stream" (vector port extract (make-bytevector 0)))))))
(reg-ctor! '("FileReader" "java.io.FileReader")
(lambda (src . _) (make-char-reader (transcoded-port (open-file-input-port (path-of src) (file-options) (buffer-mode block)) utf8-tx))))
(reg-ctor! '("FileWriter" "java.io.FileWriter")
(lambda (src . rest)
(let ((append? (and (pair? rest) (jolt-truthy? (car rest)))))
(make-char-writer (transcoded-port (open-file-output-port (path-of src)
(if append? (file-options no-fail no-truncate append) (file-options no-fail))
(buffer-mode block)) utf8-tx)))))
;; InputStreamReader / OutputStreamWriter take ownership of the wrapped byte
;; stream's port and transcode it (UTF-8 default; an explicit charset is honored
;; only as UTF-8 here).
(reg-ctor! '("InputStreamReader" "java.io.InputStreamReader")
(lambda (in . _) (make-char-reader (transcoded-port (in-stream-port in) utf8-tx))))
(reg-ctor! '("OutputStreamWriter" "java.io.OutputStreamWriter")
(lambda (out . _) (make-char-writer (transcoded-port (out-stream-port out) utf8-tx))))
;; Buffered* — Chez ports are buffered already; the wrapper is the wrapped stream.
(for-each (lambda (n) (register-class-ctor! n (lambda (inner . _) inner)))
'("BufferedReader" "java.io.BufferedReader"
"BufferedWriter" "java.io.BufferedWriter"
"BufferedInputStream" "java.io.BufferedInputStream"
"BufferedOutputStream" "java.io.BufferedOutputStream"))
;; --- integration: slurp / line-seq / with-open ------------------------------
;; a char-reader joins the reader-jhost set (drain-reader / line-seq read it via
;; its .read method).
(let ((prev reader-jhost?))
(set! reader-jhost? (lambda (x) (or (char-reader? x) (prev x)))))
;; slurp a char-reader (drain chars) or a byte in-stream (drain bytes -> decode).
(let ((prev jolt-slurp))
(set! jolt-slurp
(lambda (src . opts)
(cond
((char-reader? src) (drain-reader src))
((in-stream? src) (decode-bytevector (let ((bv (get-bytevector-all (in-stream-port src))))
(if (eof-object? bv) (make-bytevector 0) bv))
(slurp-encoding opts)))
(else (apply prev src opts)))))
(def-var! "clojure.core" "slurp" jolt-slurp))
;; with-open closes the new stream jhosts via their .close method.
(let ((prev jolt-close))
(set! jolt-close
(lambda (x)
(if (and (jhost? x) (member (jhost-tag x) '("in-stream" "out-stream" "char-reader" "char-writer")))
(begin (record-method-dispatch x "close" jolt-nil) jolt-nil)
(prev x))))
(def-var! "clojure.core" "__close" jolt-close))
;; --- clojure.java.io: byte streams + copy / make-parents / delete-file -------
;; input-stream/output-stream now yield real byte streams (were char reader/writer).
(define (jio-input-stream x)
(cond ((in-stream? x) x)
((jfile? x) (make-in-stream (open-file-input-port (jfile-fs x) (file-options) (buffer-mode block))))
((and (jolt-array? x) (eq? (jolt-array-kind x) 'byte)) (make-in-stream (open-bytevector-input-port (na-bytearray->bv x))))
((bytevector? x) (make-in-stream (open-bytevector-input-port x)))
((and (jhost? x) (string=? (jhost-tag x) "url")) (make-in-stream (open-file-input-port (url-strip-scheme (url-spec x)) (file-options) (buffer-mode block))))
((string? x) (make-in-stream (open-file-input-port (project-relative x) (file-options) (buffer-mode block))))
(else (error #f "io/input-stream: don't know how to open" x))))
(define (jio-output-stream x . rest)
(cond ((out-stream? x) x)
((or (jfile? x) (string? x))
(let ((append? (let loop ((o rest)) (cond ((or (null? o) (null? (cdr o))) #f)
((and (keyword-t? (car o)) (string=? (keyword-t-name (car o)) "append") (jolt-truthy? (cadr o))) #t)
(else (loop (cddr o)))))))
(make-out-stream (open-file-output-port (path-of x)
(if append? (file-options no-fail no-truncate append) (file-options no-fail))
(buffer-mode block)))))
(else (error #f "io/output-stream: don't know how to open" x))))
(def-var! "clojure.java.io" "input-stream" jio-input-stream)
(def-var! "clojure.java.io" "output-stream" jio-output-stream)
;; io/make-parents: create the parent directories of the last path segment.
(define (jio-make-parents . args)
(let ((p (apply-make-file-path args)))
(let loop ((i (- (string-length p) 1)))
(cond ((<= i 0) #f)
((char=? (string-ref p i) #\/) (mkdirs! (substring p 0 i)))
(else (loop (- i 1)))))))
(define (apply-make-file-path args)
(jfile-path (apply jolt-make-file args)))
(def-var! "clojure.java.io" "make-parents" jio-make-parents)
;; io/delete-file: delete the file; raise unless :silently truthy.
(define (jio-delete-file f . opts)
(let ((p (file-path-of f)))
(if (delete-path! p) jolt-nil
(if (and (pair? opts) (jolt-truthy? (car opts))) jolt-nil
(error #f (string-append "Couldn't delete " p))))))
(def-var! "clojure.java.io" "delete-file" jio-delete-file)
;; io/copy: file/path/reader/stream/string/byte[] -> writer/stream/file/path.
;; A byte source copies byte-exact to a byte/file destination (no lossy text
;; round-trip); otherwise the content is read as text. UTF-8 bridges byte<->char.
(define (input-bytes input) ; bytevector for a byte source, else #f
(cond ((in-stream? input) (let ((bv (get-bytevector-all (in-stream-port input)))) (if (eof-object? bv) (make-bytevector 0) bv)))
((bytevector? input) input)
((and (jolt-array? input) (eq? (jolt-array-kind input) 'byte)) (na-bytearray->bv input))
;; a byte-input-stream shim (host tagged-table, :jolt/input-stream — e.g.
;; http-client's ByteArrayInputStream): drain it byte-exact, like slurp.
((and (htable? input) (jolt-truthy? (jolt-ref-get input (keyword "jolt" "input-stream"))))
(drain-byte-stream input))
(else #f)))
(define (input-text input)
(cond ((string? input) input)
((or (char-reader? input) (reader-jhost? input)) (drain-reader input))
((jfile? input) (jolt-slurp input))
((input-bytes input) => (lambda (bv) (decode-bytevector bv '())))
(else (jolt-str-render-one input))))
(define (jio-copy input output . opts)
(cond
((out-stream? output)
(put-bytevector (out-stream-port output)
(or (input-bytes input) (string->utf8 (input-text input)))))
((char-writer? output) (put-string (char-writer-port output) (input-text input)))
((and (jhost? output) (member (jhost-tag output) '("writer" "file-writer" "port-writer" "print-writer")))
(record-method-dispatch output "write" (list->cseq (list (input-text input)))))
((or (jfile? output) (string? output))
(let ((bv (and (not (string? input)) (not (jfile? input)) (input-bytes input))))
(if bv
(let ((port (open-file-output-port (path-of output) (file-options no-fail) (buffer-mode block))))
(put-bytevector port bv) (close-port port))
(jolt-spit output (input-text input)))))
;; a byte-output-stream shim (a host tagged-table with :jolt/output-stream,
;; e.g. http-client's ByteArrayOutputStream): write through its .write method,
;; byte-exact for a byte source.
((and (htable? output) (jolt-truthy? (jolt-ref-get output (keyword "jolt" "output-stream"))))
(let ((bv (input-bytes input)))
(record-method-dispatch output "write"
(list->cseq (list (if bv (make-jolt-array (list->vector (bytevector->u8-list bv)) 'byte)
(input-text input)))))))
(else (error #f "io/copy: don't know how to write to" output)))
jolt-nil)
(def-var! "clojure.java.io" "copy" jio-copy)
;; --- instance? for the java.io stream taxonomy ------------------------------
(register-class-arm! in-stream? (lambda (x) "java.io.InputStream"))
(register-class-arm! out-stream? (lambda (x) "java.io.OutputStream"))
(register-class-arm! char-reader? (lambda (x) "java.io.Reader"))
(register-class-arm! char-writer? (lambda (x) "java.io.Writer"))
(register-instance-check-arm!
(lambda (type-sym val)
(if (not (symbol-t? type-sym)) 'pass
(let ((short (last-dot (symbol-t-name type-sym))))
(cond
((and (in-stream? val) (member short '("InputStream" "FileInputStream" "ByteArrayInputStream"
"BufferedInputStream" "FilterInputStream" "Closeable" "AutoCloseable"))) #t)
((and (out-stream? val) (member short '("OutputStream" "FileOutputStream" "ByteArrayOutputStream"
"BufferedOutputStream" "FilterOutputStream" "Closeable" "AutoCloseable" "Flushable"))) #t)
((and (char-reader? val) (member short '("Reader" "BufferedReader" "FileReader" "InputStreamReader"
"Closeable" "AutoCloseable" "Readable"))) #t)
((and (char-writer? val) (member short '("Writer" "BufferedWriter" "FileWriter" "OutputStreamWriter"
"Closeable" "AutoCloseable" "Flushable" "Appendable"))) #t)
(else 'pass))))))

760
host/chez/java/io.ss Normal file
View file

@ -0,0 +1,760 @@
;; java.io.File + host file I/O, implemented over Chez's filesystem
;; primitives. A File is a
;; path-backed jfile record: (instance? java.io.File f) is true, str/slurp coerce
;; it to its path, and the File method surface (getName/getPath/exists/
;; isDirectory/isFile/listFiles) dispatches through record-method-dispatch.
;;
;; Provides make-file/file?/slurp/spit/flush/dir?/
;; list-dir for the overlay file-seq (20-coll.clj), which calls __file?/__dir?/
;; __list-dir + the .isDirectory/.listFiles/.isFile method surface.
;;
;; Loaded LAST in rt.ss, after
;; dot-forms.ss (so the jfile method arm wraps the fully-built dispatch) and
;; natives-meta.ss / records.ss / printing.ss (jolt-type / instance-check /
;; jolt-str-render-one, which it extends).
(define-record-type jfile (fields path) (nongenerative jolt-jfile-v1))
(define (jolt-file? x) (jfile? x))
;; path string of any value: a jfile -> its path, else its str rendering.
(define (file-path-of x) (if (jfile? x) (jfile-path x) (jolt-str-render-one x)))
;; Resources baked into a standalone binary by `jolt build` (deps.edn
;; :jolt/build :embed). The build emits a register-embedded-resource! per file at
;; heap-build time, so the contents live in the boot image — io/resource serves
;; them with no file on disk. An embedded hit reads through slurp/reader exactly
;; like a jfile would.
(define embedded-resources (make-hashtable equal-hash equal?))
(define (register-embedded-resource! name content)
(hashtable-set! embedded-resources name content))
(define-record-type embedded-res (fields name content) (nongenerative jolt-embres-v1))
;; --- self-contained build artifacts (jolt-eaj) ------------------------------
;; A toolchain-free `jolt build` (the distributed joltc) carries the Chez
;; petite/scheme boots and a prebuilt launcher stub baked into its own boot image.
;; They live in the same table as embedded-resources, but keyed under bytevector
;; values (register-embedded-bytes!) rather than strings; resolve-on-roots /
;; io/resource only ever ask for the string-keyed source entries, so the two
;; coexist. The build driver reads them at heap-build time from files that exist
;; only on the dev machine.
(define (register-embedded-bytes! name bv) (hashtable-set! embedded-resources name bv))
(define (jolt-embedded-bytes name)
(let ((v (hashtable-ref embedded-resources name #f)))
(and (bytevector? v) v)))
;; Read a whole file as a bytevector ("" -> empty). Used to slurp boot/stub files.
(define (read-file-bytes path)
(let ((p (open-file-input-port path)))
(let ((bv (get-bytevector-all p)))
(close-port p)
(if (eof-object? bv) (bytevector) bv))))
;; Write an embedded bytevector resource out to a path. make-boot-file needs the
;; petite/scheme boots as files, so they are spilled to scratch before the call.
(define (jolt-spill-embedded! name path)
(let ((bv (jolt-embedded-bytes name)))
(unless bv (error 'jolt-spill-embedded! "no embedded bytes for" name))
(let ((p (open-file-output-port path (file-options no-fail) (buffer-mode block))))
(put-bytevector p bv)
(close-port p))))
;; Frame an app boot onto a file that already holds the stub bytes. Layout:
;; [stub][boot][boot-length:le64]["JOLTBOOT"]. The stub (host/chez/stub/launcher.c)
;; reads the trailing 16 bytes — the 8-byte magic, then the preceding 8-byte LE
;; length — to locate and register the boot, so a boot that itself contains the
;; magic bytes can't be mistaken for the frame.
(define jolt-payload-magic (string->utf8 "JOLTBOOT"))
(define (jolt-append-payload! path boot-bv)
(let ((head (read-file-bytes path))) ; the stub bytes already written
(let ((p (open-file-output-port path (file-options no-fail) (buffer-mode block)))
(lb (make-bytevector 8 0)))
(bytevector-u64-set! lb 0 (bytevector-length boot-bv) (endianness little))
(put-bytevector p head)
(put-bytevector p boot-bv)
(put-bytevector p lb)
(put-bytevector p jolt-payload-magic)
(close-port p))))
;; chmod 0755 via libc, so the produced binary is executable. load-shared-object
;; with #f pulls the running process's own symbols (chmod is in libc, linked into
;; every Chez binary) — no external toolchain. Falls back to /bin/sh chmod if the
;; symbol can't be resolved.
(define jolt-chmod-755
(let ((c (jolt-foreign-proc-safe "chmod" '(string int) 'int)))
(lambda (path)
(cond
(c (c path #o755))
;; Windows has no chmod and needs none (execute is by extension)
((let ((m (symbol->string (machine-type))))
(let loop ((i 0))
(cond ((> (+ i 2) (string-length m)) #f)
((string=? (substring m i (+ i 2)) "nt") #t)
(else (loop (+ i 1))))))
0)
(else (system (string-append "chmod 755 '" path "'")))))))
;; A user-facing relative path resolves against JOLT_PWD — the user's cwd before
;; the launcher cd'd to the jolt repo root — matching the JVM, where io/file is
;; cwd-relative. (io/resource builds jfiles from the source roots directly, so it
;; isn't routed through here.)
(define (project-relative p)
(if (or (= (string-length p) 0) (char=? (string-ref p 0) #\/))
p
(let ((pwd (getenv "JOLT_PWD")))
(if (and pwd (> (string-length pwd) 0)) (string-append pwd "/" p) p))))
;; (io/file path) / (io/file parent child) — join children with "/". The File
;; keeps the path AS GIVEN (like the JVM: new File("rel").getPath() is "rel");
;; a relative path resolves against JOLT_PWD only when the filesystem is touched
;; (jfile-fs / slurp / spit / the stream constructors).
(define (jolt-make-file path . rest)
(let loop ((p (file-path-of path)) (cs rest))
(if (null? cs) (make-jfile p)
(loop (string-append p "/" (file-path-of (car cs))) (cdr cs)))))
;; the on-disk path of a value: a relative path resolves against JOLT_PWD.
(define (jfile-fs f) (project-relative (file-path-of f)))
(define (path-last-segment p)
(let loop ((i (- (string-length p) 1)))
(cond ((< i 0) p)
((char=? (string-ref p i) #\/) (substring p (+ i 1) (string-length p)))
(else (loop (- i 1))))))
;; directory children as full paths, sorted (the __list-dir seed primitive).
(define (jolt-list-dir path)
(let ((p (project-relative (file-path-of path))))
(map (lambda (e) (string-append p "/" e))
(sort string<? (directory-list p)))))
(define (jolt-dir? path) (if (file-directory? (project-relative (file-path-of path))) #t #f))
;; absolute path string (cwd-relative paths resolved against current-directory).
(define (jfile-abs p)
(if (and (> (string-length p) 0) (char=? (string-ref p 0) #\/)) p
(string-append (current-directory) "/" p)))
;; --- file metadata over Chez filesystem ops ---------------------------------
;; byte size of a regular file (0 for a directory or a missing file).
(define (file-byte-size p)
(if (or (not (file-exists? p)) (file-directory? p)) 0
(let ((port (open-file-input-port p))) (let ((n (file-length port))) (close-port port) n))))
;; last-modified as epoch milliseconds (0 if the file is absent).
(define (file-mtime-millis p)
(if (file-exists? p)
(let ((t (file-modification-time p)))
(+ (* (time-second t) 1000) (div (time-nanosecond t) 1000000)))
0))
;; mkdir -p: create p and any missing parents. Returns #t if p ends up a dir.
(define (mkdirs! p)
(unless (or (= 0 (string-length p)) (file-exists? p))
(let loop ((i (- (string-length p) 1)))
(cond ((<= i 0) #f)
((char=? (string-ref p i) #\/)
(let ((parent (substring p 0 i))) (unless (file-exists? parent) (mkdirs! parent))))
(else (loop (- i 1)))))
(guard (e (#t #f)) (mkdir p)))
(and (file-exists? p) (file-directory? p)))
;; delete a file or an (empty) directory; #t on success.
(define (delete-path! p)
(guard (e (#t #f))
(cond ((not (file-exists? p)) #f)
((file-directory? p) (delete-directory p) #t)
(else (delete-file p) #t))))
;; --- java.net.URL (a jhost "url", state #(spec)) ----------------------------
;; A File.toURL value: .toString / .toExternalForm give the spec, .getPath /
;; .getFile strip the "file:" scheme.
(define (make-url spec) (make-jhost "url" (vector spec)))
(define (url-spec u) (vector-ref (jhost-state u) 0))
(define (url-strip-scheme spec)
(if (and (>= (string-length spec) 5) (string=? (substring spec 0 5) "file:"))
(substring spec 5 (string-length spec)) spec))
(define (url-protocol spec)
(let ((i (let loop ((j 0)) (cond ((>= j (string-length spec)) #f)
((char=? (string-ref spec j) #\:) j) (else (loop (+ j 1)))))))
(if i (substring spec 0 i) "")))
;; (java.net.URL. spec) — a basic file/http URL value (a library may register a
;; richer URL shim, which overrides this).
(register-class-ctor! "URL" (lambda (spec . _) (make-url (jolt-str-render-one spec))))
(register-class-ctor! "java.net.URL" (lambda (spec . _) (make-url (jolt-str-render-one spec))))
(register-host-methods! "url"
(list (cons "toString" (lambda (self) (url-spec self)))
(cons "toExternalForm" (lambda (self) (url-spec self)))
(cons "getProtocol" (lambda (self) (url-protocol (url-spec self))))
(cons "getPath" (lambda (self) (url-strip-scheme (url-spec self))))
(cons "getFile" (lambda (self) (url-strip-scheme (url-spec self))))))
;; --- File method surface (record-method-dispatch arm) -----------------------
(define (jfile-method f name args) ; -> boxed result, or #f to fall through
(let ((p (jfile-path f)) ; the path as given (display methods)
(fp (jfile-fs f))) ; JOLT_PWD-resolved on-disk path (FS methods)
(cond
((string=? name "getPath") (list p))
((string=? name "getName") (list (path-last-segment p)))
((string=? name "toString") (list p))
((string=? name "getAbsolutePath")(list (jfile-abs fp)))
((string=? name "getCanonicalPath")(list (jfile-abs fp)))
((string=? name "toURI") (list (string-append "file:" (jfile-abs fp))))
((string=? name "toURL") (list (make-url (string-append "file:" (jfile-abs fp)))))
;; io/resource returns a File where the JVM returns a file: URL; answer the
;; two URL methods resource-serving middleware (ring) calls on the result, so
;; it sees a "file" protocol and a path without changing the return type.
((string=? name "getProtocol") (list "file"))
((string=? name "getFile") (list (jfile-abs fp)))
((string=? name "exists") (list (if (file-exists? fp) #t #f)))
((string=? name "isDirectory") (list (if (file-directory? fp) #t #f)))
((string=? name "isFile") (list (if (and (file-exists? fp) (not (file-directory? fp))) #t #f)))
((string=? name "isAbsolute") (list (if (and (> (string-length p) 0) (char=? (string-ref p 0) #\/)) #t #f)))
((string=? name "listFiles") (list (list->cseq (map make-jfile (jolt-list-dir fp)))))
;; .list -> the child NAMES (a String[]), nil if not a directory.
((string=? name "list")
(list (if (file-directory? fp)
(apply jolt-vector (sort string<? (directory-list fp)))
jolt-nil)))
((string=? name "length") (list (->num (file-byte-size fp))))
((string=? name "lastModified") (list (->num (file-mtime-millis fp))))
((string=? name "canRead") (list (if (file-exists? fp) #t #f)))
((string=? name "canWrite") (list (if (file-exists? fp) #t #f)))
((string=? name "canExecute") (list (if (file-exists? fp) #t #f)))
((string=? name "isHidden") (list (let ((nm (path-last-segment p)))
(if (and (> (string-length nm) 0) (char=? (string-ref nm 0) #\.)) #t #f))))
((string=? name "mkdir") (list (guard (e (#t #f)) (and (not (file-exists? fp)) (begin (mkdir fp) #t)))))
((string=? name "mkdirs") (list (if (mkdirs! fp) #t #f)))
((string=? name "delete") (list (if (delete-path! fp) #t #f)))
((string=? name "deleteOnExit") (list jolt-nil))
((string=? name "setLastModified")(list #t))
((string=? name "createNewFile")
(list (if (file-exists? fp) #f
(guard (e (#t #f)) (close-port (open-output-file fp 'truncate)) #t))))
((string=? name "renameTo")
(list (let ((dst (jfile-fs (car args)))) (guard (e (#t #f)) (rename-file fp dst) #t))))
((string=? name "getParentFile")
(let loop ((i (- (string-length p) 1)))
(cond ((< i 0) (list jolt-nil))
((char=? (string-ref p i) #\/) (list (make-jfile (if (= i 0) "/" (substring p 0 i)))))
(else (loop (- i 1))))))
((string=? name "getAbsoluteFile") (list (make-jfile (jfile-abs p))))
((string=? name "getCanonicalFile") (list (make-jfile (jfile-abs p))))
((string=? name "compareTo") (list (->num (let ((o (file-path-of (car args))))
(cond ((string<? p o) -1) ((string>? p o) 1) (else 0))))))
((string=? name "equals") (list (and (jfile? (car args)) (string=? p (jfile-path (car args))))))
((string=? name "hashCode") (list (->num (string-hash p))))
((string=? name "getParent")
(let loop ((i (- (string-length p) 1)))
(cond ((< i 0) (list jolt-nil))
((char=? (string-ref p i) #\/) (list (if (= i 0) "/" (substring p 0 i))))
(else (loop (- i 1))))))
(else #f))))
(register-method-arm! 41
(lambda (obj method-name rest-args)
(if (jfile? obj)
(let* ((rest (if (jolt-nil? rest-args) '() (seq->list rest-args)))
(r (jfile-method obj method-name rest)))
(if r (car r) (error #f "no File method" method-name)))
'pass)))
;; .isDirectory / .listFiles emit to jolt-host-call (rt.ss), not record-method-
;; dispatch — the shims there assume a path STRING target. Make them jfile-aware
;; so file-seq's File branch works.
(define %io-host-call jolt-host-call)
(set! jolt-host-call
(lambda (method target . args)
(cond
((and (jfile? target) (string=? method "isDirectory"))
(if (file-directory? (jfile-fs target)) #t #f))
((and (jfile? target) (string=? method "listFiles"))
(list->cseq (map make-jfile (jolt-list-dir target))))
(else (apply %io-host-call method target args)))))
;; --- slurp / spit / flush ---------------------------------------------------
(define (read-file-string path)
(let ((p (open-input-file path)))
(let ((s (get-string-all p))) (close-port p) (if (eof-object? s) "" s))))
;; Drain a jhost reader (StringReader / PushbackReader): read code units from the
;; current position to EOF (-1) and assemble the string. Used by slurp; advances
;; the reader, as on the JVM.
(define (drain-reader r)
(let loop ((acc '()))
(let ((u (record-method-dispatch r "read" jolt-nil)))
(if (or (jolt-nil? u) (and (number? u) (< u 0)))
(list->string (reverse acc))
(loop (cons (integer->char (exact (truncate u))) acc))))))
(define (reader-jhost? x)
(and (jhost? x) (member (jhost-tag x) '("string-reader" "pushback-reader"))))
;; Refill a host reader so subsequent read/slurp see `s` (the unconsumed tail).
(define (reader-refill! r s)
(cond
((string=? (jhost-tag r) "string-reader")
(vector-set! (jhost-state r) 0 s) (vector-set! (jhost-state r) 1 0))
((string=? (jhost-tag r) "pushback-reader")
(vector-set! (jhost-state r) 0 (host-new "StringReader" s))
(vector-set! (jhost-state r) 1 '()))))
;; Read ONE form from a host reader (StringReader/PushbackReader): drain the
;; remaining chars, parse one form, push the tail back. -> (values form found?).
;; (read r) over a java.io reader — cuerdas' interpolation reads this way.
(define (host-reader-read-form r)
(let* ((s (drain-reader r)) (pr (jolt-parse-next s)))
(if (jolt-nil? pr)
(begin (reader-refill! r "") (values jolt-nil #f))
(begin (reader-refill! r (jolt-nth pr 1)) (values (jolt-nth pr 0) #t)))))
;; clojure.edn/read over a reader: drain the jhost reader to a string and read the
;; first EDN form (read-string). Re-asserted over the prelude in post-prelude.ss.
(define (chez-edn-read reader)
(jolt-invoke (var-deref "clojure.core" "read-string")
(if (reader-jhost? reader) (drain-reader reader) (jolt-str-render-one reader))))
;; line-seq: an io/reader is a jhost StringReader. Drain it (or take a string)
;; and split on newline; a trailing newline does NOT yield a final empty line
;; (like readLine -> nil at EOF). Re-asserted in post-prelude.ss.
(define (chez-lines s)
(let loop ((cs (string->list s)) (cur '()) (acc '()))
(cond ((null? cs) (reverse (if (null? cur) acc (cons (list->string (reverse cur)) acc))))
((char=? (car cs) #\newline) (loop (cdr cs) '() (cons (list->string (reverse cur)) acc)))
(else (loop (cdr cs) (cons (car cs) cur) acc)))))
(define (chez-line-seq rdr)
(list->cseq (chez-lines (cond ((string? rdr) rdr)
((reader-jhost? rdr) (drain-reader rdr))
(else (jolt-str-render-one rdr))))))
;; (slurp src :encoding "...") — pull the charset from the trailing kwargs.
(define (slurp-encoding opts)
(let loop ((o opts))
(cond ((or (null? o) (null? (cdr o))) '())
((and (keyword-t? (car o)) (string=? (keyword-t-name (car o)) "encoding"))
(list (jolt-str-render-one (cadr o))))
(else (loop (cddr o))))))
;; drain a byte input-stream shim (tagged-table) one byte at a time to a bytevector.
(define (drain-byte-stream src)
(let loop ((acc '()))
(let ((b (record-method-dispatch src "read" jolt-nil)))
(if (or (jolt-nil? b) (and (number? b) (< b 0)))
(u8-list->bytevector (reverse acc))
(loop (cons (bitwise-and (jnum->exact b) #xff) acc))))))
(define (jolt-slurp src . opts)
(cond
((jfile? src) (read-file-string (jfile-fs src)))
((embedded-res? src) (embedded-res-content src))
((reader-jhost? src) (drain-reader src))
;; bytes (a bytevector or a jolt byte-array): decode with :encoding (UTF-8
;; default). clj-http-lite slurps response-body byte arrays.
((bytevector? src) (decode-bytevector src (slurp-encoding opts)))
((and (jolt-array? src) (eq? (jolt-array-kind src) 'byte))
(decode-bytevector (na-bytearray->bv src) (slurp-encoding opts)))
;; a byte input-stream shim (e.g. clj-http-lite's :as :stream body): drain it.
((and (htable? src) (jolt-truthy? (jolt-ref-get src (keyword "jolt" "input-stream"))))
(decode-bytevector (drain-byte-stream src) (slurp-encoding opts)))
((string? src) (read-file-string (project-relative src)))
(else (error #f "slurp: unsupported source" src))))
(define (spit-append? opts)
(let loop ((o opts))
(cond ((or (null? o) (null? (cdr o))) #f)
((and (keyword-t? (car o)) (string=? (keyword-t-name (car o)) "append")
(jolt-truthy? (cadr o))) #t)
(else (loop (cddr o))))))
(define (jolt-spit path content . opts)
(let* ((p (project-relative (file-path-of path)))
(port (open-output-file p (if (spit-append? opts) 'append 'truncate))))
(put-string port (jolt-str-render-one content))
(close-port port)
jolt-nil))
(define (jolt-flush) (flush-output-port (current-output-port)) jolt-nil)
;; --- str / type / instance? integration ------------------------------------
;; str of a jfile is its path (Clojure's File.toString).
(register-str-render! jfile? jfile-path)
;; stdin line seam: the clojure.core *in* reader (50-io.clj) drives read-line /
;; read / read+string through __stdin-read-line. Return the next line (newline
;; stripped) or nil at EOF. Without this, (read-line) and the REPL call nil.
(def-var! "clojure.core" "__stdin-read-line"
(lambda () (let ((l (get-line (current-input-port)))) (if (eof-object? l) jolt-nil l))))
;; (type f) -> :jolt/file (the tagged-file :jolt/type). Re-def-var!
;; "type": natives-meta.ss already bound the var to the old jolt-type value, so the
;; set! alone (which updates the symbol for internal callers) wouldn't reach it.
(define io-kw-file (keyword "jolt" "file"))
(define %io-type jolt-type)
(set! jolt-type (lambda (x) (if (jfile? x) io-kw-file (%io-type x))))
;; (instance? java.io.File f): the instance? macro passes the class-name symbol;
;; match "File" / "java.io.File" (and any *.File) against a jfile.
(register-instance-check-arm!
(lambda (type-sym val)
(let ((tname (symbol-t-name type-sym)))
(if (and (jfile? val)
(or (string=? tname "File") (string=? tname "java.io.File")
(string=? (path-last-segment tname) "File")))
#t
'pass))))
;; --- def-var! the native names the overlay file-seq + str/slurp use ----
(def-var! "clojure.core" "__make-file" jolt-make-file)
(def-var! "clojure.core" "__file?" jolt-file?)
(def-var! "clojure.core" "__dir?" jolt-dir?)
(def-var! "clojure.core" "__list-dir" (lambda (p) (list->cseq (jolt-list-dir p))))
(def-var! "clojure.core" "slurp" jolt-slurp)
(def-var! "clojure.core" "spit" jolt-spit)
(def-var! "clojure.core" "flush" jolt-flush)
;; --- with-open's close seam (__close): a map-like value closes via its :close
;; fn; a jhost reader/writer/file via its .close method (a no-op here); anything
;; else is an error.
(define (jolt-close x)
(cond
((jolt-nil? x) jolt-nil)
((and (jhost? x) (member (jhost-tag x) '("string-reader" "pushback-reader" "writer"
"file-writer" "port-writer" "print-writer")))
(record-method-dispatch x "close" jolt-nil) jolt-nil)
;; a library's stream shim (tagged-table) closes via its registered .close
;; method (a no-op for in-memory streams); absent method -> no-op.
((htable? x) (guard (e (#t jolt-nil)) (record-method-dispatch x "close" jolt-nil)) jolt-nil)
((jfile? x) jolt-nil)
;; a deftype/defrecord that implements a `close` method (java.io.Closeable /
;; AutoCloseable, e.g. tools.reader's reader types) closes through it — the
;; same method (.close x) would dispatch to.
((and (jrec? x) (jrec-cl x "close"))
(record-method-dispatch x "close" jolt-nil) jolt-nil)
(else
(let ((closef (jolt-get x (keyword #f "close") jolt-nil)))
(if (and (not (jolt-nil? closef)) (procedure? closef))
(begin (jolt-invoke closef) jolt-nil)
(error #f "with-open: don't know how to close" x))))))
(def-var! "clojure.core" "__close" jolt-close)
;; --- clojure.java.io/reader: an in-memory java.io.Reader over the source. An
;; existing reader passes through; a File / path / URL is slurped; a char[] (or
;; any seq) becomes a reader over (apply str …). Mirrors io.clj's reader. Returns
;; a StringReader (host-static.ss jhost) so .read/.mark/.reset and slurp work.
(define (seq-source->string x)
(apply string-append (map jolt-str-render-one (seq->list x))))
;; io/reader returns an in-memory StringReader (the full Reader contract incl.
;; (read), mark/reset and pushback). The streaming java.io.FileReader /
;; BufferedReader classes (io-streams.ss) read a Chez port directly when a caller
;; wants to avoid loading the whole source.
(define (jolt-io-reader x)
(cond
((reader-jhost? x) x)
((jfile? x) (host-new "StringReader" (read-file-string (jfile-fs x))))
((embedded-res? x) (host-new "StringReader" (embedded-res-content x)))
((and (jhost? x) (string=? (jhost-tag x) "url"))
(host-new "StringReader" (read-file-string (url-strip-scheme (url-spec x)))))
((string? x) (host-new "StringReader" (read-file-string (project-relative x))))
((or (cseq? x) (empty-list-t? x) (pvec? x))
(host-new "StringReader" (seq-source->string x)))
(else (host-new "StringReader" (jolt-str-render-one x)))))
;; --- clojure.java.io/writer: an existing writer passes through; a File / path
;; gets a file-backed writer (host-static.ss "file-writer") that persists on
;; flush/close. Mirrors io.clj's writer over the host's StringWriter/file ports.
(define (jolt-io-writer x)
(cond
((and (jhost? x) (string=? (jhost-tag x) "writer")) x)
((and (jhost? x) (string=? (jhost-tag x) "file-writer")) x)
((jfile? x) (make-jhost "file-writer" (vector (jfile-path x) "")))
((string? x) (make-jhost "file-writer" (vector x "")))
(else (error #f "io/writer: don't know how to create a writer from" x))))
;; --- clojure.java.io ns -----------------------------------------------------
(def-var! "clojure.java.io" "file" jolt-make-file)
(def-var! "clojure.java.io" "as-file" (lambda (x) (if (jfile? x) x (make-jfile (file-path-of x)))))
;; "reader" is bound by natives-array.ss (loaded later) so a char[] argument is
;; handled; that binding delegates here via jolt-io-reader for everything else.
(def-var! "clojure.java.io" "writer" jolt-io-writer)
(def-var! "clojure.java.io" "input-stream" jolt-io-reader)
(def-var! "clojure.java.io" "output-stream" jolt-io-writer)
;; resource: jolt has no classpath, so a named resource is resolved against the
;; loader's source roots (a project's :paths, e.g. "resources"). Returns a File
;; (slurp/reader-able) for the first match, else nil. get-source-roots is the
;; loader's accessor (loader.ss), resolved at call time — the runtime CLI loads it.
(define (jolt-io-resource name)
(let* ((nm (jolt-str-render-one name))
(emb (hashtable-ref embedded-resources nm #f)))
(if emb (make-embedded-res nm emb)
(let loop ((roots (get-source-roots)))
(cond ((null? roots) jolt-nil)
((file-exists? (string-append (car roots) "/" nm)) (make-jfile (string-append (car roots) "/" nm)))
(else (loop (cdr roots))))))))
(def-var! "clojure.java.io" "resource" jolt-io-resource)
;; as-url honors a library-registered URL class (e.g. jolt-lang/http-client's full
;; java.net.URL shim) so io/as-url and (URL. spec) agree; else the file-only jhost.
(def-var! "clojure.java.io" "as-url"
(lambda (x)
(cond ((and (jhost? x) (string=? (jhost-tag x) "url")) x)
((htable? x) x)
(else (let ((ctor (lookup-class class-ctors-tbl "URL")))
(if ctor (ctor (jolt-str-render-one x)) (make-url (jolt-str-render-one x))))))))
;; --- java.lang.ClassLoader --------------------------------------------------
;; jolt has no classpath; a "classloader" resolves a named resource against the
;; loader's source roots (the same model as clojure.java.io/resource), returning a
;; file: URL or nil. getSystemClassLoader / a thread's contextClassLoader both hand
;; back this loader. Libraries that probe the classpath (e.g. migratus's migration-
;; dir discovery) then fall back to the filesystem when a resource isn't a root.
(define the-classloader (make-jhost "classloader" (vector)))
(define (cl-get-resource self name)
(let ((nm (jolt-str-render-one name)))
(let loop ((roots (get-source-roots)))
(cond ((null? roots) jolt-nil)
((file-exists? (string-append (car roots) "/" nm))
(make-url (string-append "file:" (car roots) "/" nm)))
(else (loop (cdr roots)))))))
;; getResources: every source root that holds the named resource, as file: URLs
;; (enumeration-seq just calls seq, so a list serves). ring's static-resource
;; symlink check enumerates these to confirm a served file sits under a root.
(define (cl-get-resources self name)
(let ((nm (jolt-str-render-one name)))
(let loop ((roots (get-source-roots)) (acc '()))
(cond ((null? roots) (list->cseq (reverse acc)))
((file-exists? (string-append (car roots) "/" nm))
(loop (cdr roots) (cons (make-url (string-append "file:" (car roots) "/" nm)) acc)))
(else (loop (cdr roots) acc))))))
(register-host-methods! "classloader"
(list (cons "getResource" cl-get-resource)
(cons "getResources" cl-get-resources)
(cons "getResourceAsStream"
(lambda (self name)
(let ((u (cl-get-resource self name)))
(if (jolt-nil? u) jolt-nil (host-new "StringReader" (jolt-slurp (url-strip-scheme (url-spec u))))))))))
(register-class-statics! "ClassLoader" (list (cons "getSystemClassLoader" (lambda () the-classloader))))
(register-class-statics! "java.lang.ClassLoader" (list (cons "getSystemClassLoader" (lambda () the-classloader))))
;; clojure.lang.RT/baseLoader — the resource-resolving class loader (RT/baseLoader
;; is how libraries reach Clojure's base loader, e.g. aws-api's resources ns).
(register-class-statics! "RT" (list (cons "baseLoader" (lambda () the-classloader))))
(register-class-statics! "clojure.lang.RT" (list (cons "baseLoader" (lambda () the-classloader))))
;; clojure.lang.RT/nextID — process-unique increasing id (AtomicInteger(1)
;; getAndIncrement), used by id generators such as core.logic's lvar.
(define rt-next-id-counter 1)
(define (rt-next-id)
(let ((v rt-next-id-counter))
(set! rt-next-id-counter (+ rt-next-id-counter 1))
v))
(register-class-statics! "RT" (list (cons "nextID" rt-next-id)))
(register-class-statics! "clojure.lang.RT" (list (cons "nextID" rt-next-id)))
;; clojure.lang.Util — hash/equality helpers libraries call directly (core.logic's
;; LCons.hashCode uses Util/hash). hash = Java hashCode (0 for nil); hasheq = the
;; value hash jolt's = uses; equiv = value equality; identical = reference identity.
(let ((util-statics
(list (cons "hash" (lambda (x) (if (jolt-nil? x) 0 (record-method-dispatch x "hashCode" jolt-nil))))
(cons "hasheq" (lambda (x) (jolt-hash x)))
(cons "equiv" (lambda (a b) (if (jolt= a b) #t #f)))
(cons "identical" (lambda (a b) (if (eq? a b) #t #f))))))
(register-class-statics! "Util" util-statics)
(register-class-statics! "clojure.lang.Util" util-statics))
;; Thread/currentThread -> a fresh thread jhost wrapping THIS thread's interrupt
;; flag (the box from current-interrupt-box, host-static.ss), so .interrupt from
;; any thread sets the target thread's flag and .isInterrupted reads it without
;; clearing (instance semantics; the static Thread/interrupted reads-and-clears).
;; getContextClassLoader hands back the loader.
(register-host-methods! "thread"
(list (cons "getContextClassLoader" (lambda (self) the-classloader))
(cons "getName" (lambda (self) "main"))
;; no reified call stack (jolt does TCO, so caller frames are erased) — an
;; empty StackTraceElement[]. clojure.spec.test.alpha's instrument reads it
;; to name the caller var; it degrades to no ::caller, the conform error
;; (the ExceptionInfo) is still thrown.
(cons "getStackTrace" (lambda (self) (jolt-vector)))
(cons "interrupt" (lambda (self)
(when (box? (jhost-state self)) (set-box! (jhost-state self) #t))
jolt-nil))
(cons "isInterrupted" (lambda (self)
(and (box? (jhost-state self)) (unbox (jhost-state self)) #t)))))
(define (current-thread-handle) (make-jhost "thread" (current-interrupt-box)))
(register-class-statics! "Thread" (list (cons "currentThread" current-thread-handle)))
(register-class-statics! "java.lang.Thread" (list (cons "currentThread" current-thread-handle)))
;; --- java.io.File / java.util.UUID constructors -----------------------------
;; (java.io.File. parent child) joins with "/"; (File. path) wraps the path.
(register-class-ctor! "File"
(lambda (a . rest)
(if (pair? rest)
(jolt-make-file (string-append (file-path-of a) "/" (file-path-of (car rest))))
(jolt-make-file a))))
;; File statics: the platform separators plus createTempFile / listRoots.
(define temp-file-counter 0)
(define (file-create-temp prefix suffix . dir)
(let* ((d (cond ((pair? dir) (file-path-of (car dir)))
((getenv "TMPDIR") => (lambda (t) t))
(else "/tmp")))
(sfx (if (or (null? (list suffix)) (jolt-nil? suffix)) ".tmp" (jolt-str-render-one suffix))))
(set! temp-file-counter (+ temp-file-counter 1))
(let loop ((n temp-file-counter))
(let ((p (string-append d "/" (jolt-str-render-one prefix)
(number->string (now-millis)) "-" (number->string n) sfx)))
(if (file-exists? p) (loop (+ n 1))
(begin (close-port (open-output-file p 'truncate)) (make-jfile p)))))))
(let ((statics (list (cons "separator" "/")
(cons "separatorChar" #\/)
(cons "pathSeparator" ":")
(cons "pathSeparatorChar" #\:)
(cons "createTempFile" file-create-temp)
(cons "listRoots" (lambda () (jolt-vector (make-jfile "/")))))))
(register-class-statics! "File" statics)
(register-class-statics! "java.io.File" statics))
(register-class-ctor! "java.io.File"
(lambda (a . rest)
(if (pair? rest)
(jolt-make-file (string-append (file-path-of a) "/" (file-path-of (car rest))))
(jolt-make-file a))))
;; UUID: randomUUID / fromString statics + a (UUID. s) string ctor.
(register-class-statics! "UUID"
(list (cons "randomUUID" (lambda () (jolt-random-uuid)))
(cons "fromString" (lambda (s) (jolt-parse-uuid (jolt-str-render-one s))))))
(register-class-statics! "java.util.UUID"
(list (cons "randomUUID" (lambda () (jolt-random-uuid)))
(cons "fromString" (lambda (s) (jolt-parse-uuid (jolt-str-render-one s))))))
;; (UUID. msb lsb): build from the most/least-significant 64-bit halves (the JVM's
;; 2-long ctor), the form test.check's uuid generator uses. (UUID. s) parses a
;; string. The 128 bits format as the canonical 8-4-4-4-12 lowercase hex string.
(define (uuid-long->hex16 n)
(let* ((u (bitwise-and (jnum->exact n) #xFFFFFFFFFFFFFFFF))
(s (string-downcase (number->string u 16)))) ; JVM UUIDs are lowercase
(string-append (make-string (- 16 (string-length s)) #\0) s)))
(define (uuid-from-halves msb lsb)
(let ((h (uuid-long->hex16 msb)) (l (uuid-long->hex16 lsb)))
(make-juuid (string-append (substring h 0 8) "-" (substring h 8 12) "-" (substring h 12 16)
"-" (substring l 0 4) "-" (substring l 4 16)))))
(define (uuid-ctor . args)
(if (= (length args) 2)
(uuid-from-halves (car args) (cadr args))
(jolt-parse-uuid (jolt-str-render-one (car args)))))
(register-class-ctor! "UUID" uuid-ctor)
(register-class-ctor! "java.util.UUID" uuid-ctor)
;; (Long. n) / (Long. "n"): a Long is just jolt's integer; return it (parse a string).
(register-class-ctor! "Long" (lambda (x) (if (string? x) (parse-int-or-throw x 10 "Long") (->num (jnum->exact x)))))
(register-class-ctor! "java.lang.Long" (lambda (x) (if (string? x) (parse-int-or-throw x 10 "Long") (->num (jnum->exact x)))))
;; (Integer. n) / (Integer. "n"): jolt's integer, range-checked like intCast.
(define (integer-ctor x)
(jolt-int-cast (if (string? x) (parse-int-or-throw x 10 "Integer") x)))
(register-class-ctor! "Integer" integer-ctor)
(register-class-ctor! "java.lang.Integer" integer-ctor)
;; (Double. x) / (Double. "x"): jolt's double.
(define (double-ctor x)
(if (string? x)
(let ((n (string->number x)))
(if n (exact->inexact n)
(jolt-throw (jolt-host-throwable "java.lang.NumberFormatException"
(string-append "For input string: \"" x "\"")))))
(jolt-double x)))
(register-class-ctor! "Double" double-ctor)
(register-class-ctor! "java.lang.Double" double-ctor)
;; (Boolean. "true") / (Boolean. b): true for the string "true" (case-insensitive,
;; anything else false) or the boolean itself — Boolean.valueOf semantics; the
;; box is jolt's plain boolean.
(define (boolean-ctor x)
(cond ((string? x) (string-ci=? x "true"))
((boolean? x) x)
(else #f)))
(register-class-ctor! "Boolean" boolean-ctor)
(register-class-ctor! "java.lang.Boolean" boolean-ctor)
;; --- java.net.URI -----------------------------------------------------------
;; A minimal RFC-3986 split into scheme/authority/host/port/path/query/fragment,
;; kept in a jhost "uri" carrying the original string. (str u)/(.toString u) give
;; the original; getHost is nil for a relative URI (hiccup.util/to-str branches on
;; it). instance? java.net.URI + extend-protocol dispatch work via value-host-tags.
(define (uri-index-of s ch from)
(let ((n (string-length s)))
(let loop ((i from)) (cond ((>= i n) #f) ((char=? (string-ref s i) ch) i) (else (loop (+ i 1)))))))
(define (uri-scheme-end s)
;; index of ':' that ends a scheme (letter then alnum/+-. before any /?#), or #f.
(let ((n (string-length s)))
(and (> n 0) (char-alphabetic? (string-ref s 0))
(let loop ((i 1))
(cond ((>= i n) #f)
((char=? (string-ref s i) #\:) i)
((let ((c (string-ref s i)))
(or (char-alphabetic? c) (char-numeric? c) (char=? c #\+) (char=? c #\-) (char=? c #\.)))
(loop (+ i 1)))
(else #f))))))
(define (uri-parse s)
(let* ((n (string-length s))
(se (uri-scheme-end s))
(scheme (and se (substring s 0 se)))
(rest-start (if se (+ se 1) 0))
;; fragment
(hash (uri-index-of s #\# rest-start))
(frag (and hash (substring s (+ hash 1) n)))
(pre-frag-end (or hash n))
;; query
(qm (uri-index-of s #\? rest-start))
(query (and qm (< qm pre-frag-end) (substring s (+ qm 1) pre-frag-end)))
(hp-end (cond ((and qm (< qm pre-frag-end)) qm) (else pre-frag-end)))
;; authority (after "//")
(has-auth (and (<= (+ rest-start 2) n)
(char=? (string-ref s rest-start) #\/)
(char=? (string-ref s (+ rest-start 1)) #\/)))
(auth-start (and has-auth (+ rest-start 2)))
(auth-end (and has-auth
(let loop ((i auth-start))
(cond ((>= i hp-end) hp-end)
((char=? (string-ref s i) #\/) i)
(else (loop (+ i 1)))))))
(authority (and has-auth (substring s auth-start auth-end)))
(path-start (if has-auth auth-end rest-start))
(path (substring s path-start hp-end)))
;; host:port from authority (strip userinfo@)
(let* ((at (and authority (uri-index-of authority #\@ 0)))
(hostport (if at (substring authority (+ at 1) (string-length authority)) authority))
(colon (and hostport (uri-index-of hostport #\: 0)))
(host (cond ((not hostport) jolt-nil)
(colon (substring hostport 0 colon))
(else hostport)))
(port (if (and colon (< (+ colon 1) (string-length hostport)))
(or (string->number (substring hostport (+ colon 1) (string-length hostport))) -1)
-1)))
(make-jhost "uri"
(list (cons 'string s)
(cons 'scheme (or scheme jolt-nil))
(cons 'authority (or authority jolt-nil))
(cons 'host (if (and host (string? host) (= 0 (string-length host))) jolt-nil host))
(cons 'port (->num port))
(cons 'path (if (= 0 (string-length path)) (if has-auth "" jolt-nil) path))
(cons 'query (or query jolt-nil))
(cons 'fragment (or frag jolt-nil)))))))
(define (uri-field u k) (let ((p (assq k (jhost-state u)))) (if p (cdr p) jolt-nil)))
(register-class-ctor! "URI" (lambda (s) (uri-parse (jolt-str-render-one s))))
(register-class-ctor! "java.net.URI" (lambda (s) (uri-parse (jolt-str-render-one s))))
;; URI/create — the static factory, same as the (URI. s) constructor.
(register-class-statics! "URI" (list (cons "create" (lambda (s) (uri-parse (jolt-str-render-one s))))))
(register-class-statics! "java.net.URI" (list (cons "create" (lambda (s) (uri-parse (jolt-str-render-one s))))))
(register-host-methods! "uri"
(list (cons "toString" (lambda (u) (uri-field u 'string)))
(cons "toASCIIString" (lambda (u) (uri-field u 'string)))
(cons "getScheme" (lambda (u) (uri-field u 'scheme)))
(cons "getAuthority" (lambda (u) (uri-field u 'authority)))
(cons "getHost" (lambda (u) (uri-field u 'host)))
(cons "getPort" (lambda (u) (uri-field u 'port)))
(cons "getPath" (lambda (u) (uri-field u 'path)))
(cons "getRawPath" (lambda (u) (uri-field u 'path)))
(cons "getQuery" (lambda (u) (uri-field u 'query)))
(cons "getRawQuery" (lambda (u) (uri-field u 'query)))
(cons "getFragment" (lambda (u) (uri-field u 'fragment)))
(cons "isAbsolute" (lambda (u) (not (jolt-nil? (uri-field u 'scheme)))))
(cons "hashCode" (lambda (u) (string-hash (uri-field u 'string))))
(cons "equals" (lambda (u o) (and (jhost? o) (string=? (jhost-tag o) "uri")
(string=? (uri-field u 'string) (uri-field o 'string)))))))
;; (= u1 u2) is value equality by string form (the .equals method above only
;; serves explicit (.equals …)); hash matches so a URI works as a map key / set
;; member (ring/hiccup compare (URI. "/") values).
(define (uri-jhost? x) (and (jhost? x) (string=? (jhost-tag x) "uri")))
(register-eq-arm! (lambda (a b) (or (uri-jhost? a) (uri-jhost? b)))
(lambda (a b) (and (uri-jhost? a) (uri-jhost? b)
(string=? (uri-field a 'string) (uri-field b 'string)))))
(register-hash-arm! uri-jhost? (lambda (x) (string-hash (uri-field x 'string))))
;; str / pr-str of a uri -> its string form.
(register-str-render! (lambda (x) (and (jhost? x) (string=? (jhost-tag x) "uri")))
(lambda (x) (uri-field x 'string)))
(register-pr-readable-arm! (lambda (x) (and (jhost? x) (string=? (jhost-tag x) "uri")))
(lambda (x) (string-append "#object[java.net.URI \"" (uri-field x 'string) "\"]")))
;; class of the host value types defined by now (uri/uuid/file).
(register-class-arm! (lambda (x) (and (jhost? x) (string=? (jhost-tag x) "uri"))) (lambda (x) "java.net.URI"))
(register-class-arm! juuid? (lambda (x) "java.util.UUID"))
(register-class-arm! jfile? (lambda (x) "java.io.File"))

2307
host/chez/java/java-time.ss Normal file

File diff suppressed because it is too large Load diff

67
host/chez/java/math.ss Normal file
View file

@ -0,0 +1,67 @@
;; clojure.math — host shim over native flonum math.
;;
;; clojure.math is registered as native bindings, NOT a .clj file — so there's no
;; source tier to emit. The def-var! shims here back each clojure.math fn over
;; Chez's native procedures. The analyzer knows the clojure.math ns exists, so a
;; ref like clojure.math/sqrt lowers to a var-deref; these cells back it at
;; runtime.
;;
;; jolt is all-flonum, so every result is a flonum (inputs arrive as flonums; Chez
;; sqrt/sin/expt/... return flonums for flonum args). Semantics match
;; Clojure 1.11 clojure.math: round = floor(x+0.5), rint = round-half-even,
;; floor/ceil/floor-div return doubles, to-degrees/to-radians via PI.
(define jolt-math-pi (acos -1.0))
(define jolt-math-e (exp 1.0))
(define (jolt-math-cbrt x)
;; sign-aware so negative inputs stay real (expt of a negative flonum to a
;; fractional power goes complex).
(if (< x 0.0)
(- (expt (- x) (/ 1.0 3.0)))
(expt x (/ 1.0 3.0))))
;; clojure.math/round returns a long (exact); floor/ceil/signum/rint return doubles.
(define (jolt-math-round x) (exact (floor (+ x 0.5))))
(define (jolt-math-signum x) (cond ((< x 0.0) -1.0) ((> x 0.0) 1.0) (else 0.0)))
(define (jolt-math-to-degrees r) (/ (* r 180.0) jolt-math-pi))
(define (jolt-math-to-radians d) (/ (* d jolt-math-pi) 180.0))
(define (jolt-math-hypot a b) (sqrt (+ (* a a) (* b b))))
(define (jolt-math-floor-div a b) (floor (/ a b)))
(define (jolt-math-floor-mod a b) (- a (* b (floor (/ a b)))))
;; clojure.math fns always return a DOUBLE; Chez's sqrt/expt/sin/floor/... return
;; EXACT for exact args ((sqrt 9) -> 3, (sin 0) -> 0), so coerce.
(define (m1 f) (lambda (x) (exact->inexact (f x))))
(define (m2 f) (lambda (a b) (exact->inexact (f a b))))
(def-var! "clojure.math" "sqrt" (m1 sqrt))
(def-var! "clojure.math" "cbrt" jolt-math-cbrt)
(def-var! "clojure.math" "pow" (m2 expt))
(def-var! "clojure.math" "exp" (m1 exp))
(def-var! "clojure.math" "expm1" (lambda (x) (- (exp x) 1.0)))
(def-var! "clojure.math" "log" (m1 log))
(def-var! "clojure.math" "log10" (lambda (x) (exact->inexact (log x 10.0))))
(def-var! "clojure.math" "log1p" (lambda (x) (log (+ 1.0 x))))
(def-var! "clojure.math" "sin" (m1 sin))
(def-var! "clojure.math" "cos" (m1 cos))
(def-var! "clojure.math" "tan" (m1 tan))
(def-var! "clojure.math" "asin" (m1 asin))
(def-var! "clojure.math" "acos" (m1 acos))
(def-var! "clojure.math" "atan" (m1 atan))
;; clojure.math/atan2 is atan2(y, x); Chez's 2-arg atan is (atan y x).
(def-var! "clojure.math" "atan2" (lambda (y x) (exact->inexact (atan y x))))
(def-var! "clojure.math" "sinh" (m1 sinh))
(def-var! "clojure.math" "cosh" (m1 cosh))
(def-var! "clojure.math" "tanh" (m1 tanh))
(def-var! "clojure.math" "floor" (m1 floor))
(def-var! "clojure.math" "ceil" (m1 ceiling))
(def-var! "clojure.math" "rint" (m1 round))
(def-var! "clojure.math" "round" jolt-math-round)
(def-var! "clojure.math" "signum" jolt-math-signum)
(def-var! "clojure.math" "to-degrees" jolt-math-to-degrees)
(def-var! "clojure.math" "to-radians" jolt-math-to-radians)
(def-var! "clojure.math" "hypot" jolt-math-hypot)
(def-var! "clojure.math" "floor-div" jolt-math-floor-div)
(def-var! "clojure.math" "floor-mod" jolt-math-floor-mod)
(def-var! "clojure.math" "E" jolt-math-e)
(def-var! "clojure.math" "PI" jolt-math-pi)

View file

@ -0,0 +1,210 @@
;; natives-array.ss — Java-style mutable arrays for the Chez host.
;;
;; A jolt-array wraps a Chez mutable vector + a `kind` tag (for bytes?). The array
;; CONSTRUCTORS are native (they build the backing); the overlay's aget/aset/alength
;; are pure over count / nth / jolt.host/ref-put!, so we extend those dispatchers
;; to see a jolt-array (backed by a Chez vector). Loaded after host-table.ss (ref-put!),
;; transients.ss, seq.ss (the dispatchers it chains).
(define-record-type jolt-array (fields (mutable vec) kind) (nongenerative jolt-array-v1))
;; JVM array class name per element kind ((class (int-array 3)) -> "[I", like the
;; JVM's Class.getName for arrays). Object arrays use the descriptor form.
(define (na-array-class-name arr)
(case (jolt-array-kind arr)
((int) "[I") ((long) "[J") ((short) "[S") ((double) "[D")
((float) "[F") ((boolean) "[Z") ((byte) "[B") ((char) "[C")
(else "[Ljava.lang.Object;")))
(define (na-idx i) (if (and (number? i) (not (exact? i))) (exact (floor i)) i))
(define (na-from-seq x kind) (make-jolt-array (list->vector (seq->list (jolt-seq x))) kind))
;; (T-array size) | (T-array size init) | (T-array seq)
(define (na-num-array a rest init kind)
(if (number? a)
(make-jolt-array (make-vector (exact (na-idx a)) (if (pair? rest) (car rest) init)) kind)
(na-from-seq a kind)))
;; numeric tower: array element defaults / masked bytes / count are
;; EXACT integers (= JVM byte/short/int), matching exact integer literals.
(define (na-byte-of v) (bitwise-and (exact (floor v)) #xff))
;; --- constructors -----------------------------------------------------------
(define (na-object-array a . rest) (na-num-array a rest jolt-nil 'object))
;; integer kinds default to exact 0 (JVM int/long/short 0 -> "0", not "0.0").
(define (na-int-array a . rest) (na-num-array a rest 0 'int))
(define (na-long-array a . rest) (na-num-array a rest 0 'long))
(define (na-short-array a . rest) (na-num-array a rest 0 'short))
(define (na-double-array a . rest) (na-num-array a rest 0.0 'double))
(define (na-float-array a . rest) (na-num-array a rest 0.0 'float))
(define (na-boolean-array a . rest) (na-num-array a rest #f 'boolean))
;; char-array is a real 'char array (instance? "[C"), seqing as chars via the
;; dispatchers below — io/reader (extended here) and str/slurp consume the seq.
(define (na-char-array a . rest)
(cond
((string? a) (make-jolt-array (list->vector (string->list a)) 'char))
((number? a) (make-jolt-array (make-vector (exact (na-idx a)) #\nul) 'char))
(else (make-jolt-array
(list->vector (map (lambda (c) (if (char? c) c (integer->char (exact (truncate c)))))
(seq->list (jolt-seq a)))) 'char))))
;; (byte-array n [init]) | (byte-array coll). Also coerces the host's OTHER byte
;; carrier — a Chez bytevector (what String/.getBytes produce) — and a string's
;; UTF-8 bytes, so bytevector and byte-array interconvert across interop seams.
(define (na-byte-array a . rest)
(cond
((number? a) (make-jolt-array (make-vector (exact (na-idx a)) (na-byte-of (if (pair? rest) (car rest) 0))) 'byte))
((bytevector? a) (make-jolt-array (list->vector (bytevector->u8-list a)) 'byte))
((string? a) (make-jolt-array (list->vector (bytevector->u8-list (string->utf8 a))) 'byte))
(else (make-jolt-array (list->vector (map na-byte-of (seq->list (jolt-seq a)))) 'byte))))
;; jolt byte-array -> Chez bytevector (for String decode / utf8->string).
(define (na-bytearray->bv arr)
(let* ((v (jolt-array-vec arr)) (n (vector-length v)) (bv (make-bytevector n)))
(do ((i 0 (+ i 1))) ((= i n)) (bytevector-u8-set! bv i (bitwise-and (exact (vector-ref v i)) #xff)))
bv))
(define (na-make-array a . rest) ; (make-array len) | (make-array type len ...)
(make-jolt-array (make-vector (exact (na-idx (if (number? a) a (car rest)))) jolt-nil) 'object))
(define (na-into-array a . rest) (na-from-seq (if (pair? rest) (car rest) a) 'object))
(define (na-to-array coll) (na-from-seq coll 'object))
(define (na-aclone arr)
(if (jolt-array? arr)
(make-jolt-array (vector-copy (jolt-array-vec arr)) (jolt-array-kind arr))
(na-from-seq arr 'object)))
;; --- typed aset (return the stored value) -----------------------------------
(define (na-aset! arr i v) (vector-set! (jolt-array-vec arr) (exact (na-idx i)) v) v)
(define (na-aset-int arr i v) (na-aset! arr i v))
(define (na-aset-long arr i v) (na-aset! arr i v))
(define (na-aset-short arr i v) (na-aset! arr i v))
(define (na-aset-double arr i v) (na-aset! arr i v))
(define (na-aset-float arr i v) (na-aset! arr i v))
(define (na-aset-char arr i v) (na-aset! arr i v))
(define (na-aset-boolean arr i v) (na-aset! arr i v))
(define (na-aset-byte arr i v)
(vector-set! (jolt-array-vec arr) (exact (na-idx i)) (na-byte-of v)) v)
;; --- coercions (identity on arrays; byte/short are masked scalar casts) ------
(define (na-bytes x) (if (and (jolt-array? x) (eq? (jolt-array-kind x) 'byte)) x (na-byte-array x)))
(define (na-bytes? x) (and (jolt-array? x) (eq? (jolt-array-kind x) 'byte)))
(define (na-identity x) x)
(define (na-byte x) (jolt-byte-cast x))
(define (na-short x) (jolt-short-cast x))
;; --- chunked seqs -----------------------------------------------------------
;; The chunked-seq accessors (chunked-seq? / chunk-first / chunk-rest / chunk-next)
;; live in seq.ss with the cseq core they read; here we only bind them plus the
;; chunk-builder API (clojure.lang.ChunkBuffer + chunk-cons). chunk-buffer collects
;; appended items, chunk seals them into a pvec chunk, and chunk-cons prepends that
;; chunk onto a rest seq as a real ChunkedCons (cseq-chunked) — empty chunk == just
;; the rest, like clojure.core/chunk-cons.
(define-record-type jolt-chunkbuf (fields (mutable items)) (nongenerative jolt-chunkbuf-v1))
(define (na-chunk-buffer cap) (make-jolt-chunkbuf '()))
(define (na-chunk-append b x) (jolt-chunkbuf-items-set! b (append (jolt-chunkbuf-items b) (list x))) b)
(define (na-chunk b) (make-pvec (list->vector (jolt-chunkbuf-items b))))
(define (na-chunk-cons chunk rest)
(if (fx=? 0 (pvec-count chunk)) rest (cseq-chunked chunk 0 rest)))
;; --- extend the collection dispatchers to see a jolt-array ------------------
(define %na-count jolt-count)
(set! jolt-count (lambda (c) (if (jolt-array? c) (vector-length (jolt-array-vec c)) (%na-count c))))
(define %na-seq jolt-seq)
(set! jolt-seq (lambda (c) (if (jolt-array? c) (list->cseq (vector->list (jolt-array-vec c))) (%na-seq c))))
(define %na-nth jolt-nth)
(set! jolt-nth
(case-lambda
((c i) (if (jolt-array? c) (vector-ref (jolt-array-vec c) (exact (na-idx i))) (%na-nth c i)))
((c i d) (if (jolt-array? c)
(let ((v (jolt-array-vec c)) (j (exact (na-idx i))))
(if (and (>= j 0) (< j (vector-length v))) (vector-ref v j) d))
(%na-nth c i d)))))
(def-var! "jolt.host" "array-value?" (lambda (x) (if (jolt-array? x) #t jolt-nil)))
(define %na-get jolt-get)
(set! jolt-get
(case-lambda
((c k) (if (jolt-array? c) (jolt-nth c k jolt-nil) (%na-get c k)))
((c k d) (if (jolt-array? c) (jolt-nth c k d) (%na-get c k d)))))
;; aset (overlay) writes through jolt.host/ref-put! — mutate the slot, return arr.
;; count/nth/seq/get above are NATIVE-OPS (inlined at call sites), so aget/alength/
;; array-seq/vec already use the set!-extended globals; ref-put! is a host var
;; (var-deref'd), so re-assert its cell to the array-aware closure.
(define %na-ref-put! jolt-ref-put!)
(set! jolt-ref-put!
(lambda (t k v)
(if (jolt-array? t) (begin (vector-set! (jolt-array-vec t) (exact (na-idx k)) v) t)
(%na-ref-put! t k v))))
(def-var! "jolt.host" "ref-put!" jolt-ref-put!)
;; --- array identity: type / class / instance? recognize arrays ---------------
;; (type arr) / (class arr) -> the JVM array class name; (class …) delegates to
;; (jolt-type …) for arrays, so extending jolt-type covers both.
(define %na-type jolt-type)
(set! jolt-type (lambda (x) (if (jolt-array? x) (na-array-class-name x) (%na-type x))))
;; instance? over an array class token ([I, [C, …). An array token reaches us as
;; a string ("[C", from (Class/forName "[C")) — the dispatcher leaves it a string
;; (non-array string tokens are already normalized to symbols there); decide it
;; here, deferring everything else.
(register-instance-check-arm!
(lambda (type-sym val)
(let ((tname (cond ((string? type-sym) type-sym)
((symbol-t? type-sym) (symbol-t-name type-sym))
(else #f))))
(if (and tname (> (string-length tname) 0) (char=? (string-ref tname 0) #\[))
(and (jolt-array? val) (string=? (na-array-class-name val) tname))
'pass))))
;; clojure.java.io/reader over a char-array reads its chars (the JVM char[] branch).
(def-var! "clojure.java.io" "reader"
(lambda (x)
(if (jolt-array? x)
(host-new "StringReader"
(apply string-append (map jolt-str-render-one (seq->list (jolt-seq x)))))
(jolt-io-reader x))))
;; --- bind into clojure.core -------------------------------------------------
(for-each (lambda (p) (def-var! "clojure.core" (car p) (cdr p)))
(list
(cons "object-array" na-object-array) (cons "int-array" na-int-array)
(cons "long-array" na-long-array) (cons "short-array" na-short-array)
(cons "double-array" na-double-array) (cons "float-array" na-float-array)
(cons "boolean-array" na-boolean-array)
(cons "byte-array" na-byte-array) (cons "char-array" na-char-array)
(cons "array?" (lambda (x) (jolt-array? x)))
(cons "make-array" na-make-array)
(cons "into-array" na-into-array) (cons "to-array" na-to-array) (cons "aclone" na-aclone)
(cons "aset-int" na-aset-int) (cons "aset-long" na-aset-long)
(cons "aset-short" na-aset-short) (cons "aset-double" na-aset-double)
(cons "aset-float" na-aset-float) (cons "aset-char" na-aset-char)
(cons "aset-boolean" na-aset-boolean) (cons "aset-byte" na-aset-byte)
(cons "bytes" na-bytes) (cons "bytes?" na-bytes?)
(cons "booleans" na-identity) (cons "ints" na-identity) (cons "longs" na-identity)
(cons "shorts" na-identity) (cons "doubles" na-identity) (cons "floats" na-identity)
(cons "chars" na-identity) (cons "byte" na-byte) (cons "short" na-short)
(cons "chunk-buffer" na-chunk-buffer) (cons "chunk-append" na-chunk-append)
(cons "chunk" na-chunk) (cons "chunk-cons" na-chunk-cons)
(cons "chunk-first" na-chunk-first) (cons "chunk-rest" na-chunk-rest)
(cons "chunk-next" na-chunk-next) (cons "chunked-seq?" na-chunked-seq?)))
;; --- clojure.java.io/copy ---------------------------------------------------
;; Copy src -> dst, JVM-style. Raw bytes (byte-array / bytevector / string) and a
;; jhost reader write in one shot; any other source (a stream shim with a .read
;; method, e.g. jolt-lang/http-client's ByteArrayInputStream) drains via .read
;; into a byte-array buffer and .write to dst — both reached through method
;; dispatch, so a library's tagged-table streams work without the host knowing
;; their layout. Lives here (not io.ss) because io.ss loads before byte-array.
(define (jolt-io-copy src dst . _opts)
(define (write-all! bytes)
(record-method-dispatch dst "write" (list->cseq (list bytes 0 (vector-length (jolt-array-vec bytes))))))
(cond
((or (bytevector? src) (string? src)
(and (jolt-array? src) (eq? (jolt-array-kind src) 'byte)))
(write-all! (na-byte-array src)))
((and (jhost? src) (member (jhost-tag src) '("string-reader" "pushback-reader")))
(write-all! (na-byte-array (drain-reader src))))
(else
(let ((buf (na-byte-array 8192)))
(let loop ()
(let ((n (record-method-dispatch src "read" (list->cseq (list buf 0 8192)))))
(when (and (number? n) (> (jnum->exact n) 0))
(record-method-dispatch dst "write" (list->cseq (list buf 0 n)))
(loop)))))))
jolt-nil)
(def-var! "clojure.java.io" "copy" jolt-io-copy)

View file

@ -0,0 +1,69 @@
;; natives-queue.ss — clojure.lang.PersistentQueue for the Chez host.
;;
;; A functional queue: a `front` Scheme list (the dequeue end, head = front of the
;; queue) + a reversed `rear` Scheme list (the enqueue end, head = most recent).
;; conj adds to rear; peek/first read front; pop drops the front, rebalancing
;; rear->front when front empties — amortized O(1). A queue is jolt-sequential?, so
;; seq=?/seq-hash give cross-type equality (= [1 2 3] (queue 1 2 3)) for free, like
;; the JVM. Loaded after seq/collections/lazy-bridge/records/host-table so every
;; dispatcher it chains is at its latest binding.
(define-record-type jolt-queue (fields front rear cnt) (nongenerative jolt-queue-v1))
(define jolt-queue-empty (make-jolt-queue '() '() 0))
(define (queue-conj q x)
(if (null? (jolt-queue-front q))
(make-jolt-queue (list x) '() (fx+ (jolt-queue-cnt q) 1))
(make-jolt-queue (jolt-queue-front q) (cons x (jolt-queue-rear q)) (fx+ (jolt-queue-cnt q) 1))))
(define (queue->list q) (append (jolt-queue-front q) (reverse (jolt-queue-rear q))))
(define (queue-peek q) (if (null? (jolt-queue-front q)) jolt-nil (car (jolt-queue-front q))))
(define (queue-pop q)
(let ((f (jolt-queue-front q)))
;; popping an empty PersistentQueue returns it (Clojure's pop: if f==null
;; return this) — unlike a vector, which throws.
(cond ((null? f) q)
((null? (cdr f)) (make-jolt-queue (reverse (jolt-queue-rear q)) '() (fx- (jolt-queue-cnt q) 1)))
(else (make-jolt-queue (cdr f) (jolt-queue-rear q) (fx- (jolt-queue-cnt q) 1))))))
;; --- extend the collection dispatchers to see a jolt-queue ------------------
(define %q-seq jolt-seq)
(set! jolt-seq (lambda (x) (if (jolt-queue? x)
(let ((l (queue->list x))) (if (null? l) jolt-nil (list->cseq l)))
(%q-seq x))))
(define %q-count jolt-count)
(set! jolt-count (lambda (x) (if (jolt-queue? x) (jolt-queue-cnt x) (%q-count x))))
(define %q-empty? jolt-empty?)
(set! jolt-empty? (lambda (x) (if (jolt-queue? x) (fx=? 0 (jolt-queue-cnt x)) (%q-empty? x))))
(define %q-peek jolt-peek)
(set! jolt-peek (lambda (x) (if (jolt-queue? x) (queue-peek x) (%q-peek x))))
(define %q-pop jolt-pop)
(set! jolt-pop (lambda (x) (if (jolt-queue? x) (queue-pop x) (%q-pop x))))
(define %q-conj1 jolt-conj1)
(set! jolt-conj1 (lambda (coll x) (if (jolt-queue? coll) (queue-conj coll x) (%q-conj1 coll x))))
;; sequential => seq=?/seq-hash handle queue equality + hashing.
(define %q-sequential? jolt-sequential?)
(set! jolt-sequential? (lambda (x) (or (jolt-queue? x) (%q-sequential? x))))
;; printing: render the elements as a parenthesized list (delegate to the seq path).
(define (jolt-seq-or-empty x) (let ((s (jolt-seq x))) (if (jolt-nil? s) jolt-empty-list s)))
(register-pr-readable-arm! jolt-queue? (lambda (x) (jolt-pr-readable (jolt-seq-or-empty x))))
(register-str-render! jolt-queue? (lambda (x) (jolt-str-render-one (jolt-seq-or-empty x))))
;; class / type / instance? recognize a queue.
(register-class-arm! jolt-queue? (lambda (x) "clojure.lang.PersistentQueue"))
(register-instance-check-arm!
(lambda (type-sym val)
(if (jolt-queue? val)
(let ((tn (cond ((string? type-sym) type-sym)
((symbol-t? type-sym) (symbol-t-name type-sym)) (else ""))))
(and (member (last-dot tn)
'("PersistentQueue" "IPersistentCollection" "Sequential" "Collection" "Object"))
#t))
'pass)))
;; clojure.lang.PersistentQueue/EMPTY + a queue? predicate.
(register-class-statics! "PersistentQueue" (list (cons "EMPTY" jolt-queue-empty)))
(register-class-statics! "clojure.lang.PersistentQueue" (list (cons "EMPTY" jolt-queue-empty)))
(def-var! "clojure.core" "queue?" (lambda (x) (jolt-queue? x)))
;; the FQ class token self-evaluates (for (instance? clojure.lang.PersistentQueue …)).
(def-var! "clojure.core" "clojure.lang.PersistentQueue" "clojure.lang.PersistentQueue")

View file

@ -0,0 +1,400 @@
;; natives-str.ss — java.lang.String method interop on Chez.
;;
;; (.method s arg*) on a string target lowers to record-method-dispatch (emit.ss),
;; which falls through to jolt-string-method here when the target is a string.
;; Covers the
;; portable java.lang.String/CharSequence methods cljc libraries actually call.
;; Case mapping is ASCII (the whole engine is byte-oriented), indexOf returns -1
;; on miss as on the JVM, indices come in as flonums, char results are Scheme
;; chars, and numeric results are flonums to match jolt's number model.
;;
;; Loaded from rt.ss AFTER regex.ss (the regex methods reuse jolt-re-pattern /
;; regex-t-irx) and records.ss (which calls jolt-string-method).
;; --- ASCII case mapping (byte-oriented) -------
(define (ascii-up-char c)
(if (and (char<=? #\a c) (char<=? c #\z))
(integer->char (fx- (char->integer c) 32)) c))
(define (ascii-down-char c)
(if (and (char<=? #\A c) (char<=? c #\Z))
(integer->char (fx+ (char->integer c) 32)) c))
(define (ascii-string-up s) (list->string (map ascii-up-char (string->list s))))
(define (ascii-string-down s) (list->string (map ascii-down-char (string->list s))))
;; --- ASCII trim: drop leading/trailing chars with code <= space (JVM .trim) ---
(define (str-trim s)
(let ((len (string-length s)))
(let scan-l ((i 0))
(cond ((fx=? i len) "")
((char<=? (string-ref s i) #\space) (scan-l (fx+ i 1)))
(else (let scan-r ((j (fx- len 1)))
(if (char<=? (string-ref s j) #\space)
(scan-r (fx- j 1))
(substring s i (fx+ j 1)))))))))
(define (str-triml s)
(let ((len (string-length s)))
(let loop ((i 0))
(cond ((fx=? i len) "")
((char<=? (string-ref s i) #\space) (loop (fx+ i 1)))
(else (substring s i len))))))
(define (str-trimr s)
(let loop ((j (fx- (string-length s) 1)))
(cond ((fx<? j 0) "")
((char<=? (string-ref s j) #\space) (loop (fx- j 1)))
(else (substring s 0 (fx+ j 1))))))
;; --- substring search: first index of `needle` in `s` at/after `from`, or -1 --
(define (str-index-of s needle from)
(let ((nlen (string-length needle)) (slen (string-length s)))
(let loop ((i (max 0 from)))
(cond ((fx>? (fx+ i nlen) slen) -1)
((string=? (substring s i (fx+ i nlen)) needle) i)
(else (loop (fx+ i 1)))))))
(define (str-last-index-of s needle)
(let ((nlen (string-length needle)) (slen (string-length s)))
(let loop ((i (fx- slen nlen)) (found -1))
(cond ((fx<? i 0) found)
((string=? (substring s i (fx+ i nlen)) needle) i)
(else (loop (fx- i 1) found))))))
;; A needle arg: a char value -> its 1-char string; a number -> the char at that
;; code point (JVM treats an int arg to indexOf as a char code); else a string.
(define (str-needle x)
(cond ((char? x) (string x))
((number? x) (string (integer->char (exact (truncate x)))))
((string? x) x)
(else (jolt-str x))))
;; literal replace-all (JVM String.replace(CharSequence,CharSequence)).
(define (str-replace-literal s a b)
(let ((alen (string-length a)) (slen (string-length s)))
(if (fx=? alen 0) s
(let loop ((i 0) (acc '()))
(cond ((fx>? (fx+ i alen) slen)
(apply string-append (reverse (cons (substring s i slen) acc))))
((string=? (substring s i (fx+ i alen)) a)
(loop (fx+ i alen) (cons b acc)))
(else (loop (fx+ i 1) (cons (substring s i (fx+ i 1)) acc))))))))
;; A compiled irregex for a plain-string Java-regex pattern (or a jolt-regex).
(define (str-irx pat) (regex-t-irx (jolt-re-pattern pat)))
;; JVM String.split: split fully, then drop trailing empty strings.
(define (str-split-drop-trailing parts)
(let loop ((p (reverse parts)))
(if (and (pair? p) (string=? (car p) "")) (loop (cdr p)) (reverse p))))
;; Encode a string to bytes (a bytevector) under a named charset. UTF-8 default;
;; ISO-8859-1/latin1/ascii are one byte per char; UTF-16/UTF-32 via Chez's codecs
;; (plain "UTF-16" emits a big-endian BOM then BE, matching the JVM). Shared by
;; .getBytes and decode-bytevector (String.).
(define (charset-encode-bv s csname)
(let ((cs (ascii-string-down (if (string? csname) csname (jolt-str-render-one csname)))))
(cond
((or (string=? cs "utf-8") (string=? cs "utf8")) (string->utf8 s))
((member cs '("iso-8859-1" "latin1" "iso8859-1" "us-ascii" "ascii"))
(let* ((n (string-length s)) (bv (make-bytevector n)))
(do ((i 0 (+ i 1))) ((= i n) bv)
(bytevector-u8-set! bv i (bitwise-and (char->integer (string-ref s i)) #xff)))))
((string=? cs "utf-16be") (string->utf16 s (endianness big)))
((string=? cs "utf-16le") (string->utf16 s (endianness little)))
((or (string=? cs "utf-16") (string=? cs "utf16") (string=? cs "unicode"))
(let ((be (string->utf16 s (endianness big))))
(let* ((n (bytevector-length be)) (bv (make-bytevector (+ n 2))))
(bytevector-u8-set! bv 0 #xfe) (bytevector-u8-set! bv 1 #xff)
(bytevector-copy! be 0 bv 2 n) bv)))
((or (string=? cs "utf-32be") (string=? cs "utf-32") (string=? cs "utf32"))
(string->utf32 s (endianness big)))
((string=? cs "utf-32le") (string->utf32 s (endianness little)))
(else (string->utf8 s)))))
;; Object.hashCode parity: Java's specified String hash and Clojure's Symbol hash
;; (Util.hashCombine), so (.hashCode s) / (.hashCode sym) match the JVM. 32-bit int.
(define (jolt-u32 x) (bitwise-and x #xFFFFFFFF))
(define (jolt-s32 x) (let ((m (jolt-u32 x))) (if (>= m #x80000000) (- m #x100000000) m)))
(define (java-string-hash s)
(let ((n (string-length s)))
(let loop ((i 0) (h 0))
(if (fx<? i n)
(loop (fx+ i 1) (jolt-s32 (+ (* 31 h) (char->integer (string-ref s i)))))
(jolt-s32 h)))))
(define (java-hash-combine seed hash)
(let* ((su (jolt-u32 seed))
(sl (bitwise-arithmetic-shift-left su 6))
(sr (bitwise-arithmetic-shift-right (jolt-s32 su) 2))
(add (+ (jolt-u32 hash) #x9e3779b9 sl sr)))
(jolt-s32 (bitwise-xor su (jolt-u32 add)))))
(define (java-symbol-hash name ns)
(java-hash-combine (java-string-hash name) (if ns (java-string-hash ns) 0)))
(define (jolt-string-method method s rest)
(define (arg n) (list-ref rest n))
(cond
((string=? method "toString") s)
((string=? method "hashCode") (java-string-hash s))
((string=? method "toLowerCase") (ascii-string-down s))
((string=? method "toUpperCase") (ascii-string-up s))
((string=? method "trim") (str-trim s))
((string=? method "length") (string-length s)) ; exact int (= JVM)
((string=? method "isEmpty") (fx=? (string-length s) 0))
((string=? method "charAt") (string-ref s (jolt->idx (arg 0))))
((string=? method "substring")
(substring s (jolt->idx (arg 0))
(if (fx>? (length rest) 1) (jolt->idx (arg 1)) (string-length s))))
((string=? method "indexOf")
(str-index-of s (str-needle (arg 0))
(if (fx>? (length rest) 1) (jolt->idx (arg 1)) 0)))
((string=? method "lastIndexOf")
(str-last-index-of s (str-needle (arg 0))))
((string=? method "startsWith")
(let ((p (arg 0))) (and (fx>=? (string-length s) (string-length p))
(string=? (substring s 0 (string-length p)) p))))
((string=? method "endsWith")
(let ((p (arg 0)) (slen (string-length s)))
(and (fx>=? slen (string-length p))
(string=? (substring s (fx- slen (string-length p)) slen) p))))
((string=? method "contains")
(fx>=? (str-index-of s (str-needle (arg 0)) 0) 0))
((string=? method "concat") (string-append s (arg 0)))
((string=? method "replace") (str-replace-literal s (str-needle (arg 0)) (str-needle (arg 1))))
((string=? method "equalsIgnoreCase")
(string=? (ascii-string-down s) (ascii-string-down (arg 0))))
((string=? method "compareTo")
(let ((o (arg 0))) (cond ((string<? s o) -1.0) ((string>? s o) 1.0) (else 0.0))))
((string=? method "getBytes")
;; (.getBytes s) / (.getBytes s charset) -> a jolt byte-array (seqable /
;; countable / alength-able, like (byte-array …)); the JVM returns byte[].
(na-byte-array
(charset-encode-bv s (if (null? rest)
"utf-8"
(if (string? (arg 0)) (arg 0) (jolt-str-render-one (arg 0)))))))
((string=? method "matches") (if (irregex-match (str-irx (arg 0)) s) #t #f))
((string=? method "replaceAll") (irregex-replace/all (str-irx (arg 0)) s (arg 1)))
((string=? method "replaceFirst") (irregex-replace (str-irx (arg 0)) s (arg 1)))
((string=? method "split")
(apply jolt-vector (str-split-drop-trailing (irregex-split (str-irx (arg 0)) s))))
;; universal object-methods that reach a string target (seed object-methods):
;; a thrown string / Exception. ctor (which keeps the message string) answers
;; getMessage with itself; equals is value equality.
((or (string=? method "getMessage") (string=? method "getLocalizedMessage")) s)
((string=? method "equals") (and (string? (arg 0)) (string=? s (arg 0))))
;; String.intern: jolt strings aren't pooled, but value equality holds, so the
;; canonical representation is the string itself.
((string=? method "intern") s)
;; A class token is its canonical-name string, so Class methods land here:
;; (.getName (.getClass x)) / (.getSimpleName …) over the name string.
((or (string=? method "getName") (string=? method "getCanonicalName")) s)
((string=? method "getSimpleName")
(let ((i (str-last-index-of s "."))) (if (>= i 0) (substring s (+ i 1) (string-length s)) s)))
;; .getChars srcBegin srcEnd dst dstBegin — copy s[srcBegin,srcEnd) into the
;; char-array dst at dstBegin (used by buffered readers, e.g. data.json).
((string=? method "getChars")
(let ((src-begin (jolt->idx (arg 0))) (src-end (jolt->idx (arg 1)))
(dv (jolt-array-vec (arg 2))) (dst-begin (jolt->idx (arg 3))))
(let loop ((i src-begin) (j dst-begin))
(when (fx<? i src-end)
(vector-set! dv j (string-ref s i))
(loop (fx+ i 1) (fx+ j 1)))))
jolt-nil)
((string=? method "subSequence")
(substring s (jolt->idx (arg 0)) (jolt->idx (arg 1))))
;; Class.isArray over a class-name string: array classes are "[…" (e.g. "[C").
((string=? method "isArray") (and (fx>? (string-length s) 0) (char=? (string-ref s 0) #\[)))
(else (error #f (string-append "No method " method " for value")))))
;; --- clojure.core str-* primitives (the substrate clojure.string.clj calls) ---
;; clojure.string.clj is pure Clojure over these
;; natives; def-var!'d here so the emitted
;; clojure.string prelude tier's var-derefs resolve:
;; string/ascii-* (ASCII), string/find (index or nil), core-str-* (regex|literal).
;; (string/split sep s) -> parts, splitting on each non-overlapping sep.
(define (str-literal-split s sep)
(let ((slen (string-length s)) (plen (string-length sep)))
(if (fx=? plen 0)
(map string (string->list s))
(let loop ((i 0) (start 0) (acc '()))
(cond ((fx>? (fx+ i plen) slen)
(reverse (cons (substring s start slen) acc)))
((string=? (substring s i (fx+ i plen)) sep)
(loop (fx+ i plen) (fx+ i plen) (cons (substring s start i) acc)))
(else (loop (fx+ i 1) start acc)))))))
(define (str-upper s) (ascii-string-up s))
(define (str-lower s) (ascii-string-down s))
(define (str-reverse-b s) (list->string (reverse (string->list s))))
;; (str-find needle haystack) -> exact int index of first occurrence, or nil.
(define (str-find needle s)
(let ((i (str-index-of s needle 0)))
(if (fx<? i 0) jolt-nil i)))
;; (str-join coll [sep]) -> stringify each element (Clojure str), join by sep.
;; str-join-strs (defined below) does the join; here we just render each element.
(define (str-join coll . opt)
(let ((sep (if (pair? opt) (jolt-str-render-one (car opt)) "")))
(str-join-strs (map jolt-str-render-one (seq->list coll)) sep)))
;; (re-split irx s limit) -> parts, splitting at each match. Keeps interior AND
;; trailing empty strings (the clojure.string wrapper drops trailing for limit 0);
;; a positive limit yields at most `limit` parts (the rest kept unsplit).
;; The clojure.string.clj split wrapper
;; layers the trailing-empty trim on top.
(define (re-split irx s limit)
(let ((len (string-length s)))
(let loop ((start 0) (last 0) (out '()))
(if (and limit (fx>=? (length out) (fx- limit 1)))
(reverse (cons (substring s last len) out))
(let ((m (and (fx<=? start len) (irregex-search irx s start))))
(if (not m)
(reverse (cons (substring s last len) out))
(let ((ms (irregex-match-start-index m 0))
(me (irregex-match-end-index m 0)))
(if (fx=? me ms) ; zero-width: step past to avoid a stall
(if (fx>=? start len)
(reverse (cons (substring s last len) out))
(loop (fx+ start 1) last out))
(loop me me (cons (substring s last ms) out))))))))))
;; (str-split pat s [limit]) -> parts. Regex or literal separator; a positive
;; limit caps the part count (the unsplit tail kept), matching core-str-split.
(define (str-split pat s . opt)
(let ((limit (if (and (pair? opt) (not (jolt-nil? (car opt)))) (jolt->idx (car opt)) #f)))
(if (jolt-regex? pat)
(apply jolt-vector (re-split (regex-t-irx pat) s limit))
(let ((parts (str-literal-split s pat)))
(apply jolt-vector
(if (and limit (fx>? limit 0) (fx>? (length parts) limit))
(append (list-head parts (fx- limit 1))
(list (str-join-strs (list-tail parts (fx- limit 1)) pat)))
parts))))))
(define (str-join-strs strs sep)
(let loop ((xs strs) (first #t) (acc '()))
(cond ((null? xs) (apply string-append (reverse acc)))
(first (loop (cdr xs) #f (cons (car xs) acc)))
(else (loop (cdr xs) #f (cons (car xs) (cons sep acc)))))))
;; $0/$1... expansion in a string replacement against an irregex match (the
;; JVM/seed replacement syntax). $N -> group N's text (dropped if non-matching).
(define (expand-dollar repl m)
(let ((len (string-length repl)))
(let loop ((i 0) (acc '()))
(if (fx>=? i len)
(apply string-append (reverse acc))
(let ((c (string-ref repl i)))
(if (and (char=? c #\$) (fx<? (fx+ i 1) len)
(char<=? #\0 (string-ref repl (fx+ i 1)))
(char<=? (string-ref repl (fx+ i 1)) #\9))
(let* ((n (fx- (char->integer (string-ref repl (fx+ i 1))) 48))
(g (and (fx<=? n (irregex-match-num-submatches m))
(irregex-match-substring m n))))
(loop (fx+ i 2) (if g (cons g acc) acc)))
(loop (fx+ i 1) (cons (string c) acc))))))))
;; One match's replacement text. A string gets $N expansion; a fn (jolt closure)
;; is called with the match result (whole string, or [whole g1 ...] when grouped)
;; and its result stringified.
(define (replacement-text replacement m)
(cond
((string? replacement) (expand-dollar replacement m))
((procedure? replacement) (jolt-str-render-one (jolt-invoke replacement (irx-result m))))
(else (jolt-str-render-one replacement))))
;; regex replace, first or all matches.
(define (re-replace irx s replacement all?)
(let ((len (string-length s)))
(let loop ((start 0) (last 0) (acc '()))
(let ((m (and (fx<=? start len) (irregex-search irx s start))))
(if (not m)
(apply string-append (reverse (cons (substring s last len) acc)))
(let ((ms (irregex-match-start-index m 0))
(me (irregex-match-end-index m 0)))
(if (fx=? me ms) ; zero-width: step past
(if (fx>=? start len)
(apply string-append (reverse (cons (substring s last len) acc)))
(loop (fx+ start 1) last acc))
(let ((acc2 (cons (replacement-text replacement m)
(cons (substring s last ms) acc))))
(if all?
(loop me me acc2)
(apply string-append (reverse (cons (substring s me len) acc2))))))))))))
;; (str-replace-all pat repl s) / (str-replace pat repl s) — regex or literal.
(define (str-replace-all pat repl s)
(if (jolt-regex? pat)
(re-replace (regex-t-irx pat) s repl #t)
;; literal match: a char/number match or replacement (str/replace s \a \b)
;; coerces to a string, as on the JVM.
(str-replace-literal s (str-needle pat) (str-needle repl))))
(define (str-replace-literal-first s a b)
(let ((alen (string-length a)) (i (str-index-of s a 0)))
(if (fx<? i 0) s
(string-append (substring s 0 i) b (substring s (fx+ i alen) (string-length s))))))
(define (str-replace pat repl s)
(if (jolt-regex? pat)
(re-replace (regex-t-irx pat) s repl #f)
(str-replace-literal-first s (str-needle pat) (str-needle repl))))
(def-var! "clojure.core" "str-upper" str-upper)
(def-var! "clojure.core" "str-lower" str-lower)
(def-var! "clojure.core" "str-trim" str-trim)
(def-var! "clojure.core" "str-triml" str-triml)
(def-var! "clojure.core" "str-trimr" str-trimr)
(def-var! "clojure.core" "str-find" str-find)
(def-var! "clojure.core" "str-reverse-b" str-reverse-b)
(def-var! "clojure.core" "str-join" str-join)
(def-var! "clojure.core" "str-split" str-split)
(def-var! "clojure.core" "str-replace" str-replace)
(def-var! "clojure.core" "str-replace-all" str-replace-all)
;; (require ...) / (use ...) at runtime: register each spec's :as alias + :refer
;; names into the runtime ns tables (chez-register-spec!, ns.ss), keyed by the
;; current ns. The spine also pre-registers these at analyze time (idempotent),
;; so ns-aliases/ns-resolve over an :as alias resolve. Specs arrive evaluated
;; (quoted).
(define (chez-runtime-require . specs)
(for-each (lambda (s) (chez-register-spec! (chez-current-ns) s)) specs)
jolt-nil)
(def-var! "clojure.core" "require" chez-runtime-require)
;; use = require + refer ALL of the target's public vars (unless an explicit
;; :only/:refer filter is given, which chez-register-spec! handles per-name).
(define (chez-runtime-use . specs)
(for-each
(lambda (spec)
(chez-register-spec! (chez-current-ns) spec)
(let* ((items (cond ((pvec? spec) (seq->list spec))
((or (cseq? spec) (empty-list-t? spec)) (seq->list spec))
((symbol-t? spec) (list spec))
(else '())))
(target (and (pair? items) (symbol-t? (car items)) (symbol-t-name (car items))))
(filtered (let scan ((xs (if (pair? items) (cdr items) '())))
(cond ((null? xs) #f)
((and (keyword? (car xs))
(member (keyword-t-name (car xs)) '("only" "refer"))) #t)
(else (scan (cdr xs)))))))
(when (and target (not filtered))
(chez-register-refer-all! (chez-current-ns) target))))
specs)
jolt-nil)
(def-var! "clojure.core" "use" chez-runtime-use)
;; import: bring a deftype/defrecord from another ns into the current one. A spec
;; [from-ns Type ...] binds each Type's ctor closure under the current ns, so its
;; (Type. ...) constructor (host-new resolves it as a var) works after :import.
(define (chez-runtime-import . specs)
(for-each
(lambda (spec)
(let ((items (cond ((pvec? spec) (seq->list spec))
((or (cseq? spec) (empty-list-t? spec)) (seq->list spec))
(else '()))))
(when (and (pair? items) (symbol-t? (car items)))
(let ((from (symbol-t-name (car items))))
(for-each
(lambda (tn)
(when (symbol-t? tn)
(let ((c (var-cell-lookup from (symbol-t-name tn))))
(when (and c (var-cell-defined? c))
(def-var! (chez-current-ns) (symbol-t-name tn) (var-cell-root c))))))
(cdr items))))))
specs)
jolt-nil)
(def-var! "clojure.core" "import" chez-runtime-import)

View file

@ -0,0 +1,163 @@
;; records-interop.ss — JVM-emulation taxonomy split out of records.ss: the
;; ex-info class accessors, the exception supertype hierarchy, and instance-check
;; / case-string (the (instance? Class x) decision table). Loaded right after
;; records.ss; instance-check forward-refs nothing in records.ss at load time.
;; pmap? guard: ex-info maps are plain hash-maps, never sorted-map htables — and a
;; bare jolt-get on a sorted-map would invoke its comparator on :jolt/type and throw.
(define (ex-info-map? v)
(and (pmap? v) (jolt=2 (jolt-get v jolt-kw-ex-type jolt-nil) jolt-kw-ex-info)))
(define (ex-info-class v)
(let ((c (jolt-get v jolt-kw-class jolt-nil)))
(if (string? c) c "clojure.lang.ExceptionInfo")))
;; Is `wanted` (simple name) `cls` or a supertype of it? The exception hierarchy
;; lives in the one class graph (class-hierarchy.ss) — resolve the simple name to
;; its graph key and ask jch-isa?, so exceptions and every other class share a
;; single source of truth (ExceptionInfo -> IExceptionInfo is a graph edge).
(define (exception-isa? cls wanted)
(jch-isa? (jch-fqn-of-simple cls) wanted))
;; A raw Chez condition (an arity or non-seqable error Chez itself raised, not a
;; jolt ex-info) carries no jolt exception class. Map the ones Clojure raises a
;; specific class for, by message, so (class e) and (instance? C e) match the JVM.
;; Returns a simple class name or #f.
(define (ri-substring? needle hay)
(let ((nl (string-length needle)) (hl (string-length hay)))
(let loop ((i 0))
(cond ((> (+ i nl) hl) #f)
((string=? needle (substring hay i (+ i nl))) #t)
(else (loop (+ i 1)))))))
(define (chez-condition-exc-class v)
(and (condition? v) (message-condition? v)
(let ((m (condition-message v)))
(and (string? m)
(cond ((ri-substring? "incorrect number of arguments" m) "ArityException")
((ri-substring? "not seqable" m) "IllegalArgumentException")
;; Chez's numeric ops raise "~s is not a real number" on a bad
;; operand. The JVM throws NullPointerException for a nil operand
;; (null deref) and ClassCastException for a non-number (can't
;; cast to Number) — clojure.spec.alpha's conform-explain relies
;; on the distinction. The offending value rides in the irritants.
((or (ri-substring? "is not a real number" m)
(ri-substring? "is not a number" m))
(if (and (irritants-condition? v)
(let loop ((xs (condition-irritants v)))
(and (pair? xs) (or (jolt-nil? (car xs)) (loop (cdr xs))))))
"NullPointerException"
"ClassCastException"))
(else #f))))))
;; instance-check: (type-sym val) — type/protocol membership. Host shims loaded
;; later (io, inst-time, natives-array, natives-queue, host-static-classes)
;; register an arm with register-instance-check-arm! instead of set!-wrapping
;; instance-check; an arm returns #t/#f to decide or 'pass to defer to the next.
;; Newest arm is checked first (matches the old outermost-wins set! order).
;; instance-check-base is the JVM taxonomy fallback when no arm decides.
(define instance-check-registry '())
(define (register-instance-check-arm! f) ; f: (type-sym val) -> #t | #f | 'pass
(set! instance-check-registry (cons f instance-check-registry)))
;; (instance? C raw-condition): match when C is the condition's mapped class or a
;; supertype of it (ArityException is also an IllegalArgumentException, etc.).
(register-instance-check-arm!
(lambda (type-sym val)
(let ((k (chez-condition-exc-class val)))
(if k (if (exception-isa? k (last-dot (symbol-t-name type-sym))) #t #f) 'pass))))
;; Object / java.lang.Object is the root of the type hierarchy: every non-nil
;; value is an instance of Object; nil is not an instance of anything.
(register-instance-check-arm!
(lambda (type-sym val)
(let ((tn (symbol-t-name type-sym)))
(if (or (string=? tn "Object") (string=? tn "java.lang.Object"))
(not (jolt-nil? val))
'pass))))
(define (instance-check-base type-sym val)
(let ((tname (symbol-t-name type-sym)))
(cond
((jrec? val)
(let ((tag (jrec-tag val)))
(or (string=? tag tname)
;; a simple name matches a qualified tag only at a `.` boundary:
;; "a.b.IntervalFD" is an IntervalFD, but "a.b.MultiIntervalFD" is NOT
;; (a raw string-suffix would wrongly match the latter).
(let ((tl (string-length tag)) (nl (string-length tname)))
(and (fx>? tl nl)
(char=? (string-ref tag (fx- (fx- tl nl) 1)) #\.)
(string=? (substring tag (fx- tl nl) tl) tname)))
;; a protocol/interface the type implements (defprotocol generates an
;; interface; (instance? SomeProtocol record) is true when the record
;; implements it — core.match dispatches on instance? IPatternCompile).
(type-satisfies? tag tname)
(type-satisfies? tag (last-dot tname)))))
((jreify? val) (let ((short (last-dot tname)))
;; every Clojure reify implements IObj/IMeta (carries metadata).
(or (member short '("IObj" "IMeta"))
(and (memp (lambda (p) (string=? (last-dot p) short)) (jreify-protos val)) #t))))
((ex-info-map? val) (exception-isa? (last-dot (ex-info-class val)) (last-dot tname)))
(else (case-string tname val)))))
(define (instance-check type-sym0 val)
;; a Class value as the type arg (instance? (class x) y) -> use its name string.
(let* ((type-sym (if (jclass? type-sym0) (jclass-name type-sym0) type-sym0))
(ts (if (and (string? type-sym)
(or (= 0 (string-length type-sym))
(not (char=? (string-ref type-sym 0) #\[))))
(jolt-symbol #f type-sym)
type-sym)))
(let loop ((rs instance-check-registry))
(if (null? rs)
(instance-check-base ts val)
(let ((r ((car rs) ts val)))
(if (eq? r 'pass) (loop (cdr rs)) r))))))
(define (case-string tname val)
(cond
((member tname '("Number" "java.lang.Number")) (number? val))
((member tname '("Long" "java.lang.Long" "Integer" "java.lang.Integer"))
(and (number? val) (exact? val) (integer? val)))
((member tname '("Double" "java.lang.Double" "Float" "java.lang.Float")) (and (number? val) (flonum? val)))
((member tname '("Ratio" "clojure.lang.Ratio")) (and (number? val) (exact? val) (rational? val) (not (integer? val))))
((member tname '("String" "java.lang.String" "CharSequence" "java.lang.CharSequence")) (string? val))
((member tname '("Boolean" "java.lang.Boolean")) (boolean? val))
((member tname '("Character" "java.lang.Character")) (char? val))
((member tname '("Keyword" "clojure.lang.Keyword")) (keyword? val))
((member tname '("Symbol" "clojure.lang.Symbol")) (jolt-symbol? val))
((member tname '("Atom" "clojure.lang.Atom")) (jolt-atom? val))
((member tname '("IFn" "clojure.lang.IFn" "Fn" "clojure.lang.Fn")) (procedure? val))
((member tname '("Pattern" "java.util.regex.Pattern")) (regex-t? val))
((member tname '("URI" "java.net.URI"))
(and (jhost? val) (string=? (jhost-tag val) "uri")))
((member tname '("File" "java.io.File")) (jfile? val))
((member tname '("UUID" "java.util.UUID")) (juuid? val))
(else #f)))
;; str of a record uses a custom (Object toString) impl if the type defines one
;; (deftype with no default toString relies on this); otherwise the map form
;; without the leading # (Clojure's record .toString). converters.ss loads before
;; records.ss, so this set! sees the registry — forward refs resolve at call time.
(def-var! "clojure.core" "instance-check" instance-check)
;; Broad-catch fallback for catch-clause dispatch (analyze-try desugars
;; (catch C e …) to (or (instance? C e) (__catch-broad? "C" e))). A jolt host
;; condition or a raw raised value carries no jolt exception class, so instance?
;; can't place it; a Clojure (catch C e) over such a value matches when C is
;; RuntimeException (or a subclass) / Exception / Throwable — most host runtime
;; errors are RuntimeExceptions. Typed throwables (ex-info, (SomeException. …)) are
;; recognized by instance? as Throwable, so untyped? is false and they dispatch
;; precisely through the instance? arm instead.
(define throwable-type-sym (jolt-symbol #f "Throwable"))
(define (simple-class-name nm)
(let loop ((i (- (string-length nm) 1)))
(cond ((< i 0) nm)
((char=? (string-ref nm i) #\.) (substring nm (+ i 1) (string-length nm)))
(else (loop (- i 1))))))
(define (jolt-catch-broad? nm v)
(and (not (instance-check throwable-type-sym v))
(let ((s (simple-class-name nm)))
(or (exception-isa? s "RuntimeException")
(string=? s "Exception")
(string=? s "Throwable")))))
(def-var! "clojure.core" "__catch-broad?"
(lambda (nm v) (if (jolt-catch-broad? nm v) #t #f)))

View file

@ -0,0 +1,122 @@
#!/bin/sh
# joltc self-build smoke (jolt-eaj): build joltc as a self-contained binary, then
# use THAT binary to compile a jolt app with Chez and cc removed from the
# environment — the whole point of the feature. The produced app must then run
# and match the same expected output as build-smoke.sh.
root="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)"
cd "$root"
# Preflight: building joltc itself needs the Chez kernel dev files (libkernel.a +
# scheme.h) and a C compiler, same as build-smoke.sh. A distro chezscheme package
# ships neither, so skip there (CI included).
csv="$JOLT_CHEZ_CSV"
if [ -z "$csv" ]; then
chez_bin="$(command -v chez || command -v scheme || command -v petite || true)"
if [ -n "$chez_bin" ]; then
base="$(cd "$(dirname "$chez_bin")/.." 2>/dev/null && pwd)"
for d in "$base"/lib/csv*/*/; do
[ -f "${d}libkernel.a" ] && csv="${d%/}" && break
done
fi
fi
if ! command -v cc >/dev/null 2>&1 || [ -z "$csv" ] || [ ! -f "$csv/scheme.h" ] || [ ! -f "$csv/libkernel.a" ]; then
echo "joltc self-build smoke: skipped (Chez kernel dev files or C compiler not available)"
exit 0
fi
export JOLT_CHEZ_CSV="$csv"
# 1. Build joltc (debug profile — faster; the self-contained app-build mechanism
# is identical to release, only Chez compile settings differ).
joltc="$root/target/debug/joltc"
echo "joltc self-build smoke: building $joltc"
if ! chez --script host/chez/build-joltc.ss debug "$joltc" >/dev/null 2>&1; then
echo " FAIL: build-joltc.ss exited non-zero"
exit 1
fi
[ -x "$joltc" ] || { echo " FAIL: no joltc executable produced"; exit 1; }
# 2. The distributed joltc must run with no Chez install: a basic eval.
got_e="$(env -i HOME="$HOME" "$joltc" -e '(reduce + (range 10))' 2>&1)"
if [ "$got_e" != "45" ]; then
echo " FAIL: joltc -e under empty env gave '$got_e', want 45"
exit 1
fi
# 2b. JOLT_TRACE must take effect in the BUILT binary. The env check runs at
# runtime (the launcher), NOT at heap-build where JOLT_TRACE is always unset — so
# an uncaught error shows a tail-frame trace recovering the TCO-elided chain, and
# exactly ONE trace block (the launcher must not double-print it).
got_tr="$(env -i HOME="$HOME" JOLT_TRACE=1 "$joltc" -e '(defn a [x] (+ x 1)) (defn b [x] (a x)) (b :x)' 2>&1)"
if ! printf '%s' "$got_tr" | grep -q ' trace:' || ! printf '%s' "$got_tr" | grep -q 'b'; then
echo " FAIL: JOLT_TRACE=1 in the built joltc produced no tail-frame trace"
echo "--- got ---"; echo "$got_tr"; exit 1
fi
if [ "$(printf '%s' "$got_tr" | grep -c ' trace:')" != "1" ]; then
echo " FAIL: built joltc double-printed the trace block"
echo "--- got ---"; echo "$got_tr"; exit 1
fi
# 3. Build an app through the distributed joltc with an EMPTY environment — no
# PATH at all, so no chez, no cc, no shell tools are reachable. This is the core
# guarantee: joltc compiles apps entirely on its own.
app="$(mktemp -d)/build-app"
cp -r "$root/test/chez/build-app" "$app"
out="$app/app"
echo "joltc self-build smoke: compiling app.core via the binary (no chez/cc on PATH)"
if ! env -i HOME="$HOME" JOLT_PWD="$app" "$joltc" build -m app.core -o "$out" >/dev/null 2>&1; then
echo " FAIL: self-contained jolt build exited non-zero"
rm -rf "$(dirname "$app")"
exit 1
fi
[ -x "$out" ] || { echo " FAIL: no app executable produced"; rm -rf "$(dirname "$app")"; exit 1; }
# 4. The produced app runs from a neutral cwd and matches build-smoke's output.
got="$(cd / && "$out" alpha bb ccc 2>&1)"
want='embedded resource ok
HELLO FROM A BUILT BINARY!
HELLO FROM A BUILT BINARY!
args: [alpha bb ccc]
sum: 10
greet-default: greet:default
greet-loud: greet:loud
greet-soft: greet:soft'
rm -rf "$(dirname "$app")"
if [ "$got" != "$want" ]; then
echo " FAIL: produced app output mismatch"
echo "--- want ---"; echo "$want"
echo "--- got ----"; echo "$got"
exit 1
fi
# 5. Static native linking through the distributed joltc: it bundles the Chez
# kernel, so with the system cc (but still no external Chez) it re-links a stub
# that bakes a :jolt/native :static archive into the app. The app then calls the
# C function with the archive removed from disk. Uses the normal PATH so cc — and
# the kernel's link deps (lz4/…) — are found, but Chez stays out of the build.
napp="$(mktemp -d)/native-app"
mkdir -p "$napp/src/app"
printf 'int jolt_static_answer(void){return 42;}\n' > "$napp/greet.c"
cc -c "$napp/greet.c" -o "$napp/greet.o" && ar rcs "$napp/libgreet.a" "$napp/greet.o"
cat > "$napp/src/app/core.clj" <<'EOF'
(ns app.core (:require [jolt.ffi :as ffi]))
(ffi/defcfn answer "jolt_static_answer" [] :int)
(defn -main [& _] (println "answer:" (answer)))
EOF
cat > "$napp/deps.edn" <<EOF
{:paths ["src"]
:jolt/native [{:name "greet" :static {:archive "$napp/libgreet.a"}}]}
EOF
nout="$napp/app"
echo "joltc self-build smoke: static-linking a native lib via the binary (no external Chez)"
if ! JOLT_PWD="$napp" "$joltc" build -m app.core -o "$nout" >/dev/null 2>&1; then
echo " FAIL: static native build via distributed joltc exited non-zero"
rm -rf "$(dirname "$napp")"; exit 1
fi
rm -f "$napp/libgreet.a" "$napp/greet.o" # nothing to load at runtime
got_n="$(cd / && "$nout" 2>&1)"
rm -rf "$(dirname "$napp")"
if [ "$got_n" != "answer: 42" ]; then
echo " FAIL: static-linked app (via distributed joltc) output mismatch"
echo "--- got ----"; echo "$got_n"; exit 1
fi
echo "joltc self-build smoke: passed (joltc runs + builds a working app with no external toolchain, incl. static native linking)"

95
host/chez/lazy-bridge.ss Normal file
View file

@ -0,0 +1,95 @@
;; lazy-seq bridge — make-lazy-seq / coll->cells.
;;
;; The `lazy-seq` macro (00-syntax.clj) expands to
;; (make-lazy-seq (fn* [] (coll->cells (do body))))
;; and `lazy-cat` to (concat (lazy-seq c) ...). These back every overlay fn
;; built on lazy-seq — repeat / iterate / cycle / dedupe / take-nth / keep /
;; interpose / reductions / tree-seq (-> flatten) / lazy-cat.
;;
;; Bridge to the cseq model (seq.ss): a `jolt-lazyseq` is a deferred seq — a 0-arg
;; thunk that, when forced once, yields a seq (cseq | nil). coll->cells coerces the
;; body result to a seq (= jolt-seq), so the thunk already returns a seq; jolt-seq
;; is extended to force a lazyseq. The one trap: (cons x (a-lazy-seq)) must NOT
;; force the tail (else (repeat x) = (lazy-seq (cons x (repeat x))) loops forever),
;; so jolt-cons defers a lazyseq tail into a lazy cseq cell.
;;
;; Loaded LAST (after host-table.ss): %ls-seq then captures the fully-extended
;; jolt-seq (sorted-aware), so a lazy body returning a sorted coll still seqs.
(define-record-type jolt-lazyseq
(fields (mutable thunk) (mutable val) (mutable realized?))
(nongenerative jolt-lazyseq-v1))
(define (jolt-make-lazy-seq thunk) (make-jolt-lazyseq thunk jolt-nil #f))
;; force once and memoize. The thunk is (fn [] (coll->cells body)); coll->cells
;; already coerced the body to a seq (cseq | nil) via the live jolt-seq, so the
;; result needs no further coercion (a nested lazyseq was forced by coll->cells).
(define (force-lazyseq x)
(if (jolt-lazyseq-realized? x)
(jolt-lazyseq-val x)
(let ((r (jolt-invoke (jolt-lazyseq-thunk x))))
(jolt-lazyseq-val-set! x r)
(jolt-lazyseq-realized?-set! x #t)
(jolt-lazyseq-thunk-set! x #f)
r)))
;; coll->cells: coerce the body result to the cell representation = a seq | nil.
(define (jolt-coll->cells c) (jolt-seq c))
;; extend jolt-seq to force a lazyseq (a lazyseq is seqable -> its realized seq).
(define %ls-seq jolt-seq)
(set! jolt-seq (lambda (x) (if (jolt-lazyseq? x) (force-lazyseq x) (%ls-seq x))))
;; (cons x lazyseq): keep the tail lazy — force it only when the cseq cell is
;; walked, so an infinite (repeat/iterate/cycle) stays productive.
(define %ls-cons jolt-cons)
(set! jolt-cons (lambda (x coll)
(if (jolt-lazyseq? coll)
(cseq-lazy x (lambda () (force-lazyseq coll)))
(%ls-cons x coll))))
;; (conj lazyseq x): conj onto a seq prepends, like any seq — (conj (rest xs) y).
;; rest returns a lazyseq, so this is a common path; without it conj reports the
;; lazyseq as an "unsupported collection".
(define %ls-conj1 jolt-conj1)
(set! jolt-conj1 (lambda (coll x)
(if (jolt-lazyseq? coll) (jolt-cons x coll) (%ls-conj1 coll x))))
;; A lazyseq is a NEW value type, so the dispatchers that DON'T route through
;; jolt-seq must learn it or a raw (unrealized) lazyseq escapes — e.g. the corpus
;; compares (= [1 3 5] (take-nth 2 …)) against the raw lazyseq, and jolt=2 would
;; see an unknown type and return false. Recognizing it as sequential is enough
;; for equality + hash (seq=? / seq-hash coerce via jolt-seq); count / empty? /
;; nth / the printers don't, so coerce those explicitly.
(define %ls-sequential? jolt-sequential?)
(set! jolt-sequential? (lambda (x) (or (jolt-lazyseq? x) (%ls-sequential? x))))
(define %ls-count jolt-count)
(set! jolt-count (lambda (x) (if (jolt-lazyseq? x) (%ls-count (jolt-seq x)) (%ls-count x))))
(define %ls-empty? jolt-empty?)
(set! jolt-empty? (lambda (x) (if (jolt-lazyseq? x) (%ls-empty? (jolt-seq x)) (%ls-empty? x))))
(define %ls-nth jolt-nth)
(set! jolt-nth (case-lambda
((coll i) (if (jolt-lazyseq? coll) (%ls-nth (jolt-seq coll) i) (%ls-nth coll i)))
((coll i d) (if (jolt-lazyseq? coll) (%ls-nth (jolt-seq coll) i d) (%ls-nth coll i d)))))
;; a lazy seq prints as its realized seq — force, then re-dispatch through the
;; printer. An empty realized lazy seq is still a sequence, printing "()" (like a
;; JVM LazySeq), not "nil" — so (lazy-seq nil) and (rest '(1)) render "()".
(register-pr-str-arm! jolt-lazyseq?
(lambda (x) (let ((s (jolt-seq x))) (if (jolt-nil? s) "()" (jolt-pr-str s)))))
(register-pr-readable-arm! jolt-lazyseq?
(lambda (x) (let ((s (jolt-seq x))) (if (jolt-nil? s) "()" (jolt-pr-readable s)))))
(register-str-render! jolt-lazyseq?
(lambda (x) (let ((s (jolt-seq x))) (if (jolt-nil? s) "()" (jolt-str-render-one s)))))
;; seq? — a lazy seq IS a seq (predicates.ss's jolt-seq? predates the lazyseq
;; record). Unlike the native-op dispatchers above (called via a direct top-level
;; reference, so the set! is enough), seq? is reached through var-deref, which
;; reads the var-cell root — so the patched closure must be re-def-var!'d, not just
;; set!. (Exposed once dynamic binding let with-in-str/line-seq reach seq?.)
(define %ls-seq? jolt-seq?)
(set! jolt-seq? (lambda (x) (or (jolt-lazyseq? x) (%ls-seq? x))))
(def-var! "clojure.core" "seq?" jolt-seq?)
(def-var! "clojure.core" "make-lazy-seq" jolt-make-lazy-seq)
(def-var! "clojure.core" "coll->cells" jolt-coll->cells)

397
host/chez/loader.ss Normal file
View file

@ -0,0 +1,397 @@
;; loader.ss — file-based namespace loading + a shell primitive.
;;
;; The corpus/CLI spine compiles one program at a time; namespaces declared in
;; that program see each other because a top-level (do …) unrolls. A real project
;; spans many FILES, so `require` must locate a namespace's source on the search
;; roots and load it — transitively, once each.
;;
;; Loaded by cli.ss AFTER compile-eval.ss (it calls jolt-compile-eval-form). The
;; gates load compile-eval.ss but NOT this file, so the corpus/unit/sci runners
;; keep their alias-only `require` and are unaffected.
;; --- search roots -----------------------------------------------------------
;; An ordered list of directory strings. `require` searches them left to right.
;; The CLI seeds this with the project's resolved deps roots (jolt.deps) plus the
;; jolt-core roots so jolt.main/jolt.deps themselves load.
(define source-roots '("."))
(define (set-source-roots! roots) (set! source-roots roots))
(define (get-source-roots) source-roots)
;; --- data readers (#tag literals) -------------------------------------------
;; A project's data_readers.{clj,cljc} at a source root maps a tag symbol to a
;; qualified reader fn (e.g. {time/date time-literals.data-readers/date}). We
;; merge those into clojure.core/*data-readers* and require each reader's
;; namespace, then while loading source rewrite a registered #tag form into a
;; call (reader-fn 'inner-form) so the value is built at runtime. #inst/#uuid and
;; #"regex" stay built-in (the analyzer lowers them); only tags present in
;; *data-readers* are rewritten. data-readers-active gates the per-form walk so
;; projects without data readers (the common case) pay nothing.
(define data-readers-active #f)
(define (data-readers-table) (var-deref "clojure.core" "*data-readers*"))
;; tag keyword (:#time/date) -> its registered reader symbol, or #f.
(define (data-reader-symbol tag)
(and (keyword? tag)
(let ((nm (keyword-t-name tag)))
(and (> (string-length nm) 0) (char=? (string-ref nm 0) #\#)
(let* ((bare (substring nm 1 (string-length nm)))
(slash (let loop ((i 0))
(cond ((>= i (string-length bare)) #f)
((char=? (string-ref bare i) #\/) i)
(else (loop (+ i 1))))))
(sym (if slash
(jolt-symbol (substring bare 0 slash) (substring bare (+ slash 1) (string-length bare)))
(jolt-symbol #f bare)))
(t (data-readers-table))
(v (and (pmap? t) (jolt-get t sym))))
(and v (not (jolt-nil? v)) v))))))
;; change-tracking walk: rewrite registered #tag forms, keep everything else
;; (and its identity/metadata) intact. Mirrors reader.ss rdr-form->data but keeps
;; set FORMS for the compiler spine instead of building real sets.
(define (ldr-conv-each xs)
(let loop ((xs xs) (acc '()) (changed #f))
(if (null? xs) (values (reverse acc) changed)
(let ((c (ldr-apply-readers (car xs))))
(loop (cdr xs) (cons c acc) (or changed (not (eq? c (car xs)))))))))
(define (ldr-apply-readers x)
(cond
((and (pmap? x) (eq? (jolt-get x rdr-kw-jolt-type) rdr-kw-jolt-tagged))
(let ((rdr (data-reader-symbol (jolt-get x rdr-kw-tag)))
(inner (ldr-apply-readers (jolt-get x rdr-kw-form))))
(cond
(rdr
;; Clojure applies a data reader at read time and substitutes its result
;; as code. A reader that returns a FORM (a list — e.g. borkdude.html's
;; #html expands to (->Html (str …))) must be compiled, so splice it in.
;; A reader that returns a VALUE (time-literals #time/date -> a Date) is
;; left as a runtime call (reader-fn 'inner): the value rebuilds at
;; startup, which also keeps a non-serializable constant out of an AOT
;; build. Apply is guarded — a reader that can't run at load time (its
;; deps not ready) falls back to the runtime call too.
(let ((result (and (symbol-t? rdr) (not (jolt-nil? (symbol-t-ns rdr)))
(guard (e (#t #f))
(let ((fn (var-deref (symbol-t-ns rdr) (symbol-t-name rdr))))
(and (procedure? fn) (jolt-invoke fn inner)))))))
(if (cseq? result)
result
(jolt-list rdr (jolt-list (jolt-symbol #f "quote") inner)))))
((eq? inner (jolt-get x rdr-kw-form)) x)
(else (rdr-make-tagged (jolt-get x rdr-kw-tag) inner)))))
((rdr-set-form? x)
(let-values (((items changed) (ldr-conv-each (seq->list (jolt-get x rdr-kw-value)))))
(if changed (rdr-carry-meta x (rdr-make-set items)) x)))
((pvec? x)
(let-values (((items changed) (ldr-conv-each (vector->list (pvec-v x)))))
(if changed (rdr-carry-meta x (apply jolt-vector items)) x)))
((pmap? x)
(let ((order (hashtable-ref rdr-map-order x #f)))
(if order
(let-values (((kvs changed) (ldr-conv-each order)))
(if changed (rdr-carry-meta x (rdr-make-map kvs)) x))
(let-values (((kvs changed) (ldr-conv-each (pmap-fold x (lambda (k v a) (cons k (cons v a))) '()))))
(if changed (rdr-carry-meta x (apply jolt-hash-map kvs)) x)))))
((cseq? x)
(let-values (((items changed) (ldr-conv-each (seq->list x))))
(if changed (rdr-carry-meta x (apply jolt-list items)) x)))
(else x)))
;; read+merge one data_readers file: a literal {tag-sym reader-sym …} map.
(define (merge-data-readers-file path)
(let* ((src (read-file-string path)))
(let-values (((m j) (rdr-read-form src 0 (string-length src))))
(when (and (not (rdr-eof? m)) (pmap? m))
(let ((cur (data-readers-table)))
(def-var! "clojure.core" "*data-readers*"
(apply jolt-assoc (if (pmap? cur) cur empty-pmap)
(pmap-fold m (lambda (k v a) (cons k (cons v a))) '()))))
(set! data-readers-active #t)
;; eagerly load each reader fn's namespace so the rewritten call resolves.
(pmap-fold m (lambda (k v a)
(when (and (symbol-t? v) (symbol-t-ns v) (not (jolt-nil? (symbol-t-ns v))))
(guard (e (#t #f)) (load-namespace (symbol-t-ns v))))
a)
#f)))))
(define (load-data-readers!)
(for-each
(lambda (root)
(let ((clj (string-append root "/data_readers.clj"))
(cljc (string-append root "/data_readers.cljc")))
(cond ((file-exists? clj) (merge-data-readers-file clj))
((file-exists? cljc) (merge-data-readers-file cljc)))))
source-roots))
;; --- namespace -> file path -------------------------------------------------
;; "app.commonmark-test" -> "app/commonmark_test": split on '.', munge '-'->'_'
;; per segment, join with '/'. Matches Clojure's ns->file munging.
(define (ns-seg-munge seg)
(list->string (map (lambda (c) (if (char=? c #\-) #\_ c)) (string->list seg))))
(define (ns-name->rel name)
(let loop ((cs (string->list name)) (seg '()) (segs '()))
(cond
((null? cs)
(let ((all (reverse (cons (list->string (reverse seg)) segs))))
(let join ((xs all) (acc ""))
(cond ((null? xs) acc)
((string=? acc "") (join (cdr xs) (ns-seg-munge (car xs))))
(else (join (cdr xs) (string-append acc "/" (ns-seg-munge (car xs)))))))))
((char=? (car cs) #\.)
(loop (cdr cs) '() (cons (list->string (reverse seg)) segs)))
(else (loop (cdr cs) (cons (car cs) seg) segs)))))
;; First existing <root>/rel.clj or <root>/rel.cljc on the search roots, else #f.
;; A self-contained joltc binary embeds jolt-core + stdlib source keyed by their
;; root-relative path ("clojure/string.clj"); those are checked first, so a
;; `require` resolves with no source on disk. The dev bin/joltc has an empty
;; source store, so the two hashtable probes miss and it falls straight to disk.
(define (resolve-on-roots rel)
(let ((eclj (string-append rel ".clj")) (ecljc (string-append rel ".cljc")))
(cond
((string? (hashtable-ref embedded-resources eclj #f)) eclj)
((string? (hashtable-ref embedded-resources ecljc #f)) ecljc)
(else
(let loop ((roots source-roots))
(if (null? roots) #f
(let ((clj (string-append (car roots) "/" rel ".clj"))
(cljc (string-append (car roots) "/" rel ".cljc")))
(cond ((file-exists? clj) clj)
((file-exists? cljc) cljc)
(else (loop (cdr roots)))))))))))
;; Read a namespace source. An embedded key (resolve-on-roots above, or the
;; build driver's app-order entries) reads its baked string; everything else is
;; a real path read off disk. Bytevector entries (the bundled boots/stub) are not
;; source, so a string? guard skips them.
(define (ldr-read-source path)
(let ((emb (hashtable-ref embedded-resources path #f)))
(if (string? emb) emb (read-file-string path))))
(define (find-ns-file name) (resolve-on-roots (ns-name->rel name)))
;; --- the loaded set ---------------------------------------------------------
;; Seeded with every namespace that already has vars at load time — the baked
;; prelude/image (clojure.core, clojure.string, jolt.analyzer, …). A `require` of
;; one of those then no-ops instead of hunting for a (nonexistent) source file.
(define loaded-ns (make-hashtable string-hash string=?))
(vector-for-each (lambda (c) (hashtable-set! loaded-ns (var-cell-ns c) #t))
(hashtable-values var-table))
;; clojure.core.async ships native channel primitives (async.ss) AND a Clojure
;; overlay (stdlib/clojure/core/async.clj) with the higher-level dataflow API
;; (alts!, pipe, mult, mix, pub/sub, map, merge, …). The primitives pre-seed the
;; namespace above, which would make a `require` no-op and skip the overlay. Drop
;; it from the loaded set so a require pulls the overlay from the source roots
;; (like clojure.test); the primitives stay defined either way.
(hashtable-delete! loaded-ns "clojure.core.async")
;; Does `name` already have vars in the var-table? A namespace baked into the
;; image after the snapshot above — an AOT'd app namespace in a `jolt build`
;; binary — exists in memory with no source file; a later `require` of it must
;; no-op rather than hunt the (absent) source.
(define (ns-has-vars? name)
(let ((found #f))
(vector-for-each
(lambda (c) (when (and (not found) (string=? (var-cell-ns c) name)) (set! found #t)))
(hashtable-values var-table))
found))
;; Called after a file-backed namespace finishes loading, with (name file). The
;; build driver sets this to record app namespaces in dependency order for AOT
;; emission; a no-op for normal runs.
(define ns-loaded-hook (lambda (name file) #f))
(define (set-ns-loaded-hook! f) (set! ns-loaded-hook f))
;; Read every form from a file and compile+eval it in turn. The first form is
;; normally (ns …), which expands to (in-ns …) and switches the current ns, so
;; later forms compile in that namespace — (chez-current-ns) is re-read each step.
;;
;; Reads by POSITION rather than via __parse-next: a top-level form that reads as
;; nothing — a :cljs-only #? with no matching branch, a #_ discard, a trailing
;; comment — yields rdr-eof but still advances. parse-next collapses that to "no
;; more forms", which would silently drop the entire rest of the file; here we
;; skip the no-op form and continue to true end-of-string.
(define (load-jolt-file path)
(let* ((src (ldr-read-source path)) (end (string-length src)))
;; parameterize (not a bare set!) so a require nested in this file's ns form
;; restores path when control returns to the rest of this file.
(parameterize ((rdr-source-file path)) ; list forms read here carry :file = path
(let loop ((i 0))
(when (< i end)
(let-values (((form j) (rdr-read-form src i end)))
(when (> j i)
(unless (rdr-eof? form)
(when (getenv "JOLT_TRACE_LOAD")
(display " [load-form] " (current-error-port))
(display (jolt-pr-str form) (current-error-port)) (newline (current-error-port)))
(jolt-compile-eval-form (if data-readers-active (ldr-apply-readers form) form)
(chez-current-ns)))
(loop j))))))))
;; load-namespace: load `name`'s source once. Marked loaded BEFORE eval so a
;; dependency cycle terminates (Clojure's behavior). The caller's current ns is
;; restored afterward, since loading the file switched it.
(define (load-namespace name)
(unless (hashtable-ref loaded-ns name #f)
(let ((file (find-ns-file name)))
(cond
(file
(hashtable-set! loaded-ns name #t) ; mark before load so a cycle terminates
(let ((saved (chez-current-ns)))
(load-jolt-file file)
;; restore the current ns (thread-local); *ns* reads derive from it.
(set-chez-ns! saved))
(ns-loaded-hook name file))
;; No source file but the namespace exists in memory (AOT'd into a built
;; binary): it's already defined — mark loaded and move on.
((ns-has-vars? name)
(hashtable-set! loaded-ns name #t))
(else
(error #f (string-append "Could not locate " (ns-name->rel name)
".clj (or .cljc) on the source roots") name))))))
;; load-file: load an explicit path (a `run FILE`), in the current ns.
(define (jolt-load-file path)
(load-jolt-file path)
jolt-nil)
;; The target ns name of a require/use spec ([ns …] / (ns …) / bare ns).
(define (spec-target-name spec)
(let ((items (cond ((pvec? spec) (seq->list spec))
((or (cseq? spec) (empty-list-t? spec)) (seq->list spec))
((symbol-t? spec) (list spec))
(else '()))))
(and (pair? items) (symbol-t? (car items)) (symbol-t-name (car items)))))
;; A libspec under a prefix joins onto it: a bare symbol `string` -> `prefix.string`,
;; a vector `[string :as s]` -> `[prefix.string :as s]` (opts preserved).
(define (prefix-join prefix lib)
(cond
((symbol-t? lib) (jolt-symbol #f (string-append prefix "." (symbol-t-name lib))))
((pvec? lib)
(let ((items (seq->list lib)))
(if (and (pair? items) (symbol-t? (car items)))
(apply jolt-vector (jolt-symbol #f (string-append prefix "." (symbol-t-name (car items)))) (cdr items))
lib)))
(else lib)))
;; The prefix-list form of a require/use spec: a LIST `(prefix lib …)` expands to
;; one spec per lib (prefix.lib), so (:require (clojure [string :as str])) means
;; clojure.string :as str. A vector / symbol spec is already a single lib.
(define (expand-spec s)
(if (or (cseq? s) (empty-list-t? s))
(let ((items (seq->list s)))
(if (and (pair? items) (symbol-t? (car items)) (pair? (cdr items)))
(map (lambda (lib) (prefix-join (symbol-t-name (car items)) lib)) (cdr items))
(list s)))
(list s)))
;; --- require/use that LOAD ---------------------------------------------------
;; Override the alias-only versions from natives-str.ss. Load each spec's target
;; (no-op if baked/already loaded), THEN register its :as/:refer under the caller
;; ns (chez-register-spec! reads the current ns, restored by load-namespace).
(define (loader-require . specs)
(for-each
(lambda (s0)
(for-each
(lambda (s)
(let ((target (spec-target-name s)))
(when target (load-namespace target)))
(chez-register-spec! (chez-current-ns) s))
(expand-spec s0)))
specs)
jolt-nil)
(def-var! "clojure.core" "require" loader-require)
(define (loader-use . specs0)
(for-each
(lambda (spec0)
(for-each
(lambda (spec)
(let ((target (spec-target-name spec)))
(when target (load-namespace target)))
(chez-register-spec! (chez-current-ns) spec)
(let* ((items (cond ((pvec? spec) (seq->list spec))
((symbol-t? spec) (list spec))
(else '())))
(target (and (pair? items) (symbol-t? (car items)) (symbol-t-name (car items))))
(filtered (let scan ((xs (if (pair? items) (cdr items) '())))
(cond ((null? xs) #f)
((and (keyword? (car xs))
(member (keyword-t-name (car xs)) '("only" "refer"))) #t)
(else (scan (cdr xs)))))))
(when (and target (not filtered))
(chez-register-refer-all! (chez-current-ns) target))))
(expand-spec spec0)))
specs0)
jolt-nil)
(def-var! "clojure.core" "use" loader-use)
(def-var! "clojure.core" "load-file" jolt-load-file)
;; The directory of a namespace's resource path: "clojure.tools.reader-test" ->
;; "clojure/tools" (drop the last segment of ns-name->rel). "" for a top-level ns.
(define (ns-rel-dir name)
(let* ((r (ns-name->rel name)))
(let loop ((k (fx- (string-length r) 1)))
(cond ((fx<? k 0) "")
((char=? (string-ref r k) #\/) (substring r 0 k))
(else (loop (fx- k 1)))))))
;; load: an arg starting with "/" is a root-relative resource path ("/app/extra");
;; otherwise it is resolved against the CURRENT namespace's directory, matching
;; Clojure — (load "common_tests") from clojure.tools.reader-test loads
;; clojure/tools/common_tests.clj. Strip the leading slash / try .clj/.cljc.
(define (jolt-load . paths)
(for-each
(lambda (p)
(let* ((rel (cond
((and (> (string-length p) 0) (char=? (string-ref p 0) #\/))
(substring p 1 (string-length p)))
(else (let ((dir (ns-rel-dir (chez-current-ns))))
(if (string=? dir "") p (string-append dir "/" p))))))
(f (resolve-on-roots rel)))
(if f (load-jolt-file f)
(error #f "Could not locate resource on source roots" p))))
paths)
jolt-nil)
(def-var! "clojure.core" "load" jolt-load)
;; --- shell primitive (jolt.host/sh, sh-out) ---------------------------------
;; `sh` runs `sh -c CMD`, inheriting stdout/stderr (so git progress shows), and
;; returns the exit code. `sh-out` captures stdout to a string (exit ignored) for
;; commands whose output we parse (git rev-parse). Used by jolt.deps for git.
(define (jolt-sh cmd) (system cmd))
(def-var! "jolt.host" "sh" jolt-sh)
(define (jolt-sh-out cmd)
(call-with-values
(lambda () (open-process-ports (string-append "exec sh -c " (sh-quote cmd))
(buffer-mode block) (native-transcoder)))
(lambda (stdin stdout stderr pid)
(close-port stdin)
(let ((out (get-string-all stdout)))
(close-port stdout) (close-port stderr)
(if (eof-object? out) "" out)))))
(define (sh-quote s) ; single-quote for the outer sh -c
(string-append "'"
(apply string-append
(map (lambda (c) (if (char=? c #\') "'\\''" (string c))) (string->list s)))
"'"))
(def-var! "jolt.host" "sh-out" jolt-sh-out)
;; Expose source-root control + ns loading to Clojure (jolt.main / jolt.deps).
(def-var! "jolt.host" "set-source-roots!"
(lambda (roots) (set-source-roots! (seq->list roots)) (load-data-readers!) jolt-nil))
(def-var! "jolt.host" "source-roots" (lambda () (list->cseq source-roots)))
(def-var! "jolt.host" "load-namespace" (lambda (n) (load-namespace n) jolt-nil))
(def-var! "jolt.host" "file-exists?" (lambda (p) (if (file-exists? p) #t #f)))
(def-var! "jolt.host" "getenv" (lambda (n) (let ((v (getenv n))) (if v v jolt-nil))))
;; jolt version string. A self-contained binary build bakes the real tag into the
;; saved heap by emitting (set! jolt-baked-version "…") in flat.ss; a dev run off
;; the seed leaves it #f and falls back to $JOLT_VERSION (bin/joltc sets it from
;; `git describe`), then "dev".
(define jolt-baked-version #f)
(def-var! "jolt.host" "jolt-version"
(lambda ()
(or jolt-baked-version
(let ((v (getenv "JOLT_VERSION"))) (and v (> (string-length v) 0) v))
"dev")))

226
host/chez/multimethods.ss Normal file
View file

@ -0,0 +1,226 @@
;; multimethods — the multimethod dispatch runtime on the Chez host.
;;
;; defmulti/defmethod are macros that expand to ctx-capturing setup CALLS
;; (defmulti-setup / defmethod-setup, + the table ops get-method/methods/
;; remove-method/prefer-method/prefers), implemented here against
;; the runtime's ns/var machinery.
;;
;; A multimethod VALUE is a jolt-multifn record carrying its dispatch fn and a
;; mutable method table (dispatch-val -> method fn, keyed with jolt= so keyword/
;; vector/number dispatch values match by value). jolt-invoke dispatches it:
;; an exact method, else an isa?/hierarchy match (resolved through prefer-method
;; and the overlay's isa?/derive/hierarchy), else the :default method.
;;
;; NS resolution: defmulti expands to (defmulti-setup (quote name) ...) with a
;; BARE symbol — the Chez RT has no compile-time current ns at the call site, so a
;; runtime `chez-current-ns` box names where to def-var! the multifn. It defaults
;; to "user" (matching the analyzer's ns for -e user code); the assembled prelude
;; sets it to "clojure.core" around its own load (program-with-prelude), so the
;; print-method/print-dup defmultis land in clojure.core. defmethod-setup and the
;; symbol-taking table ops resolve the multifn via (var-deref (chez-current-ns) …),
;; so they agree with defmulti. Loaded from rt.ss after seq.ss (jolt-invoke),
;; collections.ss (jolt=/key-hash/jolt-hash-map) and the var-cell machinery.
;; THREAD-LOCAL: a Chez thread-parameter, so each OS thread (an nREPL
;; session worker / future) has its own current ns — vars stay global, only the
;; "current ns" pointer is per-thread, matching Clojure's thread-local *ns*. A new
;; thread inherits the forking thread's value. `star-ns-cell` (the *ns* var cell,
;; captured by dyn-binding.ss once *ns* exists) lets chez-current-ns DERIVE from a
;; thread-local (binding [*ns* ..]) so a bound *ns* drives load-string/analyzer
;; resolution; bootstrap-safe (it's #f until then, so we just read the parameter).
(define chez-current-ns-param (make-thread-parameter "user"))
(define star-ns-cell #f)
(define (chez-current-ns)
(if star-ns-cell
(let ((bv (dyn-binding-value star-ns-cell)))
(if (and (not (eq? bv dyn-no-binding)) (jns? bv))
(jns-name bv)
(chez-current-ns-param)))
(chez-current-ns-param)))
(define (set-chez-ns! ns) (chez-current-ns-param ns))
(define-record-type jolt-multifn
(fields name dispatch-fn methods default hierarchy prefers)
(nongenerative jolt-multifn-v1))
(define kw-default (keyword #f "default"))
(define (new-mm-table) (make-hashtable key-hash jolt=))
;; (defmulti-setup 'name dispatch & opts) — opts is a flat :default/:hierarchy plist.
(define (parse-mm-opts opts)
(let loop ((o opts) (dk kw-default) (h #f))
(if (or (null? o) (null? (cdr o)))
(values dk h)
(let ((k (car o)) (v (cadr o)))
(cond
((and (keyword? k) (not (keyword-t-ns k)) (string=? (keyword-t-name k) "default"))
(loop (cddr o) v h))
((and (keyword? k) (not (keyword-t-ns k)) (string=? (keyword-t-name k) "hierarchy"))
(loop (cddr o) dk v))
(else (loop (cddr o) dk h)))))))
(define (jolt-defmulti-setup name-sym dispatch . opts)
(let-values (((dk h) (parse-mm-opts opts)))
(let* ((sns (symbol-t-ns name-sym))
;; the macro qualifies the name with its EXPANSION ns, so a defmulti
;; deferred inside a fn (a deftest body) still defines in the ns it
;; was written in, not whatever ns is current when it finally runs.
(ns (if (string? sns) sns (chez-current-ns)))
(mf (make-jolt-multifn (symbol-t-name name-sym) dispatch
(new-mm-table) dk h (new-mm-table))))
(def-var! ns (symbol-t-name name-sym) mf)
mf)))
;; (defmethod-setup 'mm dispatch-val impl) — add a method. Auto-creates the multifn
;; if absent (defmethod before defmulti — rare; identity dispatch as a fallback).
(define (jolt-defmethod-setup mm-sym dval impl . rest)
(let* ((nm (symbol-t-name mm-sym))
(sns (symbol-t-ns mm-sym))
(qns (and sns (not (jolt-nil? sns)) (not (null? sns)) sns))
;; the macro passes its EXPANSION ns so a defmethod deferred inside a
;; fn resolves like the JVM (against the ns it was written in, not the
;; ns current when it runs); absent (old emitted code) fall back to the
;; runtime ns.
(here (if (and (pair? rest) (string? (car rest))) (car rest) (chez-current-ns)))
;; qualified (cf.mm/ext) resolves in its own ns (cross-ns defmethod);
;; unqualified resolves in the writing ns, else a :refer's home ns (so a
;; defmethod on a referred multifn lands on the real one), else stays in
;; the writing ns (a shadow, as before).
(mns (cond
(qns (or (chez-resolve-alias here qns) qns))
((var-cell-lookup here nm) here)
((chez-resolve-refer here nm) => values)
(else here)))
(cur (var-deref mns nm))
(mf (if (jolt-multifn? cur) cur
;; auto-create: copy the dispatch fn + default from a same-named
;; clojure.core multifn (e.g. print-method's 2-arg dispatch) so a
;; (defmethod print-method ...) before naming clojure.core's still
;; dispatches right — the old 1-arg identity fallback crashed.
(let* ((core (var-deref "clojure.core" nm))
(disp (if (jolt-multifn? core)
(jolt-multifn-dispatch-fn core)
(var-deref "clojure.core" "identity")))
(deft (if (jolt-multifn? core) (jolt-multifn-default core) kw-default))
(m (make-jolt-multifn nm disp (new-mm-table) deft #f (new-mm-table))))
(def-var! mns nm m) m))))
(hashtable-set! (jolt-multifn-methods mf) dval impl)
mf))
;; --- dispatch ----------------------------------------------------------------
(define (mm-isa? mf)
;; the overlay's isa? (the hierarchy system is pure Clojure); a per-mm :hierarchy
;; is an atom (deref each dispatch, like a Clojure var) or a plain map.
(let* ((isa (var-deref "clojure.core" "isa?"))
(h (jolt-multifn-hierarchy mf))
(hval (and h (if (jolt-atom? h) (jolt-atom-val h) h))))
(lambda (x y) (jolt-truthy? (if hval (jolt-invoke isa hval x y) (jolt-invoke isa x y))))))
(define (mm-find-isa mf dv)
(let* ((methods (jolt-multifn-methods mf))
(isa? (mm-isa? mf))
(default (jolt-multifn-default mf))
(keys (filter (lambda (k) (not (jolt= k default)))
(vector->list (hashtable-keys methods))))
(matches (filter (lambda (k) (isa? dv k)) keys)))
(cond
((null? matches) #f)
((null? (cdr matches)) (hashtable-ref methods (car matches) #f))
(else
;; >1 isa-match: pick the dominant key (x dominates y when x is
;; prefer-method'd over y, or (isa? x y)); ambiguity with no dominant is an
;; error, as in Clojure.
(let* ((prefers (jolt-multifn-prefers mf))
(pref? (lambda (x y)
(let ((px (hashtable-ref prefers x #f)))
(and px (hashtable-ref px y #f) #t))))
(dom? (lambda (x y) (or (pref? x y) (isa? x y))))
(best (fold-left (lambda (b k) (if (dom? k b) k b)) (car matches) (cdr matches))))
(for-each
(lambda (k)
(when (and (not (jolt= k best)) (not (dom? best k)))
(error #f (string-append "Multiple methods in multimethod '" (jolt-multifn-name mf)
"' match dispatch value - and neither is preferred"))))
matches)
(hashtable-ref methods best #f))))))
(define (multifn-dispatch mf . args)
(let* ((dv (apply jolt-invoke (jolt-multifn-dispatch-fn mf) args))
(methods (jolt-multifn-methods mf))
(direct (hashtable-ref methods dv #f)))
(cond
(direct (apply jolt-invoke direct args))
((mm-find-isa mf dv) => (lambda (m) (apply jolt-invoke m args)))
((hashtable-ref methods (jolt-multifn-default mf) #f)
=> (lambda (m) (apply jolt-invoke m args)))
(else (error #f (string-append "No method in multimethod '" (jolt-multifn-name mf)
"' for dispatch value: " (jolt-pr-str dv)))))))
;; jolt-invoke dispatches a multifn (otherwise falls through to the prior logic).
(define %prev-jolt-invoke jolt-invoke)
(set! jolt-invoke
(lambda (f . args)
(if (jolt-multifn? f)
(apply multifn-dispatch f args)
(apply %prev-jolt-invoke f args))))
;; --- table ops ---------------------------------------------------------------
;; prefer-method/remove-method/remove-all-methods/prefers take the name QUOTED;
;; get-method/methods take the multifn VALUE (Clojure semantics).
(define (mm-of-sym sym) (let ((v (var-deref (chez-current-ns) (symbol-t-name sym))))
(and (jolt-multifn? v) v)))
(define (jolt-prefer-method-setup mm-sym dval-a dval-b)
(let ((mf (mm-of-sym mm-sym)))
(when mf
(let ((sub (or (hashtable-ref (jolt-multifn-prefers mf) dval-a #f)
(let ((h (new-mm-table)))
(hashtable-set! (jolt-multifn-prefers mf) dval-a h) h))))
(hashtable-set! sub dval-b #t)))
mf))
(define (jolt-remove-method-setup mm-sym dval)
(let ((mf (mm-of-sym mm-sym)))
(when mf (hashtable-delete! (jolt-multifn-methods mf) dval))
mf))
(define (jolt-remove-all-methods-setup mm-sym)
(let ((mf (mm-of-sym mm-sym)))
(when mf (hashtable-clear! (jolt-multifn-methods mf)))
mf))
(define (jolt-get-method-setup mf dval)
(if (jolt-multifn? mf)
(or (hashtable-ref (jolt-multifn-methods mf) dval #f)
(hashtable-ref (jolt-multifn-methods mf) (jolt-multifn-default mf) #f)
jolt-nil)
jolt-nil))
(define (jolt-methods-setup mf)
(if (jolt-multifn? mf)
(let-values (((ks vs) (hashtable-entries (jolt-multifn-methods mf))))
(let loop ((i 0) (m (jolt-hash-map)))
(if (fx>=? i (vector-length ks)) m
(loop (fx+ i 1) (jolt-assoc m (vector-ref ks i) (vector-ref vs i))))))
jolt-nil))
(define (jolt-prefers-setup mm-sym)
(let ((mf (mm-of-sym mm-sym)))
(if (not mf) (jolt-hash-map)
(let-values (((ks vs) (hashtable-entries (jolt-multifn-prefers mf))))
(let loop ((i 0) (m (jolt-hash-map)))
(if (fx>=? i (vector-length ks)) m
;; each value is an inner set of preferred-over keys -> a jolt set
(loop (fx+ i 1)
(jolt-assoc m (vector-ref ks i)
(apply jolt-hash-set
(vector->list (hashtable-keys (vector-ref vs i))))))))))))
(def-var! "clojure.core" "defmulti-setup" jolt-defmulti-setup)
(def-var! "clojure.core" "defmethod-setup" jolt-defmethod-setup)
(def-var! "clojure.core" "prefer-method-setup" jolt-prefer-method-setup)
(def-var! "clojure.core" "remove-method-setup" jolt-remove-method-setup)
(def-var! "clojure.core" "remove-all-methods-setup" jolt-remove-all-methods-setup)
(def-var! "clojure.core" "get-method-setup" jolt-get-method-setup)
(def-var! "clojure.core" "methods-setup" jolt-methods-setup)
(def-var! "clojure.core" "prefers-setup" jolt-prefers-setup)

28
host/chez/natives-coll.ss Normal file
View file

@ -0,0 +1,28 @@
;; Collection constructors + rand — host-coupled natives the overlay assumes as
;; bare clojure.core vars. The persistent-collection constructors already exist
;; in collections.ss (jolt-hash-map / jolt-hash-set / jolt-vector); this just
;; binds the public clojure.core names to them. Loaded after def-var! (rt.ss) +
;; the collections + seq tiers. hash-map/array-map/hash-set/set/rand semantics.
;; array-map: insertion-ordered, any size (Clojure's PersistentArrayMap, via
;; createAsIfByAssoc). hash-map: hash order (PersistentHashMap). The map LITERAL
;; ctor (jolt-hash-map, emitted for {...}) is array-ordered up to 8 entries and
;; hash beyond, matching RT.map.
(define (jolt-array-map . kvs) (jolt-array-map-build kvs))
(define (jolt-hash-map-fn . kvs) (jolt-hash-map-build kvs))
;; set lives in the kernel overlay tier (clojure/core/00-kernel.clj): it's a pure
;; composition (apply hash-set (seq coll)) the compiler uses only off the emit path,
;; so the Clojure version lowers to the same code without a bootstrap cycle.
;; rand: a flonum in [0, n) (n defaults to 1.0) — jolt is all-flonum, so the
;; result is a double like every other number.
(define (jolt-rand . n)
(let ((r (random 1.0)))
(if (null? n) r (* r (exact->inexact (car n))))))
(def-var! "clojure.core" "hash-map" jolt-hash-map-fn)
(def-var! "clojure.core" "hash-set" jolt-hash-set)
(def-var! "clojure.core" "array-map" jolt-array-map)
(def-var! "clojure.core" "rand" jolt-rand)
(def-var! "clojure.core" "map-entry?" jolt-map-entry?)

View file

@ -0,0 +1,62 @@
;; natives-format.ss — a small %-format engine for clojure.core `format` over the
;; all-flonum number model: %d (integer), %s (str), %f / %.Nf (fixed-point), %x/%X
;; (hex int), %o (octal), %c (char int), %b (boolean), %% (literal). Enough for the
;; corpus, not the full Java Formatter spec. Loaded after natives-misc.ss (uses
;; jolt-str-render-one via converters + jolt-truthy?).
(define (->long x) (exact (truncate x)))
(define (pad-left s n c) (if (fx>=? (string-length s) n) s (string-append (make-string (fx- n (string-length s)) c) s)))
(define (fmt-float x prec)
(let* ((neg (< x 0)) (ax (abs x))
(scale (expt 10 prec))
(scaled (round (* (inexact ax) scale)))
(i (exact (truncate (/ scaled scale))))
(frac (exact (truncate (- scaled (* i scale))))))
(string-append (if neg "-" "")
(number->string i)
(if (fx>? prec 0) (string-append "." (pad-left (number->string frac) prec #\0)) ""))))
(define (jolt-format fmt . args)
(let ((out (open-output-string)))
(let loop ((i 0) (as args))
(if (fx>=? i (string-length fmt))
(get-output-string out)
(let ((c (string-ref fmt i)))
(if (char=? c #\%)
;; parse a directive: %[-][0][width][.prec]conv
(let scan ((j (fx+ i 1)) (left #f) (zero #f) (width #f) (prec #f) (seen-dot #f))
(let ((d (string-ref fmt j)))
(cond
((char=? d #\%) (write-char #\% out) (loop (fx+ j 1) as))
((and (not seen-dot) (not width) (char=? d #\-))
(scan (fx+ j 1) #t zero width prec seen-dot))
((and (not seen-dot) (not width) (char=? d #\0))
(scan (fx+ j 1) left #t width prec seen-dot))
((char=? d #\.) (scan (fx+ j 1) left zero width 0 #t))
((and (char>=? d #\0) (char<=? d #\9))
(if seen-dot
(scan (fx+ j 1) left zero width (fx+ (fx* (or prec 0) 10) (fx- (char->integer d) 48)) seen-dot)
(scan (fx+ j 1) left zero (fx+ (fx* (or width 0) 10) (fx- (char->integer d) 48)) prec seen-dot)))
(else
(let* ((a (if (null? as) jolt-nil (car as)))
(rest (if (null? as) '() (cdr as)))
(s (case d
((#\d) (number->string (->long a)))
((#\s) (jolt-str-render-one a))
((#\f) (fmt-float a (or prec 6)))
((#\x) (string-downcase (number->string (->long a) 16)))
((#\X) (string-upcase (number->string (->long a) 16)))
((#\o) (number->string (->long a) 8))
((#\b) (if (jolt-truthy? a) "true" "false"))
((#\c) (string (integer->char (->long a))))
(else (string #\% d))))
;; pad to width: left-justify with spaces, else right-justify
;; (zero-pad only a right-justified number).
(s (if (and width (fx<? (string-length s) width))
(let ((p (fx- width (string-length s))))
(if left (string-append s (make-string p #\space))
(string-append (make-string p (if (and zero (memv d '(#\d #\f #\x #\X #\o))) #\0 #\space)) s)))
s)))
(display s out)
(loop (fx+ j 1) rest))))))
(begin (write-char c out) (loop (fx+ i 1) as))))))))
(def-var! "clojure.core" "format" jolt-format)

157
host/chez/natives-meta.ss Normal file
View file

@ -0,0 +1,157 @@
;; metadata — meta / with-meta. Chez values don't
;; carry metadata, so collections use an identity-keyed side-table: with-meta
;; returns a fresh COPY of the value (new identity) and records its meta there, so
;; the original is unchanged (Clojure's immutable-with-meta) and a copy made by a
;; later op (conj/assoc) drops the meta. Symbols carry meta in their own field.
;; meta on a non-metadatable value (number/string/keyword) is nil.
;;
;; Loaded after records.ss (jrec) + collections/seq/values (the ctors it copies).
;; Weak so a collection's metadata is reclaimed with the collection — collection
;; ops (conj/assoc/into) carry meta forward onto fresh values, so a strong table
;; would retain every meta-bearing intermediate.
(define meta-table (make-weak-eq-hashtable))
(define (jolt-meta x)
(cond
((symbol-t? x) (let ((m (symbol-t-meta x))) (if (jolt-nil? m) jolt-nil m)))
;; a var's meta is {:ns :name} (derived from the cell) + any def-time user
;; meta from rt.ss's var-meta-table.
((var-cell? x)
(let ((user (hashtable-ref var-meta-table x #f)))
(jolt-assoc (if user user (jolt-hash-map))
jolt-kw-var-ns (var-cell-ns x)
jolt-kw-var-name (var-cell-name x))))
;; a deftype implementing clojure.lang.IObj stores meta in a field and threads
;; it through its own assoc/withMeta (core.logic's Substitutions/LVar/LCons),
;; so dispatch to its meta method rather than the identity side-table — which
;; the deftype's reconstructed instances would not share.
((and (jrec? x) (jrec-cl x "meta")) => (lambda (m) (jolt-invoke m x)))
;; everything else (collections, fns, reify, atoms/agents and any reference
;; type) reads the identity side-table; a value with no entry is nil meta.
(else (hashtable-ref meta-table x jolt-nil))))
;; fresh-identity copy of a metadatable value (so attaching meta doesn't mutate
;; the original). cseq/procedure can't be copied meaningfully — keyed in place.
(define (meta-copy x)
(cond
((pvec? x) (make-pvec (pvec-v x) (pvec-ent x)))
((pmap? x) (make-pmap (pmap-root x) (pmap-cnt x) (pmap-order x)))
((pset? x) (make-pset (pset-m x)))
((jrec? x) (make-jrec (jrec-desc x) (jrec-vec-copy (jrec-vals x)) (jrec-ext x)))
;; a reify shares its (read-only) method table + protos but gets a fresh
;; identity, so attaching meta leaves the original's meta untouched. Every
;; Clojure reify implements IObj.
((jreify? x) (make-jreify (jreify-methods x) (jreify-protos x)))
;; () is a shared singleton — a fresh instance keeps meta off every other ().
((empty-list-t? x) (fresh-empty-list))
;; a list/seq node gets a fresh identity too (Clojure's PersistentList is
;; immutable — (with-meta a-list m) returns a NEW list). Keying meta on the
;; original mutated it, so (with-meta xs {:k xs}) built a self-referential
;; cycle that loops *print-meta* printing.
((cseq? x) (make-cseq (cseq-head x) (cseq-tail x) (cseq-forced? x)
(cseq-list? x) (cseq-cvec x) (cseq-ci x) (cseq-crest x)))
((jolt-lazyseq? x) (make-jolt-lazyseq (jolt-lazyseq-thunk x) (jolt-lazyseq-val x)
(jolt-lazyseq-realized? x)))
(else x))) ; procedure
(define (jolt-with-meta x m)
(cond
((symbol-t? x) (make-symbol-t (symbol-t-ns x) (symbol-t-name x) m))
;; a deftype with an explicit clojure.lang.IObj withMeta carries meta in a
;; field; dispatch to it (see jolt-meta) so the meta survives reconstruction.
((and (jrec? x) (jrec-cl x "withMeta")) => (lambda (meth) (jolt-invoke meth x m)))
((or (pvec? x) (pmap? x) (pset? x) (cseq? x) (empty-list-t? x) (jolt-lazyseq? x) (jrec? x) (jreify? x) (procedure? x))
(let ((c (meta-copy x)))
(if (jolt-nil? m) (hashtable-delete! meta-table c) (hashtable-set! meta-table c m))
c))
(else (error #f "with-meta: value does not support metadata" x))))
(def-var! "clojure.core" "meta" jolt-meta)
(def-var! "clojure.core" "with-meta" jolt-with-meta)
;; Carry SRC's collection metadata onto DST (a freshly-built collection of the
;; same kind), as Clojure's ops do — each new collection threads its receiver's
;; meta() forward. Returns DST. The size check is the fast path: programs that
;; never attach collection metadata pay one O(1) check per op, no lookup.
(define (meta-carry src dst)
(if (fx=? 0 (hashtable-size meta-table))
dst
(let ((m (hashtable-ref meta-table src #f)))
(if m
;; never attach to the shared () singleton — use a fresh instance
(let ((d (if (empty-list-t? dst) (fresh-empty-list) dst)))
(hashtable-set! meta-table d m) d)
dst))))
;; (type x) — Clojure's (or (:type (meta x)) (class x)). With no JVM classes the
;; "class" is a host taxonomy: a record yields its ns-qualified class-name SYMBOL
;; (user.TyR), everything else a keyword (:number/:vector/:seq/…).
;; MUST be total — a non-record value
;; falling through to a crash would read as a divergence, not the right keyword.
;; Forward refs (jolt-lazyseq?, the sorted-htable / wrapper predicates) all bind by
;; call time (every host .ss loads before any user expr runs).
(define ty-kw-type (keyword #f "type")) ; the :type meta key
(define ty-kw-jtype (keyword "jolt" "type")) ; tagged-map discriminator (ex-info)
(define ty-number (keyword #f "number"))
(define ty-string (keyword #f "string"))
(define ty-keyword (keyword #f "keyword"))
(define ty-symbol (keyword #f "symbol"))
(define ty-boolean (keyword #f "boolean"))
(define ty-char (keyword #f "char"))
(define ty-vector (keyword #f "vector"))
(define ty-map (keyword #f "map"))
(define ty-set (keyword #f "set"))
(define ty-seq (keyword #f "seq"))
(define ty-fn (keyword #f "fn"))
(define ty-atom (keyword "jolt" "atom"))
(define ty-volatile (keyword "jolt" "volatile"))
(define ty-regex (keyword "jolt" "regex"))
(define ty-var (keyword "jolt" "var"))
(define ty-transient (keyword "jolt" "transient"))
(define ty-uuid (keyword "jolt" "uuid"))
(define ty-sorted-set (keyword "jolt" "sorted-set"))
(define ty-object (keyword #f "object"))
(define (jolt-type x)
(let* ((m (jolt-meta x))
(override (if (jolt-nil? m) jolt-nil (jolt-get m ty-kw-type jolt-nil))))
(cond
((not (jolt-nil? override)) override) ; :type meta wins
;; record -> its ns-qualified class-name STRING (= (class x)). jolt models
;; classes as strings, so (symbol (str (type r))) is NOT (type r) — as on the
;; JVM where type is a Class, not a Symbol.
((jrec? x) (jrec-tag x))
((jolt-nil? x) jolt-nil)
((boolean? x) ty-boolean)
((number? x) ty-number)
((string? x) ty-string)
((keyword? x) ty-keyword)
((symbol-t? x) ty-symbol)
((char? x) ty-char)
;; host wrappers — keyed by their :jolt/* tags (checked before the
;; collection arms; none of these are pvec/pmap/pset).
((jolt-atom? x) ty-atom)
((jvol? x) ty-volatile)
((jolt-regex? x) ty-regex)
((var-cell? x) ty-var)
((jolt-transient? x) ty-transient)
((juuid? x) ty-uuid)
((htable-sorted-set? x) ty-sorted-set)
((htable-sorted-map? x) ty-map)
;; collections — pvec INCLUDES map entries (:vector).
((pvec? x) ty-vector)
((pmap? x) ; a :jolt/type-tagged map (ex-info) -> its tag
(let ((t (jolt-get x ty-kw-jtype jolt-nil))) (if (jolt-nil? t) ty-map t)))
((pset? x) ty-set)
((or (cseq? x) (empty-list-t? x) (jolt-lazyseq? x)) ty-seq)
((procedure? x) ty-fn)
(else ty-object))))
;; jolt-type is the keyword TAXONOMY (:string/:set/:jolt/inst/…) — jolt's native
;; value model, with no JVM in it. print-method/print-dup dispatch on it (via
;; __type-tag). The PUBLIC clojure.core/type is Clojure's (or (:type meta) (class
;; x)) — a JVM class — but that mapping belongs to the java host layer (host-class.ss
;; rebinds `type` next to `class`), so this core layer stays JVM-free.
(def-var! "clojure.core" "__type-tag" jolt-type)
(def-var! "clojure.core" "type" jolt-type)

111
host/chez/natives-misc.ss Normal file
View file

@ -0,0 +1,111 @@
;; misc scalar natives — UUID, tagged-literal, bigint, and the hash API. (format /
;; printf moved to natives-format.ss.)
;;
;; Loaded after the printers (pr-str of a uuid is #uuid "…") and converters
;; (jolt-str-render-one for %s / str of a uuid).
;; --- UUID --------------------------------------------------------------------
;; A uuid is a record wrapping its canonical 36-char lowercase string. str -> the
;; bare string; pr-str -> #uuid "…"; not map?/coll?.
(define-record-type juuid (fields s) (nongenerative chez-juuid-v1))
(define (jolt-uuid-pred? x) (juuid? x))
(define hexd "0123456789abcdef")
(define (rand-hex) (string-ref hexd (random 16)))
;; v4: 8-4-4-4-12, version nibble (index 14) = 4, variant nibble (index 19) in 8-b.
(define (random-uuid-str)
(let ((cs (make-string 36)))
(let loop ((i 0))
(if (fx=? i 36) cs
(begin
(string-set! cs i
(cond ((or (fx=? i 8) (fx=? i 13) (fx=? i 18) (fx=? i 23)) #\-)
((fx=? i 14) #\4)
((fx=? i 19) (string-ref "89ab" (random 4)))
(else (rand-hex))))
(loop (fx+ i 1)))))))
(define (jolt-random-uuid) (make-juuid (random-uuid-str)))
;; #uuid literal -> a uuid value (the emitter lowers the :uuid node to this). The
;; reader already validated the shape; lowercase for value equality.
(define (jolt-uuid-from-string s) (make-juuid (string-downcase s)))
;; parse-uuid: validate the canonical shape (8-4-4-4-12 hex), lowercase, -> uuid;
;; nil if the string doesn't conform (Clojure parse-uuid), error on a non-string.
(define (hex-char? c) (or (and (char>=? c #\0) (char<=? c #\9))
(and (char>=? c #\a) (char<=? c #\f))
(and (char>=? c #\A) (char<=? c #\F))))
(define (uuid-shape? s)
(and (string? s) (fx=? (string-length s) 36)
(let loop ((i 0))
(if (fx=? i 36) #t
(let ((c (string-ref s i)))
(cond ((or (fx=? i 8) (fx=? i 13) (fx=? i 18) (fx=? i 23)) (and (char=? c #\-) (loop (fx+ i 1))))
((hex-char? c) (loop (fx+ i 1)))
(else #f)))))))
(define (jolt-parse-uuid s)
(cond ((not (string? s)) (error #f "parse-uuid: not a string" s))
((uuid-shape? s) (make-juuid (string-downcase s)))
(else jolt-nil)))
;; uuid? / random-uuid / parse-uuid are OVERLAY fns (they read :jolt/type), so
;; the prelude would clobber a def-var! here — they're asserted in post-prelude.ss.
;; str of a uuid -> the bare 36-char string; pr-str -> #uuid "…".
(register-str-render! juuid? juuid-s)
(define (juuid-pr u) (string-append "#uuid \"" (juuid-s u) "\""))
(register-pr-arm! juuid? juuid-pr)
;; two uuids are = iff same string.
(register-eq-arm! (lambda (a b) (or (juuid? a) (juuid? b)))
(lambda (a b) (and (juuid? a) (juuid? b) (string=? (juuid-s a) (juuid-s b)))))
;; --- bigint / biginteger -----------------------------------------------------
;; jolt models every number as a double; an integer-valued double prints without
;; a ".0" (jolt-num->string), so bigint is just the number for the corpus range.
;; (Arbitrary-precision beyond 2^53 is a separate concern.)
(define (jolt-bigint x) x)
(def-var! "clojure.core" "bigint" jolt-bigint)
(def-var! "clojure.core" "biginteger" jolt-bigint)
;; --- tagged-literal ----------------------------------------------------------
;; (tagged-literal tag form): a tagged value with :tag / :form. tagged-literal? is
;; overlay (reads :jolt/type) so it's overridden in post-prelude.ss.
(define-record-type jtagged (fields tag form) (nongenerative chez-jtagged-v1))
(define (jolt-tagged-literal tag form) (make-jtagged tag form))
(define (jolt-tagged-literal-pred? x) (jtagged? x))
(define kw-tl-tag (keyword #f "tag"))
(define kw-tl-form (keyword #f "form"))
(register-get-arm! jtagged?
(lambda (coll k d)
(cond ((jolt=2 k kw-tl-tag) (jtagged-tag coll))
((jolt=2 k kw-tl-form) (jtagged-form coll))
(else d))))
(define (jtagged-pr t) (string-append "#" (jolt-pr-str (jtagged-tag t)) " " (jolt-pr-readable (jtagged-form t))))
(register-pr-arm! jtagged? jtagged-pr)
;; two tagged literals are = iff same tag and (recursively) = form, like the JVM's
;; TaggedLiteral — so they work as map keys / set members. (jolt-hash already
;; hashes the fields structurally, so eq/hash stay consistent.)
(register-eq-arm! (lambda (a b) (or (jtagged? a) (jtagged? b)))
(lambda (a b) (and (jtagged? a) (jtagged? b)
(jolt=2 (jtagged-tag a) (jtagged-tag b))
(jolt=2 (jtagged-form a) (jtagged-form b)))))
(def-var! "clojure.core" "tagged-literal" jolt-tagged-literal)
;; tagged-literal? is OVERLAY (reads :jolt/type) — asserted in post-prelude.ss.
;; --- hash family (24-bit masked so int? holds) -------------------------------
;; The public hash API over jolt-hash (values.ss). hash-ordered/-unordered-coll
;; fold the element hashes the way Clojure's IHash mixers do.
(define (nm-h24 x) (bitwise-and (jolt-hash x) #xffffff))
(define (nm-hash x) (nm-h24 x))
(define (nm-hash-combine a b)
(bitwise-and (bitwise-xor (nm-h24 a) (+ (nm-h24 b) #x9e3779)) #xffffff))
(define (nm-hash-ordered-coll coll)
(let loop ((xs (seq->list (jolt-seq coll))) (h 1))
(if (null? xs) h (loop (cdr xs) (bitwise-and (+ (* 31 h) (nm-h24 (car xs))) #xffffff)))))
(define (nm-hash-unordered-coll coll)
(let loop ((xs (seq->list (jolt-seq coll))) (h 0))
(if (null? xs) h (loop (cdr xs) (bitwise-and (+ h (nm-h24 (car xs))) #xffffff)))))
(def-var! "clojure.core" "hash" nm-hash)
(def-var! "clojure.core" "hash-combine" nm-hash-combine)
(def-var! "clojure.core" "hash-ordered-coll" nm-hash-ordered-coll)
(def-var! "clojure.core" "hash-unordered-coll" nm-hash-unordered-coll)

88
host/chez/natives-num.ss Normal file
View file

@ -0,0 +1,88 @@
;; bit ops + string->number parsers — host-coupled natives (bit family,
;; parse-long/double). jolt models every number as a double, so bit ops coerce
;; to an exact integer, operate, and return a flonum. parse-* use strict shapes
;; (Clojure 1.11: nil on malformed, throw on a non-string).
;; bit ops return EXACT integers (= JVM long). ->int coerces the operand.
(define (->int x) (exact (truncate x)))
(define (jolt-bit-and a b) (bitwise-and (->int a) (->int b)))
(define (jolt-bit-or a b) (bitwise-ior (->int a) (->int b)))
(define (jolt-bit-xor a b) (bitwise-xor (->int a) (->int b)))
(define (jolt-bit-and-not a b) (bitwise-and (->int a) (bitwise-not (->int b))))
(define (jolt-bit-not a) (bitwise-not (->int a)))
(define (jolt-bit-shift-left x n) (bitwise-arithmetic-shift-left (->int x) (->int n)))
(define (jolt-bit-shift-right x n) (bitwise-arithmetic-shift-right (->int x) (->int n)))
(define (bit-mask n) (bitwise-arithmetic-shift-left 1 (->int n)))
(define (jolt-bit-set x n) (bitwise-ior (->int x) (bit-mask n)))
(define (jolt-bit-clear x n) (bitwise-and (->int x) (bitwise-not (bit-mask n))))
(define (jolt-bit-flip x n) (bitwise-xor (->int x) (bit-mask n)))
(define (jolt-bit-test x n) (not (zero? (bitwise-and (->int x) (bit-mask n)))))
;; unsigned-bit-shift-right: LOGICAL right shift over a 64-bit long (Java >>>),
;; so a negative operand shifts in zeros from its 64-bit two's-complement window
;; ((>>> -1 1) = 2^63-1), not the sign. The shift count is taken mod 64.
(define (jolt-unsigned-bit-shift-right x n)
(bitwise-arithmetic-shift-right (bitwise-and (->int x) #xFFFFFFFFFFFFFFFF)
(bitwise-and (->int n) 63)))
;; ---- string->scalar parsers -------------------------------------------------
(define (ascii-digit? c) (and (char>=? c #\0) (char<=? c #\9)))
(define (skip-digits s i n) (let loop ((i i)) (if (and (< i n) (ascii-digit? (string-ref s i))) (loop (+ i 1)) i)))
(define (sign-at? s i n) (and (< i n) (let ((c (string-ref s i))) (or (char=? c #\+) (char=? c #\-)))))
(define (parse-long-shape? s)
(let* ((n (string-length s)) (i0 (if (sign-at? s 0 n) 1 0)))
(and (> n i0) (= (skip-digits s i0 n) n))))
(define (jolt-parse-long s)
(if (not (string? s)) (error #f "parse-long requires a string" s)
(if (parse-long-shape? s) (string->number s) jolt-nil))) ; exact long
;; strict float shape: [+-]? ( D+ (. D*)? | . D+ ) ([eE][+-]? D+)? fully anchored.
(define (parse-double-shape? s)
(let ((n (string-length s)))
(and (> n 0)
(call/cc
(lambda (no)
(let* ((i0 (if (sign-at? s 0 n) 1 0))
(after-int (skip-digits s i0 n))
(had-int (> after-int i0))
;; mantissa end
(jm (cond
((and had-int (< after-int n) (char=? (string-ref s after-int) #\.))
(skip-digits s (+ after-int 1) n)) ; D+ . D*
((and (not had-int) (< i0 n) (char=? (string-ref s i0) #\.))
(let ((k (skip-digits s (+ i0 1) n))) ; . D+
(if (> k (+ i0 1)) k (no #f))))
(had-int after-int)
(else (no #f))))
;; optional exponent
(je (if (and (< jm n) (let ((c (string-ref s jm))) (or (char=? c #\e) (char=? c #\E))))
(let* ((es (if (sign-at? s (+ jm 1) n) (+ jm 2) (+ jm 1)))
(ee (skip-digits s es n)))
(if (> ee es) ee (no #f)))
jm)))
(= je n)))))))
(define (jolt-parse-double s)
(if (not (string? s)) (error #f "parse-double requires a string" s)
(cond
((string=? s "Infinity") +inf.0)
((string=? s "-Infinity") -inf.0)
((string=? s "NaN") +nan.0)
((parse-double-shape? s) (exact->inexact (string->number s)))
(else jolt-nil))))
(def-var! "clojure.core" "__bit-and" jolt-bit-and)
(def-var! "clojure.core" "__bit-or" jolt-bit-or)
(def-var! "clojure.core" "__bit-xor" jolt-bit-xor)
(def-var! "clojure.core" "__bit-and-not" jolt-bit-and-not)
(def-var! "clojure.core" "bit-not" jolt-bit-not)
(def-var! "clojure.core" "bit-shift-left" jolt-bit-shift-left)
(def-var! "clojure.core" "bit-shift-right" jolt-bit-shift-right)
(def-var! "clojure.core" "bit-set" jolt-bit-set)
(def-var! "clojure.core" "bit-clear" jolt-bit-clear)
(def-var! "clojure.core" "bit-flip" jolt-bit-flip)
(def-var! "clojure.core" "bit-test" jolt-bit-test)
(def-var! "clojure.core" "unsigned-bit-shift-right" jolt-unsigned-bit-shift-right)
(def-var! "clojure.core" "parse-long" jolt-parse-long)
(def-var! "clojure.core" "parse-double" jolt-parse-double)

View file

@ -0,0 +1,58 @@
;; natives-reader.ss — reader/macro runtime-support natives: the #?() reader feature
;; set, the reader-conditional + re-matcher tagged-map constructors, and macroexpand.
;;
;; Loaded late (after ns.ss): macroexpand forward-refs the runtime macro table
;; (host-contract hc-macro?/hc-expand-1) + the analyzer ctx, resolved at call time
;; after the spine loads. The hash / transient? / rseq / cat natives that used to
;; live here moved to natives-misc, transients, natives-seq, and natives-transduce.
;; --- reader feature set (for #?() conditionals) — mutable list of name strings,
;; default jolt + default. __reader-features returns the strings; -set! replaces.
(define nr-reader-features (list "jolt" "default"))
(define (nr-reader-features-get) (list->cseq nr-reader-features))
(define (nr-reader-features-set! names)
(set! nr-reader-features
(map (lambda (n) (cond ((keyword-t? n) (keyword-t-name n)) ((string? n) n) (else (jolt-pr-str n))))
(seq->list (jolt-seq names))))
jolt-nil)
;; --- reader-conditional: a tagged map (reader-conditional? is an overlay
;; tagged-value predicate that reads :jolt/type). STAYS NATIVE: building a
;; :jolt/type-tagged map is part of the native value model — an overlay defn
;; returning {:jolt/type ...} silently fails to bind during the seed mint (the
;; guard around each prelude form swallows the load-time error), the same reason
;; every other tagged-value constructor (atom/volatile!/tagged-literal) is native.
;; re-matcher / re-find / re-groups are the stateful matcher API in regex.ss.
(define nr-kw-type (keyword "jolt" "type"))
(define nr-kw-rc (keyword "jolt" "reader-conditional"))
(define nr-kw-form (keyword #f "form"))
(define nr-kw-spl (keyword #f "splicing?"))
(define (nr-reader-conditional form splicing?)
(jolt-hash-map nr-kw-type nr-kw-rc nr-kw-form form nr-kw-spl splicing?))
;; --- macroexpand-1 / macroexpand: expand a (quoted) call form via the runtime
;; macro table (host-contract hc-macro?/hc-expand-1; forward-referenced, resolved
;; at call time after the spine loads). macroexpand loops until the head is no
;; longer a macro (subforms are not expanded, matching Clojure).
(define (nr-macroexpand-1 form)
(if (and (cseq? form) (cseq-list? form) (symbol-t? (seq-first form)))
(let ((ctx (make-analyze-ctx (chez-current-ns))))
(if (hc-macro? ctx (seq-first form)) (hc-expand-1 ctx form) form))
form))
(define (nr-macroexpand form)
(let loop ((cur form))
(let ((nxt (nr-macroexpand-1 cur))) (if (eq? cur nxt) cur (loop nxt)))))
(def-var! "clojure.core" "__reader-features" nr-reader-features-get)
(def-var! "clojure.core" "__reader-features-set!" nr-reader-features-set!)
(def-var! "clojure.core" "reader-conditional" nr-reader-conditional)
(def-var! "clojure.core" "macroexpand-1" nr-macroexpand-1)
;; letfn is a special form (the analyzer lowers it to letrec*, checked before any
;; macro), but on the JVM it is also a clojure.core macro that (resolve 'letfn)
;; finds — like let / loop / fn here. Intern a var so resolution matches; the value
;; is never invoked (the analyzer handles every (letfn …) form), and it is NOT
;; marked a macro, so macroexpand leaves a letfn form alone (it is special).
(def-var! "clojure.core" "letfn"
(lambda args (jolt-throw (jolt-ex-info "letfn is a special form" (jolt-hash-map)))))
(def-var! "clojure.core" "macroexpand" nr-macroexpand)

247
host/chez/natives-seq.ss Normal file
View file

@ -0,0 +1,247 @@
;; seq-native shims — native seq fns the overlay assumes are clojure.core
;; natives. Each is a pure fn over the existing seq layer (seq.ss) — collection
;; arities only; the 1-arg transducer arities follow below. Loaded last (after
;; converters.ss for jolt-compare and seq.ss for the reduced record).
;; reduced / reduced? — the box itself is the jolt-reduced record from seq.ss
;; (so the reduce machinery there can see it); these just expose the constructor
;; and predicate. (deref a-reduced) is handled in atoms.ss.
(define (jolt-reduced-new x) (make-jolt-reduced x))
(define (jolt-reduced-pred x) (jolt-reduced? x))
(define (ensure-reduced x) (if (jolt-reduced? x) x (make-jolt-reduced x)))
;; ============================================================================
;; transducers — the 1-arg arity of map/filter/take/... returns a
;; transducer (fn [rf] rf') where rf' is a reducing fn with arities
;; []=init, [acc]=complete, [acc x]=step. rf and the mapping/predicate fns are jolt values, so every
;; call routes through jolt-invoke. A `reduced` step stops the fold — reduce-seq
;; (seq.ss) already short-circuits on a jolt-reduced.
;; ============================================================================
;; The map transducer's step fn supports multiple inputs ([result input & inputs]),
;; so a multi-collection sequence/transduce — or medley's sequence-padded, which
;; calls (f acc i1 i2 …) — applies f across all of them: (rf result (apply f inputs)).
(define (td-map f)
(lambda (rf)
(lambda a
(case (length a)
((0) (jolt-invoke rf))
((1) (jolt-invoke rf (car a)))
(else (jolt-invoke rf (car a) (apply jolt-invoke f (cdr a))))))))
(define (td-filter pred)
(lambda (rf)
(lambda a
(case (length a)
((0) (jolt-invoke rf))
((1) (jolt-invoke rf (car a)))
(else (if (jolt-truthy? (jolt-invoke pred (cadr a)))
(jolt-invoke rf (car a) (cadr a))
(car a)))))))
(define (td-remove pred) (td-filter (lambda (x) (jolt-not (jolt-invoke pred x)))))
(define (td-take n)
(lambda (rf)
(let ((left n))
(lambda a
(case (length a)
((0) (jolt-invoke rf))
((1) (jolt-invoke rf (car a)))
(else (if (<= left 0)
(make-jolt-reduced (car a))
(let ((r (jolt-invoke rf (car a) (cadr a))))
(set! left (- left 1))
(if (<= left 0) (ensure-reduced r) r)))))))))
(define (td-drop n)
(lambda (rf)
(let ((left n))
(lambda a
(case (length a)
((0) (jolt-invoke rf))
((1) (jolt-invoke rf (car a)))
(else (if (> left 0) (begin (set! left (- left 1)) (car a))
(jolt-invoke rf (car a) (cadr a)))))))))
(define (td-take-while pred)
(lambda (rf)
(lambda a
(case (length a)
((0) (jolt-invoke rf))
((1) (jolt-invoke rf (car a)))
(else (if (jolt-truthy? (jolt-invoke pred (cadr a)))
(jolt-invoke rf (car a) (cadr a))
(make-jolt-reduced (car a))))))))
(define (td-drop-while pred)
(lambda (rf)
(let ((dropping #t))
(lambda a
(case (length a)
((0) (jolt-invoke rf))
((1) (jolt-invoke rf (car a)))
(else (begin
(when (and dropping (not (jolt-truthy? (jolt-invoke pred (cadr a)))))
(set! dropping #f))
(if dropping (car a) (jolt-invoke rf (car a) (cadr a))))))))))
;; (mapcat f) transducer: map f, then splice (cat) f's result into rf, honoring a
;; mid-splice `reduced`.
(define (td-mapcat f)
(lambda (rf)
(lambda a
(case (length a)
((0) (jolt-invoke rf))
((1) (jolt-invoke rf (car a)))
(else (let loop ((acc (car a))
(xs (seq->list (jolt-seq (jolt-invoke f (cadr a))))))
(if (or (null? xs) (jolt-reduced? acc)) acc
(loop (jolt-invoke rf acc (car xs)) (cdr xs)))))))))
;; (into to xform from): transduce `from` through `xform` with conj as the rf.
(define (into-xform to xform from)
(let* ((conj-rf (lambda a (if (fx=? (length a) 1) (car a) ; completion = identity
(jolt-conj1 (car a) (cadr a)))))
(xrf (jolt-invoke xform conj-rf))
(res (reduce-seq xrf to (jolt-seq from))))
(jolt-invoke xrf res)))
;; mapcat: (mapcat f) -> transducer; (mapcat f coll & colls) -> map f across the
;; colls (stops at shortest), then concat the results.
(define (jolt-mapcat f . colls)
(if (null? colls)
(td-mapcat f)
;; lazily concat the per-element results — no seq->list, so mapcat over an
;; infinite source stays lazy; the outer lazy-seq node defers the first
;; element so a side-effecting f does not fire at construction (LazySeq).
(jolt-make-lazy-seq (lambda () (jolt-seq (lazy-concat-seq (apply jolt-map f colls)))))))
;; take-while / drop-while: 1-arg -> transducer; 2-arg -> a seq over the coll.
(define (take-while-seq pred s)
(if (jolt-nil? s) jolt-empty-list
(let ((x (seq-first s)))
(if (jolt-truthy? (jolt-invoke pred x))
(cseq-lazy x (lambda () (take-while-seq pred (jolt-seq (seq-more s)))))
jolt-empty-list))))
(define jolt-take-while
(case-lambda
((pred) (td-take-while pred))
((pred coll) (jolt-make-lazy-seq (lambda () (jolt-seq (take-while-seq pred (jolt-seq coll))))))))
(define (drop-while-seq pred coll)
(let loop ((s (jolt-seq coll)))
(if (and (not (jolt-nil? s)) (jolt-truthy? (jolt-invoke pred (seq-first s))))
(loop (jolt-seq (seq-more s)))
(if (jolt-nil? s) jolt-empty-list s))))
(define jolt-drop-while
(case-lambda
((pred) (td-drop-while pred))
((pred coll) (jolt-make-lazy-seq (lambda () (jolt-seq (drop-while-seq pred coll)))))))
;; partition: (partition n coll), (partition n step coll), or
;; (partition n step pad coll). Only complete partitions of size n are kept;
;; with pad, a short final partition is padded from pad (and may be < n if pad
;; runs out). Each partition is a seq; the whole result is a lazy seq of seqs.
(define jolt-partition
(case-lambda
((n coll) (jolt-make-lazy-seq (lambda () (jolt-seq (partition* (->idx n) (->idx n) #f #f coll)))))
((n step coll) (jolt-make-lazy-seq (lambda () (jolt-seq (partition* (->idx n) (->idx step) #f #f coll)))))
((n step pad coll) (jolt-make-lazy-seq (lambda () (jolt-seq (partition* (->idx n) (->idx step) #t pad coll)))))))
(define (take-n n s) ; -> (values list-of-first-n remaining-seq taken-count)
(let loop ((n n) (s s) (acc '()))
(if (or (fx<=? n 0) (jolt-nil? s))
(values (reverse acc) s (length acc))
(loop (fx- n 1) (jolt-seq (seq-more s)) (cons (seq-first s) acc)))))
(define (partition* n step has-pad pad coll)
(let loop ((s (jolt-seq coll)))
(if (jolt-nil? s) jolt-empty-list
(let-values (((part rest taken) (take-n n s)))
(cond
;; full partition: emit it, advance `step` from its START
((fx=? taken n)
(cseq-lazy (list->cseq part)
(lambda () (loop (jolt-seq (advance-by step s))))))
;; short final partition with pad: top up to n from pad, then stop
((and has-pad (fx>? taken 0))
(let ((padded (append part (take-list (- n taken) (jolt-seq pad)))))
(cseq-lazy (list->cseq padded) (lambda () jolt-empty-list))))
;; short final partition, no pad: dropped (Clojure keeps only full ones)
(else jolt-empty-list))))))
(define (advance-by step s) ; drop `step` elements from s (seq), returns a seq
(let loop ((step step) (s s))
(if (or (fx<=? step 0) (jolt-nil? s)) s
(loop (fx- step 1) (jolt-seq (seq-more s))))))
(define (take-list n s) ; up to n elements of seq s as a Scheme list
(let loop ((n n) (s s) (acc '()))
(if (or (fx<=? n 0) (jolt-nil? s)) (reverse acc)
(loop (fx- n 1) (jolt-seq (seq-more s)) (cons (seq-first s) acc)))))
;; sort: (sort coll) uses compare; (sort cmp coll) uses cmp, whose result may be
;; a 3-way number (<0 / 0 / >0) OR a boolean (a Clojure-style less-than pred).
(define (cmp->less cmp)
(lambda (a b)
(let ((r (jolt-invoke cmp a b)))
(if (number? r) (< r 0) (jolt-truthy? r)))))
(define jolt-sort
(case-lambda
((coll) (jolt-sort* (cmp->less jolt-compare) coll))
((cmp coll) (jolt-sort* (cmp->less cmp) coll))))
(define (jolt-sort* less? coll)
(let ((s (jolt-seq coll)))
(if (jolt-nil? s) jolt-empty-list
(list->cseq (list-sort less? (seq->list s))))))
;; identical?: reference identity (Clojure ==). eq? gives pointer identity over
;; the value model — interned keywords/fixnums/nil compare equal, distinct
;; collections do not. Must NOT be value equality: a deftype whose .equals calls
;; (identical? this o) to short-circuit (e.g. core.logic's Substitutions) would
;; otherwise recur forever (identical? -> = -> equiv -> .equals -> identical?).
(define (jolt-identical? a b) (eq? a b))
;; Give the seq.ss native procedures their transducer (1-arg) arity — the emitter
;; lowers (map f)/(filter p)/(take n) at the wrong arity to the bare procedure
;; (value-position path), so widening the procedures is what makes the 1-arg form
;; work. Capture the originals (collection arities) first, then redefine.
(define %prev-jolt-map jolt-map)
(set! jolt-map (lambda (f . colls)
(if (null? colls) (td-map f) (apply %prev-jolt-map f colls))))
(define %prev-jolt-filter jolt-filter)
(set! jolt-filter (case-lambda ((pred) (td-filter pred))
((pred coll) (%prev-jolt-filter pred coll))))
(define %prev-jolt-remove jolt-remove)
(set! jolt-remove (case-lambda ((pred) (td-remove pred))
((pred coll) (%prev-jolt-remove pred coll))))
(define %prev-jolt-take jolt-take)
(set! jolt-take (case-lambda ((n) (td-take n))
((n coll) (%prev-jolt-take n coll))))
(define %prev-jolt-drop jolt-drop)
(set! jolt-drop (case-lambda ((n) (td-drop n))
((n coll) (%prev-jolt-drop n coll))))
;; into: add the 3-arg (into to xform from). The 2-arg stays the seq.ss fold.
(define %prev-jolt-into jolt-into)
(set! jolt-into (case-lambda ((to from) (%prev-jolt-into to from))
((to xform from) (into-xform to xform from))))
(def-var! "clojure.core" "reduced" jolt-reduced-new)
(def-var! "clojure.core" "reduced?" jolt-reduced-pred)
(def-var! "clojure.core" "mapcat" jolt-mapcat)
(def-var! "clojure.core" "take-while" jolt-take-while)
(def-var! "clojure.core" "drop-while" jolt-drop-while)
(def-var! "clojure.core" "partition" jolt-partition)
(def-var! "clojure.core" "sort" jolt-sort)
(def-var! "clojure.core" "identical?" jolt-identical?)
;; rseq: vectors + sorted colls only (Clojure), the reverse of the ascending seq.
(define (jolt-rseq coll)
(cond
((or (pvec? coll) (htable-sorted? coll))
(list->cseq (reverse (seq->list (jolt-seq coll)))))
;; a deftype/record implementing clojure.lang.Reversible (rseq) — e.g.
;; data.priority-map — drives rseq through its own method.
((and (jrec? coll) (find-method-any-protocol (jrec-tag coll) "rseq"))
=> (lambda (f) (jolt-invoke f coll)))
(else (jolt-throw (jolt-ex-info "rseq requires a vector or sorted collection" (jolt-hash-map))))))
(def-var! "clojure.core" "rseq" jolt-rseq)
;; clojure.core/unchecked-* — host-defined wrapping (Java long) arithmetic from
;; seq.ss. def-var!'d here because def-var! isn't bound when seq.ss loads.
(let ((d! (lambda (n v) (def-var! "clojure.core" n v))))
(d! "unchecked-add" jolt-unchecked-add) (d! "unchecked-add-int" jolt-unchecked-add)
(d! "unchecked-subtract" jolt-unchecked-sub) (d! "unchecked-subtract-int" jolt-unchecked-sub)
(d! "unchecked-multiply" jolt-unchecked-mul) (d! "unchecked-multiply-int" jolt-unchecked-mul)
(d! "unchecked-negate" jolt-uncneg) (d! "unchecked-negate-int" jolt-uncneg)
(d! "unchecked-inc" jolt-uncinc) (d! "unchecked-inc-int" jolt-uncinc)
(d! "unchecked-dec" jolt-uncdec) (d! "unchecked-dec-int" jolt-uncdec)
(d! "unchecked-divide-int" jolt-unchecked-div) (d! "unchecked-remainder-int" jolt-unchecked-rem))

View file

@ -0,0 +1,96 @@
;; natives-transduce.ss — the transducer surface: volatiles, the `cat` transducer,
;; and sequence / transduce application.
;;
;; `sequence` and `transduce` are seed natives. The stateful transducer arities
;; (take-nth/map-indexed/partition-by/dedupe/distinct, all overlay) use
;; volatile!/vswap!/vreset!/deref, shimmed here.
;;
;; Volatiles are a native mutable box (jvol) — the overlay vreset!/vswap! drive a
;; volatile through jolt.host/ref-put!+get, but a Chez volatile is a record, not a
;; tagged table, so those overlay versions are overridden natively in
;; post-prelude.ss. transduce/sequence build on the existing into-xform / reduce-
;; seq machinery (natives-seq.ss / seq.ss). Loaded after those + atoms.ss (deref).
;; --- volatiles ---------------------------------------------------------------
(define-record-type jvol (fields (mutable v)) (nongenerative chez-jvol-v1))
(define (jolt-volatile! x) (make-jvol x))
(define (jolt-vreset! vol x) (jvol-v-set! vol x) x)
(define (jolt-vswap! vol f . args)
(let ((nv (apply jolt-invoke f (jvol-v vol) args))) (jvol-v-set! vol nv) nv))
(define (jolt-volatile-pred? x) (jvol? x))
;; deref reads a volatile too (partition-all/-by transducers @-deref their box).
(define %xf-deref jolt-deref)
(set! jolt-deref (lambda (x) (if (jvol? x) (jvol-v x) (%xf-deref x))))
(def-var! "clojure.core" "volatile!" jolt-volatile!)
(def-var! "clojure.core" "deref" jolt-deref)
;; --- sequence ----------------------------------------------------------------
;; transduce lives in the overlay (clojure/core/22-coll.clj): it's a pure
;; composition (xf (reduce xf init coll)) over reduce, so the Clojure version
;; lowers to the same code the native shim did. sequence stays native (below):
;; its transformer iterator drives the reduced box + lazy realization directly.
;; (sequence coll) -> a seq; (sequence xform coll) -> a LAZY seq of coll transformed
;; by xform. A transformer iterator (mirrors clojure.core's TransformerIterator):
;; pull one input at a time through (xform rf), where rf buffers each emitted value;
;; emit the buffer lazily, pulling more input only when it drains. So an infinite or
;; expensive source is consumed incrementally — (first (sequence (map inc) (range)))
;; returns at once. Honors `reduced` (stop pulling) and runs the 1-arg completion to
;; flush a stateful xform (partition-all / dedupe / a trailing partition).
(define (sequence-xf xform coll)
(let* ((buf (box '())) ; emitted values for the current step, reversed
(rf (case-lambda
(() jolt-nil)
((acc) acc)
((acc x) (set-box! buf (cons x (unbox buf))) acc)))
(xrf (jolt-invoke xform rf)))
;; advance the source until buf holds output or the input is drained+completed.
(define (fill src acc completed)
(let loop ((src src) (acc acc) (completed completed))
(cond
((pair? (unbox buf)) (values src acc completed))
(completed (values src acc #t))
((jolt-reduced? acc)
(jolt-invoke xrf (jolt-reduced-val acc)) ; completion may flush
(loop src (jolt-reduced-val acc) #t))
(else
(let ((s (jolt-seq src)))
(if (jolt-nil? s)
(begin (jolt-invoke xrf acc) (loop src acc #t)) ; complete -> flush
(loop (seq-more s) (jolt-invoke xrf acc (seq-first s)) completed)))))))
;; Resolve the next chunk now (one fill pulls just enough input to emit or to
;; exhaust), so the result is a real cseq | empty — `empty` is jolt-empty-list
;; at the top (so an empty result still prints "()") and jolt-nil inside a tail
;; (the cseq terminator). The TAILS stay lazy, so an infinite source is fine.
(define (step src acc completed empty)
(let-values (((src2 acc2 comp2) (fill src acc completed)))
(let ((out (reverse (unbox buf))))
(set-box! buf '())
(if (null? out)
empty
(let build ((o out))
(if (null? (cdr o))
(cseq-lazy (car o) (lambda () (step src2 acc2 comp2 jolt-nil)))
(cseq-lazy (car o) (lambda () (build (cdr o))))))))))
(step coll jolt-nil #f jolt-empty-list)))
(define jolt-sequence
(case-lambda
((coll) (jolt-seq coll))
((xform coll) (sequence-xf xform coll))))
(def-var! "clojure.core" "sequence" jolt-sequence)
;; --- cat ---------------------------------------------------------------------
;; cat transducer: each input item is itself a collection, concatenated into the
;; downstream reducing fn.
(define (jolt-cat rf)
(lambda a
(cond
((null? a) (jolt-invoke rf))
((null? (cdr a)) (jolt-invoke rf (car a)))
(else
(let loop ((xs (seq->list (jolt-seq (cadr a)))) (acc (car a)))
(if (null? xs) acc (loop (cdr xs) (jolt-invoke rf acc (car xs)))))))))
(def-var! "clojure.core" "cat" jolt-cat)

391
host/chez/ns.ss Normal file
View file

@ -0,0 +1,391 @@
;; namespaces — the namespace value model.
;;
;; The namespace ops (find-ns/resolve/in-ns/…) work over the rt.ss var-table
;; (cells carry ns + name + defined?) and the multimethods.ss chez-current-ns
;; box. A namespace VALUE is a `jns` record carrying its name string — distinct
;; from a map/record so (map? ns) is #f, but the overlay's `ns-name` reads
;; (get ns :name); that's overridden natively in post-prelude.ss (loads after
;; the overlay clobbers it).
;;
;; Loaded LAST from rt.ss. The analyzer bakes a def's target ns at compile time,
;; so a runtime in-ns redirects only *ns* / str-of-ns, not later defs in the
;; same program.
(define-record-type jns (fields name) (nongenerative chez-jns-v1))
;; registry: name-string -> jns. Seeded with the two always-present namespaces;
;; grown by in-ns / create-ns. find-ns ALSO derives existence from the var-table
;; (any cell with that ns), so a namespace that only ever had vars def'd into it
;; is still found.
(define ns-registry (make-hashtable string-hash string=?))
(define (intern-ns! name)
(or (hashtable-ref ns-registry name #f)
(let ((n (make-jns name))) (hashtable-set! ns-registry name n) n)))
(intern-ns! "user")
(intern-ns! "clojure.core")
;; --- namespace aliases ------------------------------------------------------
;; (require '[ns :as a]) registers a -> ns so the analyzer can resolve a/foo to
;; ns/foo. Keyed by (compile-ns . alias). The requires are pre-registered at
;; analyze time (compile-eval.ss) — analysis precedes eval, so a runtime require
;; no-op is fine. Also drives jolt-ns-aliases below.
(define ns-alias-table (make-hashtable equal-hash equal?))
(define (chez-register-alias! cns alias target)
(hashtable-set! ns-alias-table (cons cns alias) target))
(define (chez-resolve-alias cns alias)
(hashtable-ref ns-alias-table (cons cns alias) #f))
;; :refer brings an UNQUALIFIED name into cns, resolving to target-ns/name.
(define ns-refer-table (make-hashtable equal-hash equal?))
(define (chez-register-refer! cns name target)
(hashtable-set! ns-refer-table (cons cns name) target))
;; refer-all (a bare `use`): cns -> list of fully-referred target ns names. A name
;; not found per-name resolves to the first refer-all target that defines it.
(define ns-refer-all-table (make-hashtable equal-hash equal?))
(define (chez-register-refer-all! cns target)
(let ((cur (hashtable-ref ns-refer-all-table cns '())))
(unless (member target cur)
(hashtable-set! ns-refer-all-table cns (cons target cur)))))
(define (chez-resolve-refer cns name)
(or (hashtable-ref ns-refer-table (cons cns name) #f)
(let loop ((ts (hashtable-ref ns-refer-all-table cns '())))
(cond ((null? ts) #f)
((let ((c (var-cell-lookup (car ts) name))) (and c (var-cell-defined? c))) (car ts))
(else (loop (cdr ts)))))))
;; parse a require/use spec FORM and register its :as alias + :refer names under
;; `cns`. spec: [ns :as a :refer [x y] ...] / (ns ...) / bare ns. opts are
;; keyword/value pairs after the ns symbol.
(define (chez-register-spec! cns spec)
(let ((items (cond ((pvec? spec) (seq->list spec))
((or (cseq? spec) (empty-list-t? spec)) (seq->list spec))
(else '()))))
(when (and (pair? items) (symbol-t? (car items)))
(let ((target (symbol-t-name (car items))))
(let loop ((xs (cdr items)))
(when (and (pair? xs) (pair? (cdr xs)))
(let ((k (car xs)) (v (cadr xs)))
(when (keyword? k)
(cond
((string=? (keyword-t-name k) "as")
(when (symbol-t? v) (chez-register-alias! cns (symbol-t-name v) target)))
;; :refer (require) and :only (use) both bring unqualified names
;; into cns resolving to target/name.
((or (string=? (keyword-t-name k) "refer") (string=? (keyword-t-name k) "only"))
(cond
;; :refer :all — bring in every public var (require :refer :all)
((and (keyword? v) (string=? (keyword-t-name v) "all"))
(chez-register-refer-all! cns target))
;; :refer [a b] or :refer (a b) — both forms list names to bring in.
((or (pvec? v) (cseq? v) (empty-list-t? v))
(for-each (lambda (n)
(when (symbol-t? n) (chez-register-refer! cns (symbol-t-name n) target)))
(seq->list v))))))))
(loop (cddr xs))))))))
;; a namespace designator -> its name string (a jns or a symbol; the corpus never
;; passes a bare string).
(define (ns-desig->name d)
(if (jns? d) (jns-name d) (symbol-t-name d)))
(define (ns-has-vars? nm)
(let ((found #f))
(vector-for-each
(lambda (c) (when (and (not found) (string=? (var-cell-ns c) nm)) (set! found #t)))
(hashtable-values var-table))
found))
(define (jolt-find-ns desig)
(let ((nm (ns-desig->name desig)))
(or (hashtable-ref ns-registry nm #f)
(and (ns-has-vars? nm) (intern-ns! nm))
jolt-nil)))
(define (jolt-the-ns desig)
(if (jns? desig) desig
(let ((n (jolt-find-ns desig)))
(if (jns? n) n (error #f "No namespace" desig)))))
(define (jolt-create-ns desig) (intern-ns! (ns-desig->name desig)))
;; in-ns: register + switch the current ns + re-bind *ns* + return the jns. This
;; updates only the RUNTIME current ns — subsequent defs in the same program were
;; already ns-baked by the analyzer, so it does not redirect them. It is enough
;; for *ns* / str-of-ns to track the switch.
(define (jolt-in-ns desig)
(let* ((nm (ns-desig->name desig)) (n (intern-ns! nm)))
;; set the THREAD-LOCAL current ns; *ns* reads derive from it (dyn-binding.ss),
;; so this is per-thread — concurrent nREPL sessions don't clobber each other.
(set-chez-ns! nm)
n))
;; ns-name: a namespace's name as a (no-ns) symbol. Overrides the overlay (which
;; reads (get ns :name) = nil on a jns record) — wired in via post-prelude.ss.
(define (jolt-ns-name desig)
(jolt-symbol #f (jns-name (jolt-the-ns desig))))
(define (jolt-all-ns)
(let ((seen (make-hashtable string-hash string=?)))
(vector-for-each (lambda (k) (hashtable-set! seen k #t)) (hashtable-keys ns-registry))
(vector-for-each (lambda (c) (hashtable-set! seen (var-cell-ns c) #t)) (hashtable-values var-table))
(list->cseq (map intern-ns! (vector->list (hashtable-keys seen))))))
;; ns-publics / ns-map / ns-interns: a {sym -> var-cell} jolt map built by scanning
;; the var-table for defined cells in the namespace. ns-interns/ns-map keep every
;; var; ns-publics drops the ones marked ^:private (defn-/def ^:private), like the
;; JVM. ns-aliases is an empty map (map? is true).
(define (var-private? c)
(let ((m (hashtable-ref var-meta-table c #f)))
(and m (jolt-truthy? (jolt-get m (keyword #f "private"))))))
(define (ns-vars-pmap-when nm keep?)
(let ((m (jolt-hash-map)))
(vector-for-each
(lambda (c)
(when (and (string=? (var-cell-ns c) nm) (var-cell-defined? c) (keep? c))
(set! m (jolt-assoc m (jolt-symbol #f (var-cell-name c)) c))))
(hashtable-values var-table))
m))
(define (ns-vars-pmap nm) (ns-vars-pmap-when nm (lambda (c) #t)))
(define (jolt-ns-publics desig) (ns-vars-pmap-when (ns-desig->name desig) (lambda (c) (not (var-private? c)))))
(define (jolt-ns-interns desig) (ns-vars-pmap (ns-desig->name desig)))
;; ns-aliases: the {alias-sym -> ns-value} registered under `desig`
;; (default the current ns) via require :as / alias. Reads ns-alias-table.
(define (jolt-ns-aliases . desig)
(let ((cns (if (pair? desig) (ns-desig->name (car desig)) (chez-current-ns)))
(m (jolt-hash-map)))
(vector-for-each
(lambda (k)
(when (string=? (car k) cns)
(set! m (jolt-assoc m (jolt-symbol #f (cdr k))
(intern-ns! (hashtable-ref ns-alias-table k #f))))))
(hashtable-keys ns-alias-table))
m))
;; ns-refers: the {sym -> var} referred into `desig` via refer/use.
(define (jolt-ns-refers desig)
(let ((cns (ns-desig->name desig)) (m (jolt-hash-map)))
(vector-for-each
(lambda (k)
(when (string=? (car k) cns)
(let* ((target (hashtable-ref ns-refer-table k #f))
(c (and target (var-cell-lookup target (cdr k)))))
(when c (set! m (jolt-assoc m (jolt-symbol #f (cdr k)) c))))))
(hashtable-keys ns-refer-table))
m))
;; ns-imports: clojure.core auto-imports the 96 public java.lang classes into
;; every ns. jolt has no classloader, but returns that map (short symbol ->
;; canonical class-name token) so (count (ns-imports 'user)) = 96 like the JVM.
(define jolt-default-import-names
'("AbstractMethodError" "Appendable" "ArithmeticException" "ArrayIndexOutOfBoundsException"
"ArrayStoreException" "AssertionError" "BigDecimal" "BigInteger" "Boolean" "Byte"
"Callable" "CharSequence" "Character" "Class" "ClassCastException" "ClassCircularityError"
"ClassFormatError" "ClassLoader" "ClassNotFoundException" "CloneNotSupportedException"
"Cloneable" "Comparable" "Compiler" "Deprecated" "Double" "Enum"
"EnumConstantNotPresentException" "Error" "Exception" "ExceptionInInitializerError" "Float"
"IllegalAccessError" "IllegalAccessException" "IllegalArgumentException"
"IllegalMonitorStateException" "IllegalStateException" "IllegalThreadStateException"
"IncompatibleClassChangeError" "IndexOutOfBoundsException" "InheritableThreadLocal"
"InstantiationError" "InstantiationException" "Integer" "InternalError" "InterruptedException"
"Iterable" "LinkageError" "Long" "Math" "NegativeArraySizeException" "NoClassDefFoundError"
"NoSuchFieldError" "NoSuchFieldException" "NoSuchMethodError" "NoSuchMethodException"
"NullPointerException" "Number" "NumberFormatException" "Object" "OutOfMemoryError" "Override"
"Package" "Process" "ProcessBuilder" "Readable" "Runnable" "Runtime" "RuntimeException"
"RuntimePermission" "SecurityException" "SecurityManager" "Short" "StackOverflowError"
"StackTraceElement" "StrictMath" "String" "StringBuffer" "StringBuilder"
"StringIndexOutOfBoundsException" "SuppressWarnings" "System" "Thread" "Thread$State"
"Thread$UncaughtExceptionHandler" "ThreadDeath" "ThreadGroup" "ThreadLocal" "Throwable"
"TypeNotPresentException" "UnknownError" "UnsatisfiedLinkError" "UnsupportedClassVersionError"
"UnsupportedOperationException" "VerifyError" "VirtualMachineError" "Void"))
(define jolt-default-imports
(let loop ((ns jolt-default-import-names) (m (jolt-hash-map)))
(if (null? ns) m
(loop (cdr ns)
(jolt-assoc m (jolt-symbol #f (car ns)) (string-append "java.lang." (car ns)))))))
(define (jolt-ns-imports . _) jolt-default-imports)
;; resolve: an unqualified symbol resolves in the current ns then clojure.core; a
;; qualified one in its own ns. Returns the var iff genuinely defined, else nil —
;; never interns an empty cell (var-cell-lookup is non-creating).
;; resolve `sym` in the current ns: a qualified ns part is read as an :as alias
;; (then a real ns); an unqualified name resolves in the current ns, its :refers,
;; then clojure.core. (ns-resolve does the same against an explicit ns.)
(define (jolt-resolve sym)
(let* ((cns (chez-current-ns))
(sns (symbol-t-ns sym)) (nm (symbol-t-name sym))
(c (if (string? sns)
(var-cell-lookup (or (chez-resolve-alias cns sns) sns) nm)
(or (var-cell-lookup cns nm)
(let ((ref (chez-resolve-refer cns nm))) (and ref (var-cell-lookup ref nm)))
(var-cell-lookup "clojure.core" nm)))))
(if (and c (var-cell-defined? c)) c jolt-nil)))
(define (jolt-find-var sym)
(let ((sns (symbol-t-ns sym)) (nm (symbol-t-name sym)))
(if (string? sns)
(let ((c (var-cell-lookup sns nm))) (if (and c (var-cell-defined? c)) c jolt-nil))
(error #f "find-var requires a fully-qualified symbol" sym))))
;; ns-unmap: clear the mapping — drop defined? and reset the root to unbound, so a
;; later resolve returns nil.
(define (jolt-ns-unmap ns-desig sym)
(let ((c (var-cell-lookup (ns-desig->name ns-desig) (symbol-t-name sym))))
(when c (var-cell-defined?-set! c #f) (var-cell-root-set! c jolt-unbound)))
jolt-nil)
;; --- ns runtime fns ---------------------------------------------------------
;; ns-resolve: resolve `sym` as if reading it in namespace `ns-desig`. Qualified
;; syms consult that ns's :as aliases; unqualified resolve in the ns, its :refers,
;; then clojure.core. Returns the var or nil (never interns).
(define (jolt-ns-resolve ns-desig sym)
(let* ((cns (ns-desig->name ns-desig))
(sns (symbol-t-ns sym)) (nm (symbol-t-name sym))
(c (if (string? sns)
(var-cell-lookup (or (chez-resolve-alias cns sns) sns) nm)
(or (var-cell-lookup cns nm)
(let ((ref (chez-resolve-refer cns nm))) (and ref (var-cell-lookup ref nm)))
(var-cell-lookup "clojure.core" nm)))))
(if (and c (var-cell-defined? c)) c jolt-nil)))
;; remove-ns: drop the namespace from the registry AND its vars, so find-ns
;; (which also derives existence from the var-table) returns nil afterward.
(define (jolt-remove-ns desig)
(let ((nm (ns-desig->name desig)))
(hashtable-delete! ns-registry nm)
(vector-for-each
(lambda (k) (let ((c (hashtable-ref var-table k #f)))
(when (and c (string=? (var-cell-ns c) nm)) (hashtable-delete! var-table k))))
(hashtable-keys var-table))
jolt-nil))
;; intern: create/set a var ns/sym to val (or an unbound cell). Returns the var.
(define (jolt-intern ns-desig sym . vopt)
(let ((nm (ns-desig->name ns-desig)) (s (symbol-t-name sym)))
;; the namespace must exist (Namespace.find), like the JVM's intern
(unless (hashtable-ref ns-registry nm #f)
(jolt-throw (jolt-ex-info (string-append "No namespace: " nm " found") empty-pmap)))
(if (pair? vopt) (def-var! nm s (car vopt)) (declare-var! nm s))))
;; alias / ns-unalias: register/drop an :as alias under the current (or given) ns.
;; A runtime alias is registered into the SAME table the analyzer consults, so a
;; later form in the program resolves alias/foo (the spine analyzes form by form).
(define (jolt-alias alias-sym ns-sym)
(chez-register-alias! (chez-current-ns) (symbol-t-name alias-sym) (ns-desig->name ns-sym))
jolt-nil)
(define (jolt-ns-unalias ns-desig alias-sym)
(hashtable-delete! ns-alias-table (cons (ns-desig->name ns-desig) (symbol-t-name alias-sym)))
jolt-nil)
;; refer: bring every public var of `ns-sym` into the current ns as an unqualified
;; name (filters accepted/ignored — the corpus uses the bare form). refer-clojure
;; is a no-op (clojure.core always resolves on Chez).
(define (jolt-refer ns-sym . _filters)
(let ((target (ns-desig->name ns-sym)) (cns (chez-current-ns)))
(vector-for-each
(lambda (c) (when (and (string=? (var-cell-ns c) target) (var-cell-defined? c))
(chez-register-refer! cns (var-cell-name c) target)))
(hashtable-values var-table))
jolt-nil))
;; (:refer-clojure :exclude [names…]) — clojure.core always resolves on Chez, so
;; the only thing to track is the EXCLUDE set: an excluded name is not
;; clojure.core/name, so syntax-quote qualifies it to the current ns instead (a ns
;; that excludes and defines its own, e.g. core.logic.fd's ==).
(define ns-core-exclude-table (make-hashtable equal-hash equal?)) ; cns -> (name -> #t)
(define (chez-register-core-exclude! cns name)
(let ((h (or (hashtable-ref ns-core-exclude-table cns #f)
(let ((h (make-hashtable string-hash string=?)))
(hashtable-set! ns-core-exclude-table cns h) h))))
(hashtable-set! h name #t)))
(define (chez-core-excluded? cns name)
(let ((h (hashtable-ref ns-core-exclude-table cns #f)))
(and h (hashtable-ref h name #f) #t)))
(define (jolt-refer-clojure . args)
(let ((cns (chez-current-ns)))
(let loop ((a args))
(when (and (pair? a) (pair? (cdr a)))
(when (and (keyword? (car a)) (string=? (keyword-t-name (car a)) "exclude"))
(for-each (lambda (n) (when (symbol-t? n)
(chez-register-core-exclude! cns (symbol-t-name n))))
(seq->list (cadr a))))
(loop (cddr a)))))
jolt-nil)
;; alter-meta! / reset-meta!: a var's metadata lives in var-meta-table (rt.ss);
;; any other reference (atom/agent/namespace) uses the identity meta side-table
;; jolt-meta reads.
(define (jolt-alter-meta! ref f . args)
(if (var-cell? ref)
(let* ((cur (or (hashtable-ref var-meta-table ref #f) (jolt-hash-map)))
(new (apply jolt-invoke f cur args)))
(hashtable-set! var-meta-table ref new)
new)
(let* ((cur (let ((m (jolt-meta ref))) (if (jolt-nil? m) (jolt-hash-map) m)))
(new (apply jolt-invoke f cur args)))
(hashtable-set! meta-table ref new)
new)))
(define (jolt-reset-meta! ref m)
(if (var-cell? ref)
(hashtable-set! var-meta-table ref m)
(hashtable-set! meta-table ref m))
m)
;; --- RESOLVE FRICTION: native-op cells -------------------------------------
;; Native-op primitives (+ map reduce …) are INLINED at emit, so they have no
;; var-cell and (resolve '+) would be nil — diverging from Clojure where it is a
;; var. def-var! each to its value-position procedure so it has a real, defined
;; cell (calls still inline, so no perf hit; #'+ deref and ((resolve '+) 1 2) also
;; work now). The clojure.core prelude, loaded AFTER rt.ss, overwrites the cells
;; for names it also defines in the overlay (map/filter/…); the purely-inlined
;; scalars (+/-/</inc/…) keep these.
(for-each
(lambda (p) (def-var! "clojure.core" (car p) (cdr p)))
(list
(cons "+" jolt-add) (cons "-" jolt-sub) (cons "*" jolt-mul) (cons "/" jolt-div)
(cons "<" <) (cons ">" >) (cons "<=" <=) (cons ">=" >=)
(cons "=" jolt=) (cons "inc" jolt-inc) (cons "dec" jolt-dec) (cons "not" jolt-not)
(cons "min" min) (cons "max" max)
(cons "mod" modulo) (cons "rem" remainder) (cons "quot" quotient)
(cons "vector" jolt-vector) (cons "hash-map" jolt-hash-map) (cons "hash-set" jolt-hash-set)
(cons "conj" jolt-conj) (cons "get" jolt-get) (cons "nth" jolt-nth) (cons "count" jolt-count)
(cons "assoc" jolt-assoc) (cons "dissoc" jolt-dissoc) (cons "contains?" jolt-contains?)
(cons "empty?" jolt-empty?) (cons "peek" jolt-peek) (cons "pop" jolt-pop)
(cons "first" jolt-first) (cons "rest" jolt-rest) (cons "next" jolt-next) (cons "seq" jolt-seq)
(cons "cons" jolt-cons) (cons "list" jolt-list) (cons "reverse" jolt-reverse) (cons "last" jolt-last)
(cons "map" jolt-map) (cons "filter" jolt-filter) (cons "remove" jolt-remove)
(cons "reduce" jolt-reduce) (cons "into" jolt-into) (cons "concat" jolt-concat) (cons "apply" jolt-apply)
(cons "range" jolt-range) (cons "take" jolt-take) (cons "drop" jolt-drop)
(cons "keys" jolt-keys) (cons "vals" jolt-vals)
(cons "even?" jolt-even?) (cons "odd?" jolt-odd?) (cons "pos?" jolt-pos?) (cons "neg?" jolt-neg?)
(cons "zero?" jolt-zero?) (cons "identity" jolt-identity)
(cons "ex-info" jolt-ex-info)))
;; --- bindings + *ns* --------------------------------------------------------
(def-var! "clojure.core" "find-ns" jolt-find-ns)
(def-var! "clojure.core" "the-ns" jolt-the-ns)
(def-var! "clojure.core" "create-ns" jolt-create-ns)
(def-var! "clojure.core" "in-ns" jolt-in-ns)
(def-var! "clojure.core" "all-ns" jolt-all-ns)
(def-var! "clojure.core" "ns-publics" jolt-ns-publics)
(def-var! "clojure.core" "ns-map" jolt-ns-interns)
(def-var! "clojure.core" "ns-interns" jolt-ns-interns)
(def-var! "clojure.core" "ns-aliases" jolt-ns-aliases)
(def-var! "clojure.core" "ns-refers" jolt-ns-refers)
(def-var! "clojure.core" "ns-imports" jolt-ns-imports)
(def-var! "clojure.core" "resolve" jolt-resolve)
(def-var! "clojure.core" "ns-resolve" jolt-ns-resolve)
(def-var! "clojure.core" "find-var" jolt-find-var)
(def-var! "clojure.core" "ns-unmap" jolt-ns-unmap)
(def-var! "clojure.core" "remove-ns" jolt-remove-ns)
(def-var! "clojure.core" "intern" jolt-intern)
(def-var! "clojure.core" "alias" jolt-alias)
(def-var! "clojure.core" "ns-unalias" jolt-ns-unalias)
(def-var! "clojure.core" "refer" jolt-refer)
(def-var! "clojure.core" "refer-clojure" jolt-refer-clojure)
(def-var! "clojure.core" "alter-meta!" jolt-alter-meta!)
(def-var! "clojure.core" "reset-meta!" jolt-reset-meta!)
;; *ns* starts at the user namespace (the current ns for -e user code). in-ns
;; re-binds it. (ns-name is overridden natively in post-prelude.ss.)
(def-var! "clojure.core" "*ns*" (intern-ns! "user"))
;; --- printer patches: a namespace renders as its name (str / pr-str / -e) ----
(register-pr-arm! jns? jns-name)
(register-str-render! jns? jns-name)

122
host/chez/png.ss Normal file
View file

@ -0,0 +1,122 @@
;; png.ss — jolt.png: a minimal PNG writer, the built-in the
;; ray-tracer-multi example renders through. Truecolor (8-bit RGB), no
;; compression: the IDAT zlib stream uses DEFLATE "stored" (uncompressed) blocks,
;; so there is no compressor to carry — just CRC-32 / Adler-32 framing over Chez
;; bytevectors. def-var!'d into the jolt.png namespace, so a (require '[jolt.png])
;; resolves it as a baked namespace (no source file).
;; --- CRC-32 (PNG chunk checksum) --------------------------------------------
(define png-crc-table
(let ((t (make-vector 256)))
(do ((n 0 (+ n 1))) ((= n 256) t)
(let loop ((c n) (k 0))
(if (= k 8)
(vector-set! t n c)
(loop (if (odd? c) (bitwise-xor #xedb88320 (bitwise-arithmetic-shift-right c 1))
(bitwise-arithmetic-shift-right c 1))
(+ k 1)))))))
(define (png-crc32 bv)
(let ((len (bytevector-length bv)))
(let loop ((i 0) (c #xffffffff))
(if (= i len)
(bitwise-xor c #xffffffff)
(loop (+ i 1)
(bitwise-xor (bitwise-arithmetic-shift-right c 8)
(vector-ref png-crc-table
(bitwise-and (bitwise-xor c (bytevector-u8-ref bv i)) #xff))))))))
;; --- Adler-32 (zlib checksum) -----------------------------------------------
(define (png-adler32 bv)
(let ((len (bytevector-length bv)))
(let loop ((i 0) (a 1) (b 0))
(if (= i len)
(bitwise-ior (bitwise-arithmetic-shift-left b 16) a)
(let ((a* (modulo (+ a (bytevector-u8-ref bv i)) 65521)))
(loop (+ i 1) a* (modulo (+ b a*) 65521)))))))
;; --- byte helpers -----------------------------------------------------------
(define (png-u32be n)
(let ((bv (make-bytevector 4)))
(bytevector-u8-set! bv 0 (bitwise-and (bitwise-arithmetic-shift-right n 24) #xff))
(bytevector-u8-set! bv 1 (bitwise-and (bitwise-arithmetic-shift-right n 16) #xff))
(bytevector-u8-set! bv 2 (bitwise-and (bitwise-arithmetic-shift-right n 8) #xff))
(bytevector-u8-set! bv 3 (bitwise-and n #xff))
bv))
(define (png-bytes . bs) (u8-list->bytevector bs))
(define (png-cat . bvs)
(let* ((total (apply + (map bytevector-length bvs)))
(out (make-bytevector total)))
(let loop ((bvs bvs) (off 0))
(if (null? bvs) out
(let ((n (bytevector-length (car bvs))))
(bytevector-copy! (car bvs) 0 out off n)
(loop (cdr bvs) (+ off n)))))))
;; one PNG chunk: length(4) + type(4) + data + crc32(type+data)(4)
(define (png-chunk type-str data)
(let ((type (string->utf8 type-str)))
(png-cat (png-u32be (bytevector-length data)) type data
(png-u32be (png-crc32 (png-cat type data))))))
;; DEFLATE "stored" stream of raw: ≤65535-byte blocks, each 1 header byte
;; (BFINAL bit) + LEN(2 LE) + NLEN(2 LE) + raw. Wrapped as zlib (0x78 0x01 …
;; adler32).
(define (png-deflate-stored raw)
(let ((len (bytevector-length raw)))
(let loop ((off 0) (acc (list (png-bytes #x78 #x01))))
(if (>= off len)
(apply png-cat (reverse (cons (png-u32be (png-adler32 raw)) acc)))
(let* ((n (min 65535 (- len off)))
(final (if (>= (+ off n) len) 1 0))
(block (png-cat (png-bytes final)
(png-bytes (bitwise-and n #xff) (bitwise-arithmetic-shift-right n 8))
(png-bytes (bitwise-and (bitwise-not n) #xff)
(bitwise-and (bitwise-arithmetic-shift-right (bitwise-not n) 8) #xff))
(let ((b (make-bytevector n))) (bytevector-copy! raw off b 0 n) b))))
(loop (+ off n) (cons block acc)))))))
;; --- the image value --------------------------------------------------------
(define-record-type pimg (fields w h data (mutable cur)) (nongenerative jolt-png-img-v1))
(define (png-clamp-byte n)
(let ((x (cond ((and (number? n) (exact? n) (integer? n)) n)
((number? n) (exact (floor n)))
(else 0))))
(cond ((< x 0) 0) ((> x 255) 255) (else x))))
(define (png-image w h) (make-pimg w h (make-bytevector (* w h 3) 0) 0))
(define (png-put! img r g b)
(let ((d (pimg-data img)) (c (pimg-cur img)))
(when (<= (+ c 3) (bytevector-length d))
(bytevector-u8-set! d c (png-clamp-byte r))
(bytevector-u8-set! d (+ c 1) (png-clamp-byte g))
(bytevector-u8-set! d (+ c 2) (png-clamp-byte b))
(pimg-cur-set! img (+ c 3)))
jolt-nil))
;; scanlines with a 0 (None) filter byte per row -> raw -> zlib -> IDAT
(define (png-raw img w h)
(let* ((stride (* w 3)) (raw (make-bytevector (* h (+ 1 stride)))) (src (pimg-data img)))
(do ((y 0 (+ y 1))) ((= y h) raw)
(let ((ro (* y (+ 1 stride))))
(bytevector-u8-set! raw ro 0) ; filter: None
(bytevector-copy! src (* y stride) raw (+ ro 1) stride)))))
(define png-signature (png-bytes #x89 #x50 #x4e #x47 #x0d #x0a #x1a #x0a))
(define (png-ihdr w h)
(png-cat (png-u32be w) (png-u32be h)
(png-bytes 8 2 0 0 0))) ; bitdepth 8, colortype 2 (RGB), deflate, filter 0, no interlace
(define (png-write img w h path)
(let* ((idat (png-deflate-stored (png-raw img w h)))
(bytes (png-cat png-signature
(png-chunk "IHDR" (png-ihdr w h))
(png-chunk "IDAT" idat)
(png-chunk "IEND" (make-bytevector 0))))
(p (open-file-output-port path (file-options no-fail) (buffer-mode block))))
(put-bytevector p bytes)
(close-port p)
jolt-nil))
(def-var! "jolt.png" "image" png-image)
(def-var! "jolt.png" "put!" png-put!)
(def-var! "jolt.png" "write" png-write)

150
host/chez/post-prelude.ss Normal file
View file

@ -0,0 +1,150 @@
;; post-prelude overrides — loaded AFTER the assembled clojure.core
;; prelude, so these win over the overlay's own def-var!.
;;
;; A few clojure.core predicates are implemented in the overlay by inspecting a
;; tagged value's :jolt/type key (e.g. (get x :jolt/type)). That key doesn't
;; exist for native representations: a jolt char is a Scheme char, an atom is a
;; Chez record. The overlay's def-var! loads after rt.ss, so it clobbers the
;; correct native shims (predicates.ss / atoms.ss) with versions that return
;; false on every Chez value. Re-assert the native versions here.
(def-var! "clojure.core" "char?" jolt-char-pred?)
(def-var! "clojure.core" "atom?" jolt-atom?)
;; atom watches/validators: the overlay drives these via jolt.host/ref-put! on a
;; tagged table (get a :watches), which a Chez atom record is not — re-assert the
;; native versions (defined in atoms.ss), and swap!/reset! notify+validate there.
(def-var! "clojure.core" "add-watch" jolt-add-watch)
(def-var! "clojure.core" "remove-watch" jolt-remove-watch)
(def-var! "clojure.core" "set-validator!" jolt-set-validator!)
(def-var! "clojure.core" "get-validator" jolt-get-validator)
;; volatiles: a Chez volatile is a jvol record, but the overlay vreset!/vswap!/
;; volatile? drive it via jolt.host/ref-put!+get / :jolt/type (tagged-table only).
;; Override with the native versions (defined in natives-transduce.ss).
(def-var! "clojure.core" "vreset!" jolt-vreset!)
(def-var! "clojure.core" "vswap!" jolt-vswap!)
(def-var! "clojure.core" "volatile?" jolt-volatile-pred?)
;; bound?: the overlay reads (get v :root) — nil on a Chez var-cell record, so it
;; would wrongly report every var unbound. Native version (defined in vars.ss).
(def-var! "clojure.core" "bound?" jolt-bound?)
;; uuid?/random-uuid/parse-uuid/tagged-literal? are overlay (read :jolt/type or
;; build tagged tables) — re-assert the native versions (natives-misc.ss).
(def-var! "clojure.core" "uuid?" jolt-uuid-pred?)
(def-var! "clojure.core" "random-uuid" jolt-random-uuid)
(def-var! "clojure.core" "parse-uuid" jolt-parse-uuid)
(def-var! "clojure.core" "tagged-literal?" jolt-tagged-literal-pred?)
;; ns-name: the overlay reads (get ns :name) — nil on a jns namespace record.
;; Native version (defined in ns.ss) returns the namespace's name symbol.
(def-var! "clojure.core" "ns-name" jolt-ns-name)
;; concurrency: the overlay's future-done?/future-cancelled?/realized? read a
;; future-map's :cached/:cancelled keys, and promise/deliver are a non-blocking
;; atom shim. A Chez future/promise is a record, and we want JVM (blocking,
;; shared-heap) semantics — re-assert the native versions. realized?
;; wraps the overlay (which still handles delay/lazy-seq/atom) for non-futures.
(def-var! "clojure.core" "future-done?" jolt-native-future-done?)
(def-var! "clojure.core" "future-cancelled?" jolt-native-future-cancelled?)
(def-var! "clojure.core" "future?" jolt-future?)
(def-var! "clojure.core" "promise" jolt-promise-new)
(def-var! "clojure.core" "deliver" jolt-deliver)
;; agents: the overlay (50-io) is a synchronous shim (agent = atom, send applies
;; immediately). Re-assert the native async agents (per-agent serialized worker),
;; matching the JVM. await/restart-agent are new (the overlay has neither).
(def-var! "clojure.core" "agent" jolt-agent-new)
(def-var! "clojure.core" "agent?" jolt-agent?)
(def-var! "clojure.core" "send" jolt-agent-send)
(def-var! "clojure.core" "send-off" jolt-agent-send)
(def-var! "clojure.core" "await" jolt-agent-await)
(def-var! "clojure.core" "agent-error" jolt-agent-error)
(def-var! "clojure.core" "restart-agent" jolt-agent-restart)
(def-var! "clojure.core" "deref" jolt-deref)
(let ((overlay-realized? (var-deref "clojure.core" "realized?")))
(def-var! "clojure.core" "realized?"
(lambda (x)
(cond
((or (jolt-future? x) (jolt-promise? x) (jolt-delay? x)) (jolt-conc-realized? x))
;; a lazy-seq carries its own realized? flag (lazy-bridge.ss). The overlay
;; realized? reads :jolt/type and throws on a jolt-lazyseq record.
((jolt-lazyseq? x) (jolt-lazyseq-realized? x))
;; a seq cell answers by its forced flag: the rest of a realized lazy
;; chain is a cseq under jolt's seq model, and (realized? (rest s)) after
;; a next must be true like the JVM's realized LazySeq — never a throw
;; whose message renders the (possibly infinite) seq.
;; a PLAIN seq (list/cons/range — not a lazy-seq wrapper) is not an
;; IPending on the JVM: realized? throws.
((or (cseq? x) (empty-list-t? x))
(jolt-throw (jolt-host-throwable
"java.lang.ClassCastException"
(string-append "class " (guard (e (#t "?")) (jolt-class-name x))
" cannot be cast to class clojure.lang.IPending"))))
(else (jolt-invoke overlay-realized? x))))))
;; clojure.edn/read over a reader: drain the jhost reader, then read through the
;; overlay read-string so the opts map (:readers/:default/:eof) is honored.
(def-var! "clojure.edn" "read"
(case-lambda
((reader) (chez-edn-read reader))
((opts reader)
(jolt-invoke (var-deref "clojure.edn" "read-string") opts
(if (reader-jhost? reader) (drain-reader reader) (jolt-str-render-one reader))))))
;; line-seq: a jhost reader (io/reader result) -> drain+split; a map-reader (the
;; overlay's :read-line-fn model, e.g. with-in-str) -> the overlay version.
(let ((overlay-line-seq (var-deref "clojure.core" "line-seq")))
(def-var! "clojure.core" "line-seq"
(lambda (rdr)
(if (reader-jhost? rdr) (chez-line-seq rdr) (jolt-invoke overlay-line-seq rdr)))))
;; JVM-parity numeric tower. integer?/float? are on the compiler emit/inference
;; path (so they stay native) but the overlay (20-coll.clj) still carries an
;; all-flonum int?/double? (int? -> integer?, double? -> not-integer) that
;; misclassifies exact rationals (e.g. (double? 1/2) -> true). Re-assert the
;; native tower-correct versions so they win over those overlay defs. int?/double?
;; alias integer?/float?. == is value-equality. (ratio?/rational? are now correct
;; in the overlay, built on jolt.host tower tests, so they need no re-assertion.)
(def-var! "clojure.core" "integer?" jolt-integer?)
(def-var! "clojure.core" "int?" jolt-integer?)
(def-var! "clojure.core" "float?" jolt-float?)
(def-var! "clojure.core" "double?" jolt-float?)
;; ratio?/rational? now live (correctly) in the overlay, so they no longer need a
;; native re-assertion here. decimal? stays (bigdec re-binds it).
(def-var! "clojure.core" "decimal?" jolt-decimal?)
(def-var! "clojure.core" "==" jolt-num-equiv)
;; chunked-seq? is true for a vector's seq (a real chunked-seq); the overlay's
;; always-false stub loaded over the host fn, so re-assert it.
(def-var! "clojure.core" "chunked-seq?" na-chunked-seq?)
;; record? is a host type check — true only for a defrecord, not a bare deftype
;; (jrec-record?), matching the JVM (instance? IRecord). The overlay's
;; (some? (get x :jolt/deftype)) get-trick would invoke a sorted-map comparator.
(def-var! "clojure.core" "record?" (lambda (x) (jrec-record? x)))
;; read / read+string over a HOST reader jhost (java.io StringReader/PushbackReader):
;; the overlay's IReader protocol only covers the reify map-reader, so a (read
;; pushback-reader) — cuerdas' string interpolation — would miss. Intercept a host
;; reader; everything else (the *in* reify) delegates to the overlay.
(let ((ov-read (var-deref "clojure.core" "read")))
(def-var! "clojure.core" "read"
(case-lambda
(() (jolt-invoke ov-read))
((stream)
(if (reader-jhost? stream)
(let-values (((form found?) (host-reader-read-form stream)))
(if found? form (jolt-throw (jolt-ex-info "EOF while reading" empty-pmap))))
(jolt-invoke ov-read stream)))
((stream e? ev)
(if (reader-jhost? stream)
(let-values (((form found?) (host-reader-read-form stream)))
(cond (found? form)
((jolt-truthy? e?) (jolt-throw (jolt-ex-info "EOF while reading" empty-pmap)))
(else ev)))
(jolt-invoke ov-read stream e? ev))))))
(let ((ov-rps (var-deref "clojure.core" "read+string")))
(def-var! "clojure.core" "read+string"
(case-lambda
(() (jolt-invoke ov-rps))
((stream) (jolt-invoke (var-deref "clojure.core" "read+string") stream #t jolt-nil))
((stream e? ev)
(if (reader-jhost? stream)
(let* ((s (drain-reader stream)) (pr (jolt-parse-next s)))
(if (jolt-nil? pr)
(begin (reader-refill! stream "")
(if (jolt-truthy? e?) (jolt-throw (jolt-ex-info "EOF while reading" empty-pmap))
(jolt-vector ev "")))
(let ((rest (jolt-nth pr 1)))
(reader-refill! stream rest)
(jolt-vector (jolt-nth pr 0) (substring s 0 (- (string-length s) (string-length rest)))))))
(jolt-invoke ov-rps stream e? ev))))))

110
host/chez/predicates.ss Normal file
View file

@ -0,0 +1,110 @@
;; type predicates + simple accessors — host-coupled natives.
;;
;; These are host primitives (not clojure.core overlay fns), so they're never
;; def-var!'d by the assembled prelude; the Chez host must provide them.
;; map?/vector?/set? are STRICT over the persistent-collection records, seq? is
;; true only for real sequences, coll? is the union. Record arms are added by
;; records.ss, which extends these dispatchers.
(define (jolt-map? x) (pmap? x))
;; a map entry is a pvec under the hood AND is vector? — Clojure's MapEntry
;; implements IPersistentVector, so (vector? (first {:a 1})) is true.
(define (jolt-vector? x) (pvec? x))
(define (jolt-set? x) (pset? x))
(define (jolt-seq? x) (or (cseq? x) (empty-list-t? x)))
;; list? lives in the overlay (clojure/core/20-coll.clj) — see jolt.host/cseq? etc.
(define (jolt-coll-pred? x)
(or (pvec? x) (pmap? x) (pset? x) (cseq? x) (empty-list-t? x) (jolt-lazyseq? x)))
(define (jolt-number? x) (number? x))
(define (jolt-string? x) (string? x))
(define (jolt-char-pred? x) (char? x))
;; JVM-parity number-type predicates over the Chez numeric tower. integer? is the
;; INTEGER TYPE (exact integer = Long/BigInt), NOT integer-VALUED: (integer? 3.0)
;; is false on the JVM (3.0 is a Double). float? = flonum (double). ratio? = exact
;; non-integer (= JVM Ratio). rational? = exact (integer or ratio; jolt has no
;; BigDecimal). decimal? is always false (no BigDecimal type).
(define (jolt-integer? x) (and (number? x) (exact? x) (integer? x)))
(define (jolt-float? x) (and (number? x) (flonum? x)))
;; ratio?/rational? live in the overlay (clojure/core/20-coll.clj), built on the
;; jolt.host tower tests. decimal? stays native: the optional bigdec module
;; (java/bigdec.ss) re-binds it to jbigdec?, so it can't be a static overlay const.
(define (jolt-decimal? x) #f)
(define (jolt-fn? x) (procedure? x))
(define (jolt-boolean-pred? x) (boolean? x))
;; (boolean x) coerces truthiness (nil/false -> false, else true). MUST stay native:
;; the backend's emit path calls clojure.core/boolean for every :if node
;; (backend_scheme.clj bool tracking), so it has to exist before ANY compilation,
;; including the kernel overlay tier (whose own fns contain `if`). Migrating it even
;; to the kernel tier deadlocks: compiling the tier that defines boolean needs boolean.
(define (jolt-boolean x) (if (jolt-truthy? x) #t #f))
;; (name x): keyword/symbol -> name string; string -> itself.
(define (jolt-name x)
(cond
((keyword? x) (keyword-t-name x))
((symbol-t? x) (symbol-t-name x))
((string? x) x)
(else (error #f "name: expected string/symbol/keyword" x))))
;; (namespace x): keyword/symbol ns string, or nil when unqualified.
(define (jolt-namespace x)
(let ((ns (cond ((keyword? x) (keyword-t-ns x))
((symbol-t? x) (symbol-t-ns x))
(else (error #f "namespace: expected symbol/keyword" x)))))
(if (or (jolt-nil? ns) (not ns) (eq? ns '())) jolt-nil ns)))
(def-var! "clojure.core" "nil?" jolt-nil?)
(def-var! "clojure.core" "number?" jolt-number?)
(def-var! "clojure.core" "string?" jolt-string?)
(def-var! "clojure.core" "char?" jolt-char-pred?)
(def-var! "clojure.core" "integer?" jolt-integer?)
(def-var! "clojure.core" "float?" jolt-float?)
(def-var! "clojure.core" "decimal?" jolt-decimal?)
;; == numeric value-equality (ignores exactness, unlike =): (== 3 3.0) -> true.
;; 1-arity is trivially true; 2+ args must all be numbers (Numbers.equiv throws
;; otherwise). Uses Scheme = (value across the tower), not jolt= (category-aware).
(define (jolt-num-equiv . xs)
;; 1-arity short-circuits to true for ANY value (Clojure's == 1-arg returns true
;; before the number check); 2+ args must all be numbers.
(if (and (pair? xs) (null? (cdr xs)))
#t
(let all-num? ((ys xs))
(cond
((null? ys) (or (null? xs) (apply = xs)))
((number? (car ys)) (all-num? (cdr ys)))
(else (error #f "== requires numbers" xs))))))
(def-var! "clojure.core" "==" jolt-num-equiv)
(def-var! "clojure.core" "symbol?" jolt-symbol?)
(def-var! "clojure.core" "keyword?" keyword?)
(def-var! "clojure.core" "map?" jolt-map?)
(def-var! "clojure.core" "vector?" jolt-vector?)
(def-var! "clojure.core" "set?" jolt-set?)
(def-var! "clojure.core" "seq?" jolt-seq?)
(def-var! "clojure.core" "coll?" jolt-coll-pred?)
(def-var! "clojure.core" "fn?" jolt-fn?)
(def-var! "clojure.core" "boolean?" jolt-boolean-pred?)
(def-var! "clojure.core" "boolean" jolt-boolean)
(def-var! "clojure.core" "name" jolt-name)
(def-var! "clojure.core" "namespace" jolt-namespace)
;; --- jolt.host raw type-test primitives -------------------------------------
;; Some clojure.core predicates bottom out at host tests overlay Clojure can't
;; reach. Expose the ones the migratable predicates need so the overlay versions
;; lower to exactly these calls — no perf loss. rational-type? is the Chez TYPE
;; test (exact rational), distinct from clojure.core/rational? (which gates on
;; number? first). exact? is wrapped TOTAL (Chez's raw exact? errors on a
;; non-number); rational-type? already returns #f for a non-match.
;;
;; Only the tests consumed by the migrated predicates (ratio?/rational? -> exact?,
;; rational-type?; list? -> cseq?/cseq-list?/empty-list?) are exposed. The rest of
;; the predicate web stays native and is NOT exposed: map?/set?/seq?/coll? are
;; extended at runtime with sorted/record/lazy arms, decimal? is extended by the
;; optional bigdec module, integer?/float? are on the compiler emit/inference path,
;; and vector? is reached by the kernel-tier peek during bootstrap.
(define (jh-exact? x) (and (number? x) (exact? x)))
(def-var! "jolt.host" "exact?" jh-exact?)
(def-var! "jolt.host" "rational-type?" rational?)
(def-var! "jolt.host" "cseq?" cseq?)
(def-var! "jolt.host" "empty-list?" empty-list-t?)
(def-var! "jolt.host" "cseq-list?" cseq-list?)

144
host/chez/printing.ss Normal file
View file

@ -0,0 +1,144 @@
;; readable printer + output seams — the __pr-str1 / __write / __with-out-str
;; host seams the overlay's pr-str/pr/prn/print/println/*-str family is built on
;; (jolt-core/clojure/core/20-coll.clj).
;;
;; jolt-pr-str (rt.ss) is STR-style: strings render raw. pr-str needs READABLE
;; (pr) style: strings quoted+escaped at every nesting level. This adds the
;; readable renderer; it mirrors jolt-pr-str but quotes strings and recurses into
;; itself, delegating scalars (nil/bool/number/keyword/symbol/char/regex) to
;; jolt-pr-str (already readable for those). The canonical ORDERED printer is
;; still future work — unordered colls render in HAMT order, compared via `=`.
;; inner string escape (no surrounding quotes): " \ newline tab return.
(define (jolt-str-escape s)
(let loop ((cs (string->list s)) (acc '()))
(if (null? cs)
(list->string (reverse acc))
(loop (cdr cs)
(let ((c (car cs)))
(case c
((#\") (cons #\" (cons #\\ acc)))
((#\\) (cons #\\ (cons #\\ acc)))
((#\newline) (cons #\n (cons #\\ acc)))
((#\tab) (cons #\t (cons #\\ acc)))
((#\return) (cons #\r (cons #\\ acc)))
(else (cons c acc))))))))
;; A host shim registers a type's readable rendering via register-pr-readable-arm!,
;; or register-pr-arm! for types whose str and readable forms match (most host types:
;; inst, uuid, record, var, …). Disjoint types, checked before the base cases.
(define jolt-pr-readable-arms '())
(define (register-pr-readable-arm! pred render)
(set! jolt-pr-readable-arms (cons (cons pred render) jolt-pr-readable-arms)))
(define (register-pr-arm! pred render)
(register-pr-str-arm! pred render)
(register-pr-readable-arm! pred render))
(define (jolt-pr-readable-base x)
(cond
((string? x) (string-append "\"" (jolt-str-escape x) "\""))
;; pr renders the infinities / NaN in READABLE form (##Inf reads back), unlike
;; str's "Infinity"/"-Infinity"/"NaN". Applies at every nesting level.
((and (flonum? x) (fl= x +inf.0)) "##Inf")
((and (flonum? x) (fl= x -inf.0)) "##-Inf")
((and (flonum? x) (not (fl= x x))) "##NaN")
;; transients print as a cold tagged type (print-method routes this through a
;; multimethod; the readable fallback renders it directly).
;; forward refs to transients.ss (loaded later) — resolved at call time.
((jolt-transient? x)
(case (jolt-transient-kind x)
((vec) "#<transient vector>") ((set) "#<transient set>") (else "#<transient map>")))
((pvec? x) (if (jolt-print-hash?) "#"
(with-deeper-print
(string-append "[" (jolt-str-join (jolt-limited-vec-strs x jolt-pr-readable)) "]"))))
((pset? x) (if (jolt-print-hash?) "#"
(with-deeper-print
(string-append "#{" (jolt-str-join (jolt-limited-list-strs
(pset-fold x (lambda (e a) (cons (jolt-pr-readable e) a)) '()))) "}"))))
((pmap? x) (if (jolt-print-hash?) "#"
(with-deeper-print
(string-append "{" (jolt-str-join (jolt-limited-list-strs
(pmap-fold x (lambda (k v a)
(cons (string-append (jolt-pr-readable k) " " (jolt-pr-readable v)) a)) '()))) "}"))))
((empty-list-t? x) (if (jolt-print-hash?) "#" "()"))
((cseq? x) (if (jolt-print-hash?) "#"
(with-deeper-print
(string-append "(" (jolt-str-join (jolt-limited-seq-strs x jolt-pr-readable)) ")"))))
(else (jolt-pr-str x))))
(define (jolt-pr-readable-dispatch x)
(let loop ((as jolt-pr-readable-arms))
(cond ((null? as) (jolt-pr-readable-base x))
(((caar as) x) ((cdar as) x))
(else (loop (cdr as))))))
;; *print-meta* support. The var is def'd after this file loads, so capture its
;; cell lazily; jolt-var-get (patched by dyn-binding.ss) honors a `binding`.
(define pr-meta-cell #f)
(define (pr-print-meta?)
(unless pr-meta-cell (set! pr-meta-cell (jolt-var "clojure.core" "*print-meta*")))
(jolt-truthy? (jolt-var-get pr-meta-cell)))
;; The metadata to print before x, or jolt-nil. A var prints as #'ns/name (its
;; {:ns :name} is derived, not user metadata) and a procedure is opaque — skip both.
(define (pr-user-meta x)
(if (or (var-cell? x) (procedure? x)) jolt-nil (jolt-meta x)))
(define (jolt-pr-readable x)
(if (pr-print-meta?)
(let ((m (pr-user-meta x)))
(if (jolt-nil? m)
(jolt-pr-readable-dispatch x)
(string-append "^" (jolt-pr-readable-dispatch m) " " (jolt-pr-readable-dispatch x))))
(jolt-pr-readable-dispatch x)))
;; __pr-str1: render ONE value readably (the overlay's pr-str joins these).
(define (jolt-pr-str1 x) (jolt-pr-readable x))
;; __write: push a string to output. Normally this goes to the current Chez port
;; (so __with-out-str's redirect captures it). When clojure.pprint is active it
;; installs __pprint-write-hook; jolt-write then offers each string to the hook,
;; which routes it column-aware into a clojure.pprint pretty-writer if *out* is
;; bound to one (returns truthy) and otherwise declines (returns nil) so the
;; string falls through to the port. This is the JVM behaviour where core print
;; honours *out*; jolt only needs it for the pretty-printer.
(define jolt-pprint-write-hook jolt-nil)
;; suppressed while __with-out-str captures output to a string port: there the
;; redirect, not *out*, defines where text goes (pr-str / print-str rely on it).
(define jolt-pprint-hook-suppressed (make-thread-parameter #f))
(define (jolt-write s)
(if (and (not (jolt-nil? jolt-pprint-write-hook))
(not (jolt-pprint-hook-suppressed))
(jolt-truthy? (jolt-invoke jolt-pprint-write-hook s)))
jolt-nil
(begin (display s) jolt-nil)))
(def-var! "clojure.core" "__set-pprint-write-hook!"
(lambda (f) (set! jolt-pprint-write-hook f) jolt-nil))
;; clojure.pprint wraps its writing in this so core print routes into the active
;; pretty-writer even under an outer with-out-str (which sets suppressed). A
;; pr-str/print-str nested inside then re-suppresses, so its capture still works.
(def-var! "clojure.core" "__with-pprint-routing"
(lambda (thunk)
(parameterize ((jolt-pprint-hook-suppressed #f)) (jolt-invoke thunk))))
;; __with-out-str: run a jolt thunk with *out* rebound to a string port, return
;; the captured text.
(define (jolt-with-out-str thunk)
(with-output-to-string
(lambda () (parameterize ((jolt-pprint-hook-suppressed #t)) (jolt-invoke thunk)))))
;; __eprint / __eprintf: stderr seams. Flush each write — like the JVM's
;; auto-flushing System.err — so a long-running process (a server that never
;; returns from -main) shows its log lines instead of leaving them in a buffer
;; that only drains at exit.
(define (jolt-eprint s)
(display s (current-error-port))
(flush-output-port (current-error-port))
jolt-nil)
(define (jolt-eprintf fmt . args)
(apply fprintf (current-error-port) fmt args)
(flush-output-port (current-error-port))
jolt-nil)
(def-var! "clojure.core" "__pr-str1" jolt-pr-str1)
(def-var! "clojure.core" "__write" jolt-write)
(def-var! "clojure.core" "__with-out-str" jolt-with-out-str)
(def-var! "clojure.core" "__eprint" jolt-eprint)
(def-var! "clojure.core" "__eprintf" jolt-eprintf)

1007
host/chez/reader.ss Normal file

File diff suppressed because it is too large Load diff

Some files were not shown because too many files have changed in this diff Show more