jolt is Clojure, so a dep on org.clojure/clojure is always satisfied
intrinsically — the "skipping unsupported coordinate" warning on its
:mvn/version coordinate was just noise. Other unsupported mvn coords
still warn.
The nREPL server bound its loopback socket through libc process symbols,
which don't carry the socket entry points on Windows — they live in
ws2_32.dll and aren't in joltc.exe's export table, so --nrepl-server died
with "no entry for socket". Load ws2_32 before the foreign bindings there,
call WSAStartup once, and use Winsock's closesocket and int-typed
recv/send. Also fix SOL_SOCKET/SO_REUSEADDR, which the old macos-only
check got wrong on Windows.
Bake a version string into the self-contained binary at build time (from
$JOLT_VERSION, else git describe) and expose it via jolt.host/jolt-version:
--version / -V print it, and it shows in --help, the repl banner, and the
nREPL startup line. Dev runs off bin/joltc read it from git describe.
Add -e to the help output.
Auditing the remaining cts baseline for R7 exposed real contract gaps hiding
among the model residue — all fixed to reference behavior:
- stale no-ratio-era stubs: numerator/denominator now work over jolt's exact
rationals (non-ratio is the Ratio cast failure); rational? includes decimals
- casts and pending: peek/pop demand an IPersistentStack (pop nil is nil),
realized? demands an IPending (a plain list/range throws), transient demands
an editable COLLECTION (non-colls throw; the RFC 0003 sorted/list/seq
superset keeps the copy-on-write fallback), empty on a plain record throws
- nil and empties: (nth nil i) is nil, (nth nil i d) is d, a nil index is NPE,
keys/vals of anything empty are nil, (conj nil) is nil
- lookups: contains? on a string is index-only (other keys IAE), get on an
array is lenient (nth still throws), a VECTOR invocation has nth semantics
(([1 2] 5) throws — call position and jolt-invoke both)
- into only transients editable collections; a PersistentQueue/sorted target
folds through conj (RT's IEditableCollection split)
- numbers: number?/num accept BigDecimal, quot/rem throw on an Infinite/NaN
quotient, even?/odd? demand integers
- ordering: keywords compare namespace-first with nil first (Symbol.compareTo)
- misc: run! honors reduced, eval self-evaluates non-form values, intern
demands an existing namespace, counted? excludes strings, seqable? includes
arrays, shuffle rejects maps, sort-by rejects a collection comparator,
when-let demands one binding pair, case*/deftype*/letfn*/reify*/& are
special symbols
Two mis-certified corpus rows fixed (they threw on the JVM too and hid in the
tolerated bucket): a raw \d string escape and duplicate literal map keys.
SPEC.md gains the baseline-traceability section: every one of the 146
remaining suite failures maps to a documented divergence (integer-box,
no-single-float, RFC 0003 transients, seq/chunking model, stm-refs,
parse-uuid strictness, vec-array adoption). cts baseline 5955 -> 6042 pass,
5 errors, 30 namespaces. 9 JVM-certified corpus rows.
Arithmetic and comparisons lowered to raw Chez ops, so an operand outside
Chez's tower (BigDecimal) crashed with a raw condition, and Chez contagion
leaked: (* 1.0 0) gave exact 0 where the JVM gives 0.0, (* ##Inf 0) gave 0
instead of ##NaN, (/ 1 0) raised an untyped error.
One seam now (host/chez/seq.ss): call position emits jolt-n* macros with the
both-Chez-numbers fast path open-coded; value position folds through the same
binary ops. Anything outside the tower falls to per-op slow hooks that
java/bigdec.ss extends, so bigdec arithmetic works in every position (the old
static-only :bigdec typing limitation is gone). JVM rules patched into the
fast path: a double operand wins, an exact zero divisor throws
ArithmeticException while a double zero divisor yields Inf/NaN, quot/rem/mod
cover ratios and doubles, min/max return the original operand with NaN
winning, a nil operand is NPE and a non-number CCE, zero-arg -// throw
ArityException at runtime instead of failing expansion.
Also: with-precision now binds *math-context* and bigdec results round with
real RoundingMode semantics (UNNECESSARY throws; division rounds to precision
instead of throwing); rationalize goes through the shortest decimal print
like BigDecimal.valueOf (the identity stub is gone); ratios coerce to bigdec
like Numbers.toBigDecimal; min/max int-literal operands no longer coerce to
flonum in the numeric pass.
Perf neutral: fib and seq benches unchanged (the fast path is two type checks
the optimizer folds); hinted fl/fx paths untouched. 19 JVM-certified corpus
rows; cts baseline 5614->5730 pass, 192->88 errors, 84->79 baselined
namespaces.
An unknown reader tag produced the reader's internal form
{:jolt/type :jolt/tagged :tag :#foo :form bar}, which tagged-literal? didn't
recognize and which leaked as a raw map when printed:
(tagged-literal? (read-string "#foo bar")) => false ; want true
(pr-str (quote [#foo bar])) => "[{:jolt/type :jolt/tagged ...}]"
Both the data path (rdr-construct-tag) and the compile path (emit-quoted) now
build a real tagged-literal for a tag with no registered reader, like Clojure's
*default-data-reader-fn*, so tagged-literal? / :tag / :form / printing all work.
clojure.edn reads raw forms through a separate __read-form-raw path and applies
:readers/:default itself, so it is unaffected.
Re-mint (backend + reader are seed sources); prelude byte-identical, image only.
make test green (selfhost holds, 0 new/stale), +2 unit rows.
Running the whole rewrite-clj test suite (159 tests) surfaced seven more bugs;
with these it passes 3377/0/0. Each is a general jolt/JVM divergence:
- *out* was pinned to the startup stdout port, so (.write *out* …) escaped a
with-out-str capture (z/print writes via *out*). It now resolves the live
current-output-port, like print/__write, so a redirect is seen.
- nth / assoc past the end of a vector or seq threw a bare Chez error (class
:object). Throw IndexOutOfBoundsException, matching the JVM.
- A number's .toString(radix) ignored the base. Render in the base, lowercase
(rewrite-clj rebuilds 0xff / 0377 / 2r1001 through it).
- A required namespace's own :as aliases leaked into its requirer: the loaded ns
form compiles while (chez-current-ns) is still the requirer, so ce-scan-requires!
registered the loaded ns's aliases under the wrong ns and clobbered a same-named
alias there. Register an (ns NAME …) form's aliases under NAME.
- A quoted collection dropped its metadata; now it keeps USER metadata (drops the
reader's :line/:column/:file), like a Clojure quoted constant.
- enumeration-seq only did (seq e); it now drives a java.util.Enumeration through
hasMoreElements/nextElement, and StringTokenizer implements them.
Regressions: corpus rows (with-out-str/*out*, nth/assoc bounds, toString radix,
quote metadata, enumeration-seq) certified against JVM; a smoke fixture for the
alias leak (a required ns's alias must not leak). tools.reader + rewrite-clj added
to docs/libraries.md. make test green.
Running the rewrite-clj test suite under jolt exposed six bugs, each fixed here:
- `for`/`doseq` `:let` bindings never went through `destructure`, so a
destructuring pattern (`:let [{:keys [y]} x]`) hit `let*` raw and failed to
compile. Emit `let`, like Clojure.
- `with-open` couldn't close a deftype/defrecord that implements a `close` method
(java.io.Closeable / AutoCloseable, e.g. tools.reader's readers) — `__close`
only knew jhost readers and map `:close` fns. Dispatch a record's `close`.
- A deftype/defrecord method param named like a field didn't shadow the field
(the field's let-binding wrapped the params). Params now shadow, as in Clojure.
- A deftype whose simple name collided with a built-in host class clobbered it in
the global ctor table, so `(java.io.PushbackReader. …)` built tools.reader's
same-named deftype. Register deftypes/built-ins by FQN, don't let a deftype
overwrite a built-in's simple name, and qualify a bare `(Name. …)` to the
deftype's FQN only in the ns that defined it.
- `clojure.walk` was lazy over a non-list seq (missing `doall`), so a walk whose
fn has side effects read stale state. Make it eager, like Clojure.
- `Character/isWhitespace` used an ASCII-only check that missed U+2028 and other
Unicode whitespace. Use the JVM's Unicode set (minus the no-break spaces it
excludes).
Regressions: corpus rows (for-let destructure, method-param shadow, walk eager,
isWhitespace), a unit row (with-open closes a record), and smoke checks (the
class-name collision, run in a fresh -e process so the deftype doesn't leak).
One divergence remains unfixed: a submatch from a losing regex alternation branch
leaks when the winning branch has a quantified group (a bug in the vendored
irregex engine, not jolt) — tracked separately.
A :jolt/native spec can now carry a :static archive; `jolt build` links it
into the executable, so the app calls the C code with no shared object on the
target. --dynamic (or :jolt/build {:dynamic-natives true}) keeps the old
runtime load-shared-object behavior; a spec with no :static is unchanged.
The cc link force-loads the archive (-force_load on macOS, --whole-archive on
Linux) and exports the executable's symbols (-rdynamic on Linux) so the baked-in
symbols resolve via (load-shared-object #f) + foreign-procedure at startup. Build
step 1 evaluates the app's foreign-procedure forms in-process, so a static
archive is preloaded there as a throwaway shared object to resolve them.
The distributed self-contained joltc has no external cc/Chez but must build these
apps, so it now bundles the Chez kernel (libkernel.a + scheme.h) and the launcher
source and re-links a custom stub with the archives baked in — needing only a
system cc, no Chez. run/repl skip static-only specs (nothing to load); keep a
:darwin/:linux candidate to use such a lib interpreted.
Adds static-native-smoke (cc path) to ci and a static phase to the joltc
self-build smoke (distributed path).
repl-form-complete? entered the :regex state on '#' but only consumed the
'#', so the opening '"' was then read by the :regex handler as the CLOSING
quote. The regex body got scanned in :code state, and any delimiter or quote
inside it (a group like #"(a)", a char class #"[0-9]+") threw off the
paren/string count — so a one-line regex form was judged incomplete and the
REPL hung waiting for continuation lines. Consume the '#"' together.
Adds a self-checking predicate test (test/chez/repl-reader-test.clj, run via
joltc so jolt.main resolves) and an end-to-end regex REPL case in smoke.sh.
The REPL evaluated one line at a time, so a form split across lines
(e.g. `(+` then `1 2)`) raised instead of waiting. The read loop now
accumulates lines until delimiters are balanced — skipping string,
char, regex and comment context — printing a `... ` continuation prompt
for each extra line.
Reader/runtime errors rendered as Chez's "attempt to apply
non-procedure #[chez-pmap...]" instead of their real message. Two causes:
jolt-throw raised the thrown value raw. When a throw crossed the host
`eval` boundary, Chez re-wrapped the non-condition into a compound
condition whose message extraction applies the value, losing the message
and crashing on ex-info's empty-map :data. jolt-throw now raises a
&jolt-throw condition wrapping the value; catch (lowered to `guard`),
jolt-report-uncaught and jolt-render-throwable unwrap it back via
jolt-unwrap-throw, so ex-data/ex-message and the backtrace tag survive.
Every reader/post-prelude EOF-throw site used `(empty-pmap)` (with
parens), applying the empty-map value as a procedure and crashing during
ex-info construction before jolt-throw ran. Fixed to `empty-pmap`.
Re-minted the seed; smoke 23/23, unit 574/574.
^C to a running `joltc --nrepl-server` aborted with "thread does not
own mutex" because the accept-loop thread absorbed SIGINT in its foreign
accept() call, where Chez can't run the keyboard-interrupt handler, and
run-main-pump's tight condition-wait loop wasn't interruptible anyway.
Block SIGINT in the primordial thread before starting the server so the
accept loop inherits a blocked mask, park in a single interruptible
condition-wait via the new park-until-interrupt, and run registered
shutdown hooks (newest-first, each isolated) from the keyboard-interrupt
handler before (exit 0). The stop fn now drops .nrepl-port via the new
jolt.host/delete-file seam — clojure.java.io/delete-file doesn't exist
in Jolt and silently no-ops, so .nrepl-port was never removed.
^D (EOF) exits cleanly in canonical mode but some terminals and editors
don't deliver it, leaving the user stuck. Accepting :repl/quit or :exit
as the first form of a line gives a reliable keyword exit that works
everywhere. The check parses the line with read-string rather than
checking the evaluated value, so a nested value that happens to print
as the keyword can't trigger an exit.
Two pre-existing issues in the nrepl command, exposed when #269 moved the accept
loop into a future.
jolt.nrepl/start was invoked inside (future ...), so binding the socket happened
on a background thread. A bind failure such as the port already being in use was
captured into a future that nothing derefs and silently swallowed, leaving the
main thread parked in run-main-pump forever with no server and no error. start
now binds the socket synchronously before returning, so that failure propagates
to the caller and the process exits with a visible message. Only the blocking
accept loop runs on a worker thread.
There was also no shutdown path: the accept loop ran forever and the listen
socket was never closed. start now returns a stop fn that breaks the accept
loop, closes the socket to free the port, and removes .nrepl-port. main.clj runs
run-main-pump and then calls the stop fn, so a stop-main-pump (from a glimmer app
on quit, or from nrepl-evaluated code) shuts the server down cleanly and the
process exits.
Match babashka's spelling: the nREPL server now starts with
`bin/joltc --nrepl-server [port]` instead of `bin/joltc nrepl`. Port
parsing and JOLT_NREPL_PORT are unchanged.
Also wire up --help/-h to print usage (previously only the no-arg
invocation did), and fix the usage listing to show the real flag.
Smoke now asserts --help mentions --nrepl-server. Docs updated to match.
Profiling jolt-i5if showed <=60-bit arithmetic is already native-fast; the real
general overhead in the run/-e/-m path is var resolution. Every var reference
compiled to (var-deref ns name), which builds + hashes a fresh "ns/name" string
and does a hashtable lookup per access (~45ns). The var cell is interned and
def-var! mutates it in place, so caching the resolved cell is sound under
redefinition.
Generalize the devirt per-site cache-cell mechanism to var value references: a
ref inside a fn resolves its cell once into the def's closure, then reads it via
var-cell-deref (a field read after the first). var-cell-deref is the cell-based
var-deref — binding-aware (dynamic vars + *ns* still resolve) and lenient on an
unbound root (a forward-declared var doesn't throw, unlike jolt-var-get).
Gated by a runtime flag: ON for runtime-compiled code (compile-eval.ss), OFF for
the seed mint and AOT build (emit-image.ss) so the seed stays a byte-fixpoint --
prelude.ss is unchanged, only image.ss picks up the new backend. ~5x on a
var-ref-heavy loop (1058ms->205ms); ~1.2x on test.check (its generators are more
deftype/dispatch-bound than var-deref-bound). No C/FFI.
Corpus rows pin redefinition / dynamic binding / forward ref through a cached
ref. make test + shakesmoke green, selfhost holds, SCI 211/218, certify 0-new.
Profiling the test.check distribution/large-sample slowness (jolt-i5if): the
hot path is the SplitMix PRNG, dominated by 64-bit mix arithmetic, and the
bitwise ops (bit-and/or/xor/not, shifts) were NOT in the backend native-ops
table — so (bit-xor a b) compiled to a var-deref through the variadic overlay
(__bit-xor) instead of a direct call, the way +/-/* already emit.
Map bit-and/or/xor/not to the Chez bitwise-and/ior/xor/not primitives (inlined
to native code; a non-integer operand now errors like the JVM instead of being
silently truncated) and the shifts to a direct helper call. bit-and-not stays on
its overlay — its only Scheme impl is 2-arg, so a value-position arity-3 use
would mis-emit.
mix-64 arithmetic 2.7x faster, raw split+rand-long 2.4x, gen/vector ~1.4x. The
remaining gap is the bignum-vs-native-long floor (~20x, substrate) plus the
generator machinery (deftype/fn dispatch, separate). Corpus rows added for value
position, bit-not, apply, and a full-64-bit unsigned shift.
Close clojure.spec.alpha's remaining gaps — its conform/explain/describe/multi-spec
suite (clojure.test-clojure.spec, multi-spec) now passes fully.
- (get reify k) / (:k reify) routes to a reify's clojure.lang.ILookup valAt. spec
reifies fspec/regex specs as ILookup and reads (:args spec) off them, so before
this instrument never saw the args spec.
- A failed numeric comparison reports the JVM class: a nil operand is
NullPointerException, a non-number is ClassCastException (was an opaque :object
condition). conform-explain checks the thrown class.
- A quoted / macro-form #inst / #uuid literal constructs its Date/UUID value, like
the JVM reader (which builds it at read time). emit-quoted was emitting the raw
tagged form, so #inst "1939" and #inst "1939-01-01T00:00:00.000-00:00" weren't =.
- An anonymous fn reports class clojure.lang.AFunction$fn (the $fn marker), so
spec's fn-sym returns ::s/unknown for it, matching the JVM's ns$fn__N.
- A fn with & {:as m} kwargs accepts a trailing map (Clojure 1.11): (f :a 1 {:b 2})
and (f {:a 1}) both bind m, by merging an odd trailing map over the pairs.
- A thread responds to .getStackTrace (empty — jolt does TCO).
clojure.test-clojure.instr does not fully pass: its ::caller assertions need the
calling fn's stack frame, which TCO erases (an inherent host divergence, like the
JVM keeping tail frames).
make test green (+4 corpus rows, 0 new divergences), shakesmoke byte-identical.
Re-mint (backend emit-quoted + the destructure macro).
Two general fixes shaken out by clojure.test.check's own suite (its splittable
PRNG mixes 64-bit longs and binds locals named min/max).
- *unchecked-math* now wraps arithmetic a macro emits. The analyzer rewrote a
bare (+/-/*) to its wrapping unchecked-* under *unchecked-math*, but a macro's
syntax-quote produces clojure.core/* (qualified), which was skipped — so e.g.
test.check's mix-64 multiply grew to a bignum instead of a 64-bit long. The
rewrite now also fires on the clojure.core-qualified form.
- A local binding named like a bare-emitted native op no longer shadows it. ops
where native-ops maps the name to itself (+ - * / < > min max …) emit as the
bare Scheme name; a local `max` emitted the same token, so
(fn [max] (clojure.core/max …)) called the param. munge-name now prefixes such
locals, like reserved words (derived from native-ops so they can't drift).
make test green (+1 corpus row, 0 new divergences), shakesmoke byte-identical.
One re-mint (analyzer + backend).
jolt modelled letfn as a special form directly, so (macroexpand-1 '(letfn …))
returned the form unchanged. Clojure's letfn is a macro that expands to letfn*,
and macroexpansion tooling (tools.macro, tools.analyzer) depends on that — its
special-form handlers key on letfn*, not letfn.
Split it the Clojure way:
- letfn* is now the special form (analyzer), taking flat name/fn-form pairs
[name1 fn1 name2 fn2 …] — the letrec :let lowering is unchanged.
- letfn is a macro (00-syntax) turning each (name [params] body*) spec into a
name + (fn name [params] body*) binding, so it expands to letfn*.
So (macroexpand-1 '(letfn [(f [x] x)] (f 1))) now yields
(letfn* [f (fn f [x] x)] (f 1)), and clojure.tools.macro passes its whole suite
(macrolet / symbol-macrolet / mexpand-all). Listed in docs + site.
make test green (+1 corpus row, 0 new divergences), shakesmoke byte-identical.
One re-mint (analyzer + the letfn macro); selfhost holds.
Completes the JVM long-compatibility gap so clojure.test.check (and the
property-based suites built on it, e.g. data.codec) run on jolt.
A ^long is 64-bit but a Chez fixnum is only 61-bit, so the backend's fast fx
comparison / quot / min / max / inc / dec ops raised on a full-width long (one
from the PRNG or wrapping arithmetic). They now go through the jolt-l* macros
(host/chez/seq.ss): the fx fast path when the operands ARE fixnums, the generic
op otherwise — so e.g. ((fn [^long a ^long b] (< a b)) Long/MAX 1) is false, not
an error. Arithmetic +/-/* keep the raw fx ops (under *unchecked-math* they're
already the wrapping unchecked-*).
Also fixes unsigned-bit-shift-right: it was an arithmetic (sign-propagating)
shift, now a logical shift over the 64-bit two's-complement window, so
(unsigned-bit-shift-right -1 1) is 2^63-1 like the JVM.
Result: test.check 1.1.3 loads and runs (generators, quick-check, shrinking);
data.codec's base64 property suite passes (12/12 defspecs; the 2 deftests check
clojure.lang.IFn$OLLOL, a JVM primitive-fn interface, N/A). Both added to
docs/libraries.md + the site.
re-mint (backend/seed). make test green (+3 corpus rows, 0 new divergences,
numeric gate updated to the jolt-l* ops), shakesmoke byte-identical.
clojure.core's unchecked-* (and +/-/*/inc/dec under *unchecked-math*) are long
ops that WRAP on overflow; jolt's checked arithmetic is arbitrary-precision and
its unchecked-* were plain non-wrapping (+ x y), diverging from the JVM. Now they
truncate to the low 64 bits as a signed long, matching Clojure:
(unchecked-add 9223372036854775807 1) => -9223372036854775808
(unchecked-multiply 9223372036854775807 …) => 1
- host/chez/seq.ss: jolt-wrap64 + binary jolt-unc{add,sub,mul,inc,dec,neg}2 and
the variadic clojure.core/unchecked-* fns (def-var!'d in natives-seq.ss, where
def-var! is bound). The overlay's plain unchecked-* defns are removed.
- backend lng-ops: unchecked-+/-/* emit the wrapping jolt-unc* helpers (the
raising fx ops can't wrap on Chez's 61-bit fixnums); unchecked-inc/dec too.
- *unchecked-math* is honored: the analyzer reads it (jolt.host/unchecked-math?)
and rewrites +/-/*/inc/dec to their unchecked-* for the rest of a file that
(set!)s it, like the JVM.
- jolt->fx: a ^long value that overflows the 61-bit fixnum range passes through
as an exact integer instead of erroring (a full-width long from wrapping math).
Also adds Long/bitCount / numberOfLeadingZeros / reverse and Math/getExponent /
scalb (test.check's splittable PRNG uses them).
This lets clojure.test.check load and run quick-check on jolt. re-mint (analyzer/
backend/overlay are seed sources). make test green (+6 corpus rows, 0 new
divergences, numeric gate updated), shakesmoke byte-identical.
clojure.core.contracts (over core.unify) now runs its whole suite on jolt —
14/14 across contracts/constraints/with-constraints/provide tests. Two general
gaps fixed:
- Symbol and Keyword now report IFn (and Fn/Runnable/Callable) in the modeled
class hierarchy, so a (class x)-dispatched multimethod with an IFn method
matches a symbol or keyword, like the JVM (both implement IFn — they're
callable). core.contracts' funcify* dispatches on (class constraint) and a
bare predicate symbol must hit the IFn arm. Runtime, no re-mint.
- A live Var value spliced into a form by a macro (defcurry-from resolves a var
and emits (~v l r)) now compiles: analyze treats a var-cell form as a
:the-var reference by ns+name, the same node as (var ns/name), mirroring the
existing spliced-namespace (~*ns*) case. analyzer.clj + host-contract.ss,
re-mint (prelude stays byte-identical; only the analyzer image changes).
Listed in docs/libraries.md + the site.
make test green (+2 corpus rows, 0 new divergences), shakesmoke byte-identical.
A macro body can now read &form (the call form) and &env (a map of the in-scope
local symbols), like Clojure. This is what core.logic's matche/defne use to tell
a pattern symbol that names an enclosing local from a fresh pattern var — so
locals-membero and the recursive checko in `matches` now compute correctly. The
suite reaches 535/2/0 (the last two are constraint reification ORDER, where the
constraint set is right but it is spliced from a set whose iteration order differs
from the JVM — a host set-ordering divergence, not a bug).
&form/&env are clojure.core dynamic vars bound around each expander call rather
than prepended params, so the macro calling convention is unchanged and the mint
stays consistent (the seed prelude is byte-identical; only the analyzer carries
the env into form-expand-1). macroexpand-1 passes an empty env.
corpus.edn: the ~@ unquote row is now a boolean compare (a bare clojure.core/
unquote-splicing symbol evaluates to an unbound var, not the symbol).
Follow-on to the core.logic relational-engine work. These clear every crash in
core.logic's constraint-logic-programming and unifier layers (33 errors -> 0) and
most of the value mismatches; the suite goes 504 -> 523 passing assertions. All
are general gaps, not core.logic-specific.
- symbols intern their ns/name strings (JVM Symbol.intern .intern()s them): two
separately-read `?a` symbols now share one name-string object. core.logic's
non-unique lvars compare names by identity (via (str sym)), so without this a
term's lvar and a constraint's lvar built from different `?a` reads never matched
and constraints silently never fired.
- (str x) of a single arg returns its rendering directly instead of copying through
string-append, and a symbol stringifies to its (interned) name — JVM (str x) is
x.toString(). Needed for the identity comparison above.
- a clojure.core-qualified special form dispatches correctly: syntax-quote
namespace-qualifies a macro like letfn to clojure.core/letfn (matching Clojure,
where it's a macro), and the analyzer now maps that back to the special form
instead of treating it as an invoke of a nil var. core.logic's fnc/defnc emit
(clojure.core/letfn ...). Re-mint.
- (disj nil ...) is nil (JVM), instead of crashing in the set path — core.logic's
constraint store does (disj (get km v) id) where the get can be nil.
corpus.edn: 4 JVM-certified rows. make test + shakesmoke green, 0 new divergences,
self-host fixpoint holds.
Running clojure/core.logic's own suite surfaced a batch of general jolt gaps.
None are core.logic-specific; each is a language/host behavior that was wrong or
missing. With these, the core relational engine (unify, run/fresh/conde,
conso/membero/appendo, reification to _0/_1, lcons) runs; the remaining failures
are in core.logic's constraint-logic-programming and finite-domain layers
(tracked separately).
- analyzer: accept the list-member dot form (. target (method args)), sugar for
(. target method args). Re-mint.
- identical? is reference identity (eq?), not value equality. It was aliased to =,
which infinite-loops when a deftype's .equals short-circuits on (identical? this o)
(core.logic's Substitutions) and is wrong for distinct equal collections.
- jrecs use a deftype's declared hashCode/equals/equiv for map/set keying instead
of structural field comparison, so metadata-wrapped keys still match (core.logic
keys substitutions on lvar id, ignoring metadata).
- meta/with-meta dispatch to a deftype's clojure.lang.IObj meta/withMeta methods
when present, so metadata threaded through the type's own assoc/withMeta survives
(previously kept in an identity side-table the reconstructed instances didn't share).
- coll?/seqable? on a deftype require IPersistentCollection (cons) or ISeq (first);
ILookup(valAt)/Indexed(nth)/Counted(count)/Seqable(seq) alone no longer qualify,
matching the JVM.
- syntax-quote resolves a bare symbol to the compile ns's own def before
clojure.core, so a name the ns excluded and redefined (core.logic's == after
:refer-clojure :exclude) qualifies correctly in macro output.
- reader: record literals #ns.Type{...} / #ns.Type[...] expand to the map->/->
factory call.
- structmap API: defstruct/create-struct/struct-map/struct/accessor (map-backed,
insertion-ordered). Re-mint.
- .hashCode on strings/symbols (Java String.hashCode, Symbol Util.hashCombine);
Class.isInstance; java.util.Collection.contains over vector/list/set;
clojure.lang.RT/nextID and clojure.lang.Util hash/hasheq/equiv/identical statics.
corpus.edn: 8 JVM-certified rows. unit.edn: a Counted+Seqable deftype is coll?=false
(was a stale expectation encoding the old behavior).
jolt maps were HAMTs with hash iteration order; Clojure keeps small maps as
PersistentArrayMap (insertion order), converting to PersistentHashMap past a
threshold. Map literals, array-map, assoc, into/transient, merge, zipmap,
select-keys, update-keys/vals, frequencies and group-by now iterate in insertion
order for <=8 entries, matching the JVM. hash-map and >8-entry maps stay hash
order; sets stay hash order.
The pmap record gains an order field (the insertion-order key list, or #f once
hashed); the HAMT still backs the values so equality/hash/lookup are unchanged.
pmap-fold visits an array-mode map last-to-first so the runtime's cons-accumulate
idiom reconstructs insertion order without touching its many call sites, and
hash-mode output stays byte-identical; pmap-fold-fwd visits in order for the few
sites that build a value directly. Transient maps track insertion order and
promote to hash past max(8, source-count), matching TransientArrayMap.
The hash-map native-op retargets to a hash-order builder so (hash-map ...) stays
hash-ordered while {...} literals are ordered; syntax-quote builds maps via the
hash builder (Clojure expands `{...} to apply hash-map). The core overlay map
builders seed from {} instead of (hash-map) to keep order.
Threshold is 8 for any key (the keyword exception in newer Clojure isn't in
1.12.5). honeysql now passes 832/0/0; 19 JVM-certified corpus rows added.
A devirtualized protocol call resolved its impl with devirt-resolve on EVERY call
— but the tag/proto/method are compile-time constants, so the resolved fn is a
runtime constant (closed world). That per-call find-protocol-method (three
hashtable lookups) was the cost: on mono-dispatch, dispatch was ~75% of the time
(ablation: same arithmetic direct-call 166ms vs dispatch 673ms).
Resolve once. When emitting a direct-link def, each devirt site gets a fresh cache
cell, bound to #f in a let wrapping the def (so it persists across calls and is
shared by every invocation); the site resolves into it on first use ((or cell
(let ((_f (devirt-resolve ..))) (set! cell _f) _f))) and reuses it after — the
inline cache the JVM gets for free. First call still passes the real receiver, so
the Object/host-tag fallback (devirt-resolve) is unchanged.
mono-dispatch 673ms -> 214ms (~3.15x), 47.5x -> ~15x JVM, near the 166ms
direct-call floor. run-devirt.ss gains the cached-path checks (cell present, 1st
call caches + 2nd reuses, both == dispatch). make test / shakesmoke green, selfhost
holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
A record-or-nil (a protocol method whose impls return a record in one branch and
nil in another, or an `if` over a ctor and nil) now types as a NILABLE record
instead of widening to :any. A nilable record still bare-indexes its field reads
(jrec-field-at falls back to jolt-get on nil), but some?/nil? do NOT fold on it, so
a runtime guard is preserved — and inside (if (some? x) ..) / (if x ..) the then-
branch narrows x to the non-nil record, so its reads bare-index AND unbox there.
This is what lets the bounced ray type without a hint: scatter returns
ScatterResult-or-nil (Metal absorbs some rays), and the consumer reads
(:ray scattered) only under (if (some? scattered) ..). The narrowing proves
scattered non-nil there.
lattice: :nil type; :nil ∨ struct -> nilable struct, ∨ anything else -> :any;
nilability is contagious through a struct join, which also now preserves :type when
both sides agree (needed so a record ∨ its nilable self stays that record).
truthy-type?/field-type/pred-on treat a nilable struct as maybe-nil. types: nil
literal -> :nil; an `if` whose test is (some? x)/(nil? x)/x narrows the nilable
local x in the proven branch.
Ray tracer with NO hints: 38.4s -> 23.9s (~1.6x) — hit-sphere now types fully
(0 jolt-get, 57 jrec-field-at, 38 fl-ops), identical to the hand-hinted build.
run-narrow.ss gate, incl. the load-bearing check that the nil case still takes the
else branch (the guard is not folded away). make test / shakesmoke green, selfhost
holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
A protocol method whose impls all return the same record type has a monomorphic
return. collect-pm-rets! scans the unit's (register-(inline-)method ..) forms,
infers each impl fn's return type, and joins them per method; call-ret-type then
types a (method recv ..) call as that record, so a field read off the result
bare-indexes — e.g. (:ray (scatter m ..)) reads off a Ray. A disagreeing impl
joins to :any and keeps the generic path.
run-protoret.ss: a method with all-record impls bare-indexes + unboxes the field
read; a mixed-return method (one impl returns a number) stays generic. make test /
shakesmoke green, selfhost holds, 0 new divergences.
Foundation for auto-typing record values that flow through protocol dispatch. Does
not yet move the ray tracer: its scatter returns ScatterResult-or-nil (Metal
absorbs some rays), and the nil widens the join to :any — typing a nullable return
soundly needs flow-sensitive narrowing (a guarded (some? x) proves non-nil), filed
separately.
Co-authored-by: Yogthos <yogthos@gmail.com>
The whole-program fixpoint collects a self-recursive call's arg types into the
fn's own params. When a recursive call threads a param straight through unchanged
(same arg, same position — e.g. ray-cast passing `hittables` to itself), that arg's
type is the param's own current type: :any until external callers determine it. And
:any is absorbing, so collecting it pinned the param at :any forever — the type a
caller supplied (a vector of records) was lost, and the fn's field reads stayed
generic.
Skip a same-position pass-through arg in the self-recursion collection (contribute
the join identity). It can't add information — param i ⊇ param i is trivial — so
dropping it is sound; the param is still constrained by every external caller and
by any non-pass-through recursive arg. Applies to both self-recursion paths: a
`defn` recursing through its var, and a named fn literal recursing via its
self-local.
This is why ray-cast's `ray` typed (its recursion passes a fresh ray) but
`hittables` didn't (passed through). With the fix, hittables keeps its
vec<Sphere> element type, so hit-all's reduce element — and hit-sphere's reads —
type without any hint: ray tracer 38.4s -> 31.3s (~1.23x) with no annotations.
run-wp.ss: a recursive fn threading a vec param through keeps its element type.
make test / shakesmoke green, selfhost holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
A record field tagged ^double now reads back as a flonum and feeds the numeric
pass, so hintless arithmetic over those fields lowers to fl-ops — the leaf-numeric
analog of the ^Vec3 nested-field hints. Combined with the whole-program :double
param inference, a vec3-dot over a ^double-fielded record unboxes end to end with
no per-fn hints.
records.ss: a ^double field tag passes through resolution, and the ctor (and a
mutable-field set!) coerce a ^double field to a flonum — JVM primitive-field
parity (jolt returned an exact 1, not 1.0, before), and what makes reading the
field back as :double sound for an fl-op.
types.clj: field-type-from-tag maps "double" -> :double, and a keyword/get lookup
whose result is :double annotates the node :num-read :double. numeric.clj reads
that annotation and classifies the field read as a :double operand, so the
enclosing arithmetic specializes — the read itself keeps its jrec-field-at/jolt-get
emit.
run-fieldnum.ss gate: ctor coercion (int field -> flonum), field-field arithmetic
emitting fl*/fl+, and an untagged field staying generic. make test / shakesmoke
green, selfhost holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
The closed-world fixpoint (#226) flowed record types across fn boundaries; this
adds a numeric refinement so a hintless fn whose every call site passes a flonum
has its param unboxed to fl-ops, no ^double hint needed.
Lattice gains :double, a flonum refinement of :num: two doubles join to :double,
a double joined with anything else widens to :num — so a param is :double only
when every contributing value is a flonum, which is what makes the fl-op sound.
infer types a flonum literal and flonum arithmetic (+ - * / min max inc dec over
double/int-literal operands) as :double, and the fixpoint joins those across call
sites and return types like any other lattice value.
The bridge to the existing hint-directed pass is a synthetic [param :double]
nhint: wp-infer! stashes the :double params separately from the structural seeds,
and run-passes injects them as nhints before numeric/annotate, so the fl-op
emission and the exact->inexact entry coercion (a no-op on a proven flonum) apply
unchanged.
Sound subset only: :double, never :long — an untyped integer can be a bignum and
fx-ops would overflow/diverge from jolt's arbitrary precision. So an integer
caller leaves a param generic; an escaped fn (unknown callers) keeps :any.
run-numwp.ss gate: cross-fn :double propagation incl. through a flonum-returning
helper, the integer-caller and escape negatives, and the full run-passes path
emitting fl* + entry coercion. make test / shakesmoke green, selfhost holds, 0
new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
The devirtualized protocol call emitted find-protocol-method on the inferred
record tag, but a record can satisfy a protocol via an Object/host-tag default
rather than a direct impl — find-protocol-method on its own tag misses that,
while protocol-resolve walks to the default. So a record relying on
(extend-protocol P Object ...) resolved under ordinary dispatch but applied #f
under devirt and crashed. Closed-world opt builds only; the gate previously
covered just direct inline/extend-type impls so it shipped green.
Emit devirt-resolve, which tries the static tag and falls back to
protocol-resolve on a miss — same fast path, correct regardless of how the
record satisfies the protocol. Mirrors jrec-field-at falling back to jolt-get.
The receiver binds to one temp so it feeds the resolve and the application
without double-evaluating a side-effecting arg 0.
Also widen the whole-program fixpoint to :any on hitting the iteration cap: a
non-converged pre-fixpoint is more specific than the least fixpoint, so seeding
it would be unsound. Not reached in practice (~2 passes); a defensive floor.
run-devirt.ss gains an Object-default case. make test / shakesmoke green,
selfhost holds, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
When the inference types a keyword-lookup receiver as a record — it carries the
field-order :shape and :hint :struct from the whole-program fixpoint — the back
end reads the field by its declared slot via jrec-field-at instead of jolt-get.
That skips the jolt-get case-lambda, the dispatch fn, and the field-key
hashtable lookup, leaving a jrec? check + a static-index vector-ref.
jrec-field-at falls back to jolt-get when the receiver isn't the expected record
(a map downgraded by dissoc, or a value the inference mistyped), so it stays
correct if the static type is wrong. Only the no-default form takes the bare
path (a declared field is always present).
Sound only for non-nil records: a self-recursive param that can be nil (e.g.
binary-trees check-tree, whose untagged child is nil at leaves) types :any and
keeps jolt-get — the whole-program fixpoint demotes it. The target is non-nil
record params, like a Vec3 dot product (~5% there; boxed-flonum arithmetic
dominates the rest, a separate numeric lever).
run-fieldread.ss gate: emitted form uses jrec-field-at at the right slot and
matches jolt-get for each declared field; a non-field key and a default-arg form
keep the generic path. make test / shakesmoke green, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
When the inference proves a protocol call's receiver is one record type, the
back end resolves the impl by that static tag (find-protocol-method) instead of
routing through the protocol var -> jolt-invoke -> protocol-resolve, which
re-derives the tag and walks the type table. Same table lookup, minus the
var-deref, the rest-cons, and the receiver-type computation.
Fires only on a monomorphic site: a megamorphic receiver joins to :any and
carries no :devirt-type, so it keeps ordinary dispatch (the dispatch bench is
unaffected). The annotation comes from the whole-program fixpoint typing a
reduce/HOF element or a ctor return as a specific record.
Modest on the dispatch benchmarks (~6% on mono-dispatch) — float boxing in the
reduce accumulator dominates there, a separate numeric lever — but it removes
the dispatch overhead wherever a typed receiver is known.
run-devirt.ss gate: emitted form uses find-protocol-method, and evaluating it
matches ordinary dispatch for an inline impl, an extend-type impl, and the
non-devirt path. make test / shakesmoke green, 0 new divergences.
Co-authored-by: Yogthos <yogthos@gmail.com>
Re-derive each app fn's param types from its call sites under --opt, so a
record type flows across fn boundaries: a ctor's return reaches a callee
param, and a typed vector's element reaches a HOF closure's param. The back
end can then bare-index field reads and devirtualize protocol calls at those
sites (it reads the resulting :hint/:devirt annotations; consuming them is
separate work).
This rebuilds the inter-procedural driver the Janet host had — the API
(infer-body/reinfer-def) survived the rehost but nothing drove it, and the
record-shapes/protocol-methods registries were empty stubs.
- records.ss: populate record-shapes (ctor key -> fields/tags/type, resolving
nested record field tags) and protocol-methods (method var -> [proto method])
registries at deftype/defprotocol load time; jolt.host accessors materialize
them.
- passes/types.clj: wp-infer! runs a closed-world fixpoint joining call-site
arg types into callee params; reinfer-def re-seeds each def at emit. Self-
recursive calls and fn-level recur are collected so a recursive fn's params
are constrained by its recursion, not just external callers — else a param
the recursion widens (e.g. binary-trees check-tree, whose untagged child can
be nil) would be unsoundly typed non-nil. A fn used in value position keeps
:any params (callers unknown). Megamorphic sites join to :any.
- build.ss: analyze all app forms and run the fixpoint before per-form emit.
- run-wp.ss: gate (cross-fn propagation, escape soundness, self-recursion).
make test / shakesmoke green, 0 new divergences, selfhost holds.
Co-authored-by: Yogthos <yogthos@gmail.com>
A loop test like (or (>= i cap) (> ... 4.0)) desugars to
(let* [g (>= i cap)] (if (truthy? g) g (> ... 4.0))) and the whole thing was
wrapped in jolt-truthy? because returns-scheme-bool? only looked at :const and
:invoke nodes, not the let*/if an or/and expands to. The wrapper defeats Chez's
branch inlining on the hot loop edge.
Make returns-scheme-bool? recursive over :if (both branches bool), :let (body
bool, tracking which bound locals hold a Scheme boolean), and :local (in that
set). or/and over bool-returning ops then read as Scheme booleans and the outer
wrapper drops. Still sound: eliding only when the value is provably #t/#f — a
jolt-nil is a truthy record in Chez, so a false positive would be a real bug, and
the recursion only proves bool-ness through ops already known to return one.
No bench regression; the win lands on hinted float loops where the branch, not
boxed arithmetic, is the cost.
Co-authored-by: Yogthos <yogthos@gmail.com>
defprotocol emitted one variadic (fn [this & rest] (protocol-dispatch P m this
(list->cseq rest))) per method, so every protocol call — even a no-extra-arg one
like (area s) — consed a rest list, wrapped it in a cseq, var-deref'd
protocol-dispatch, and jolt-invoke'd it (consing again). On mono-dispatch that was
2.07GB of allocation, ~65% of the benchmark.
Emit one fixed-arity clause per declared arglist instead. The 1/2/3-param arities
call positional protocol-dispatch{1,2,3}, which resolve the impl (by record tag,
reify method, or host-tag extension — factored into protocol-resolve) and apply it
directly; no rest-list, no seq round-trip. The dispatchN entry points are in the
native-op table so the shim calls bind straight to the records.ss procedures
rather than var-deref. 4+ params fall back to the variadic protocol-dispatch.
mono-dispatch 1.5s/2.07GB -> 0.69s/280MB; dispatch 26x -> 12.2x, mono-dispatch
111x -> 51x vs JVM. 5 new corpus rows pin multi-arity methods, host-type args,
and protocol-method-as-value against JVM Clojure.
Co-authored-by: Yogthos <yogthos@gmail.com>
Records were a jrec holding an alist of (kw . val) conses: ~113B/node, built
fresh per construction, field reads a list scan. Replace that with a shared
per-type descriptor (tag + field keywords + an eq?-keyed keyword->index table)
plus a flat per-instance value vector and an extension map for any non-field
keys assoc'd on (jolt-nil when there are none). Construction now allocates one
vector instead of a cons chain and a field read is an index lookup. binary-trees
construction allocation drops 2.085GB -> 1.19GB.
That alone barely moved binary-trees wall-time: profiling showed the read loop,
not allocation, dominates, and the read loop's own allocation came from (nil? l)
lowering to (jolt-invoke (var-deref "clojure.core" "nil?") l), which conses its
args every call. Add nil?/some? to the backend native-op table so they inline to
jolt-nil?/jolt-some? (and drop the truthy wrapper, like the other predicates).
check-tree's read loop goes from 1.476GB allocated to zero; binary-trees 18.9x
-> 9.7x vs JVM. The remaining gap is the field-read dispatch chain (jolt-c3mw).
Two JVM divergences fixed along the way, both certified:
- dissoc of a declared field downgrades a record to a plain map (was kept as a
record); an extension key still drops cleanly.
- map->R keeps extension keys (was dropping anything outside the declared basis).
16 new corpus rows pin assoc/dissoc/count/keys/seq/=/hash/extension-field
behavior against JVM Clojure.
Co-authored-by: Yogthos <yogthos@gmail.com>
* Make the benchmark harness build optimized binaries on Chez
bench/run.sh was Janet-era: it invoked a 'jolt' binary and set
JOLT_DIRECT_LINK/JOLT_WHOLE_PROGRAM, none of which exist on Chez, where
'joltc run -m' runs fully unoptimized (direct-link and inline default off). So
the suite was measuring jolt's unoptimized path.
run.sh now compiles each benchmark to an optimized AOT binary (joltc build
--direct-link --opt) and times it against JVM Clojure on the same portable
source, auto-detecting the Chez kernel dev files like build-smoke.sh. Adds
bench/deps.edn so joltc resolves the namespaces, NO_JVM to skip the reference.
mandelbrot.clj dropped its jolt.png require so the JVM reference can run it; the
picture demo moved to mandelbrot_png.clj (jolt-only). README scorecard refreshed
with current Chez numbers and the two-regime read (compute ~8-10x substrate floor;
dispatch/alloc ~120-330x architectural gaps the passes don't touch). Stale
'jolt -m' header lines point at bench/run.sh.
* Emit direct self-calls for named-fn self-recursion
A self-recursive call to a named fn compiled to (jolt-invoke fib ...) instead of
a direct (fib ...): emit-invoke handled a :local callee only when it was NOT a
known proc, so a :local that IS in *known-procs* (the letrec-bound self-name) fell
through to the :else jolt-invoke branch. Now a :local known proc emits a direct
Scheme call — no jolt-invoke, no per-call arg-list consing; case-lambda handles
arity.
fib 30: 63.3ms -> 4.7ms (faster than JVM Clojure's 7.1ms; was 9x slower). The win
is on every self-recursive non-loop fn, including the compiler's own. No semantic
change — selfhost holds, make test green, shakesmoke/buildsmoke byte-identical.
Re-mint (backend is seed). Corpus rows pin self-recursion across fixed/multi/
variadic arities.
* Intern no-ns keywords without per-call allocation
(keyword #f name) built a fresh combined-key string (string-append) on every
call just to do the intern-table lookup — ~80 bytes of garbage per (:kw x), map
literal, keyword arg, etc. A no-ns keyword now interns in a table keyed by the
name string directly, so a lookup of an already-interned keyword is one
hashtable-ref with no allocation. The ns table keeps the combined key; both share
the keyword-t khash (equal-hash of the combined key) so hash values are unchanged.
Small time win on its own (the field-read dispatch dominates hot record code —
see jolt-unx4) but removes per-call keyword allocation everywhere. Runtime .ss,
no re-mint; identity/=/hash unchanged, make test green.
* Fast record field reads: single eq? scan, skip the get-arm walk
(:field rec) / (get rec :field) lowers to (jolt-get rec kw), which walked the
get-arm list to reach the jrec arm, then did jrec-has? + jrec-lookup — TWO linear
scans, each comparing keys through the generic jolt=2 equality dispatcher. Field
keys are interned keywords, so:
- jrec-key=? compares a keyword query by eq? (jolt=2 only for non-keyword keys),
- jrec-ref does ONE scan (vs has?+lookup) and runs a deftype's ILookup valAt only
when the field is genuinely absent (present-nil still returns nil, not default),
- jolt-get-dispatch checks jrec? first, skipping the get-arm walk for the hottest
get target. jrec-lookup/jrec-has? (used by =, contains?, etc.) get the fast
compare too.
binary-trees 135x->18.9x, dispatch 121x->26.4x, mono-dispatch 327x->108x vs JVM.
Runtime .ss (collections.ss + records.ss), no re-mint; make test + shakesmoke +
buildsmoke green, record get/assoc/keys/=/count semantics unchanged.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
* Reader records source line/column on list forms
The reader stamps 1-based :line/:column metadata on every list form (plus
:file when load-jolt-file is reading a file), and jolt.host/form-position
reads it back so the analyzer's :pos scaffold finally gets real data. A
left-to-right cursor counts newlines over the delta between successive forms,
so it stays O(n). Vector/map/set literals are untouched (their metadata is a
runtime value the analyzer would have to wrap in with-meta); empty () can't
carry meta. ^meta now merges onto the position keys instead of clobbering them.
Re-mint is byte-identical (the backend doesn't emit :pos), so this is a pure
scaffold for the error-location work that follows.
* Report source location on uncaught errors
Each top-level form records its source position (thread-local) before it
compiles+evals, and cli.ss jolt-report-uncaught appends 'at file:line:col'
when an error propagates out. Covers joltc -e, joltc run <file>, and
load-string — every interpreted path. Top-level granularity, one set per
form; deeper frames come from the Phase 2 frame walk.
Runtime .ss only, no re-mint.
* Clojure stack traces via source registry + native frame walk
A direct-link build emits (jolt-register-source! short-name ns name file line)
once per fn def — at definition time, so zero per-call cost. On an uncaught
error the reporter walks Chez's native continuation frames (jolt-throw captures
the live continuation via call/cc; host conditions carry their own
&continuation), maps each frame's procedure name through the registry, and
prints a Clojure backtrace 'ns/name (file:line)'. Wired into both the cli and a
built binary's launcher.
Frames are keyed by the short munged fn name Chez actually reports (emit-fn's
letrec self-binding), not jv$ns$name; a cross-namespace collision degrades to
the bare frame name rather than a wrong attribution. The analyzer carries the
original form's position through defn macroexpansion onto the def node.
Calling a non-fn now throws a catchable ClassCastException (via jolt-throw)
naming the operator, instead of a raw Chez error.
Caveats (documented in source-registry.ss): names map only in direct-link/AOT
closed-world builds — the open-world -e/repl/run path falls back to the
top-level location; and pervasive TCO erases tail-call frames, so a mapped
trace shows only the non-tail spine. JOLT_DEBUG_FRAMES dumps raw frame names.
Re-mint (analyzer + backend); prelude byte-identical (direct-link off during
mint). Corpus rows certified, build-smoke asserts the trace.
* Propagate source position through macroexpansion
hc-expand-1 now carries the macro call form's :line/:column onto the top of a
list expansion that has none of its own (merged under any meta the macro set),
so errors and stack traces in macro-generated code point at the call site —
Clojure parity. The analyze recursion re-expands inner macros, so each level's
top form picks it up, matching the reference compiler. (meta (macroexpand-1
'(when x y))) now reports the call-site line.
A direct-link fn defined through a user macro (build-app's defguarded) registers
with a real line, so build-smoke's trace assertion covers macro-defined fns.
Runtime .ss (host-contract.ss) — no re-mint; selfhost holds.
Phase 3's optional items are deferred: :line-in-ex-data has no clean consumer
(it would pollute ex-data, break = and printing, and positions already surface
via the trace + top-level location), and Chez source-object emission is a large
backend change the jv$-name registry already sidesteps.
* Review fixes: registration key, thread-locals, debug flag timing
- Register a fn under the name Chez actually reports for its frame, not the def
name: a named fn literal whose name differs from the def (def foo (fn bar …))
is framed as 'bar', and an anonymous fn def (def foo (fn …)) as jv$ns$foo.
Both previously registered under the def name and so never appeared in traces.
- rdr-source-file / rdr-pos-cursor are thread parameters, so concurrent compiles
(futures, core.async) don't clobber each other's file/line attribution.
- Read JOLT_DEBUG_FRAMES at call time: a built binary evaluates top-level forms
at heap-build time, where a load-time getenv is always unset.
Re-mint (backend + reader); prelude byte-identical, selfhost holds.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Review found (< 1M 2M) worked but (min 1M 2M) threw — incoherent. Wire min/max
the same way as the other ops: value-position jolt-min/jolt-max shims (new in
seq.ss, added to core-value-procs) and call-position via bd-spec/bd-ops ->
jbd-min/jbd-max.
min/max return the original operand by value, not a coerced copy, matching
Clojure: (min 1M 2.0) -> 1M, (max 1M 2.0) -> 2.0, (min 1.50M 2M) -> 1.50M; a tie
keeps the second operand ((max 1.5M 1.50M) -> 1.50M). bigdec mixed with a flonum
in call position stays in the documented :any/contagion gap (value position
handles it). Re-mint; 6 more JVM-certified rows.
A direct (+ 1.5M 2.5M) emits a raw Chez + that rejects the bigdec record. Rather
than guard every arithmetic call site (measured 2-4x on unhinted fixnum loops),
let the analyzer dispatch where it can prove the type.
jolt.passes.numeric seeds a :bigdec kind from the M-literal and flows it through
let/loop/if like the existing :double/:long kinds; an arithmetic/comparison invoke
whose operands are all bigdec (integer literals allowed) gets :num-kind :bigdec.
The back end (bd-ops + emit-numeric) lowers those to the bigdec.ss engine
(jbd-add/-sub/-mul/-div, jbd-lt?/…, jbd-zero?/-pos?/-neg?, jbd-quot/-rem).
Zero cost on non-bigdec code: with no bigdec literals present the kind never
arises, so emission is byte-identical — the re-mint leaves prelude.ss unchanged,
only image.ss (the compiler) moves. Gaps (filed): a bigdec mixed with a flonum in
call position, and a bigdec the analyzer types :any, still hit the raw op and
throw; use value position or a literal-typed let.
Re-mint (numeric/backend are seed sources). 16 JVM-certified corpus rows.
The host/chez directory mixed jolt's own runtime (value model, seq, reader,
vars, ns, multimethods) with the shims that emulate the JVM: java.* / javax.*
classes, clojure.lang interfaces, and the host-class registry they hang off.
Move that JVM-emulation layer into host/chez/java/ so it reads as a distinct
unit instead of being interleaved with the platform runtime.
Moved (content unchanged): host-static, host-static-methods,
host-static-classes, host-class, dot-forms, records-interop, byte-buffer,
io, io-streams, inst-time, java-time, bigdec, natives-queue, natives-str,
natives-array, math, concurrency, async, ffi.
The load paths in rt.ss/cli.ss and the build.ss runtime manifest are updated
to point at java/; the build inliner follows the (load ...) strings, so the
AOT path needs no other change. All runtime shims, no seed source touched
(the three .clj edits are doc comments), so no re-mint.
Gate green: make test (selfhost fixpoint, certify 0-new, sci 211, infer),
shakesmoke (4 apps byte-identical).
Shaking out clojure.core.memoize (207 assertions, 0 fail) cleared several
general gaps:
- deref/@ on a deftype or reify implementing clojure.lang.IDeref dispatches to
its deref method (RetryingDelay / make-derefable).
- deftype mutable fields (^:unsynchronized-mutable / ^:volatile-mutable) are
read live: a set! within a method is observed by a later read in the same
invocation, not the entry-time capture. Needed for double-checked locking.
Immutable fields stay let-bound. Field reads rewrite to (.-field inst) with
lexical-shadow tracking.
- def metadata values are evaluated, like Clojure: ^{:k (f)} stores (f)'s
result and ^{:af some-fn} the fn. :tag stays a literal hint.
- try dispatches catch clauses by class in order via the exception supertype
hierarchy; a non-matching value re-throws, an untyped host condition is caught
by a RuntimeException/Exception/Throwable clause. Previously the last clause
won and the class was ignored.
- locking takes a real per-object monitor (recursive Chez mutex) now that
futures/agents/threads share one heap; it was a no-op.
- supers/ancestors reflect a small modeled JVM interface hierarchy, so
(ancestors (class f)) yields Runnable/Callable (core.memoize's arg check).
- AssertionError / Error constructors.
JOLT_FEATURES is gone from the docs: it isn't read anywhere on Chez, and the
reader already includes :clj in its default feature set. RFC 0002's
{:jolt :default} design was reverted in the reader; docs now match the code.
Raises the SCI floor 205 -> 210.
Shaken out getting ring-defaults (and its ring-core/anti-forgery/session stack)
to load and serve static resources on jolt. All general fixes, all runtime:
- Class/forName throws a catchable ClassNotFoundException for a class jolt can't
back (it returned a broken truthy value for any name, and crashed on use). Lets
the common (try (Class/forName "optional.Dep") (catch ...)) probe libraries use
to detect an absent dependency work — e.g. ring's joda-time check.
- deps: reconcile native libs (and source roots) in one step, deduped by library
identity, instead of the ad-hoc distinct at each call site. An app pulling two
libs that declare the same shared object (libcrypto via both jolt-crypto and
http-client) now includes and loads it once.
- io: a File answers getProtocol ("file") / getFile so resource-serving
middleware that expects io/resource to hand back a file: URL works; the
classloader gains getResources (every source root holding the resource).
- clojure.string/replace accepts a char match/replacement, like the JVM.
JVM-certified corpus rows for the Class/forName and string/replace behavior.
Finishes core.match — its full test suite (115/115) now passes, including the
two patterns the earlier work left out:
- Regex-literal patterns. A #"…" now reads as a regex VALUE (Clojure parity: the
reader constructs the Pattern, so a macro receives a regex, not jolt's tagged
form), and the analyzer compiles a regex value to the same :regex IR leaf via
its source. emit-quoted handles a quoted regex; a regex value carries the
java.util.regex.Pattern host tag so extend-protocol/instance? dispatch on it.
- Primitive-array patterns. A ^Type hint's :tag is now the SYMBOL (e.g. `ints`),
matching the JVM, so core.match's array-tag lookup engages the array
specialization (alength/aget). jolt's :tag consumers already tolerate a symbol
(hc-cell-num-ret normalizes; tag->nkind/def-meta handle both).
Also: a library-conformance directive in CLAUDE.md, and the supported-libraries
list (docs + site) simplified to one-line entries — a listed library is assumed
to work fully, so no tallies or feature enumerations. core.match + transit-jolt
added to the list.
Seed change (reader/backend/30-macros) -> re-minted; the rest runtime. JVM-
certified corpus rows; the stale `symbol hint -> :tag` divergence is dropped from
the allowlist (jolt now matches the JVM). make test + shakesmoke green.
Four general gaps, shaken out by loading clojure.spec.alpha:
- Special forms were shadowable by a same-named macro. analyze-list
macroexpanded before checking special forms, so a ns that redefs def/and/or
(spec excludes them via :refer-clojure :exclude) made a bare def resolve to
the macro instead of the special form, breaking every defn after. Now a head
in the special-form set is never macroexpanded, matching the reference
macroexpand1 isSpecial check.
- reify dropped all but the last arity of a multi-arity protocol method (spec
reifies (specize* [s]) and (specize* [s _])). The macro keyed methods by name
and overwrote; now it groups arities into one multi-arity fn.
- reify instances did not implement IObj: with-meta threw and (instance?
clojure.lang.IObj r) was false. Every Clojure reify carries metadata. with-meta
now copies the reify to a fresh identity (shared method table) and keys its
meta; instance? IObj/IMeta is true for any reify. This was the registry bug —
spec's with-name returned nil for specs, so get-spec missed.
- (set! (. Class field) val) was rejected. spec toggles
clojure.lang.RT/checkSpecAsserts this way; the analyzer now lowers it to a
jolt.host/set-static-field! call over a mutable-statics table, and a plain
Class/field read consults that table.
Also: .name/.getName on a Namespace and .ns/.sym on a Var (spec's ns-qualify /
->sym). analyzer + reify are seed sources (re-minted). spec.alpha now does
valid?/conform/cat/keys/explain-str/check-asserts. tick.alpha.interval-test still
needs time-literals data readers (separate).
Duration (ISO PT.. toString, between, full arithmetic), Period (between with
borrow, P.. toString, normalized), full Month/DayOfWeek enums (named constants,
print as their name — fixes the Phase-1 raw-jhost print), Year, YearMonth
(2020-02 toString, leap, atDay/atEndOfMonth), ChronoUnit (between/getDuration)
and ChronoField. The temporal machinery on the Phase-1 types now works with
ChronoUnit/ChronoField: (.plus t n DAYS), (.until t1 t2 unit), (.get/.getLong
t field), (.with t field v), (.isSupported ..), (.truncatedTo ..).
Analyzer: (. Class method args) with a class target lowers to a static call
(Class/method args) instead of mis-dispatching as an instance call on the arg
— matches JVM; needed by cljc.java-time.year. Seed re-minted; selfhost holds.
The Phase-2 cljc.java-time namespaces load; tick.core advances to a Phase-3
zone gap. 10 corpus rows certified vs JVM. make test + shakesmoke green, 0 new
divergences, data.json stays 138/139.
