Shaking out clojure.core.memoize (207 assertions, 0 fail) cleared several
general gaps:
- deref/@ on a deftype or reify implementing clojure.lang.IDeref dispatches to
its deref method (RetryingDelay / make-derefable).
- deftype mutable fields (^:unsynchronized-mutable / ^:volatile-mutable) are
read live: a set! within a method is observed by a later read in the same
invocation, not the entry-time capture. Needed for double-checked locking.
Immutable fields stay let-bound. Field reads rewrite to (.-field inst) with
lexical-shadow tracking.
- def metadata values are evaluated, like Clojure: ^{:k (f)} stores (f)'s
result and ^{:af some-fn} the fn. :tag stays a literal hint.
- try dispatches catch clauses by class in order via the exception supertype
hierarchy; a non-matching value re-throws, an untyped host condition is caught
by a RuntimeException/Exception/Throwable clause. Previously the last clause
won and the class was ignored.
- locking takes a real per-object monitor (recursive Chez mutex) now that
futures/agents/threads share one heap; it was a no-op.
- supers/ancestors reflect a small modeled JVM interface hierarchy, so
(ancestors (class f)) yields Runnable/Callable (core.memoize's arg check).
- AssertionError / Error constructors.
JOLT_FEATURES is gone from the docs: it isn't read anywhere on Chez, and the
reader already includes :clj in its default feature set. RFC 0002's
{:jolt :default} design was reverted in the reader; docs now match the code.
Raises the SCI floor 205 -> 210.
Navigability groundwork from the architecture review — zero behaviour change.
- docs/MODULES.md: the repo map. Area -> directory -> key files -> re-mint?, plus
per-feature touch points (tree-shaking, direct-linking, numeric fl/fx, inlining,
multimethods, deps) and where a given clojure.core fn lives. Answers "where does
X live / what's related to Y" in one read.
- docs/rfc/README.md: index the 7 RFCs; flags RFC 0007's stale "no code yet" status
(direct-linking + tree-shaking shipped) and the undocumented inlining/numeric work.
- CLAUDE.md: document the var-deref calling convention (public defns reached from the
.ss runtime by string lookup aren't dead), the def-var! native pattern, and the
overlay shadowing rule; point at MODULES.md.
- REFACTOR_PLAN.md: the prioritized, risk-tiered plan (working doc for this branch).
Restores the standalone-binary capability the Janet host had. `bin/joltc build
-m NS -o OUT` AOT-compiles an app into a single self-contained executable — the
whole runtime, clojure.core, stdlib and compiler embedded, no Chez install or
jolt source needed at runtime.
Pipeline (host/chez/build.ss, host primitive jolt.host/build-binary driven by
jolt.main's build command): resolve deps, load the entry namespace recording the
app namespaces in dependency order, re-emit each to Scheme, textually inline the
cli.ss runtime load sequence into one flat source + the app + a launcher, then
compile-file -> make-boot-file -> embed the boot as C bytes -> cc-link against
libkernel.a.
Two non-obvious bits: the compile pass runs in a fresh Chez, not the loaded
runtime (regex.ss shadows top-level `error`, which otherwise bakes a broken
reference into the boot); and the launcher installs scheme-start rather than
running -main at top level, since boot top-level forms execute during heap build
before argv is set, so args only reach -main through scheme-start.
Loader: a require of an in-memory namespace with no source file now no-ops, so
AOT'd app namespaces satisfy require in a built binary.
Mode flags (--opt/--dev, default release) are plumbed; the optimization passes
they gate come in a later stage. RFC 0007 has the design. Gated by `make
buildsmoke`.
Rename src/jolt -> stdlib (the runtime-loaded layer; jolt-core stays the
seed-baked layer) and update the loader / emit-image / doc paths. Drop dead
code: the spike/ experiments, the duplicate clojuredocs-export.edn (json moves
to tools/), the Janet-era jolt.http binding, and the orphaned
persistent_vector.clj whose ns/path didn't even match.
Strip porting residue from comments and docstrings across host/chez, jolt-core,
stdlib, tests, and docs: internal issue ids, "Phase N" markers, and the "vs
Janet" historical exposition, leaving present-tense descriptions and the real
JVM-Clojure semantic contrasts. Same pass over the corpus suite labels. The seed
is unchanged (docstrings/comments aren't emitted), so the self-host fixpoint and
corpus are untouched.
Port tools/spec_coverage.py off the dead janet probe to bin/joltc and regenerate
coverage.md; drop the dead :host/janet rule from certify.clj and regenerate the
conformance profile. Add docs/host-interop.md (the JVM shims and how to register
your own host class from a library) and a writing-style note in CLAUDE.md.
Stabilize the four racy concurrency corpus cases (future-cancel and agent
send/send-off): give the future a sleeping body and the agent a slow action, so
cancel reliably catches an in-flight future and deref reliably reads the
pre-update snapshot. They certify deterministically now, so drop their :flaky
allowlist entries and the orphaned legend.
Bring the docs in line with the actual implementation now that Chez is the sole
substrate.
Deleted the migration/spike/handoff artifacts that only documented the Janet
era or the port effort: the port plan, phase-0 and foundational-runtime spike
writeups (+ the stray root-level copy), the self-hosting design notes, the
architecture-refactor plan, and spike/chez/RESULTS.md.
Rewrote the current reference docs against the Chez facts: building-and-deps and
tools-deps (no jpm/build step — bin/joltc off the checked-in seed, deps via
jolt.deps into ~/.jolt/gitlibs), libraries (SQLite is built-in jdbc.core over
libsqlite3, not a Janet driver), the conformance/spec test-flow docs (the Chez
corpus runner + certify, no .janet harnesses), and the transient / type-hint /
seed-overlay design notes (Chez representations: mutable transients, flat
copy-on-write vectors, HAMT maps, the seed/overlay twin). Fixed the README
collections line (vectors aren't 32-way tries) and added the ffi/transient gate
targets. rfc 0001's numerics open-question is resolved (the Scheme tower).
Renamed the built-in HTTP adapter to jolt.http.server only (dropped the
ring-janet.adapter alias — a Janet-era name).
* Bind *command-line-args* after the deps-image cache swap (jolt-4mui)
Under whole-program (deps-image cache active), `jolt -m NS ARG` dropped ARG:
run-main set *command-line-args* on the current ctx, but a cache HIT then
replaced ctx with the saved image (via `set ctx cached`), whose *command-line-
args* was whatever got baked when the image was saved. The stale binding won at
`(apply NS/-main *command-line-args*)`, so -main ran with the wrong (usually
default) args — silently, for any optimized -m program.
Move set-command-line-args to AFTER the cache swap so it binds on the final ctx.
Repro/regression in deps-cache-args-test.janet: first run builds the image
(arg "first"), second run (cache hit) must echo "second", not the baked "first".
* docs: RFC 0003 — phm is a HAMT, sorted colls a red-black tree
The transients RFC described phm as "bucket-based copy-on-write" and mused about
"if it ever becomes a HAMT" — it is one now (jolt-684u), and sorted maps/sets are
a red-black tree (jolt-0hbr). Update the deviation/future-work notes accordingly.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Running a program is a closed world — every namespace is required, then it runs
to completion — so make it direct-link by default (inlining, record shapes, the
inference's specialization), and for a -m/-M entry auto-enable the whole-program
cross-namespace inference pass. A decomposed multi-namespace program was ~3.7x
slower than the same code in one namespace purely because per-namespace
inference can't see a caller in a not-yet-loaded namespace; this closes that for
the common case with no flags and no hints.
Interactive modes (repl, -e, nrepl-server) stay indirect/open — they have to let
you redefine vars, which direct-linking seals against. Opt-outs:
JOLT_NO_DIRECT_LINK forces the open path even for a program run (hot-reload,
runtime redefinition); JOLT_NO_WHOLE_PROGRAM keeps direct-linking but per-ns;
JOLT_DIRECT_LINK / JOLT_WHOLE_PROGRAM still force-on. Namespaces required inside
-main (after the batch pass) fall back to per-ns inference.
The success checker (RFC 0006) rides on the inference for free, but a casual
program run shouldn't spam type warnings just because it now direct-links, so its
default-on is suppressed when direct-linking was auto-enabled (:direct-link-auto?);
an explicit JOLT_DIRECT_LINK or JOLT_TYPE_CHECK still turns it on. whole-program-
test and devirt-test opt their per-ns baseline out of the new auto-default.
Docs: RFC 0005 gains 'Compilation modes and defaults' + 'Cross-namespace
inference'; RFC 0004 documents cross-ns/param hints; self-hosting-compiler and
--help updated. Full gate green.
Update the status, strictness levels, and open questions to reflect what
landed: bounded unions (jolt-pz5), user-fn domains behind
JOLT_TYPE_CHECK_USER (jolt-zo1), precise file:line:col (jolt-fqy), and the
checker folded into one inference walk that piggybacks on direct-link
specialization (on by default there, opt-in in plain builds). Align the
error-reporting example with the actual output format.
0005 proposes replacing the ad-hoc inference lattice with one recursive
structural type (a struct carries its field types, a vector its element type,
recursively), so a lookup returns its field's type and nested access is typed
end to end. It unifies :struct tracking with field tracking, subsumes the
current inference phases, and is the soft-typing (HM + a dynamic top) design:
structural types + core-fn type schemes, solved by lattice join with :any as
top instead of unify-or-fail. Includes the depth cap for termination and an
explicit design-problems section.
0006 (follow-up, depends on 0005) reuses the inference as a loose type checker
in the success-typing discipline (Dialyzer): report only PROVABLY-wrong code
(a concrete type in an operation's throwing error-domain), accept everything
ambiguous, never a false positive. Curated error-domain table, strictness
levels (off/warn/error), clear located messages, and the soundness boundaries
(closed-world, macros, unions).
Builds on the ^:struct keyword-lookup hint:
- ^TypeName for records. A tag naming a defrecord/deftype now resolves to the
struct fast path: record instances are tables tagged :jolt/deftype (not
:jolt/type), so a raw keyword get is correct for them. A new host contract fn
record-type? detects a record by its ->Name constructor; a non-record tag
(^String, ^long, ...) is ignored, as before.
- (get m :k) and (get m :k default) now get the same inlined keyword lookup as
(:k m): the representation guard fast path when unhinted, and the bare get
when the subject is ^:struct/^Record. A variable/number/string key still
falls through to core-get. The two call shapes share one emitter
(emit-kw-lookup).
- JOLT_CHECK_HINTS=1 turns a violated hint into a clear runtime error (naming
the local and key) by keeping the guard and throwing on the tagged arm. It is
off by default with zero cost to normal builds (a hinted lookup still emits a
bare get), and is part of the image-cache fingerprint. This is the answer to
"a lying hint is silent": opt into checking during development.
- Docs: RFC 0004 records the design, soundness contract, and measurements; the
reader spec gains S12b (hints are semantically transparent; jolt recognizes
^:struct and ^Record as lookup-optimization assertions).
There is no Clojure keyword equivalent for "plain map / fast keyword access"
(Clojure hints are class names), so ^:struct stays a jolt-specific flag,
analogous to ^:dynamic.
Verified: conformance 335/335 in all three modes and the full jpm test pass; a
seeded ray-tracer render is byte-identical hinted vs unhinted; the struct-hint
test covers record hints, the get-form, inline propagation, and the checked-mode
error. Full render with hints holds at 13.3s -> 10.9s (1.22x).
Fixed arities now throw Clojure's ArityException shape — 'Wrong number of
args (N) passed to: name' — on any count mismatch; variadic arities on fewer
than their fixed params. The compiled path already enforced fixed arities via
janet's native fn check and multi-arity dispatch; this adds the check to the
interpreter's single-arity closures (the oracle was silently dropping extra
args and giving a raw tuple-index error for missing ones) and guards the
compiled single-variadic wrapper's minimum. Messages carry the fn name when
there is one. 16 spec rows; the update.cljc suite row flipped green (4703 ->
4704).
Enforcement exposed that seq-to-map-for-destructuring had drifted: the spec
row called the 1-arity fn with two args, and the body silently dropped a
trailing unpaired element. Replaced with the canonical Clojure 1.11 version
(even pairs build the map, a single trailing element passes through — so
(f {:b 2}) kwargs calls work — and an unpaired key throws).
Also: transients RFC notes tuple support from the seed-shrink rounds.
Pins down what a transient is in Jolt (tagged table over a native Janet
array/table, canonical-keyed for maps/sets), where behavior deviates from the
JVM (O(n) transient/persistent! edges with O(1) native ops between, no
owner-thread check — same as Clojure 1.7+, transient-of-list leniency), and
the three reasons the machinery is seed-resident rather than a migration
candidate: it IS the mutation kernel, it sits under the seed's own dispatch,
and the value layer is declared irreducible. Exists so the kernel-shrink
ladder (jolt-tzo) doesn't revisit transients every round.
jolt no longer satisfies :clj in reader conditionals. The shortcut was a
measured net liability: :clj branches carry JVM interop and JVM-specific
test expectations jolt fails, and they shadowed :default branches jolt
passes. A/B over the suite: clj,default = 4967 assertions / 4324 pass / 119
errors; jolt,default = 5069 / 4470 / 81 (+146 pass, -38 errors, +8 clean
files). Baselines raised to 4470/86.
Matching is now by CLAUSE order like Clojure — the first clause whose key is
in the feature set wins (#?(:default 5 :clj 6) is 5 everywhere); the old code
scanned for :clj first, then :default, regardless of position.
Foreign clj-targeted libraries are a property of the LOADING CONTEXT, not the
platform: reader-features-set! opts a load into a compatibility set, and the
SCI bootstrap/runtime tests load SCI under ["jolt" "clj" "default"] (its
.cljc selects implementations via :clj with no :jolt branches).
JOLT_FEATURES remains the process-wide override.
RFC 0002 records the decision with the measured data; spec 02-reader S18 is
now normative (clause order, documented feature set, per-context override).
Reader tests updated to the portable set + an opt-in round-trip.
RFC 0001 proposes a normative, implementation-independent Clojure language
spec (the reader, evaluation model, special forms, data types, seq/laziness
contracts, namespaces/vars, and the portable clojure.core surface) to the
standard of R7RS/Racket — Clojure has none, and every alternative
implementation re-derives semantics from the reference and folklore. The
spec is executable-first: every numbered normative statement cites its
conformance test or is marked UNVERIFIED.
docs/spec/ carries the front matter (conformance terms, entry format, host
classification), the special-form catalog with worked normative entries for
if and let*, the core-library entry format with worked entries for first,
reduce, and parse-uuid, and a generated coverage dashboard over the 694-var
ClojureDocs inventory (tools/spec_coverage.py cross-references the surface
against jolt's interned+resolvable vars and the test suites).
Measured baseline: 380 implemented+tested, 154 implemented-untested, 35
portable-but-missing (filed), 22 resolvable-but-not-interned (filed — seed
fns invisible to resolve/ns-publics), rest classified host/JVM/concurrency.
