jolt/REFACTOR_PLAN.md

Architecture refactor plan

Working doc for the spike/arch-refactor branch. Goal: make the codebase easier to understand and safely modify — optimized for both human and LLM maintainers. Derived from a four-part review (compiler core, Chez runtime, build/tooling, top-level organization) plus call-graph analysis of the compiler.

The codebase is in better shape than its 58-file runtime suggests: rt.ss is a real load-order map, the passes/ split is clean, filename prefixes encode tier order, and the Makefile self-documents the gate. The work below targets the few genuine liabilities.

The re-mint constraint shapes risk

• Seed sources (jolt-core/jolt/*.clj, jolt-core/clojure/core/*.clj, host/chez/reader.ss): a change needs make remint + the byte-fixpoint make selfhost. Behaviour-preserving refactors are verified by selfhost, but every touch costs a re-mint. Higher risk, batch them.
• Runtime .ss + runtime-loaded .clj (build.ss, emit-image.ss, main.clj, the dispatcher shims, …): no re-mint; verified by make test + make shakesmoke. Lower risk.
• Docs / renames of non-seed files: zero behaviour risk.

Tier 0 — navigability (zero risk, highest LLM value, do first)

1. docs/MODULES.md — the repo map. One table: area → directory → key files → re-mint? Plus a per-feature "touch points" list for cross-cutting features (tree-shaking, direct-linking, numeric fl/fx, multimethods, the deps resolver). Collapses the current 3–4-source lookup ("where does X live?") into one read.
2. docs/rfc/README.md — index the 7 RFCs (number, title, status), mirroring docs/spec/README.md.
3. Document the shipped-but-undocumented compiler features: IR inlining and fl*/fx* numeric lowering (the passes/ pipeline). A short "compilation passes" doc or an RFC, referenced from RFC 0007.
4. CLAUDE.md — one "Invariants you must preserve" block: the var-deref calling convention (the compiler is reached from .ss by string lookup), the def-var! native-binding pattern, and the seed dual-home shadowing rule (today only in seed-overlay-registry.md).
5. First-line purpose comment on every .ss (audit the laggards) and note the reader's surprising location (host/chez/reader.ss, Scheme, re-mint applies).

Tier 1 — runtime structure (no re-mint; verify with make test + shakesmoke)

6. Registry pattern for the core dispatchers — the biggest runtime win. jolt-pr-str, jolt-pr-readable, jolt=2, jolt-get, jolt-class, jolt-hash are extended by set!-capture-and-rebind across ~8 files (e.g. jolt-pr-str is wrapped 9× in 8 files). The registry pattern already exists in tree (register-str-render!, register-instance-check-arm!) and is strictly better: type-disjoint arms gathered in one walker, load-order-independent, greppable. Add register-pr-arm! / register-eq-arm! / register-get-arm! / register-hash-arm! / register-class-arm!; convert the ~35 set! sites to one-line registrations. Eliminates "read 8 files to understand one dispatcher."
7. Extract a host/chez/dce.ss module for tree-shaking. The DCE logic is split across emit-image.ss (the dce-* helpers + record producer) and build.ss (bld-shake-all + the manifest splice). Pull every dce-* def + bld-shake-all (→ dce-shake) into dce.ss; give the record a named accessor API (dce-rec-keep?/-fqn/-refs/-str) instead of (vector-ref r 0..3); split bld-shake-all's five jobs (root-seed, edge-build, BFS, bail-detect, partition) into named steps. This is recently-added code and the loosest contract in the build.
8. Tag the runtime manifest. bld-emit-runtime decides "splice shaken core" / "drop compiler" by substring-matching (load "…seed/prelude.ss") strings. Replace the manifest with tagged entries ((prelude)/(image)/(compile-eval)/ (load path)) and dispatch on the tag — removes a silent-failure coupling.
9. build-binary options map. It takes 8 positional args ending in two bare booleans (direct-link? tree-shake?); a swap compiles and misbehaves silently. Pass one options map from main.clj; new flags become additive.
10. dynamic-wind the compiler-global set/reset in build-binary so set-optimize!/set-direct-link! always revert (today they leak on a build error; harmless for the CLI, a trap for any in-process caller).
11. Re-split host-static-{,statics,objects}.ss on a real axis — static methods/fields vs host object classes — instead of the current "grew too big" chain (one file is 502 lines, the largest non-seed runtime file). Move the System-property/env plumbing out to io.ss / a host-system.ss.
12. Consolidate the seq/transducer/parity natives and dissolve natives-parity.ss (a literal leftovers file). One natives-transduce.ss for td-*/into-xform/transduce/sequence/cat; relocate parity's hash / macroexpand / reader-conditional pieces to their real homes.
13. Smaller renames/moves: dynamic-vars.ss (5 constants) vs dyn-binding.ss (the binding stack) — rename the former; pull the format engine out of natives-misc.ss into natives-format.ss.

Tier 2 — compiler core (seed sources; re-mint + selfhost + corpus)

14. Split the success-type checker out of types.clj (716 lines = inference + checker + driver). Move the checker (check-*, the error-domain predicates) to jolt.passes.types.check. Highest reduction in edit blast-radius in the compiler — inference and checker stop being able to break each other.
15. Decompose infer's :invoke arm (≈120 lines, 8 hand-coded call patterns) into named infer-<pattern> helpers (as numeric.clj's an-invoke already does), and add ty/nd accessors for the positional [type node] tuple (a transposition is currently silent and type-correct).
16. Single-source the const-keyword-lookup recognizer. It's implemented three times divergently (inline.clj, types.clj ×2 arms, backend_scheme.clj). Lift one predicate into jolt.passes.fold (the shared-predicate home) for the two analysis copies; the backend's value-emission copy can stay.
17. Co-locate the numeric op tables. backend_scheme.clj (dbl-ops/lng-ops, the Scheme strings) and numeric.clj (dbl-spec/lng-spec, specializable names) must agree or emit-numeric splices a nil op string. Cross-link or share the name→op table.
18. Collapse local-escapes?'s mechanical recursion (≈12 of its arms are just "does any child escape" = reduce-ir-children); keep only the binder/lookup arms explicit. Removes ~50 lines and a new-op soundness hazard. Fold recur-kinds/recur-arg-lists (numeric) into one recur-tails.
19. Touch-when-nearby: drop the dead form-char? refer in analyzer.clj:21; standardize on (get node :k) in the pass files; promote a few inline-comment soundness arguments to function docstrings (an-invoke :wild, dbl/lng-spec).

Sequencing

Tier 0 lands immediately (docs only). Tier 1 is the bulk of the maintainability win and carries no re-mint — do 6 (registry) and 7–10 (DCE module + build hygiene) first; they target the code most likely to be edited and most recently grown. Tier 2 batches into one or two re-mints once Tier 1 is stable, each verified by make selfhost + make corpus.

Every code change in Tiers 1–2 is behaviour-preserving and gated; no feature work.