* Add architecture refactor plan
Synthesizes a six-part architectural review into phased, gate-validated cleanup
work. Targets LLM-maintainability: one home per feature, no god-files, explicit
checked contracts, no copy-paste dispatch. No code changes yet — the plan only.
* Refactor phase 0: dead code + isolated bugs
Pure cleanup ahead of the structural phases (docs/architecture-refactor-plan.md).
No behavior change except the two bug fixes, which are covered by a regression row.
Dead code (all verified zero-reference or overridden):
- core-resolve / core-satisfies? / core-type->str seed stubs + bindings —
resolve and satisfies? are interned by install-stateful-fns! (the seed copies
were shadowed); type->str was an inert SCI stub with no callers.
- find defined twice in 20-coll.clj; the dead copy returned a plain vector
(wrong — the live def at :787 returns a real map-entry) with a comment that
contradicted it.
- mark-hint (passes.clj), phs-to-struct (phm), shape-vals / ns-imports-fn
(types) — unreferenced.
- redundant local pad2 in javatime (module-level one already in scope).
Bugs:
- File.toURL stored :url but every :jolt/url method reads :spec, so a URL from
(.toURL file) returned nil from all its methods. Now stores :spec (+ spec row).
- pl-rest had a no-op (if (plist? r) r r); collapsed to r.
- :map-shapes? was missing from the deps-image cache key — two runs differing
only in map-shapes could reuse each other's image.
Also dropped read-quote's unused pos param. Full gate green.
---------
Co-authored-by: Yogthos <yogthos@gmail.com>
Pivot from a jolt reimplementation to running the upstream library verbatim.
Vendors the real clojure/tools/logging.clj; jolt provides the backend and the
host primitives it needs. Language features (broadly useful for real Clojure
libs), all covered in 3-mode conformance + spec suites:
- defmacro: multi-arity dispatch (jolt-q8l) and a docstring + attr-map + params
head (jolt-qnr) — the 4-arity log macro and every level macro need these.
- syntax-quote resolves an alias-qualified symbol to its target ns (jolt-9av),
so a macro template (impl/get-logger) resolves at the use site.
- the ns macro unwraps ^{:map} metadata on the ns name (jolt-8w2 workaround,
matching def/defn/defmacro).
- a namespace object self-evaluates, so ~*ns* can be spliced into a template.
Host shims (ported from / modeled on clojure where applicable):
- clojure.string/trim-newline (ported, CharSequence interop -> count/subs)
- agent/send-off/send (minimal synchronous stubs; jolt has no thread pool/STM)
- clojure.lang.LockingTransaction/isRunning -> false
- a minimal clojure.pprint (pprint/with-pprint-dispatch/code-dispatch, for spy)
- clojure.tools.logging.impl: a jolt stderr LoggerFactory backend (the library's
designed pluggable extension point)
docs/libraries.md lists tools.logging; grammar.ebnf metadata note clarified.
Conformance 355/355 x3 modes; full jpm test gate green.
Follow-ups for running reitit alongside default-feature libraries in one app:
- __reader-features / __reader-features-set! exposed to Clojure so a namespace
can load a clj-targeted lib (reitit, under :clj) WITHOUT forcing the whole
process to :clj — set features, require, restore. Honeysql/selmer/ring stay
on the default set they were validated under. (jolt vector args coerced to a
janet array — janet map over a pvec iterates keys otherwise.)
- clojure.template added to the stdlib (verbatim, pure Clojure over
clojure.walk) — honeysql's :clj branch requires it.
- java.util.Locale registered as a qualified alias (+ ROOT) for selmer's :clj
Locale/US use.
Everything reitit-core needs to load unmodified from git under :clj features:
- The baked binary now re-reads JOLT_FEATURES at startup (like JOLT_PATH).
reader-features-set! runs at module load = BUILD time for a binary, so a
process opting into :clj (to read a lib's :clj branches) was ignored, and
unmatched #?(...) forms silently spliced to nothing — defn of a fn with an
empty arglist, hence the cryptic index errors.
- (get s i) indexes a string and returns the char, as in Clojure (nth did;
get returned nil). reitit's path parser is (get path i)-based — without
this every route read as static.
- Class-shim registration exposed to Clojure: __register-class-statics! /
__register-class-methods! / __register-class-ctor!, so a library can mirror
a Java class jolt doesn't ship (the reitit.Trie mirror lives in jolt-lang/
router on top of these).
- Java surface reitit's :clj branches call: .getMessage (on exceptions and
strings) and a small universal object-method set, .intern, java.util.HashMap
(a mutable map wrapper). Plus defprotocol already took keyword options.
Gate green; clojure-test-suite 4715 -> 4718 (the get-on-strings fix).
A native-executable build bakes the jolt ctx, and env-reading libraries
(config.core/load-env) snapshot the ENTIRE build environment into it — jpm
marshals that into the binary. GitHub push protection caught real API
tokens inside an example's build output this way.
With JOLT_BAKE_ENV_ALLOWLIST set (comma-separated names — a project's
build.sh exports it for the bake), System/getenv serves only the listed
variables: single-name reads of unlisted vars return nil and the full
snapshot is filtered. Unset — every normal runtime — reads stay live and
unfiltered, so a baked binary that re-reads env at startup (config.core/
reload-env) sees the real runtime environment.
Verified A/B on ring-app: a planted token appears in the unscrubbed binary
(strings | grep: 1 hit) and not in the scrubbed one (0), which still serves.
Direct janet.os/environ bridge calls remain unfiltered host access, as
documented.
Class names evaluate to their canonical class-name STRINGS (the same values
class returns), so a (defmulti m (comp class :body)) matches (defmethod m
String ...) — ring.util.request dispatches exactly this way. Constructor
sugar and the new special form resolve the actual ctor from the registry
when given a token; dispatch-only names (InputStream, File, ISeq, ...) are
interned for defmethod position. nil is a legal multimethod dispatch value
now (sentinel-keyed: janet tables drop nil keys) — ring keys body-string's
no-body case on it.
spork/http is VENDORED (vendor/spork/http.janet, MIT) and baked into the
image, reaching the jolt layer as janet.spork.http/* through the janet.*
bridge — whose lookup is now a chain (runtime fiber env, module env, the
vendored registry), which also fixes janet/* resolution inside net/server
connection fibers (they carry a foreign env). jolt.http is rewritten over
the spork client (its old net/request never existed; also fixes its own
get shadowing clojure.core/get).
Also: slurp accepts opts and DRAINS reader shims (ring middleware slurps
request bodies), clojure.string/replace takes fn replacements with Clojure's
match-or-groups argument, .indexOf int needles are char codes, .getBytes on
the String surface, and ^bytes-style return hints on param vectors parse
(the fn macro unwraps the with-meta form).
Suite steady at 4715/5348; conformance x3 green; deps-conformance medley +
cuerdas green (the stuartsierra/dependency failure predates this change).
The interop surface ring.util.codec needs (registered through the javatime
shim registries): URLEncoder/URLDecoder (www-form-urlencoded in pure janet),
Charset/forName, Base64 encoder/decoder, Integer/valueOf with radix +
parseInt, StringTokenizer, clojure.lang.MapEntry (a 2-tuple), a String ctor
from bytes, .getBytes on the String surface, and a java.lang.Number method
surface (byteValue and friends).
Protocol fixes: extend-protocol on java.util.Map/Set/List now dispatches
(maps — phm/struct/sorted/records — never produced host tags and fell to
Object), lazy seqs gained their ISeq tags, and a nil extension arm works
(group-by-head and extend-type both choked on the nil head). reduce
dispatches to a reified clojure.lang.IReduceInit's own reduce method, which
is how ring-codec tokenizes.
jolt-deps learns :deps/root (tools.deps monorepo subdirectory checkouts —
ring-core lives inside ring-clojure/ring). spork/http, when jpm-installed,
reaches the jolt layer as janet.spork.http/* through the janet.* bridge
(soft: nothing requires it unless used).
The protocol fixes alone let 29 more clojure-test-suite assertions execute:
5319 -> 5348 run, 4706 -> 4715 pass.
yogthos/config now loads and runs end to end: config.edn/.lein-env
deep merge, env vars keywordized and type-converted, PushbackReader
over io/reader, reload-env via alter-var-root. New shims:
java.io.PushbackReader (read/unread), Long/parseLong, BigInteger.,
Boolean/parseBoolean, System/getProperties; clojure.edn/read now
drains an actual reader (it used to read one LINE from a raw janet
file handle, so multi-line config files and shim readers both broke).
Three real bugs shaken out, each with regression specs:
- An empty rest arg bound () instead of nil. ((fn [& r] (if r 1 2)))
returned 1; the truthy () sent config's (or (.exists f) required)
down the wrong branch. Fixed in the interpreter and the compiled-fn
emission. Internal apply boundaries (protocol dispatch, core-apply)
now accept a nil seq like Clojure's (apply f x nil).
- seq/map over a raw janet table (System/getenv, os/environ) yielded
nothing, so config's read-system-env came back empty. Raw host
tables now seq as kv entries like any map, in core-seq,
realize-for-iteration, and coll->cells. The old spec row hid this
behind a vacuous (every? pred empty) — replaced with one that
asserts non-emptiness.
- edn/read single-line limitation, as above.
test/integration/config-lib-test.janet runs the real library from
~/src/config (skips when absent).
Selmer now loads and renders templates on jolt: variables, filters
(upper, date with JVM patterns), if/for tags, nested lookups, HTML
escaping, and render-file with its last-modified template cache.
New src/jolt/javatime.janet provides the java.time surface Selmer's
date filters use (DateTimeFormatter/Instant/ZoneId/LocalDateTime/
FormatStyle/Locale, epoch-ms backed, host-local timezone) plus the
java.io/java.lang/java.net shims its template reader needs
(StringReader, StringBuilder, URL, File/separator, Class/forName).
Everything registers through three new evaluator registries
(class-statics, tagged-methods, class-ctors), so the module is data
plus an install call.
Fixes shaken out along the way, each load-bearing for Selmer and
correct on their own:
- :refer :all silently referred nothing (it iterated the :all keyword)
- ns :import ignored vector specs and didn't share deftype ctor vars
- dot calls on deftype/reify instances never consulted the protocol
registry, so (.render-node node ctx) failed where (render-node ...)
worked
- instance? rejected expression type args like (Class/forName "[C")
- char-array didn't accept a string
- io/resource now searches the loader's source roots (the classpath
analog); io/reader handles char arrays, URLs, readers, and returns
an in-memory reader with :read-line-fn for file paths
- String .split (regex, JVM trailing-empty semantics), file-path
methods (.toURI/.toURL/.getPath/.lastModified/.exists)
- System/getProperty (os.name & co), the janet/* bridge now works
inside env-less fibers, and qualified class names that syntax-quote
mangles (selmer.util/StringBuilder) fall back to the ctor registry
Spec rows cover the shim surface; test/integration/selmer-test.janet
runs the real Selmer from ~/src/selmer (skips cleanly when absent).
\p{L}/\p{Lu}/\p{Ll}/\p{N}/\p{Z}/\p{Ps}/\p{Pe} (+\P negation) land in
both escape positions of the regex compiler, mapped onto the byte PEGs:
ASCII exact, any high byte (inside a UTF-8 sequence) counts as a LETTER —
so ^\p{L}+$ accepts UTF-8 words while \p{N}/\p{Z} stay ASCII. (?u) was
already a tolerated no-op flag. Unknown property names error at compile.
Chasing the acceptance target (cuerdas via deps-conformance) pulled in the
rest of its clj-compat chain, each a real gap:
- the deps-conformance harness reads libraries under clj-compat reader
features (deps are clj/cljc by definition — without :clj, cuerdas's
#?(:clj (instance? Pattern x)) branches resolved to NIL bodies)
- instance? knows Pattern/java.util.regex.Pattern (regex values) and
Character (cuerdas's rx/regexp? gate on split)
- the java.lang.String method surface: .toLowerCase/.toUpperCase/.trim/
.indexOf(-1 on miss)/.lastIndexOf/.substring/.charAt/.startsWith/
.endsWith/.contains/.replace/.equalsIgnoreCase/... — ASCII case mapping,
unknown methods error (the old path silently returned nil)
- the (.method obj args) SUGAR now desugars to (. obj method args) in the
interpreter — it was never implemented (bare .method heads resolved as
vars, hence 'Cannot call nil')
- Long/MAX_VALUE / MIN_VALUE statics (f64 approximations)
deps-conformance: medley ok, cuerdas ok (was check-error); dependency now
loads its clj branches and fails only on its single-segment ns resolution.
30 new spec rows (11 regex, 19 interop). Gate exit 0.
adds self-hosted compiler is functionally:
- The default compile path is the portable pipeline using jolt.analyzer (Clojure) → host-neutral IR → backend.janet.
- The analyzer is itself Clojure, compiled by jolt for true self-hosting.
- bootstrap-fixpoint passes (stage1 == stage2 == stage3): rebuilding the compiler on its own output.
- clojure.core is now self-hosted in the overlay.
- Stateful forms (defmacro/ns/deftype/defmulti/require/in-ns) are interpreted by design.
- host-interop-spec: add an 'interop / janet bridge' suite — janet/<name> and
janet.<module>/<name> resolution, the value-representation boundary (a Jolt
vector crosses as a Janet table), explicit-only (unprefixed module not
exposed), and unknown-symbol errors.
- nrepl-test: assert a def's value renders as #'ns/name (pr-str loops on a
var's cyclic ns refs, so jolt.nrepl renders vars itself).
Final spec areas. Bugs caught and fixed:
- (def name docstring value) used the docstring as the value; now the 3-arg
docstring form binds the value and records :doc meta
- resolve was a nil stub; now a special form resolving a symbol to its var
(nil if unresolved). Added find-ns (non-creating lookup) and ns-name.
- in-ns didn't evaluate its arg, so (in-ns 'foo) failed; now evaluates it per
Clojure (the integration test's unquoted form updated to the quoted idiom)
- #(... %& ...) built %& as a positional param instead of a & rest param;
now emits (fn* [... & gen] ...) so %& captures the rest
Full public-API spec layer now in place. conformance 218/218, jpm test green.
