A native-executable build bakes the jolt ctx, and env-reading libraries
(config.core/load-env) snapshot the ENTIRE build environment into it — jpm
marshals that into the binary. GitHub push protection caught real API
tokens inside an example's build output this way.
With JOLT_BAKE_ENV_ALLOWLIST set (comma-separated names — a project's
build.sh exports it for the bake), System/getenv serves only the listed
variables: single-name reads of unlisted vars return nil and the full
snapshot is filtered. Unset — every normal runtime — reads stay live and
unfiltered, so a baked binary that re-reads env at startup (config.core/
reload-env) sees the real runtime environment.
Verified A/B on ring-app: a planted token appears in the unscrubbed binary
(strings | grep: 1 hit) and not in the scrubbed one (0), which still serves.
Direct janet.os/environ bridge calls remain unfiltered host access, as
documented.
Running a checkout's build/jolt-deps by path failed with ENOENT unless
build/ was also on PATH: exec-jolt spawned a bare "jolt". Resolution is
now $JOLT_BIN, then the jolt sitting beside this binary (argv[0]'s
directory — the pair is built together), then PATH.
The vendored spork/http is gone — jpm owns janet packages. In its place:
- The janet.* bridge autoloads jpm-installed modules on first reference:
janet.spork.http/server requires spork/http from the module path and
caches its bindings (failures are negatively cached). Works for any
module, in every mode, including inside net/server connection fibers.
- deps.edn grows a :jpm/module coordinate: jolt-deps verifies the module is
importable at resolve time, optionally running `jpm install` on the
:jpm/install package once when it isn't, and otherwise fails with the
install hint. Contributes no source roots. ring-app declares spork/http
this way.
Docs: README's interop section, docs/tools-deps.md (:jpm/module reference),
and the ring-app README (including the jpm-version caveat for spork HEAD's
.janet native sources, which older jpm rejects).
(^bytes [b])-style return hints reach fn as a (with-meta [b] {:tag ...})
form; unhint sheds the wrapper through the rebuild path so the clause
representation never changes. The host-interop hint rows exercise it.
Class names evaluate to their canonical class-name STRINGS (the same values
class returns), so a (defmulti m (comp class :body)) matches (defmethod m
String ...) — ring.util.request dispatches exactly this way. Constructor
sugar and the new special form resolve the actual ctor from the registry
when given a token; dispatch-only names (InputStream, File, ISeq, ...) are
interned for defmethod position. nil is a legal multimethod dispatch value
now (sentinel-keyed: janet tables drop nil keys) — ring keys body-string's
no-body case on it.
spork/http is VENDORED (vendor/spork/http.janet, MIT) and baked into the
image, reaching the jolt layer as janet.spork.http/* through the janet.*
bridge — whose lookup is now a chain (runtime fiber env, module env, the
vendored registry), which also fixes janet/* resolution inside net/server
connection fibers (they carry a foreign env). jolt.http is rewritten over
the spork client (its old net/request never existed; also fixes its own
get shadowing clojure.core/get).
Also: slurp accepts opts and DRAINS reader shims (ring middleware slurps
request bodies), clojure.string/replace takes fn replacements with Clojure's
match-or-groups argument, .indexOf int needles are char codes, .getBytes on
the String surface, and ^bytes-style return hints on param vectors parse
(the fn macro unwraps the with-meta form).
Suite steady at 4715/5348; conformance x3 green; deps-conformance medley +
cuerdas green (the stuartsierra/dependency failure predates this change).
The interop surface ring.util.codec needs (registered through the javatime
shim registries): URLEncoder/URLDecoder (www-form-urlencoded in pure janet),
Charset/forName, Base64 encoder/decoder, Integer/valueOf with radix +
parseInt, StringTokenizer, clojure.lang.MapEntry (a 2-tuple), a String ctor
from bytes, .getBytes on the String surface, and a java.lang.Number method
surface (byteValue and friends).
Protocol fixes: extend-protocol on java.util.Map/Set/List now dispatches
(maps — phm/struct/sorted/records — never produced host tags and fell to
Object), lazy seqs gained their ISeq tags, and a nil extension arm works
(group-by-head and extend-type both choked on the nil head). reduce
dispatches to a reified clojure.lang.IReduceInit's own reduce method, which
is how ring-codec tokenizes.
jolt-deps learns :deps/root (tools.deps monorepo subdirectory checkouts —
ring-core lives inside ring-clojure/ring). spork/http, when jpm-installed,
reaches the jolt layer as janet.spork.http/* through the janet.* bridge
(soft: nothing requires it unless used).
The protocol fixes alone let 29 more clojure-test-suite assertions execute:
5319 -> 5348 run, 4706 -> 4715 pass.
The first per-type migration print-method unlocked: uuid, regex, transient,
and channel rendering move from host pr-render branches to io-tier
defmethods (exact same output). The renderer's tagged fallthrough now
dispatches ANY remaining :jolt/* value through the print-method hook before
the raw pairs view — so every tagged type is user-overridable, atoms
included ((defmethod print-method :jolt/atom ...) fires nested), and future
per-type migrations are pure overlay additions.
Hot types (numbers, strings, symbols, collections) stay native, and inst/
namespace/var stay host for now — their formatters (rfc3339, display names)
live there anyway. A transient's :kind is read with jolt.host/ref-get: get
on a transient is the dispatched collection lookup (same trap as sorted
colls).
Before the hook is wired (init-time error messages) tagged values fall
through to the pairs view — bootstrap rendering only.
print-method/print-dup are now multimethods in the io tier with Clojure's
exact dispatch ((:type meta) keyword, else type — core.clj 3693). On jolt the
dispatch value for a record is its quoted full-name symbol, since class names
aren't values here.
Records used to pr-str as the raw janet table; the renderer's record branch
now prints Clojure's #ns.Type{:k v} syntax, and first consults a callback the
api wires up after the overlay loads — so a user defmethod on a record type
fires everywhere: top level, nested in collections, through pr/prn/pr-str.
Builtin overrides (a :number method) fire only on direct print-method calls;
pr keeps the native fast path (documented divergence).
java.io.Writer arrives as a shim beside the StringReader/StringBuilder ones:
a :jolt/writer tagged value with write/append/flush/toString, a StringWriter
ctor, and a sink variant the renderer callback uses.
Two latent host bugs fixed on the way: the interpreted syntax-quote splice
blew up on ~@nil (an interpreted macro's empty & rest binds nil — first tier
user of defmulti found it; d-realize now treats nil as the empty seq), and
(print-method x nil) now throws like the JVM instead of returning nil.
10 spec rows; bench dead even (sandwich run); greeter green on a fresh
binary.
Batch 2 of the post-shrink sweep, all pure compositions or documented stubs:
enumeration-seq/iterator-seq (seq), promise/deliver (an atom — deref of an
undelivered promise stays nil, single-threaded host), bean, uri?,
special-symbol? (an evaluated set of quoted symbols — a QUOTED set literal
stays an unevaluated reader form on jolt, which the first version tripped
over), print-method/print-dup (inert until jolt-g1r), and the whole proxy
surface (mappings/call-with-super/init/update pass through, the constructive
half throws). The seed loses 16 defns and bindings; nothing kept.
Three canonical-conformance fixes from the post-shrink batch:
- bit-and/bit-or/bit-xor/bit-and-not get Clojure's variadic arities as
20-coll shells folding the binary host ops (now __bit-* seams). 2-arg call
sites still compile to the native janet op via the backend's native-ops
table. The passes.clj constant-fold table now names the seams — the public
fns are overlay and don't exist when the compiler loads (this briefly broke
every compile-mode init).
- core-set? recognizes the :jolt/sorted-set representation (jolt-dpn):
(set? (sorted-set 1)) was false, and ifn? on sorted sets inherited the bug.
- (if) / (if test) / (if test then else extra) throw in both the analyzer
and the interpreter — spec 03-special-forms X1, now marked verified.
Suite 4704 -> 4706; bench and the greeter example benchmark are flat.
Fixed arities now throw Clojure's ArityException shape — 'Wrong number of
args (N) passed to: name' — on any count mismatch; variadic arities on fewer
than their fixed params. The compiled path already enforced fixed arities via
janet's native fn check and multi-arity dispatch; this adds the check to the
interpreter's single-arity closures (the oracle was silently dropping extra
args and giving a raw tuple-index error for missing ones) and guards the
compiled single-variadic wrapper's minimum. Messages carry the fn name when
there is one. 16 spec rows; the update.cljc suite row flipped green (4703 ->
4704).
Enforcement exposed that seq-to-map-for-destructuring had drifted: the spec
row called the 1-arity fn with two args, and the body silently dropped a
trailing unpaired element. Replaced with the canonical Clojure 1.11 version
(even pairs build the map, a single trailing element passes through — so
(f {:b 2}) kwargs calls work — and an unpaired key throws).
Also: transients RFC notes tuple support from the seed-shrink rounds.
Round 6 of the seed shrink (the printer round, scoped by the perf wall). The
five wrappers move to 20-coll over two new host seams: __write (push a string
to *out*) and __pr-str1 (render one value readably). The renderer itself
stays in the seed — it's representation-coupled (pvec/phm/phs/sorted
internals) and shared with the hot str, and rendering through overlay calls
would pay the per-element call cost everywhere big values get printed.
print-method as a real multimethod is follow-up work.
The new spec rows caught a renderer bug: string bodies were never escaped, so
(pr-str "a\"b") didn't round-trip through the reader. pr-render now
escapes quote/backslash/control chars per Clojure.
Round 5 of the seed shrink. transduce is the canonical 5-liner over reduce
(which already honors reduced and steps lazy seqs); eduction composes with
comp and stays eager into a vector (documented divergence, as before);
td-comp — eduction's last caller — is deleted from the seed. transient
accepts tuples now (reader vectors / map entries), so (into [] (first {:a 1}))
keeps working everywhere a vector does.
into was moved, benched, and moved back: the overlay call layers cost the
into-vec suite ~11% back-to-back (536 vs 480ms), the same per-call wall that
sent even?/odd? home in round 4. A transient conj! fast path didn't pay for
itself either (jolt call overhead dominates, not the per-element conj). The
seed keeps core-into + its private transduce machinery; the binding count
still drops by three.
Round 4 of the seed shrink. zero?, pos?, every? move to the syntax tier
(empty? and the analyzer use them — raw def+fn* per the file constraint);
char? joins the tagged-value predicates in 20-coll. coll? stays seed: host
set? doesn't cover sorted sets (filed jolt-dpn) and the tag check from the
overlay would hit the sorted-coll get trap. pos? guards number? explicitly —
the staged recompile emits bare > as the native janet op, which orders
strings (zero? gets the same guard; spec rows lock both plus neg?).
The canonical every? seq-walks its coll, which exposed that rest/next over
sets, phms, struct maps and sorted colls fell into core-rest's indexed
fall-through and walked the wrapper table's INTERNAL fields — (next #{1 2})
was (nil nil), (clojure.set/subset?) broke. core-rest now seqs those
representations (branches placed AFTER the hot vector/lazy paths; the first
ordering cost seq-pipe 4x). Suite rises 4700 -> 4703; baseline 4660 -> 4695.
even?/odd? are back in the seed after the bench A/B: (filter even? ...) pays
an extra call layer per element through the overlay (seq-pipe 262 -> 1100ms).
They join the perf-wall list with the lazy hot fns.
Round 3 of the seed shrink. To the overlay: identity, constantly, neg?,
even?, odd? (20-coll, ahead of their first in-tier uses), not= and unreduced
(00-syntax — the kernel and seq tiers use them), ==, ensure-reduced,
halt-when, parse-boolean, parse-uuid, newline, seque, array-seq, to-array-2d,
and the masking unchecked-byte/short/char/float/double coercions. parse-uuid
validates via re-matches over a new __make-uuid host binding (overlay source
can't write :jolt/type map literals). memfn moves to 30-macros as a working
macro over the .method call sugar instead of a fn that throws.
Behavior fixes toward Clojure, each with spec rows: == now throws on
non-numbers instead of comparing them, and halt-when is the canonical
::halt-map version (the halt value replaces the whole reduction result, no
double completion). list? and map-entry? stay in the seed — both are
representation-coupled (plist/tuple checks).
clojure-test-suite goes 4701 -> 4700: update.cljc expects
(update {:k 1} :k identity 1 2 3 4) to throw an arity error, and jolt fns
don't enforce fixed arity anywhere (pre-existing, language-wide — the seed's
Janet identity threw natively). Filed as jolt-6xn; fixing it should flip
several suite rows at once.
Jolt numbers don't overflow, so +'/-'/*'/inc'/dec' and the whole unchecked-*
family are just the checked ops — now one-line defs in core/20-coll.clj
instead of ~25 seed bindings. int? and num move the same way.
unchecked-divide-int now goes through quot, so dividing by zero throws like
the JVM instead of silently truncating infinity. unchecked-int/long gain
char handling via int, matching Clojure ((unchecked-int \a) => 97). The
masking byte/short/char coercions are not aliases and stay in the seed for
a later round.
Also drops a second duplicate set of unchecked defns that was shadowing the
first at module load.
The seed copies of inst?/inst-ms and the multimethod table ops
(get-method/methods/remove-method/remove-all-methods/prefer-method) are dead
code — the overlay redefines all of them (20-coll.clj, 30-macros.clj) — so
they're deleted along with duplicate bindings-table entries (unreduced,
eduction, unchecked-inc/dec/add/subtract).
New spec rows for methods/remove-all-methods caught two real bugs in the
evaluator's setup fns: methods-setup returned the live host table (count
rejected it, and callers could mutate dispatch state), and
remove-all-methods-setup swapped in a fresh table the dispatch closure never
saw. methods now returns a phm snapshot; remove-all-methods clears in place.
Loading these libs via require worked (load-ns-source interprets, macros
expand lazily) but the same code inlined by uberscript routes through
eval-toplevel and compiled, surfacing four gaps:
- a ^{:map} metadata def name reads as (def (with-meta name m) v); the
analyzer died extracting the name (config.core's defonce env). It now
throws uncompilable so the interpreter, which handles it, takes over.
- declare was a no-op, so a compiled forward reference to a declared
name that collides with a janet root binding bound to the host fn
(selmer.parser's (declare parse) compiled to janet's 1-arg parse).
declare now expands to no-init defs, the interpreter interns them,
and the analyzer routes no-init def to the interpreter.
- class? was missing (selmer.util's exception macro calls it at
expansion time). Always false, like ratio? — no Class objects here.
- require of an unlocatable namespace silently left an empty ns behind,
deferring the failure to an unresolved symbol far from the cause. It
now throws like Clojure's FileNotFoundException. Namespaces entered
in-session count as loaded (Clojure puts them in *loaded-libs*), and
the SCI bootstrap opts out via :lenient-require? since its
clj-targeted requires can't all exist on this host.
The cache key was built with janet's (hash ...), which is seeded per
process, so it never matched across invocations and every jolt-deps run
re-resolved and re-fetched. Store the raw key material instead.
Run the jpm git calls silenced (:silent) with a one-line progress note
on stderr — the checkout chatter ("HEAD is now at ...") was landing on
stdout and corrupting the documented JOLT_PATH=$(jolt-deps path) capture.
Covered by two new deps-resolve checks: a cross-process cache-hit probe
(sentinel-tampered cache file) and a stdout-cleanliness check against a
local file:// git dep.
yogthos/config now loads and runs end to end: config.edn/.lein-env
deep merge, env vars keywordized and type-converted, PushbackReader
over io/reader, reload-env via alter-var-root. New shims:
java.io.PushbackReader (read/unread), Long/parseLong, BigInteger.,
Boolean/parseBoolean, System/getProperties; clojure.edn/read now
drains an actual reader (it used to read one LINE from a raw janet
file handle, so multi-line config files and shim readers both broke).
Three real bugs shaken out, each with regression specs:
- An empty rest arg bound () instead of nil. ((fn [& r] (if r 1 2)))
returned 1; the truthy () sent config's (or (.exists f) required)
down the wrong branch. Fixed in the interpreter and the compiled-fn
emission. Internal apply boundaries (protocol dispatch, core-apply)
now accept a nil seq like Clojure's (apply f x nil).
- seq/map over a raw janet table (System/getenv, os/environ) yielded
nothing, so config's read-system-env came back empty. Raw host
tables now seq as kv entries like any map, in core-seq,
realize-for-iteration, and coll->cells. The old spec row hid this
behind a vacuous (every? pred empty) — replaced with one that
asserts non-emptiness.
- edn/read single-line limitation, as above.
test/integration/config-lib-test.janet runs the real library from
~/src/config (skips when absent).
Git clones now default to a shared, sha-immutable cache —
$JOLT_GITLIBS, else <config-dir>/gitlibs — instead of a per-project
./jpm_tree, the tools.gitlibs ~/.gitlibs model. Passing tree
explicitly still works (tests do). The resolved-roots cache moves
out of the clone tree to the project-local .cpcache/jolt-deps.jdn,
since roots depend on the project while clones don't.
deps.edn grows :tasks, the honest subset of babashka's: a string
task is a shell command, a map task is {:main-opts [...] :doc}.
jolt-deps tasks lists them (merged user+project), jolt-deps task
NAME runs one. Bare-expression tasks are out of scope: the reader
hands back parsed data and round-tripping to source is fragile.
Also fixes load-config skipping the symbol-key normalization when
only one config file existed — :tasks/:deps keys stayed raw reader
symbols (which embed positions and never compare equal), so lookups
missed. Regression rows in deps-tasks-test; docs updated for the
whole tools.deps surface (aliases, -A/-M, user config, conflicts,
gitlibs cache, tasks).
deps.edn :aliases now work the tools.deps way, scoped to what jolt
supports (git/:local, no maven): :extra-paths and :extra-deps
accumulate across selected aliases, :main-opts is last-wins. The CLI
grows -A:dev:test (selects aliases for path/run/repl/-e) and
-M:alias (runs the alias :main-opts through jolt). A user-level
deps.edn ($JOLT_CONFIG, else $XDG_CONFIG_HOME/jolt, else ~/.jolt)
merges under the project file: :deps and :aliases merge per key with
the project winning, everything else replaces.
Resolution is now breadth-first so a top-level coordinate always
beats a transitive one for the same lib (it was DFS first-wins —
a dep's pin could shadow the project's own). Conflicting coordinates
for one lib warn on stderr with both coords and which won. Also
fixes dedup keying: it hashed the symbol struct's string repr, which
carries reader position metadata, so the same lib from two files
never deduped.
resolve-deps-cached keys on project edn + user edn + aliases.
Tests: deps-aliases-test and deps-conflicts-test, local deps only.
Selmer now loads and renders templates on jolt: variables, filters
(upper, date with JVM patterns), if/for tags, nested lookups, HTML
escaping, and render-file with its last-modified template cache.
New src/jolt/javatime.janet provides the java.time surface Selmer's
date filters use (DateTimeFormatter/Instant/ZoneId/LocalDateTime/
FormatStyle/Locale, epoch-ms backed, host-local timezone) plus the
java.io/java.lang/java.net shims its template reader needs
(StringReader, StringBuilder, URL, File/separator, Class/forName).
Everything registers through three new evaluator registries
(class-statics, tagged-methods, class-ctors), so the module is data
plus an install call.
Fixes shaken out along the way, each load-bearing for Selmer and
correct on their own:
- :refer :all silently referred nothing (it iterated the :all keyword)
- ns :import ignored vector specs and didn't share deftype ctor vars
- dot calls on deftype/reify instances never consulted the protocol
registry, so (.render-node node ctx) failed where (render-node ...)
worked
- instance? rejected expression type args like (Class/forName "[C")
- char-array didn't accept a string
- io/resource now searches the loader's source roots (the classpath
analog); io/reader handles char arrays, URLs, readers, and returns
an in-memory reader with :read-line-fn for file paths
- String .split (regex, JVM trailing-empty semantics), file-path
methods (.toURI/.toURL/.getPath/.lastModified/.exists)
- System/getProperty (os.name & co), the janet/* bridge now works
inside env-less fibers, and qualified class names that syntax-quote
mangles (selmer.util/StringBuilder) fall back to the ctor registry
Spec rows cover the shim surface; test/integration/selmer-test.janet
runs the real Selmer from ~/src/selmer (skips cleanly when absent).
\p{L}/\p{Lu}/\p{Ll}/\p{N}/\p{Z}/\p{Ps}/\p{Pe} (+\P negation) land in
both escape positions of the regex compiler, mapped onto the byte PEGs:
ASCII exact, any high byte (inside a UTF-8 sequence) counts as a LETTER —
so ^\p{L}+$ accepts UTF-8 words while \p{N}/\p{Z} stay ASCII. (?u) was
already a tolerated no-op flag. Unknown property names error at compile.
Chasing the acceptance target (cuerdas via deps-conformance) pulled in the
rest of its clj-compat chain, each a real gap:
- the deps-conformance harness reads libraries under clj-compat reader
features (deps are clj/cljc by definition — without :clj, cuerdas's
#?(:clj (instance? Pattern x)) branches resolved to NIL bodies)
- instance? knows Pattern/java.util.regex.Pattern (regex values) and
Character (cuerdas's rx/regexp? gate on split)
- the java.lang.String method surface: .toLowerCase/.toUpperCase/.trim/
.indexOf(-1 on miss)/.lastIndexOf/.substring/.charAt/.startsWith/
.endsWith/.contains/.replace/.equalsIgnoreCase/... — ASCII case mapping,
unknown methods error (the old path silently returned nil)
- the (.method obj args) SUGAR now desugars to (. obj method args) in the
interpreter — it was never implemented (bare .method heads resolved as
vars, hence 'Cannot call nil')
- Long/MAX_VALUE / MIN_VALUE statics (f64 approximations)
deps-conformance: medley ok, cuerdas ok (was check-error); dependency now
loads its clj branches and fails only on its single-segment ns resolution.
30 new spec rows (11 regex, 19 interop). Gate exit 0.
ifn? (jolt-1vx) is the canonical IFn set in the overlay: fns, keywords,
symbols, maps (sorted included), sets, vectors, and vars — NOT lists. The
seed version said true for lists and false for struct maps and vars.
Mutable-mode caveat documented (vectors and lists share the array repr
there). 13 predicate rows.
Multimethod dispatch (jolt-heo) now collects EVERY isa-matching method key
and picks the dominant one — x dominates y when prefer-method'd over it or
(isa? x y) — and two matches with no dominant is an ambiguity ERROR, as in
Clojure. It used to take whichever key the table yielded first, silently
ignoring prefer-method. The prefers store upgrades to Clojure's
{x -> set-of-dominated} shape, shared between the dispatch closure and
prefer-method-setup via the var; prefers becomes a macro over a setup fn
(the store lives on the VAR — the multifn value can't carry it, so the old
fn read {} forever). 6 multimethod rows + the conformance row updated to
the canonical shape (335x3).
The reader (jolt-ou8) kept the pending KEY when a comment or #_ sits in a
map's VALUE slot: the old code dropped both, desyncing kv pairing — the
real value became the next key and the closing brace landed in value
position ('Unmatched closing brace'). Selmer's deps.edn (a '; for
development (REPL, etc)' comment between key and value) now parses; 6
reader rows incl. nested commented maps.
Gate: jpm exit 0, conformance 335x3, all tests passed.
