diff --git a/docs/seed-overlay-registry.md b/docs/seed-overlay-registry.md new file mode 100644 index 0000000..5bbcbcf --- /dev/null +++ b/docs/seed-overlay-registry.md @@ -0,0 +1,80 @@ +# Seed ↔ Overlay Registry + +Jolt is "Clojure on Janet": a shrinking **Janet seed** (`src/jolt/*.janet`) +hosts a **Clojure overlay** (`jolt-core/clojure/core/NN-*.clj`). Both define +`clojure.core`-facing functions, and for a handful of names *both* tiers carry a +definition. Which copy is authoritative has been tribal knowledge. This document +is the single source of truth; `test/unit/seed-overlay-registry-test.janet` is a +build-time drift check that fails if reality diverges from what is written here. + +## The registration mechanism + +Seed core functions are named with a `core-` prefix (`core-into`, `core-conj`, +`core-transduce`) and registered into the `clojure.core` namespace by the +`core-bindings` table in `src/jolt/core.janet`. Each entry maps a **public +Clojure name** (the string key) to a seed function value: + +```janet +(def- core-bindings + @{"into" core-into + "reduce" core-reduce + ...}) +``` + +`init-core!` (`src/jolt/core.janet`) interns every pair into `clojure.core`. +The overlay tiers load afterwards (`api.janet`: 00-syntax, 00-kernel, 10-seq, +20-coll, 25-sorted, 30-macros, 40-lazy, 50-io). When an overlay tier `(defn X …)` +for a name that `core-bindings` already registered, the **overlay def shadows the +seed binding** — the seed `core-X` then survives only if some other seed code +still calls it directly. + +So a name's *home* is determined by two facts: + +1. is it a key in `core-bindings`? (registered ⇒ the seed `core-X` is reachable) +2. does an overlay tier `(defn X …)`? (defined ⇒ the overlay copy shadows) + +## Dispatch-only seed helpers: the `__` prefix + +Seed functions that are **not** public Clojure vars but must be reachable by +name from compiled/overlay code (compiler hooks, macro-expansion targets) are +registered under a `__`-prefixed key — e.g. `"__sq1"`, `"__write"`, +`"__bit-and"`, `"__jdbc-conn-raw"`. The `__` prefix is unreadable as a +user-level symbol, so these never collide with or masquerade as public API. When +you add a dispatch-only hook, give it a `__` key; do not register it under a bare +name. + +## Dispatch twins + +A **twin** is a name with *both* a seed `core-X` defn and an overlay `(defn X …)`. +There are exactly five. Each seed site carries a greppable `SEED-TWIN:` comment. + +| name | overlay (authoritative public) | seed copy (`core-X`) | registered? | role of the seed copy | +|---------------|----------------------------------------|--------------------------------------|-------------|------------------------| +| `char?` | `20-coll.clj` `char?` | `core_types.janet` `core-char?` | no | internal type dispatch | +| `sorted-map?` | `25-sorted.clj` `sorted-map?` | `core_types.janet` `core-sorted-map?`| no | internal dispatch (sorted-op) | +| `sorted-set?` | `25-sorted.clj` `sorted-set?` | `core_types.janet` `core-sorted-set?`| no | internal dispatch | +| `sorted?` | `25-sorted.clj` `sorted?` | `core_types.janet` `core-sorted?` | no | internal dispatch | +| `transduce` | `20-coll.clj` `transduce` | `core_coll.janet` `core-transduce` | no | internal helper for `core-into` only | + +None of the five is registered in `core-bindings`: the overlay copy is the public +one, and the seed copy is reached only by other *seed* code (so editing the seed +copy alone will not change what user code sees — change both, or move the logic). + +## The surprising asymmetry: `into` vs `transduce` + +`into` and `reduce` are **seed-public**: registered in `core-bindings`, and the +overlay deliberately does *not* redefine them (they sit on the perf wall — see +the "into stays in the seed" note in `20-coll.clj`). `transduce`, by contrast, is +**overlay-public**: the overlay `transduce` is the real one, and `core-transduce` +remains only because `core-into` calls it directly. So two functions that read as +a matched pair have opposite homes. That asymmetry is intentional and is the +reason this registry exists. + +## Drift check + +`test/unit/seed-overlay-registry-test.janet` recomputes the twin set from source +(names with both a seed `core-X` defn and an overlay `defn X`) and asserts it +equals the five above. It also asserts none of the twins is registered in +`core-bindings`, and that every non-`__` `core-bindings` key is a plausible +public name (no accidental `__`-less dispatch helper). If you add, remove, or +re-home a twin, update this table and that test together. diff --git a/src/jolt/core_coll.janet b/src/jolt/core_coll.janet index f02c8c4..6ce4901 100644 --- a/src/jolt/core_coll.janet +++ b/src/jolt/core_coll.janet @@ -1,5 +1,11 @@ # Jolt Core — collections, transducers, seqs, HOFs, constructors # Extracted from core.janet (jolt-nma8, phase 2b split). +# +# REP vs API: this file holds the Clojure-facing collection ops and dispatches +# on `:jolt/type` over the internal persistent structures, whose representations +# live elsewhere: persistent vector → pv.janet, list → plist.janet, hash map → +# phm.janet, set → phs.janet, lazy seq → lazyseq.janet. Grep a structure's file +# header for the primitive (pv-*/pl-*/phm-*/phs-*/ls-*) it exposes. (use ./types) (use ./phm) @@ -535,6 +541,10 @@ [rf init coll] (reduce-with-reduced rf init coll)) +# SEED-TWIN: transduce is overlay-public (jolt-core/clojure/core/20-coll.clj); +# this seed copy is NOT registered in core-bindings. It survives only as the +# helper core-into calls below — user `transduce` resolves to the overlay. The +# asymmetry with `into` (seed-public) is intentional; docs/seed-overlay-registry.md. (defn core-transduce "(transduce xform f coll) or (transduce xform f init coll)." [xform f & rest] diff --git a/src/jolt/core_types.janet b/src/jolt/core_types.janet index b25d28e..c23d2d5 100644 --- a/src/jolt/core_types.janet +++ b/src/jolt/core_types.janet @@ -50,6 +50,10 @@ # Sorted-coll tag checks + entries view, defined this early because canon-key, # empty?, and jolt-equal? (all below) need them. The sorted-coll SEMANTICS are # pure Clojure (core/25-sorted.clj); see the dispatch section further down. +# SEED-TWIN: sorted-map?/sorted-set?/sorted? also live in the overlay +# (jolt-core/clojure/core/25-sorted.clj); the overlay copies are the public ones +# (NOT in core-bindings). These seed copies exist only for earlier-tier seed +# dispatch. Change both copies together — docs/seed-overlay-registry.md. (defn core-sorted-map? [x] (and (table? x) (= :jolt/sorted-map (x :jolt/type)))) (defn core-sorted-set? [x] (and (table? x) (= :jolt/sorted-set (x :jolt/type)))) (defn core-sorted? [x] (or (core-sorted-map? x) (core-sorted-set? x))) @@ -194,6 +198,9 @@ # Predicates # ============================================================ +# SEED-TWIN: char? is also defined in the overlay (jolt-core/clojure/core/ +# 20-coll.clj) and the overlay copy is the public one (NOT in core-bindings). +# This seed copy is internal type dispatch only. docs/seed-overlay-registry.md. (defn core-char? [x] (and (struct? x) (= :jolt/char (x :jolt/type)))) (defn char-code [c] (c :ch)) (defn char->string [c] (string/from-bytes (c :ch))) diff --git a/src/jolt/lazyseq.janet b/src/jolt/lazyseq.janet index 6ad5ead..097072a 100644 --- a/src/jolt/lazyseq.janet +++ b/src/jolt/lazyseq.janet @@ -3,7 +3,11 @@ # Model: a thunk returns nil (empty) or a [first-val, rest-thunk] pair; each # step produces one element + a thunk for the rest. Supports self-referencing # sequences like fib-seq. Self-contained (janet builtins only) — the Clojure -# seq layer (core.janet) and the interpreter build on these primitives. +# seq layer (core_coll.janet) and the interpreter build on these primitives. +# +# REP vs API: this file is ONLY the lazy-cell representation (ls-* primitives). +# The Clojure-facing seq ops (first/rest/seq/cons/count dispatch, realization) +# live in core_coll.janet, branching on `:jolt/type :jolt/lazy-seq`. # # Extracted from phm.janet (jolt-bvek): a lazy sequence has nothing to do with # hash maps; both were tagged tables, which is why they shared a file. diff --git a/src/jolt/phm.janet b/src/jolt/phm.janet index 9ba69ee..85fd3f5 100644 --- a/src/jolt/phm.janet +++ b/src/jolt/phm.janet @@ -5,6 +5,12 @@ # ~12-entry linear scan per get (the jolt-s3y map-read regression). The bucket # count is derived from (length (m :buckets)), so marshaled images from before # this change keep working. +# +# REP vs API: this file is ONLY the hash-map representation (phm-* primitives). +# The Clojure-facing map ops (assoc/dissoc/get/conj/count/seq dispatch, nil-key +# handling, merge) live in core_coll.janet / core_types.janet, which recognize +# phm by its `:jolt/deftype` string. PersistentHashSet is layered on top in +# phs.janet; LazySeq (historically here) now lives in lazyseq.janet. (def- initial-buckets 8) diff --git a/src/jolt/phs.janet b/src/jolt/phs.janet index 6d907fb..536ba50 100644 --- a/src/jolt/phs.janet +++ b/src/jolt/phs.janet @@ -1,6 +1,11 @@ # PersistentHashSet — a set backed by a PersistentHashMap (members are keys # mapped to true). Extracted from phm.janet (jolt-bvek) so phm.janet is purely # the hash map; the set is a thin layer over it. +# +# REP vs API: this file is ONLY the set representation (phs-* primitives). The +# Clojure-facing set ops (conj/disj/contains?/count/seq dispatch, set-as-fn +# membership) live in core_coll.janet / core_types.janet, branching on +# `:jolt/type :jolt/set`. (use ./phm) diff --git a/src/jolt/plist.janet b/src/jolt/plist.janet index 680f3b0..a32a284 100644 --- a/src/jolt/plist.janet +++ b/src/jolt/plist.janet @@ -12,6 +12,10 @@ # # `:rest` may be a plain array/tuple so `(conj some-list x)` needn't copy the # original list — the node just references it. pl->array materializes the chain. +# +# REP vs API: this file is ONLY the cons-list representation (pl-* primitives). +# The Clojure-facing list/seq ops (conj/cons/first/rest/count/seq dispatch) live +# in core_coll.janet, which branches on `:jolt/type :jolt/plist`. (defn plist? [x] (and (table? x) (= :jolt/plist (get x :jolt/type)))) diff --git a/src/jolt/pv.janet b/src/jolt/pv.janet index 8e643ed..11e3587 100644 --- a/src/jolt/pv.janet +++ b/src/jolt/pv.janet @@ -11,6 +11,11 @@ # :tail tail} a tuple of up to 32 trailing elements (append fast-path) # # Trie nodes are immutable tuples so unchanged subtrees are shared by identity. +# +# REP vs API: this file is ONLY the trie representation (pv-* primitives). The +# Clojure-facing vector ops (nth/conj/assoc/subvec, the count/seq/first/rest +# dispatch, and tuple↔pvec polymorphism) live in core_coll.janet / +# core_types.janet, which dispatch on `:jolt/type :jolt/pvec`. (def- bits 5) (def- width 32) # 2^bits diff --git a/test/unit/seed-overlay-registry-test.janet b/test/unit/seed-overlay-registry-test.janet new file mode 100644 index 0000000..b89315c --- /dev/null +++ b/test/unit/seed-overlay-registry-test.janet @@ -0,0 +1,104 @@ +# Drift check for the seed ↔ overlay boundary (refactor phase 5d). +# +# Recomputes the dispatch-twin set from source and asserts it matches what +# docs/seed-overlay-registry.md documents. A twin is a name with BOTH a seed +# `core-X` defn (src/jolt/*.janet) and an overlay `(defn X …)` +# (jolt-core/clojure/core/*.clj). If you add, remove, or re-home a twin, update +# the doc table and the EXPECTED-TWINS set below together. +# +# This is source analysis, not Clojure eval — a plain Janet test (no harness). + +(def repo-root + # test/unit/ -> repo root is two levels up + (let [d (os/cwd)] d)) + +(defn- slurp-safe [path] + (try (slurp path) ([_] nil))) + +(defn- files-with-ext [dir ext] + (def acc @[]) + (each name (os/dir dir) + (def full (string dir "/" name)) + (case (os/stat full :mode) + :directory (array/concat acc (files-with-ext full ext)) + :file (when (string/has-suffix? ext name) (array/push acc full)))) + acc) + +# Collect bare names from `(defn NAME` / `(defn- NAME`, optionally stripping a +# `prefix` (e.g. "core-") and keeping only those that had the prefix. +(defn- defn-names [src prefix] + (def names @{}) + (def pat (peg/compile + ~{:sym (capture (some (+ (range "az" "AZ" "09") (set "*?!<>=+/.&_-")))) + :defn (* "(defn" (? "-") (some (set " \t")) :sym) + :main (some (+ :defn 1))})) + (each m (or (peg/match pat src) @[]) + (when (string? m) + (if prefix + (when (string/has-prefix? prefix m) + (put names (string/slice m (length prefix)) true)) + (put names m true)))) + names) + +(defn- merge-into [dst src] (eachk k src (put dst k true)) dst) + +# --- seed core-X names across src/jolt/*.janet --- +(def seed-names @{}) +(each f (files-with-ext (string repo-root "/src/jolt") ".janet") + (merge-into seed-names (defn-names (slurp f) "core-"))) + +# --- overlay public defns across jolt-core/clojure/core/*.clj --- +(def overlay-names @{}) +(each f (files-with-ext (string repo-root "/jolt-core/clojure/core") ".clj") + (merge-into overlay-names (defn-names (slurp f) nil))) + +# --- twins = intersection --- +(def twins @{}) +(eachk k seed-names (when (get overlay-names k) (put twins k true))) + +(def EXPECTED-TWINS + {"char?" true "sorted?" true "sorted-map?" true "sorted-set?" true + "transduce" true}) + +(defn- keyset [t] (sort (keys t))) + +(unless (deep= (keyset twins) (keyset EXPECTED-TWINS)) + (error (string "seed↔overlay twin drift!\n" + " computed: " (string/join (keyset twins) ", ") "\n" + " expected: " (string/join (keyset EXPECTED-TWINS) ", ") "\n" + " Update docs/seed-overlay-registry.md and EXPECTED-TWINS together."))) + +# --- core-bindings registered keys (the public/dispatch registration table) --- +(def core-src (slurp (string repo-root "/src/jolt/core.janet"))) +(def bind-start (string/find "@{" core-src (string/find "core-bindings" core-src))) +# brace-match from the `@{` to its close +(defn- match-brace [src open] + (var depth 0) (var i open) (var end nil) + (while (and (< i (length src)) (nil? end)) + (case (src i) + (chr "{") (++ depth) + (chr "}") (do (-- depth) (when (= depth 0) (set end i)))) + (++ i)) + end) +(def bind-end (match-brace core-src (+ bind-start 1))) +(def bind-region (string/slice core-src bind-start (+ bind-end 1))) +(def registered @{}) +(each m (or (peg/match ~(some (+ (* "\"" (capture (some (if-not "\"" 1))) "\"") 1)) bind-region) @[]) + (put registered m true)) + +# Twins must NOT be registered — the overlay copy shadows; the seed copy is +# internal-only. A twin sneaking into core-bindings means the seed copy is +# masquerading as the public binding. +(eachk t EXPECTED-TWINS + (when (get registered t) + (error (string "twin '" t "' is registered in core-bindings — it should be " + "overlay-public only (see docs/seed-overlay-registry.md)")))) + +# The seed-public anchors must stay registered (guards the into/transduce +# asymmetry the registry documents). +(each anchor ["into" "reduce"] + (unless (get registered anchor) + (error (string "seed-public anchor '" anchor "' missing from core-bindings")))) + +(print "seed↔overlay registry: " (length (keys twins)) " twins, " + (length (keys registered)) " registered bindings — OK")