Phase 11: Fix pre-existing failures — 316/317 passing

- types.janet: ns? now accepts both structs and tables
  (defensive for namespace-like tables created via @{...})
- core.janet: wire comment macro into core-bindings
  (was in core-macro-names but missing from core-bindings)
- sci/lang_stubs.clj: minimal SCI type stubs for bootstrap
  (IBox, HasName, IVar, DynVar protocols, SciUnbound/Var/Namespace deftypes)
- test-load-sci.janet: load stubs before SCI source files,
  removed broken preprocessor

Before: 315 ok, 2 fail (SciVar + array/buffer)
After:  316 ok, 1 fail (only deftype in lang.cljc remains)

The remaining failure is lang.cljc's SciVar deftype with #?@ inserts
for JVM/CLJS-specific protocol implementations — a known limitation
for Phase 15 (SCI bootstrap).
This commit is contained in:
Yogthos 2026-06-03 12:30:11 -04:00
parent 1c1100e2f0
commit 09b4e3e1bd
10 changed files with 179 additions and 69 deletions

View file

@ -1,9 +1,7 @@
REPL: `main.janet` initializes context and sets current ns to "user". `print-value` renders scalars inline (prin) and collections via `print-collection` which recursively calls print-value for nested rendering. Collections: tuples→[v1 v2], arrays→(v1 v2), structs→{k v}, deftype tables→{k v} (filtering :jolt/deftype :cnt :buckets :_meta :jolt/type :phm), sets→#{v}. Jolt symbol structs render as `name` or `ns/name`.
§
REPL collection rendering: print-value in main.janet uses cond with print-value→print-collection mutual recursion (needs forward var declaration). Tuples→[v1 v2], arrays→(v1 v2), structs→{k v}, sets→#{v}, keywords→:kw, symbols→name or ns/name. Use prin for scalars, print only after collection closing bracket. REPL starts in wrong ns after loading persistent structures — must ctx-set-current-ns to "user" after init.
§
fn* special form dispatch: fn* form emitted by macros MUST be @[...] (array) to enter eval-list's special form match. If wrapped in [...] (tuple), eval-form hits the (tuple? form) branch which maps over items instead of dispatching fn*. Same applies to register-method, protocol-dispatch, and other special form calls emitted by macros — all must be array-wrapped.
§
Janet's struct? returns true for tuples — cond forms in print-value/eval-form MUST check (tuple? x) before (struct? x) or (get x :key). Otherwise Janet sees a tuple, says yes to struct?, and calls (get tuple :name) which fails with "expected integer key for tuple in range [0, N), got :name". This hit us in print-value rendering and eval-form struct handling.
§
Protocol system: Type registry in context env (:type-registry) maps type-tag→proto-name→method-name→fn. Three dispatch special forms: protocol-dispatch (resolves method via registry or reified methods), register-method (stores impl in registry), make-reified (creates anonymous object with :jolt/protocol-methods). fn* forms emitted by extend-type/extend-protocol MUST be @[...] (array) for eval-list dispatch. Protocols are maps with :jolt/type :jolt/protocol and :methods map.
§
Project stats after Phase 10: 7,370 total lines across 35 source/test files. Key sources: compiler.janet (869 lines), evaluator.janet (869 lines), core.janet (1373 lines), types.janet (441 lines), reader.janet (463 lines), main.janet (125 lines), api.janet (93 lines), loader.janet (79 lines), phm.janet (199 lines). Test suite: 315 ok, 2 fail (pre-existing SciVar bootstrap + array/table/buffer errors). 9 .clj standard library modules under src/jolt/clojure/ and src/jolt/jolt/. All tests run via `jpm test`.
§
REPL print-value uses buffer-based output: write-value/v buf appends formatted strings via buffer/push-string, then print-value creates buffer, builds string, and does a single (print (string buf)). This prevents Janet C runtime from interleaving native <tuple 0x...> output between prin statements in jpm build executables. Cond catch-all must use true clause: Janet's cond treats plain expressions as tests, so (push-str buf (string v)) at end of cond would be evaluated as a test — need true before it.

View file

@ -1,5 +1,5 @@
Janet's `case` for multi-arity dispatch: `(defn f [& args] (case (length args) 1 ... 2 ...))`. Used in core-derive, core-isa?, core-ancestors, core-descendants because Janet doesn't support Clojure-style `([arg1] body1) ([arg1 arg2] body2)` multi-arity defn syntax.
Janet's `break` does NOT return a value from a loop — `(break val)` in a `while` loop just exits, discarding val. To return from a loop, set a variable before break: `(do (set result val) (break))`. Hit us in phm-bucket-assoc where `(break nb)` silently returned nil instead of the new bucket. Fixed by capturing index before break and building the result array after the loop.
§
Janet's boolean function doesn't exist — use (if x true false). Janet's defn doesn't support Clojure-style multi-arity syntax ([args] body) — use [& args] with case (length args) dispatch. fn? exists as Janet builtin (not Jolt core fn) — use (or (function? x) (cfunction? x)) in tests.
Clojure .clj source files loaded via eval-form cannot have docstrings. If a defn form has 5 elements (defn, name, docstring, params, body), the evaluator's defn macro handler gets 4 args instead of 3, breaking with "macro arity mismatch". All .clj files in src/jolt/clojure/ must use 4-element defn forms: (defn name [params] body). Docstrings on defn are a Clojure feature not supported by Jolt's defn macro.
§
Janet's `cond` treats the last position as a test clause, NOT a catch-all body. A bare expression like `(push-str buf val)` in the last position runs as a test (always truthy, but executed for side effects between other cond clauses). Use `true (push-str buf val)` to make it a proper catch-all body. Hit us in buffer-based write-value — raw tuple addresses leaked into output because `(push-str buf (string v))` ran as a test clause between other branches.
PHM internal key leak: Core functions iterating over PHM maps with keys/pairs get internal metadata keys (:jolt/deftype, :cnt, :buckets, :_meta). Always check for set?/phm? first and use type-aware helpers (phm-to-struct, phs-seq, phm-keys, phm-entries) before generic iteration. Hit us in core-merge, core-reduce, core-every?, core-filter which all needed set?/phm? checks added.

View file

@ -1,70 +1,70 @@
{
"jolt-compiler": {
"created_by": "agent",
"use_count": 10,
"view_count": 146,
"patch_count": 9,
"last_used_at": "2026-06-03T13:32:50.977662+00:00",
"last_viewed_at": "2026-06-03T14:17:26.562948+00:00",
"last_patched_at": "2026-06-03T03:03:27.524946+00:00",
"created_at": "2026-06-02T17:54:38.690279+00:00",
"state": "active",
"pinned": false
},
"jolt-bootstrap": {
"created_by": "agent",
"use_count": 9,
"view_count": 156,
"view_count": 243,
"patch_count": 10,
"last_used_at": "2026-06-02T18:45:23.336653+00:00",
"last_viewed_at": "2026-06-03T14:17:26.556567+00:00",
"last_viewed_at": "2026-06-03T16:29:39.837221+00:00",
"last_patched_at": "2026-06-02T03:44:45.935676+00:00",
"created_at": "2026-06-01T21:49:51.101718+00:00",
"state": "active",
"pinned": false
},
"jolt-gotchas": {
"created_by": "agent",
"use_count": 4,
"view_count": 222,
"patch_count": 4,
"last_used_at": "2026-06-03T16:12:06.850822+00:00",
"last_viewed_at": "2026-06-03T16:29:39.856662+00:00",
"last_patched_at": "2026-06-03T16:12:23.546648+00:00",
"created_at": "2026-06-03T03:50:41.474730+00:00",
"state": "active",
"pinned": false
},
"jolt-dev": {
"created_by": "agent",
"use_count": 38,
"view_count": 272,
"patch_count": 46,
"last_used_at": "2026-06-03T16:12:27.833941+00:00",
"last_viewed_at": "2026-06-03T16:29:39.851827+00:00",
"last_patched_at": "2026-06-03T16:12:38.766457+00:00",
"created_at": "2026-06-01T21:26:06.614465+00:00",
"state": "active",
"pinned": false
},
"jpm-build": {
"created_by": "agent",
"use_count": 0,
"view_count": 147,
"view_count": 234,
"patch_count": 0,
"last_viewed_at": "2026-06-03T14:17:26.587368+00:00",
"last_viewed_at": "2026-06-03T16:29:39.867842+00:00",
"created_at": "2026-06-01T20:56:39.144222+00:00",
"state": "active",
"pinned": false
},
"jolt-compiler": {
"created_by": "agent",
"use_count": 11,
"view_count": 234,
"patch_count": 10,
"last_used_at": "2026-06-03T16:12:44.265835+00:00",
"last_viewed_at": "2026-06-03T16:29:39.843393+00:00",
"last_patched_at": "2026-06-03T16:13:25.725903+00:00",
"created_at": "2026-06-02T17:54:38.690279+00:00",
"state": "active",
"pinned": false
},
"jolt-persistent-structures": {
"created_by": "agent",
"use_count": 1,
"view_count": 133,
"patch_count": 0,
"last_used_at": "2026-06-03T03:49:43.520839+00:00",
"last_viewed_at": "2026-06-03T14:17:26.582273+00:00",
"created_at": "2026-06-03T03:35:04.130959+00:00",
"state": "active",
"pinned": false
},
"jolt-dev": {
"created_by": "agent",
"use_count": 37,
"view_count": 184,
"patch_count": 44,
"last_used_at": "2026-06-03T13:32:50.995757+00:00",
"last_viewed_at": "2026-06-03T14:17:26.569463+00:00",
"last_patched_at": "2026-06-03T13:34:25.654327+00:00",
"created_at": "2026-06-01T21:26:06.614465+00:00",
"state": "active",
"pinned": false
},
"jolt-gotchas": {
"created_by": "agent",
"use_count": 3,
"view_count": 134,
"patch_count": 3,
"last_used_at": "2026-06-03T13:35:01.902830+00:00",
"last_viewed_at": "2026-06-03T14:17:26.576167+00:00",
"last_patched_at": "2026-06-03T13:35:14.859032+00:00",
"created_at": "2026-06-03T03:50:41.474730+00:00",
"view_count": 222,
"patch_count": 0,
"last_used_at": "2026-06-03T16:13:31.440242+00:00",
"last_viewed_at": "2026-06-03T16:29:39.861788+00:00",
"created_at": "2026-06-03T03:35:04.130959+00:00",
"state": "active",
"pinned": false
}

View file

@ -44,7 +44,7 @@ Order: qualified ns → local binding → core-symbol → bare symbol
core-renames maps Clojure name strings → Janet function name strings (used by both emitter paths).
core-fn-values maps Janet function name strings → actual function values (used by data-structure emitter only).
MUST keep both in sync. When adding a new core fn, update BOTH.
MUST keep both in sync. When adding a new core fn, update BOTH tables.
**Missing entries → symbol treated as unknown global, returns nil.**
Key name mappings:
@ -68,9 +68,9 @@ Key name mappings:
## Throw/try compilation
- `throw``(error val)` in Janet
- `try/catch``(try body ([err] handler-body))` — Janet uses `([sym] handler)` format
- `try/catch``(try body ([err] handler-body))` — Janet uses `([sym] handler)` format, NOT `(catch sym handler)`
## emi-vector-expr critical fix
## emit-vector-expr critical fix
**Bare tuples in Janet eval are function calls**: `[1 2 3]` tries to call `1` as a function. Always emit `['tuple ...]` or `(tuple ...)`.
@ -106,12 +106,48 @@ raw-form->janet converts Jolt symbols to Janet symbols, recursively for arrays/t
(compile-and-eval form ctx)) ; compile
```
## Protocol dispatch macros
`core-extend-type` and `core-extend-protocol` emit `register-method` call forms. The fn* form MUST be `@[...]` (array) for `eval-list` to recognize it as a special form. Same for the outer call form wrapping `register-method`:
```janet
(defn core-extend-type [type-sym proto-sym & impls]
(each method-spec impls
(def fn-form @[{:name "fn*"} arg-vec ;body]) ; @[...] array
(array/push result @[
{:name "register-method"} ; @[...] array
type-sym proto-sym method-name fn-form])))
```
## defprotocol method dispatch
Protocol methods emit fn forms that delegate to `protocol-dispatch` special form. The fn form uses `@[...]` for fn* and its body so eval-list dispatches correctly:
```janet
(def fn-form @[
{:name "fn*"}
@[{:name "this"} {:name "&"} {:name "rest-args"}]
@[{:name "protocol-dispatch"}
{:name "quote"} protocol-name
{:name "quote"} method-name
{:name "this"}
{:name "rest-args"}]])
```
In register-method, args (type-sym, proto-sym, method-name) are passed raw — evaluator resolves via `(in form N)`.
## PHM integration in core functions
~16 core functions have PHM-aware branches before generic struct/table handling:
`core-get`, `core-assoc`, `core-dissoc`, `core-conj`, `core-contains?`, `core-count`, `core-keys`, `core-vals`, `core-merge`, `core-merge-with`, `core-empty?`, `core-seq`, `core-into`, `core-map?`, `core-=`, `core-hash-map`. Sets later added to 6 of these.
## Test files
- `test/compiler-test.janet` — Phases 0-4 tests (source output + compile-eval + macro tests)
- `test/phase6-final.janet` — Phase 6 comprehensive compile-mode tests (47 assertions)
- `test/phase5-test.janet` — Phase 5 multimethod + hierarchy tests
- `test/phase6-test.janet` — Phase 6 reader extension tests
- `test/phase8-test.janet` — Phase 8 protocol system tests
- `test/hash-map-test.janet` — Phase 2 PersistentHashMap tests
Run: `janet test/<file>.janet` or `jpm test`

View file

@ -18,7 +18,37 @@ jpm test # runs all tests
janet test/foo.janet # run a single test file from project root
```
## Special Form Checklist
## Testing Patterns
```bash
# Single test file
janet test/compiler-test.janet
# Full suite
jpm test
# Phase-specific tests
janet test/phase5-test.janet # multimethods
janet test/phase8-test.janet # protocol system
janet test/phase10-test.janet # standard library
# REPL test — pipe expressions in
printf "(range 10)\n[1 2 3]\n{:a 1}\n" | janet src/jolt/main.janet
```
## Loading .clj Files
`.clj` files are loaded via `eval-form` in the interpreter:
```janet
(def src (slurp "src/jolt/clojure/string.clj"))
(var remaining src)
(while (> (length (string/trim remaining)) 0)
(def [form rest] (parse-next remaining))
(set remaining rest)
(when form (eval-form ctx @{} form)))
```
**Critical constraint:** .clj files must NOT have docstrings on defn forms. Jolt's defn macro only handles 4-element forms: `(defn name [params] body)`. A 5-element form `(defn name "doc" [params] body)` causes "macro arity mismatch".
To add a new special form to the evaluator:
@ -31,7 +61,7 @@ The match arm receives `ctx`, `bindings`, and `form` (the full list). Use `(in f
**Non-symbol heads** (keywords, etc.): `eval-list` first checks `(and (struct? first-form) (= :symbol (...)))` before extracting `name`. If not a symbol, falls through to default function application.
### Current special forms (37):
`quote`, `syntax-quote`, `unquote`, `unquote-splicing`, `do`, `if`, `def`, `defmacro`, `fn*`, `let*`, `loop*`, `recur`, `throw`, `try`, `set!`, `var`, `locking`, `instance?`, `defmulti`, `defmethod`, `deftype`, `new`, `.`, `var-get`, `var-set`, `var?`, `alter-var-root`, `find-var`, `intern`, `alter-meta!`, `reset-meta!`, `disj`, `set?`, `protocol-dispatch`, `register-method`, `make-reified`, `satisfies?`
`quote`, `syntax-quote`, `unquote`, `unquote-splicing`, `do`, `if`, `def`, `defmacro`, `fn*`, `let*`, `loop*`, `recur`, `throw`, `try`, `set!`, `var`, `locking`, `instance?`, `defmulti`, `defmethod`, `deftype`, `new`, `.`, `var-get`, `var-set`, `var?`, `alter-var-root`, `find-var`, `intern`, `alter-meta!`, `reset-meta!`, `disj`, `set?`, `satisfies?`, `protocol-dispatch`, `register-method`, `make-reified`
## Compiler Architecture

View file

@ -54,15 +54,14 @@ A bare expression in the last position of `cond` is treated as a **test** clause
Without `true`, the last expression executes as a side-effect test between branches. Hit us in buffer-based write-value — raw tuple addresses leaked into REPL output.
## Janet `cond` Requires `true` Guard for Catch-All
## REPL: Buffer-Based Output Prevents C-Runtime Interleaving
A bare expression in the last position of `cond` is treated as a **test** clause (not body). It executes between other branches as a side-effect test. Use `true` as the test to make it a proper catch-all:
Janet's C runtime in `jpm build` executables interleaves native `<tuple 0x...>` output between `prin` statements. Solution: build entire output string in a buffer, then output atomically with a single `print` call. Use `write-value/v buf` + `print-value` creates buffer → `print (string buf)`.
```janet
(cond
(nil? x) (buf "nil")
(number? x) (buf (string x))
true (buf (string x))) ; ← `true` required
```
## Janet `struct?` Returns `true` for Tuples
Without `true`, `(push-str buf (string v))` in the last position leaked raw tuple addresses into REPL output.
Always check `(tuple? x)` BEFORE `(struct? x)` in cond forms. Otherwise `(get tuple :key)` fails with "expected integer key for tuple in range [0, N), got :key". Hit us in `print-value` (symbol check on tuples) and `eval-form` struct handling.
## PHM Internal Key Leakage
PHM and set internal keys (`:jolt/deftype`, `:cnt`, `:buckets`, `:_meta`, `:jolt/type`, `:phm`) leak into `pairs`/`keys` iteration. Core fns that iterate collections (`core-merge`, `core-reduce`, `core-every?`, `core-filter`) must check for `set?`/`phm?` first and use type-aware helpers (`phm-to-struct`, `phs-seq`, `phm-keys`, `phm-entries`) before generic iteration.

View file

@ -0,0 +1,32 @@
; Minimal SCI type stubs for Jolt bootstrap
; Provides the deftype definitions that SCI's Clojure source references
; before the full SCI runtime is loaded.
; Protocol stubs for SCI's type system
(defprotocol IBox (setVal [this v]) (getVal [this]))
(defprotocol HasName (getName [this]))
(defprotocol IVar (bindRoot [this v]) (getRawRoot [this]) (toSymbol [this])
(isMacro [this]) (setThreadBound [this v]) (unbind [this]) (hasRoot [this]))
(defprotocol DynVar (dynamic? [this]))
(defprotocol IReified (getInterfaces [this]) (getMethods [this])
(getProtocols [this]) (getFields [this]))
; Unbound sentinel for vars
(deftype SciUnbound [the-var])
; Core SCI types — keep minimal for bootstrap
(deftype Namespace [name mappings aliases imports])
(deftype Var [root name meta macro dynamic ns])
; Store stub (sci.ctx-store provides a global context atom)
(def ctx-store (atom nil))
; Macro helpers from sci.impl.macros
(defn deftime [& body] nil)
(defn usetime [& body] (eval (first body)))
(defmacro ? [& args]
(if (contains? &env (quote &env))
(let [form (first args)]
(if (= :clj (first form))
(second form)
(if (= :cljs (first form)) nil)))))

View file

@ -1334,6 +1334,7 @@
"IllegalStateException" core-IllegalStateException
"definterface" core-definterface
"defrecord" core-defrecord
"comment" core-comment
"resolve" core-resolve
"update" core-update
"copy-core-var" core-copy-core-var

View file

@ -219,7 +219,7 @@
(defn ns?
"Check if x is a Jolt Namespace."
[x]
(and (struct? x) (= :jolt/namespace (x :jolt/type))))
(and (or (struct? x) (table? x)) (= :jolt/namespace (x :jolt/type))))
(defn ns-name
"Return the name symbol of a namespace."

View file

@ -5,6 +5,20 @@
(def ctx (init))
(printf "Loading SCI stubs...\n")
(defn load-stubs [ctx filepath]
(var s (slurp filepath))
(var count 0)
(while (> (length (string/trim s)) 0)
(def [form rest] (parse-next s))
(set s rest)
(++ count)
(when (not (nil? form))
(eval-form ctx @{} form)))
(printf " Loaded %d stub forms\n" count))
(load-stubs ctx "src/jolt/clojure/sci/lang_stubs.clj")
(defn load-file [ctx path]
(var s (slurp path))
(var count 0)