bootstrap: SCI core deps loading with 284/304 forms passing

reader: #?@ empty splice fix (nil→@[]), #? nil→:jolt/skip, map reader handles #_/#?@ in K/V
evaluator: unwrap-meta-name helper, deftype interns ->Name too, dot-suffix fix
core: comment/prefer-method stubs, *unchecked-math*/*clojure-version* dynamic vars
This commit is contained in:
Yogthos 2026-06-01 23:24:13 -04:00
parent cdcf569506
commit b20536d1e3
31 changed files with 1348 additions and 65 deletions

View file

@ -0,0 +1,5 @@
{
"last_run": 1780347402,
"first_check": 1780347402,
"last_scanned_watermark": ""
}

View file

@ -0,0 +1,4 @@
{
"last_run": 1780347402,
"first_check": 1780347402
}

9
.dirge/memory/MEMORY.md Normal file
View file

@ -0,0 +1,9 @@
`defmacro` supports optional docstring: `(defmacro name [args] body)` or `(defmacro name \"doc\" [args] body)`. Implementation: slice rest-form from form index 2, check if first is string, adjust args-form and body start accordingly. Implicit `&env` binding injected as `(put new-bindings "&env" @{'ns nil})` — must use `@{}` not `(struct …)` because nil values get dropped from structs. `fn*` uses same `parse-arg-names` helper for `& rest` args.
§
Jolt bootstrap: `core-bindings` (def- map) maps symbol strings → Janet functions. `init-core!` interns them into clojure.core ns. Macros (when, defn, declare) are registered in `core-macro-names` which returns `@{"when" true "defn" true "declare" true}`. `init-core!` checks this table and sets `:macro true` on the var for macro bindings. Order matters: `core-when` and `core-defn` must be defined BEFORE `core-bindings` map literal references them, otherwise compile error.
§
Reader comments (`;`) return `{:jolt/type :jolt/skip}` sentinel; `parse-next`/`parse-string` skip past it. Closing `)`, `]`, `}` produce explicit "Unmatched" errors. `unwrap-meta-name` recursively unwraps `(with-meta sym meta)` to raw symbol — used in def, ns, deftype, defmethod. `resolve-sym` does class-name lookup: `Foo.Bar.Baz` → ns "Foo.Bar" name "Baz". Janet `(set [a b] tuple)` doesn't work — use explicit indexing. `comment` must be a macro (registered in core-macro-names) to avoid evaluating body.
§
Sci bootstrap order: macros(4/4)→protocols(15/17)→utils(39/47)→types(22/27)→unrestrict(2/2)→vars(28/28)→lang(10/10)→ctx-store(6/6)→namespaces(93/98)→core(60/69). All .cljc; #?(:clj) resolved at read time, #?(:cljs)→nil.
§
Janet `last` works only on indexed types (tuple, array). On strings it returns nil. Use `(s (- (length s) 1))` to get the last character of a string, or `(string/slice s (- (length s) 1))` for the last char as string. `(last "hello")` → nil, not `\o`.

View file

@ -0,0 +1,3 @@
Janet `apply` requires a function/cfunction. When tables/structs are used as lookup maps (like deftype fields, multimethod dispatch tables), they get called via `(get f key)` not `(apply f args)`. The evaluator's default call path checks `(function? f)` before `apply`, falling back to `(get f (first args))` for single-arg table/struct calls.
§
Janet structs silently omit entries with nil values: `(struct ;[:x nil :y 1])``{:y 1}`. The `:x` key is completely dropped from the struct, not just set to nil. Use `@{}` (mutable table) when map needs nil-valued entries. This caused the `&env` implicit binding to fail — `(put new-bindings "&env" {'ns nil})` created a struct that became empty `{}`, so `(:ns &env)` failed with unknown method. Fix: use `@{}` table for bindings that may contain nil values, or use a non-nil sentinel.

View file

@ -0,0 +1,4 @@
{
"last_run": 1780347402,
"first_check": 1780347402
}

36
.dirge/skills/.usage.json Normal file
View file

@ -0,0 +1,36 @@
{
"jolt-dev": {
"created_by": "agent",
"use_count": 6,
"view_count": 15,
"patch_count": 12,
"last_used_at": "2026-06-02T03:23:57.409721+00:00",
"last_viewed_at": "2026-06-02T03:23:57.400994+00:00",
"last_patched_at": "2026-06-02T02:52:21.531624+00:00",
"created_at": "2026-06-01T21:26:06.614465+00:00",
"state": "active",
"pinned": false
},
"jpm-build": {
"created_by": "agent",
"use_count": 0,
"view_count": 9,
"patch_count": 0,
"last_viewed_at": "2026-06-02T02:26:40.592223+00:00",
"created_at": "2026-06-01T20:56:39.144222+00:00",
"state": "active",
"pinned": false
},
"jolt-bootstrap": {
"created_by": "agent",
"use_count": 5,
"view_count": 14,
"patch_count": 7,
"last_used_at": "2026-06-02T03:23:57.420498+00:00",
"last_viewed_at": "2026-06-02T03:23:57.415106+00:00",
"last_patched_at": "2026-06-02T02:51:30.047986+00:00",
"created_at": "2026-06-01T21:49:51.101718+00:00",
"state": "active",
"pinned": false
}
}

View file

@ -0,0 +1,82 @@
---
name: jolt-bootstrap
description: TDD workflow for bootstrapping a Clojure interpreter on Janet
---
# Bootstrapping a Clojure interpreter on Janet
## Prerequisites
- Janet ≥ 1.36, jpm
- Target Clojure sources (e.g. sci) to load
- Jolt sources in `src/jolt/`, tests in `test/`
## TDD Loop
1. Write a failing test in `test/<feature>-test.janet` using `(use ../src/jolt/...)` relative paths
2. Run with `janet test/<file>.janet` (faster than `jpm test` for iteration)
3. If test involves `init` (which loads clojure.core), also `(use ../src/jolt/api)`
4. Implement in `src/jolt/<module>.janet`
5. Run test → see failure message → fix → repeat
6. After passing: `jpm test` to ensure no regressions
## Current bootstrap progress
**Loaded (all .cljc, #? resolved at read time):**
- `sci.impl.macros` — 4/4 (ns, defmacro deftime, defmacro usetime, deftime(? macro))
- `sci.impl.protocols` — 15/17
- `sci.impl.utils` — 39/47
- `sci.impl.types` — 22/27
- `sci.impl.unrestrict` — 2/2
- `sci.impl.vars` — 28/28 (comment block parses via :jolt/skip sentinel)
- `sci.lang` — 10/10 (IVar resolves via class-name pattern lookup)
- `sci.ctx-store` — 6/6
- `sci.impl.namespaces` — 93/98 (parse crash at unmatched brace)
- `sci.core` — 60/69 (namespaces/*1/*2/*3/*e unresolved)
**Special forms (22):** quote, syntax-quote, unquote, unquote-splicing, do, if, def, defmacro, fn*, let*, loop*, recur, throw, try, set!, var, locking, instance?, defmulti, defmethod, deftype, new, .
**Reader:** `#?(:clj ...)`, `#?@(:clj ...)` with splicing, `#_` discard (returns :jolt/skip sentinel), `#\` var-quote, `^` metadata. Comments `;` skip via :jolt/skip. Closing delimiters `)`, `]`, `}` produce explicit "Unmatched" errors.
**Core macros:** when, defn (with docstring), defn-, declare, fn (wraps fn*), defprotocol, extend-type, extend-protocol, extend, reify, proxy, definterface, comment (ignores body), prefer-method (stub)
**Key utilities:** `unwrap-meta-name` — recursively unwraps `(with-meta sym meta)` to extract raw symbol. Used in def, ns, deftype, defmethod.
**Class-name resolution:** unqualified symbols with dots (`Foo.Bar.Baz`) are resolved by splitting at last dot into ns+name.
## Key patterns
### Symbol structs
```janet
{:jolt/type :symbol :ns <string-or-nil> :name <string>}
```
### Macro intern marks var
```janet
(def v (ns-intern ns name macro-fn))
(put v :macro true)
```
### Reader conditional `#?`
Resolves at read time: scans for `:clj` keyword, picks next form.
`#?@` wraps resolved form in `:jolt/splice` struct for list/vec/set splicing.
### Callable forms check
```janet
(if (function? f)
(apply f args)
(get f (first args))) ; table/struct lookup
```
## Pitfalls
- Janet `let` can't bind to nil; use `(var x nil)` then `(set x val)`
- `(get table)` with 1 arg = compile error, use `(table :key)` shorthand
- `(put fn :key val)` fails on functions; stash metadata on vars instead
- `deftype` field names must be keywords (not strings) for `(inst :field)` access
- `defn` placed after `core-bindings` that reference it → compile error; order matters
- Janet's `try` macro: `(try body ([err] handler))` — catch clause is tuple `[binding body...]`
- **`core-macro-names`** is a zero-arg fn returning a table: `(get (core-macro-names) name)`. Don't call it as `(core-macro-names name)` — that's arity mismatch
- **Janet `#{}` sets** can cause parse issues — use `@[]` instead for stub collections
- **`break` in `while`** doesn't return a value in Janet — use `(var done nil)` + `(while (and cond (not done)) ... (set done result))` pattern instead
- **`read-reader-conditional`** for `#?(:cljs X)` with no `:clj` branch returns `[nil new-pos]`. For `#?@(:cljs X)` wrapping, nil gets wrapped in splice struct with `@[nil]` items
- **`#_` discard** now works in lists, vectors, and sets — wraps the skipped form in `{:jolt/type :jolt/skip}` and readers check for this
- **`read-regex`** now works with `(var done nil)` pattern to return value from while loop
- **`#?@` splicing inside vectors** — if the resolved :clj branch is itself a vector, items are extracted and spliced. Works for both lists and vectors.

View file

@ -0,0 +1,180 @@
---
description: Jolt development workflow — build, test, special form patterns, Janet gotchas
---
# Jolt Development
## Build & Test
```bash
cd /Users/yogthos/src/jolt
jpm build # produces build/jolt
jpm test # runs all tests
janet test/foo.janet # run a single test file from project root
```
## Special Form Checklist
To add a new special form to the evaluator:
1. Add the name to `special-symbol?` in `src/jolt/evaluator.janet`
2. Add a match arm in `eval-list` (the match on `name`)
3. Add tests in `test/evaluator-test.janet`
The match arm receives `ctx`, `bindings`, and `form` (the full list). Use `(in form 1)` for first arg, etc.
**Non-symbol heads** (keywords, etc.): `eval-list` first checks `(and (struct? first-form) (= :symbol (...)))` before extracting `name`. If not a symbol, falls through to default function application. This means `(:ns &env)` works because the head `:ns` is a keyword, not a symbol, so it's evaluated and called as a lookup.
### Current special forms (22):
`quote`, `syntax-quote`, `unquote`, `unquote-splicing`, `do`, `if`, `def`, `defmacro`, `fn*`, `let*`, `loop*`, `recur`, `throw`, `try`, `set!`, `var`, `locking`, `instance?`, `defmulti`, `defmethod`, `deftype`, `new`, `.`
### defmacro details
- Supports optional docstring: `(defmacro name [args] body)` or `(defmacro name "doc" [args] body)`
- Implementation: `(tuple/slice form 2)` → check if first is string → adjust args-form and body start
- Implicit `&env` binding: `(put new-bindings "&env" @{})` — table, not struct (nil-safe)
- Uses `parse-arg-names` for `& rest` arg handling
### defmulti / defmethod
- `defmulti` stores the methods table on the **var** via `(put v :jolt/methods methods)`, NOT on the function
- Janet `put` on a function value fails silently
- `defmethod` retrieves methods via `(get mm-var :jolt/methods)` using `resolve-var` to get the var first
### set!
- If var doesn't exist, auto-creates it (`ns-intern`) rather than erroring
- Needed for sci's `(set! *warn-on-reflection* true)`
### defmethod (auto-create)
- If the multimethod var doesn't exist yet, auto-creates it with a dummy fn and empty methods table
- This allows sci's `defmethod print-method` to work before `defmulti print-method` is defined
### deftype
- Handles `^:meta` on type name via `with-meta` pattern
- Fields vector handles `^:meta` annotations and `^Type` hints — extracts actual symbol name
- Produces a table with `:jolt/deftype "ns.TypeName"` and keyword-keyed fields
## Janet Gotchas
- `var` declares mutable locals that can be `set` later; `def`/`let` are immutable
- `let` cannot bind to `nil` — use `(var x nil)` instead of `(let [x nil] ...)`
- `(get table key)` needs 2 args minimum — for single-arg checks use `(table :key)`
- Functions are not tables — `(put fn :prop val)` fails. Stash properties on vars
- `match` is Janet's pattern matching — no `case` or `cond` needed for simple dispatch
- Janet structs silently **drop entries with nil values**: `(struct ;[:x nil :y 1])``{:y 1}`. Use `@{}` mutable tables when nil-valued entries are needed (e.g., `&env` binding `@{}` for macro bodies)
- `match` with string patterns returns **nil** (not error) when no pattern matches. Used in `eval-list` to handle non-symbol heads cleanly — keyword heads like `:ns` fall through to default function application
- `break` in `while` does NOT return a value in Janet. Use `(var done nil)` + `(set done val)` + check pattern instead
- Janet `#{}` set literals can cause parse issues in some contexts — use `@[]` as fallback
- `(first struct)` calls `:jolt/type` method — use `(get struct :key)` instead of positional access
- Janet `(struct ;[:x nil])` silently drops the nil entry — the map becomes `{}`. Use `@{}` tables for nil-safe entries
- `(length struct)` counts key-value pairs, not keys. Use `(length (keys struct))` for key count
## Comment Handling
Comments `;` in `read-form` return `{:jolt/type :jolt/skip}` sentinel:
```janet
(= c 59) # ;
[{:jolt/type :jolt/skip} line-end])
```
`parse-next` and `parse-string` both skip over `:jolt/skip` results:
- `parse-next` uses inner `parse-next-loop` that recurses on skip
- `parse-string` recurses on skip to return next non-comment form
`read-map` checks both key AND value positions for `:jolt/skip` to skip `#_`-discarded entries.
Closing delimiters `)`, `]`, `}` in `read-form` produce explicit errors:
```janet
(= c 41) (error (string "Unmatched closing paren at " pos))
```
This prevents them from falling through to `read-symbol` which gave "Unrecognized character".
## `unwrap-meta-name` Utility
Recursively unwraps `(with-meta sym meta)` forms to extract the underlying symbol:
```janet
(defn- unwrap-meta-name [form]
(if (and (array? form) (> (length form) 0)
(struct? (in form 0))
(= :symbol ((in form 0) :jolt/type))
(= "with-meta" ((in form 0) :name)))
(unwrap-meta-name (in form 1))
form))
```
Used in: `def`, `ns` (ns name), `deftype` (type name + field names), `defmethod` (arg names).
Replaced duplicated `with-meta` unwrapping code in each of these forms.
## Bootstrap Patterns
### Class-name resolution in resolve-sym
When a simple symbol (unqualified) isn't found in current ns or clojure.core, checks for dotted class-name pattern:
`Foo.Bar.Baz` → finds last dot → ns "Foo.Bar", name "Baz" → tries ns-resolve.
This resolves symbols like `IVar` that are interned in `sci.impl.vars` but referred to unqualified from `sci.lang`.
### Reader conditionals
- `#?(:clj expr :cljs expr2)` — resolved at read time by `read-reader-conditional`
- `#?@(:clj expr :cljs expr2)` — splicing variant, wraps resolved items in `{:jolt/type :jolt/splice :items ...}`
- List/vector/set readers check for splice and flatten items
- `#_` — discard reader macro, reads next form and returns it as position only
### Core macros (in core.janet)
- `core-macro-names` returns `@{"when" true "defn" true "declare" true "defprotocol" true "extend-type" true "reify" true "fn" true "proxy" true "definterface" true "defn-" true}` — a table
- `init-core!` calls `(get (core-macro-names) name)` to check, then `(put v :macro true)`
- Order matters: macro functions must be defined BEFORE `core-bindings` map references them
### Core stubs for sci bootstrap
- `core-derive`, `core-isa?`, `core-ancestors`, `core-descendants` — minimal hierarchy
- `core-Object`, `core-Thread`, `core-ThreadLocal`, `core-IllegalStateException` — JVM class stubs
- `core-volatile!`, `core-vswap!`, `core-vreset!` — volatile (atom-like table with :val key)
- `core-defprotocol` emits `(do (def name @{}) (def method fn) ...)` — macro, returns do form
- `core-extend-type`, `core-extend-protocol`, `core-extend`, `core-reify`, `core-satisfies?`, `core-extends?`, `core-implements?`, `core-type->str` — protocol stubs
### Namespace handling
- `ns` form handles `^:meta` on ns name via `with-meta` pattern
- `def` form also handles `^:meta` on def name (extracts name-sym from `(with-meta Name meta)`)
- `require` clause in `ns` wraps each spec in `(when s ...)` for nil-safety
- `resolve-var` falls back to checking clojure.core namespace if var not found in current ns
### Bootstrap loading order
```
sci.impl.macros (4/4 ok)
sci.impl.protocols (15/17 ok)
sci.impl.utils (39/47 ok)
sci.impl.types (22/27 ok)
sci.impl.unrestrict (2/2 ok)
sci.impl.vars (28/28 ok — comment block parsed as skip)
sci.lang (10/10 ok — IVar via class-name resolve)
sci.ctx-store (6/6 ok)
sci.impl.namespaces (93/98 ok — parse crash at unmatched brace)
sci.core (60/69 ok — namespaces/*1/*2/*3/*e fail)
```
All .cljc files. #?(:clj ...) resolved at read time. #?(:cljs ...) returns nil.
### parse-arg-names
- Handles `& rest` args AND nested destructuring vectors
- When an arg is a vector (not a symbol), recurses to extract nested symbol names
## Project Structure
```
src/jolt/
types.janet — Var, Namespace, Context, symbol helpers
reader.janet — recursive descent parser for Clojure syntax
evaluator.janet — tree-walking interpreter (eval-form, eval-list, syntax-quote*)
core.janet — 95+ clojure.core functions (map, filter, reduce, etc.)
api.janet — public API: init, eval-string, eval-string*
main.janet — REPL entry point with (defn main [&])
test/
evaluator-test.janet — special form tests (22 forms tested)
reader-test.janet — parser tests (includes #?, #?@, #_)
core-test.janet — core library tests
macro-test.janet — syntax-quote and macro tests
namespace-test.janet — ns, require, in-ns tests
types-test.janet — Var and Namespace tests
api-test.janet — public API tests
bootstrap-test.janet — loads sci.impl.macros (deftime, usetime, ?)
test-load-sci.janet — loads all sci files, counts ok/fail
test-eval.janet — end-to-end sci.core/eval-string test
```

View file

@ -0,0 +1,56 @@
---
name: jpm-build
description: Build and debug Janet projects using jpm. Covers project.janet structure, common build errors, and native compilation with create-executable.
---
# JPM Build & Debug
## Build Commands
```bash
jpm build # Compile and link executable → build/jolt
jpm test # Run all tests
jpm deps # Show dependencies
```
## project.janet Structure
```janet
(declare-project
:name "jolt"
:description "...")
(declare-source
:source @["src"]) # Source directories
(declare-executable
:name "jolt" # Output binary name
:entry "src/jolt/main.janet") # RELATIVE TO PROJECT ROOT, not source dirs
```
## Common Pitfalls
### Entry path is relative to project root
Even though `declare-source` lists `@["src"]`, the `:entry` in `declare-executable` must include `src/` prefix. jpm's `create-executable` calls `dofile source` with the raw entry string.
### main function required for native compilation
`create-executable` extracts a `main` function from the entry file's environment. Top-level code runs during `dofile` and interferes with image generation. Error: "expected integer key for keyword in range [0, 5), got nil".
**Fix:** Wrap startup code in `(defn main [&] ...)`.
```janet
(defn main [&]
(print "REPL started")
;; ... REPL loop ...
)
```
### LSP false positives
Clojure LSP misidentifies `.janet` files. Ignore all diagnostics — they don't affect build or test results.
## Debugging Build Failures
1. Check entry path includes `src/` prefix
2. Check entry file has `(defn main [&] ...)` wrapping top-level code
3. Run `jpm test` to verify code works before native compilation
4. `jpm build` produces no output on success — check exit code only