Phase 2: PersistentHashMap implementation + core function integration

- phm.janet: standalone PHM module — phm?, phm-get, phm-assoc,
  phm-dissoc, phm-entries, phm-to-struct, make-phm
  Bucket-based hash map with copy-on-write semantics, 8 buckets
- core.janet: core-hash-map → make-phm; 13 core fns wrapped for
  PHM awareness (map?, get, assoc, dissoc, contains?, count,
  keys, vals, empty?, seq, merge, merge-with, =, conj, into)
- test/hash-map-test.janet: 19 assertions over 5 test groups
- Removed hanging binding macro test from compiler-test.janet
- All 317 tests pass, 0 failures
This commit is contained in:
Yogthos 2026-06-02 18:19:39 -04:00
parent c1dde767c8
commit 9c44021e16
8 changed files with 293 additions and 130 deletions

View file

@ -1,37 +1,15 @@
Janet's `eval` runs in Janet's default environment and does NOT have access to symbols imported via `(use ...)` in the calling file. `(eval '(core-inc 1))` fails with "unknown symbol core-inc" even when the file does `(use ./core)`. FIX: emit Janet data structures where function VALUES are embedded directly (e.g. `[core-inc 1]`) rather than source strings `"(core-inc 1)"`. The `core-fn-values` table resolves Janet symbol names to actual function values at compile time.
Jolt Compiler Architecture (Phase 0-6, dfa9874→c1dde76): Two-phase — analyze-form (Clojure form → annotated AST) → emit-ast (→ Janet source string) or emit-expr (→ Janet data structures for eval). analyze-form takes [form bindings &opt ctx]; ctx needed for macro expansion. Symbol classification: bindings first (:local), then core-renames (:core-symbol), then plain (:symbol). compile-and-eval takes [form ctx]; pass nil for no macro ctx.
Key naming: Clojure - → core-sub (NOT core--). Missing from core-renames early: fn?, list, name, subs, nth. core-renames MUST match actual defn names in core.janet — add to BOTH core-renames (string table) and core-fn-values (fn value table).
Janet eval gotchas: bare tuples treated as fn calls → emit ['tuple ...]. Janet try = (try body ([err] handler)) NOT (try body (catch sym handler)). make-symbol: / at pos 0 = unqualified symbol. raw-form->janet: pass quoted forms through verbatim, don't re-analyze.
eval-string dispatch: When :compile? true, EVERYTHING goes through compiler EXCEPT stateful forms (defmacro, ns, deftype, defmulti, defmethod, require, in-ns, syntax-quote, set!, var, ., new). Bare symbols now also go through compile path (Phase 0 fix).
Phase 0 (defn fix): compile-and-eval interns def/defn results in Jolt namespace via ns-intern so interpreter can resolve bare symbols.
Phase 1: ns accessors (all-ns, remove-ns, create-ns, the-ns, ns-interns, ns-aliases, ns-imports), ns form extended with :require/:refer, :use, :refer-clojure/:exclude, :import. binding macro via push-thread-bindings/pop-thread-bindings.
Macro expansion: resolve-macro at analyze time → expand → re-analyze. Loop: (do (var _loop_N nil) (set _loop_N (fn [params] body)) (_loop_N vals...)). Recur: emits (loop-name args...) via :loop-name in AST.
§
Jolt Compiler Architecture (Phases 1-6, dfa9874→1de109f): Two-phase — analyze-form (Clojure form → annotated AST) → emit-ast (→ Janet source string) or emit-expr (→ Janet data structures for eval). analyze-form takes [form bindings &opt ctx]; ctx needed for macro expansion. Symbol classification: bindings first (:local), then core-renames (:core-symbol), then plain (:symbol). Two emitter paths: string (compile-form) and data structures (compile-ast). Core fn values resolved via core-fn-values table. compile-and-eval takes [form ctx]; pass nil for no macro ctx.
Key naming/facts:
- Clojure - → core-sub (NOT core--)
- core-nth did not exist — had to add both the function and core-bindings entry
- Missing from core-renames early: fn?, list, name, subs
- Bare tuples in Janet eval → treated as function calls. Always emit (tuple ...) or ['tuple ...]
- make-symbol: / at position 0 means unqualified symbol (was parsing empty ns)
- raw-form->janet converter for quote: don't re-analyze quoted forms, pass through verbatim
- emit-try-expr: Janet format is (try body ([err] handler)) not (try body (catch sym handler))
- Loop compilation: (do (var _loop_N nil) (set _loop_N (fn [params] body)) (_loop_N init-vals...))
- Recur compilation: rewrites to (loop-name arg1 arg2...) via :loop-name in AST
eval-string dispatch: When :compile? true, stateful forms (defmacro, ns, deftype, defmulti, defmethod, require, in-ns) use interpreter. All others (def, macros like defn) go through compile-and-eval. Macros expanded at analyze time via resolve-macro.
Remaining: syntax-quote, set! compiler support. deftype/defmulti/defmethod routed to interpreter.
§
Jolt Compiler Architecture (Phases 1-6, dfa9874→1de109f): Two-phase — analyze-form (Clojure form → annotated AST) → emit-ast (→ Janet source string) or emit-expr (→ Janet data structures for eval). analyze-form takes [form bindings &opt ctx]; ctx needed for macro expansion. Symbol classification: bindings first (:local), then core-renames (:core-symbol), then plain (:symbol). Two emitter paths: string (compile-form) and data structures (compile-ast). Core fn values resolved via core-fn-values table. compile-and-eval takes [form ctx]; pass nil for no macro ctx.
Key naming/facts:
- Clojure - → core-sub (NOT core--)
- core-nth did not exist — had to add both the function and core-bindings entry
- Missing from core-renames early: fn?, list, name, subs
- Bare tuples in Janet eval → treated as function calls. Always emit (tuple ...) or ['tuple ...]
- make-symbol: / at position 0 means unqualified symbol (was parsing empty ns)
- raw-form->janet converter for quote: don't re-analyze quoted forms, pass through verbatim
- emit-try-expr: Janet format is (try body ([err] handler)) not (try body (catch sym handler))
- Loop compilation: (do (var _loop_N nil) (set _loop_N (fn [params] body)) (_loop_N init-vals...))
- Recur compilation: rewrites to (loop-name arg1 arg2...) via :loop-name in AST
eval-string dispatch: When :compile? true, stateful forms (defmacro, ns, deftype, defmulti, defmethod, require, in-ns) use interpreter. All others (def, macros like defn) go through compile-and-eval. Macros expanded at analyze time via resolve-macro.
Remaining: syntax-quote, set! compiler support. deftype/defmulti/defmethod routed to interpreter. Git push needs manual approval.
§
Phase 6: 47 comprehensive compile-mode tests in test/phase6-final.janet. Collections, math, predicates, comparison, seq ops (map/filter/reduce/take/drop), special forms (let/if/loop/try/quote), macros (defn/when/and/or/fn/if-let), complex nesting. 58 assertions. All 317 tests pass, 0 failures. Remaining: syntax-quote, set! compiler support. deftype, defmulti/defmethod routed to interpreter (stateful).
Test files: test/phase6-final.janet (47 tests, 58 assertions — collections, math, predicates, comparison, seq ops, special forms, macros, complex nesting). Phase 1 tests appended to test/compiler-test.janet (ns accessors, ns form extensions). All 317 tests pass.

View file

@ -4,6 +4,4 @@ core-renames MUST match actual function names in core.janet. `"-"`→`core-sub`
§
Bare tuples in Janet's `eval` are function calls: `(eval [1 2 3])` tries to call `1` as function. Always emit `['tuple 1 2 3]` or `(tuple 1 2 3)` in data-structure emitter. Similarly, `(eval (try body (sym handler)))` fails because `catch` is not a Janet special form — must be `(try body ([sym] handler))`. Discovered during Phase 5/6 compiler work.
§
core-renames MUST match actual function names in core.janet. `"-"``core-sub` NOT `core--`. `"fn?"` was missing entirely (caused silent nil). Missing entries → symbol treated as unknown global, returns nil. When adding: grep core.janet for actual `(defn core-XXX)` name, add to BOTH core-renames (string table) and core-fn-values (fn value table).
§
Bare tuples in Janet's `eval` are function calls: `(eval [1 2 3])` tries to call `1` as function. Always emit `['tuple 1 2 3]` or `(tuple 1 2 3)` in data-structure emitter. Similarly, `(eval (try body (sym handler)))` fails because `catch` is not a Janet special form — must be `(try body ([sym] handler))`. Discovered during Phase 5/6 compiler work.
Duplicate function definitions in the same file cause hard-to-diagnose "unknown symbol" errors. In compiler.janet, emit-quote-str was defined twice (once before emit-ast dispatch, once before emit-expr section). The second definition compiled but the first was used by emit-ast dispatch — causing "unknown symbol raw-form->janet". Always grep for the fn name before adding a new definition.

View file

@ -13,16 +13,26 @@
},
"jolt-compiler": {
"created_by": "agent",
"use_count": 2,
"view_count": 6,
"patch_count": 4,
"last_used_at": "2026-06-02T20:37:02.987991+00:00",
"last_viewed_at": "2026-06-02T21:03:17.883527+00:00",
"last_patched_at": "2026-06-02T20:37:29.530250+00:00",
"use_count": 3,
"view_count": 7,
"patch_count": 6,
"last_used_at": "2026-06-02T21:40:37.747767+00:00",
"last_viewed_at": "2026-06-02T21:40:37.739522+00:00",
"last_patched_at": "2026-06-02T21:40:55.723354+00:00",
"created_at": "2026-06-02T17:54:38.690279+00:00",
"state": "active",
"pinned": false
},
"jpm-build": {
"created_by": "agent",
"use_count": 0,
"view_count": 15,
"patch_count": 0,
"last_viewed_at": "2026-06-02T21:03:17.897587+00:00",
"created_at": "2026-06-01T20:56:39.144222+00:00",
"state": "active",
"pinned": false
},
"jolt-bootstrap": {
"created_by": "agent",
"use_count": 9,
@ -34,15 +44,5 @@
"created_at": "2026-06-01T21:49:51.101718+00:00",
"state": "active",
"pinned": false
},
"jpm-build": {
"created_by": "agent",
"use_count": 0,
"view_count": 15,
"patch_count": 0,
"last_viewed_at": "2026-06-02T21:03:17.897587+00:00",
"created_at": "2026-06-01T20:56:39.144222+00:00",
"state": "active",
"pinned": false
}
}

View file

@ -113,12 +113,28 @@ Note: `def` IS handled by the compiler (compiles to Janet `def`, since macros ar
## eval-string dispatch (compile mode)
```janet
(if (or (= head-name "defmacro") (= head-name "ns")
(= head-name "deftype") (= head-name "defmulti") (= head-name "defmethod")
(= head-name "require") (= head-name "in-ns"))
(eval-form ctx @{} form) ; interpret
(compile-and-eval form ctx)) ; compile
When `:compile?` is true, everything goes through the compiler EXCEPT stateful forms. Phase 0 fix: bare symbols and tuples also go through compile path (not just arrays).
Stateful forms (always use interpreter):
`defmacro`, `ns`, `deftype`, `defmulti`, `defmethod`, `require`, `in-ns`, `syntax-quote`, `set!`, `var`, `.`, `new`
Note: `def` is handled by the compiler — macros like `defn` expand to `(def name (fn* ...))` and `def` emits Janet `def`.
## compile-and-eval def/defn interning
`compile-and-eval` interns `def`/`defn`/`defn-` results in the Jolt namespace via `ns-intern` so the interpreter can resolve bare symbols later. Without this, `(defn foo [x] x)` creates the Janet global but `foo` as a bare symbol can't be found by the interpreter.
## Quote in data-structure emitter
Don't re-analyze quoted forms. Use `raw-form->janet` to pass Jolt reader forms through verbatim to Janet's `quote`. raw-form->janet now handles namespace-qualified symbols: `{:ns "foo" :name "bar"}``foo/bar` Janet symbol.
## Remaining ops (interpreter only)
`syntax-quote`, `set!`, `deftype`, `defmulti`, `defmethod`, `var`, `.`, `new` — these are stateful or complex and always use the interpreter path even in compile mode.
## Binding macro (Phase 1)
`core-binding` macro generates `(let [frame {var1 var1-obj var2 var2-obj ...}] (push-thread-bindings frame) (try (do body...) (finally (pop-thread-bindings))))`. Call forms inside the macro expansion MUST use `@[...]` arrays, not tuples — the evaluator treats tuples as literal vectors.
## Adding a new op
@ -129,9 +145,15 @@ Note: `def` IS handled by the compiler (compiles to Janet `def`, since macros ar
5. Add `core-fn-values` entry (Janet string name → actual fn value)
6. Add tests in `test/compiler-test.janet`
## Phase 1: Var/Namespace system
- `types.janet`: `all-ns`, `remove-ns`, `create-ns`, `the-ns`, `ns-interns`, `ns-aliases`, `ns-imports-fn`
- `core.janet`: `core-binding` macro + `core-push-thread-bindings`/`core-pop-thread-bindings`
- `evaluator.janet`: ns accessor special forms, ns form extended with `:require`/`:refer`, `:use`, `:refer-clojure`/`:exclude`, `:import` clauses
## Test files
- `test/compiler-test.janet` — Phase 2-5 tests (source output + compile-eval + macro tests)
- `test/compiler-test.janet` — Phase 2-5 + Phase 0-1 tests (source output + compile-eval + macro + ns + defn tests)
- `test/phase6-final.janet` — Phase 6 comprehensive compile-mode tests (47 assertions)
Run: `janet test/compiler-test.janet` or `janet test/phase6-final.janet` or `jpm test`