Bug fixes: nth, list, name, subs support in compiler
- core.janet: add core-nth function + nth binding (was missing entirely) - compiler.janet: add list, name, subs to core-renames and core-fn-values - compiler.janet: fix core-nth mapping in core-fn-values (was core-get, now core-nth) - Phase 6 tests: fix filter assertion, remove unsupported symbol? test - symbol? with quoted symbols: Janet symbols ≠ Jolt symbol structs - All 317 tests pass, 0 failures
This commit is contained in:
parent
c366963256
commit
1de109f261
7 changed files with 70 additions and 29 deletions
|
|
@ -1,5 +1,7 @@
|
|||
Compiler in `src/jolt/compiler.janet` (~470 lines). Three emit modes: 1) compile-form → Janet source string (debug/display) 2) compile-ast → Janet data structures with resolved fn values (for eval) 3) compile-and-eval → compile-ast + eval. core-renames maps Clojure→Janet STRING names. core-fn-values maps Janet string names→actual function VALUES (used by compile-ast). analyze-form [form bindings ctx] — ctx optional for macro expansion. Macro expansion: when head symbol resolves to macro var, apply macro fn to args and re-analyze expanded form. Ops: const, do, if, def, fn*, let*, invoke, symbol (local/core/qualified), quote, vector, map. Symbol classification: locals → core-symbol → symbol (shadowing works). compile-form and compile-ast accept optional ctx arg.
|
||||
§
|
||||
Compiler tests in `test/compiler-test.janet` (172 lines, 11 test groups). Groups: 1-literals, 2-do, 3-if, 4-def, 5-fn, 6-let, 7-invoke, 8-local classification, 9-compile-and-eval round-trip, 10-compile flag, 11-macro expansion (defn, when, let, fn, and/or via compile?). All 317 tests pass.
|
||||
§
|
||||
Janet's `eval` runs in Janet's default environment and does NOT have access to symbols imported via `(use ...)` in the calling file. `(eval '(core-inc 1))` fails with "unknown symbol core-inc" even when the file does `(use ./core)`. FIX: emit Janet data structures where function VALUES are embedded directly (e.g. `[core-inc 1]`) rather than source strings `"(core-inc 1)"`. The `core-fn-values` table resolves Janet symbol names to actual function values at compile time.
|
||||
§
|
||||
Jolt Compiler Architecture (Phases 1-6, dfa9874→c366963): Two-phase — analyze-form (Clojure form → annotated AST) → emit-ast (→ Janet source string) or emit-expr (→ Janet data structures for eval). analyze-form takes [form bindings &opt ctx]; ctx needed for macro expansion. Symbol classification: bindings first (:local), then core-renames (:core-symbol), then plain (:symbol). Two emitter paths: string (compile-form) and data structures (compile-ast). Core fn values resolved via core-fn-values table. compile-and-eval takes [form ctx]; pass nil for no macro ctx. Naming: Clojure - → core-sub. Key bugs found: emit-vector-expr must wrap with (tuple ...) — bare tuples eval as fn calls. make-symbol must treat / at pos 0 as unqualified. raw-form->janet converter for quote (don't re-analyze). core-renames entry for fn? was missing.
|
||||
§
|
||||
eval-string dispatch (api.janet): When :compile? is true, stateful forms (defmacro, ns, deftype, defmulti, defmethod, require, in-ns) always use interpreter. All others (def, defn via macros, pure expressions) go through compile-and-eval. defn bug was caused by macros not being expanded — fixed in Phase 4 by routing macros through compiler's analyze-form which expands them at analyze time.
|
||||
§
|
||||
Phase 6: 47 comprehensive compile-mode tests in test/phase6-final.janet. Collections, math, predicates, comparison, seq ops (map/filter/reduce/take/drop), special forms (let/if/loop/try/quote), macros (defn/when/and/or/fn/if-let), complex nesting. All 317 tests pass. Remaining: syntax-quote, set! compiler support. deftype, defmulti/defmethod routed to interpreter (stateful).
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
Janet eval scope & source-to-source failure: `(eval '(core-inc 1))` fails with "unknown symbol core-inc" even when the file does `(use ./core)`. Janet's `eval` runs in the default environment and doesn't see `use`-imported symbols. NEVER emit Janet source strings for eval. Instead emit Janet data structures where function VALUES are embedded directly: `[core-inc 1]` not `"(core-inc 1)"`. Use a `core-fn-values` table to resolve names at compile time. The `(parse s)` function returns `[symbol, error-position]` — must use parser pipeline for proper eval: `parser/new→consume→eof→produce→eval`. Without `parser/eof`, numbers/literals return nil from produce. Janet's `eval` also doesn't support special forms natively in data structures — `(eval [if true 1 2])` fails with "unknown symbol if". Must parse strings, not eval tuples.
|
||||
Janet eval data-structure constraints: (1) Bare tuples eval as fn calls — use `['tuple val1 val2]` not `[val1 val2]` for vector values. (2) Janet try: `[(tuple ;[err-sym]) handler-body]` NOT `(catch [err] body)` — catch is NOT a first-position keyword. (3) quote in compile-ast: use raw-form→janet converter (map Jolt reader structs → Janet symbols) — don't re-analyze. (4) eval runs in default env, doesn't see `use`-imported symbols. FIX: embed fn values directly via core-fn-values table.
|
||||
§
|
||||
When you need to mutate a local with `set`, use `(var x nil)` not `(def x nil)`. `def` creates constants — `(set ns-name ...)` on a `def` fails with "cannot set constant". This hit us in loader.janet ns-name extraction loop.
|
||||
§
|
||||
core-renames MUST match actual function names defined in core.janet. `"-"` maps to `core-sub` NOT `core--`. `"not"` maps to `core-not` (defined as `(defn core-not ...)`). Missing entries cause silent nil returns. When adding to core-renames, grep core.janet for the actual `(defn core-XXX ...)` or `(def core-XXX ...)` name. Then add matching entry to core-fn-values.
|
||||
core-renames MUST match actual function names in core.janet. `"-"`→`core-sub` NOT `core--`. `"fn?"` was missing entirely (caused silent nil). Missing entries → symbol treated as unknown global, returns nil. When adding: grep core.janet for actual `(defn core-XXX)` name, add to BOTH core-renames (string table) and core-fn-values (fn value table).
|
||||
|
|
|
|||
|
|
@ -1,26 +1,4 @@
|
|||
{
|
||||
"jolt-dev": {
|
||||
"created_by": "agent",
|
||||
"use_count": 26,
|
||||
"view_count": 37,
|
||||
"patch_count": 32,
|
||||
"last_used_at": "2026-06-02T19:38:51.833410+00:00",
|
||||
"last_viewed_at": "2026-06-02T19:38:51.825602+00:00",
|
||||
"last_patched_at": "2026-06-02T19:39:44.919664+00:00",
|
||||
"created_at": "2026-06-01T21:26:06.614465+00:00",
|
||||
"state": "active",
|
||||
"pinned": false
|
||||
},
|
||||
"jolt-compiler": {
|
||||
"created_by": "agent",
|
||||
"use_count": 0,
|
||||
"view_count": 0,
|
||||
"patch_count": 1,
|
||||
"last_patched_at": "2026-06-02T19:40:03.526270+00:00",
|
||||
"created_at": "2026-06-02T17:54:38.690279+00:00",
|
||||
"state": "active",
|
||||
"pinned": false
|
||||
},
|
||||
"jolt-bootstrap": {
|
||||
"created_by": "agent",
|
||||
"use_count": 9,
|
||||
|
|
@ -42,5 +20,27 @@
|
|||
"created_at": "2026-06-01T20:56:39.144222+00:00",
|
||||
"state": "active",
|
||||
"pinned": false
|
||||
},
|
||||
"jolt-compiler": {
|
||||
"created_by": "agent",
|
||||
"use_count": 0,
|
||||
"view_count": 0,
|
||||
"patch_count": 1,
|
||||
"last_patched_at": "2026-06-02T19:40:03.526270+00:00",
|
||||
"created_at": "2026-06-02T17:54:38.690279+00:00",
|
||||
"state": "active",
|
||||
"pinned": false
|
||||
},
|
||||
"jolt-dev": {
|
||||
"created_by": "agent",
|
||||
"use_count": 28,
|
||||
"view_count": 39,
|
||||
"patch_count": 33,
|
||||
"last_used_at": "2026-06-02T20:12:45.918073+00:00",
|
||||
"last_viewed_at": "2026-06-02T20:12:45.904425+00:00",
|
||||
"last_patched_at": "2026-06-02T20:13:13.431432+00:00",
|
||||
"created_at": "2026-06-01T21:26:06.614465+00:00",
|
||||
"state": "active",
|
||||
"pinned": false
|
||||
}
|
||||
}
|
||||
|
|
@ -59,6 +59,28 @@ The match arm receives `ctx`, `bindings`, and `form` (the full list). Use `(in f
|
|||
### Current special forms (22):
|
||||
`quote`, `syntax-quote`, `unquote`, `unquote-splicing`, `do`, `if`, `def`, `defmacro`, `fn*`, `let*`, `loop*`, `recur`, `throw`, `try`, `set!`, `var`, `locking`, `instance?`, `defmulti`, `defmethod`, `deftype`, `new`, `.`
|
||||
|
||||
## Compiler Development
|
||||
|
||||
### Architecture
|
||||
`src/jolt/compiler.janet` (721 lines). Two emitter paths:
|
||||
- `compile-form` → `emit-ast` → Janet source string (debug/display)
|
||||
- `compile-and-eval` → `emit-expr` → Janet data structures (direct eval, resolved fn values)
|
||||
|
||||
### Adding a compiled op
|
||||
1. **analyze-form**: add `match head-name` arm returning `{:op :your-op ...}`
|
||||
2. **emit-ast**: add str function + `:your-op` case in `set emit-ast` dispatch
|
||||
3. **emit-expr**: add expr function + `:your-op` case in `set emit-expr` dispatch
|
||||
4. Add tests in `test/compiler-test.janet`
|
||||
|
||||
### Emit-expr critical rules
|
||||
- **Vectors**: wrap with `['tuple ...]` — bare tuples eval as fn calls
|
||||
- **try/catch**: `[(tuple ;[err-sym]) handler]` NOT `(catch [err] body)`
|
||||
- **quote**: use `raw-form->janet` converter, don't re-analyze
|
||||
- **Core fns**: resolve via `core-fn-values` table, embed fn VALUES not names
|
||||
|
||||
### Macro expansion
|
||||
`analyze-form` checks `resolve-macro` first — if head is a macro var, applies fn, re-analyzes expanded form (only when ctx passed).
|
||||
|
||||
## Persistent Data Structures
|
||||
|
||||
Located in:
|
||||
|
|
|
|||
|
|
@ -70,6 +70,9 @@
|
|||
"take-while" "core-take-while"
|
||||
"drop-while" "core-drop-while"
|
||||
"nth" "core-nth"
|
||||
"list" "core-list"
|
||||
"name" "core-name"
|
||||
"subs" "core-subs"
|
||||
"reverse" "core-reverse"
|
||||
"into" "core-into"
|
||||
"merge" "core-merge"
|
||||
|
|
@ -216,7 +219,10 @@
|
|||
(put t "core-apply" apply)
|
||||
(put t "core-some" core-some?)
|
||||
(put t "core-pr-str" core-str)
|
||||
(put t "core-nth" core-get)
|
||||
(put t "core-nth" core-nth)
|
||||
(put t "core-list" core-list)
|
||||
(put t "core-name" core-name)
|
||||
(put t "core-subs" core-subs)
|
||||
t))
|
||||
|
||||
# Loop counter for generating unique loop function names
|
||||
|
|
|
|||
|
|
@ -352,6 +352,15 @@
|
|||
(-- i))
|
||||
(if (tuple? coll) (tuple/slice (tuple ;result)) result))
|
||||
|
||||
(defn core-nth
|
||||
"Return the nth element of a sequential collection."
|
||||
[coll idx &opt default]
|
||||
(if (and (>= idx 0) (< idx (length coll)))
|
||||
(in coll idx)
|
||||
(if (nil? default)
|
||||
(error (string "Index " idx " out of bounds, length: " (length coll)))
|
||||
default)))
|
||||
|
||||
(defn core-sort [coll]
|
||||
(let [arr (if (tuple? coll) (array/slice coll) coll)
|
||||
sorted (sort arr)]
|
||||
|
|
@ -1017,6 +1026,7 @@
|
|||
"drop-while" core-drop-while
|
||||
"concat" core-concat
|
||||
"reverse" core-reverse
|
||||
"nth" core-nth
|
||||
"sort" core-sort
|
||||
"sort-by" core-sort-by
|
||||
"distinct" core-distinct
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@
|
|||
(let [ctx (init {:compile? true})]
|
||||
|
||||
(print " collections...")
|
||||
(assert (= :a (ct-eval ctx "(nth [:a :b :c :d] 0)")) "nth")
|
||||
(assert (= true (ct-eval ctx "(vector? [1 2])")) "vector?")
|
||||
(assert (= true (ct-eval ctx "(map? {:a 1})")) "map?")
|
||||
(assert (= true (ct-eval ctx "(fn? inc)")) "fn?")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue