Phase 3: Compile-aware loader, :compile? flag, compile-and-eval

- loader.janet: load-ns with compile? dispatch, compiled-cache, compiled?, clear-compiled-cache
- compiler.janet: compile-ast (Janet data structures for direct eval), compile-and-eval
- api.janet: compile-string, compile-file, eval-string dispatches to compile-and-eval when :compile? is true
- types.janet: :compile? flag and :compiled-cache table in context env
- Stateful forms (def, defmacro, ns, deftype, defmulti, defmethod) fall back to interpreter
- All 317 tests pass + 16 new Phase 3 tests
This commit is contained in:
Yogthos 2026-06-02 15:24:28 -04:00
parent dfa98746ee
commit ab7ff85816
9 changed files with 326 additions and 176 deletions

View file

@ -1,7 +1,5 @@
Compiler implementation plan (6 phases): Phase 1 — compiler.janet skeleton + emit* for const/do/if/let/fn/def/invoke. Phase 2 — symbol classifier (resolve locals/vars/core at analyze time). Phase 3 — loader.janet + api.janet wiring (:compile? flag, load-ns, caching). Phase 4 — macro integration (expand at analyze time via interpreter). Phase 5 — remaining ops (loop/recur, try/throw, quote, syntax-quote, set!, deftype, .). Phase 6 — tests + benchmarks. New files: src/jolt/compiler.janet, src/jolt/loader.janet, test/compiler-test.janet. Modified files: evaluator.janet (compiler fast-path), types.janet (compiled-cache table), api.janet (compile-string, load-ns). Reader and core unchanged.
Compiler Phases 1-3 COMPLETE. src/jolt/compiler.janet (342 lines): analyze-form→emit-ast two-phase. Phase 2: symbol classification (locals→core→symbol, shadowing works). Phase 3: compile-and-eval via parser→consume→eof→produce→eval pipeline, :compile? flag in types.janet context, loader.janet (80 lines) for namespace loading, api.janet updated (compile-string, compile-file, compile-aware eval-string). Stateful forms (def, defmacro, ns, deftype, defmulti, defmethod, require, in-ns) always route to interpreter. All 317 tests pass. Next: Phase 4 (macro integration), Phase 5 (remaining ops: loop/recur, try/throw, quote, syntax-quote, set!, .), Phase 6 (benchmarks).
§
Compiler is in `src/jolt/compiler.janet` (352 lines). Two-phase architecture: `analyze-form` produces annotated AST with `:op` keys, `emit-ast` dispatches on `:op` to emit Janet source, `compile-form` ties them together returning a Janet source string. Source-to-source (not bytecode). `core-renames` table maps Clojure core names to Janet function names (e.g., "inc" → "core-inc"). Phase 1 covers: const, do, if, def, fn*, let*, invoke, symbol, core-symbol, qualified-symbol, quote, vector, map. Tests in `test/compiler-test.janet` (79 lines). Phase 2 (symbol classification — locals/vars/core at analyze time) is the next step.
Compiler is in `src/jolt/compiler.janet` (342 lines). Two-phase: `analyze-form [form bindings]` → annotated AST with `:op` keys, `emit-ast` dispatches on `:op` → Janet source string via StringBuffer. `compile-form` for source output, `compile-and-eval` for direct eval via `eval-janet-source` (parser/new→consume→eof→produce→eval pipeline). `core-renames` table maps Clojure→Janet names. Ops covered: const, do, if, def, fn*, let*, invoke, symbol (local/core-symbol/qualified classification), quote. Symbol classification order: locals → core-symbol → symbol (shadowing works). Stateful forms (def, defmacro, etc.) must use interpreter path, not compile-and-eval.
§
After compiler Phase 1: 317 tests pass, 0 failures (was 253 before). Compiler tests add 8 test groups (literals, do, if, def, fn, let, invoke — 19 assertions). Run with `jpm test`.
§
Compiler development workflow: (1) add match arm in `analyze-form` → produces `{:op :new-op ...}` AST, (2) add `emit-*` helper + dispatch arm in `emit-ast`, (3) add test assertions in `test/compiler-test.janet` using `compile-str` helper, (4) run `janet test/compiler-test.janet` then `jpm test`. Tests compare source-to-source output strings, not execution results. Any fn added to `core-bindings` in core.janet must also be added to `core-renames` in compiler.janet.
Compiler tests in `test/compiler-test.janet` (143 lines, 10 test groups). Phase 1: 7 groups, 19 string-output assertions. Phase 2: local classification (3 assertions: shadowing, param shadowing, nested let). Phase 3: compile-and-eval round-trip (9 assertions) + :compile? flag tests (7 assertions). All 317 tests pass. Run with `janet test/compiler-test.janet` or `jpm test`.