Phase 1+2: Clojure→Janet source compiler with symbol classification

- analyze-form: Clojure forms → annotated AST nodes (:op keys)
- emit-ast: AST dispatch → Janet source via StringBuffer
- compile-form: analyze → emit → Janet source string
- Symbol classification: locals, core refs, qualified refs
- Ops: const, do, if, def, fn, let, invoke, quote, vector, map
- Local binding awareness in fn* and let* (shadowing works)
- All 317 existing tests pass, 24 new compiler tests
This commit is contained in:
Yogthos 2026-06-02 14:59:13 -04:00
parent 679cc1d4ef
commit dfa98746ee
6 changed files with 594 additions and 22 deletions

View file

@ -1,7 +1,7 @@
`:mutable?` compile flag in `api.janet` `init`: persistent Clojure data structures loaded by default. Pass `{:mutable? true}` to use Janet native mutable tuples/tables instead. `load-persistent-structures` function in api.janet loads `src/jolt/clojure/lang/persistent_vector.clj` (17 forms) and swaps `vec`/`vector`/`vector?` bindings in `clojure.core`. PersistentHashMap WIP in `src/jolt/clojure/lang/persistent_hash_map.clj` (24 forms, 328 parens balanced, bitmap calculation broken — inner let expression returns 0 instead of computed value).
Compiler implementation plan (6 phases): Phase 1 — compiler.janet skeleton + emit* for const/do/if/let/fn/def/invoke. Phase 2 — symbol classifier (resolve locals/vars/core at analyze time). Phase 3 — loader.janet + api.janet wiring (:compile? flag, load-ns, caching). Phase 4 — macro integration (expand at analyze time via interpreter). Phase 5 — remaining ops (loop/recur, try/throw, quote, syntax-quote, set!, deftype, .). Phase 6 — tests + benchmarks. New files: src/jolt/compiler.janet, src/jolt/loader.janet, test/compiler-test.janet. Modified files: evaluator.janet (compiler fast-path), types.janet (compiled-cache table), api.janet (compile-string, load-ns). Reader and core unchanged.
§
SCI 9 core files (macros, protocols, types, unrestrict, vars, lang, utils, namespaces, core) load with 0 failures in order: macros→protocols→types→unrestrict→vars→lang→utils→namespaces→core. 46 namespaces populated. Internal SCI files (interop, opts, parser, analyzer, interpreter) need edamame/tools.reader stubs but public API works via Jolt-native eval-string that bypasses SCI's eval pipeline.
Compiler is in `src/jolt/compiler.janet` (352 lines). Two-phase architecture: `analyze-form` produces annotated AST with `:op` keys, `emit-ast` dispatches on `:op` to emit Janet source, `compile-form` ties them together returning a Janet source string. Source-to-source (not bytecode). `core-renames` table maps Clojure core names to Janet function names (e.g., "inc" → "core-inc"). Phase 1 covers: const, do, if, def, fn*, let*, invoke, symbol, core-symbol, qualified-symbol, quote, vector, map. Tests in `test/compiler-test.janet` (79 lines). Phase 2 (symbol classification — locals/vars/core at analyze time) is the next step.
§
Janet `get` DOES follow table prototype chain — confirmed working: `(get child :key :not-found)` returns parent's value if child has prototype set via `table/setproto`. Earlier confusion about `get` not following prototypes was incorrect. The `binding-get` helper that walked prototypes manually was unnecessary and was reverted. Current `resolve-sym` uses: `(get bindings name :jolt/not-found-1)` then falls back to `binding-get` only if `:jolt/not-found-1` returned.
After compiler Phase 1: 317 tests pass, 0 failures (was 253 before). Compiler tests add 8 test groups (literals, do, if, def, fn, let, invoke — 19 assertions). Run with `jpm test`.
§
Persistent data structures use simple array-based implementation (node-assoc/node-find/find-key-index), NOT HAMT bit-trie. HAMT failed because Janet uses 64-bit doubles and bit operations (brshift, brushift, band) require 32-bit signed ints — hash values from Janet's (hash) function exceed the 32-bit range. Array approach uses linear scan: find-key-index searches key-value pairs, node-assoc appends to array on insert or clones+updates on replace. O(n) lookup, acceptable for small maps.
Compiler development workflow: (1) add match arm in `analyze-form` → produces `{:op :new-op ...}` AST, (2) add `emit-*` helper + dispatch arm in `emit-ast`, (3) add test assertions in `test/compiler-test.janet` using `compile-str` helper, (4) run `janet test/compiler-test.janet` then `jpm test`. Tests compare source-to-source output strings, not execution results. Any fn added to `core-bindings` in core.janet must also be added to `core-renames` in compiler.janet.