diff --git a/.dirge/memory/MEMORY.md b/.dirge/memory/MEMORY.md index 499b960..e790f18 100644 --- a/.dirge/memory/MEMORY.md +++ b/.dirge/memory/MEMORY.md @@ -1,7 +1,5 @@ -Compiler implementation plan (6 phases): Phase 1 — compiler.janet skeleton + emit* for const/do/if/let/fn/def/invoke. Phase 2 — symbol classifier (resolve locals/vars/core at analyze time). Phase 3 — loader.janet + api.janet wiring (:compile? flag, load-ns, caching). Phase 4 — macro integration (expand at analyze time via interpreter). Phase 5 — remaining ops (loop/recur, try/throw, quote, syntax-quote, set!, deftype, .). Phase 6 — tests + benchmarks. New files: src/jolt/compiler.janet, src/jolt/loader.janet, test/compiler-test.janet. Modified files: evaluator.janet (compiler fast-path), types.janet (compiled-cache table), api.janet (compile-string, load-ns). Reader and core unchanged. +Compiler Phases 1-3 COMPLETE. src/jolt/compiler.janet (342 lines): analyze-form→emit-ast two-phase. Phase 2: symbol classification (locals→core→symbol, shadowing works). Phase 3: compile-and-eval via parser→consume→eof→produce→eval pipeline, :compile? flag in types.janet context, loader.janet (80 lines) for namespace loading, api.janet updated (compile-string, compile-file, compile-aware eval-string). Stateful forms (def, defmacro, ns, deftype, defmulti, defmethod, require, in-ns) always route to interpreter. All 317 tests pass. Next: Phase 4 (macro integration), Phase 5 (remaining ops: loop/recur, try/throw, quote, syntax-quote, set!, .), Phase 6 (benchmarks). § -Compiler is in `src/jolt/compiler.janet` (352 lines). Two-phase architecture: `analyze-form` produces annotated AST with `:op` keys, `emit-ast` dispatches on `:op` to emit Janet source, `compile-form` ties them together returning a Janet source string. Source-to-source (not bytecode). `core-renames` table maps Clojure core names to Janet function names (e.g., "inc" → "core-inc"). Phase 1 covers: const, do, if, def, fn*, let*, invoke, symbol, core-symbol, qualified-symbol, quote, vector, map. Tests in `test/compiler-test.janet` (79 lines). Phase 2 (symbol classification — locals/vars/core at analyze time) is the next step. +Compiler is in `src/jolt/compiler.janet` (342 lines). Two-phase: `analyze-form [form bindings]` → annotated AST with `:op` keys, `emit-ast` dispatches on `:op` → Janet source string via StringBuffer. `compile-form` for source output, `compile-and-eval` for direct eval via `eval-janet-source` (parser/new→consume→eof→produce→eval pipeline). `core-renames` table maps Clojure→Janet names. Ops covered: const, do, if, def, fn*, let*, invoke, symbol (local/core-symbol/qualified classification), quote. Symbol classification order: locals → core-symbol → symbol (shadowing works). Stateful forms (def, defmacro, etc.) must use interpreter path, not compile-and-eval. § -After compiler Phase 1: 317 tests pass, 0 failures (was 253 before). Compiler tests add 8 test groups (literals, do, if, def, fn, let, invoke — 19 assertions). Run with `jpm test`. -§ -Compiler development workflow: (1) add match arm in `analyze-form` → produces `{:op :new-op ...}` AST, (2) add `emit-*` helper + dispatch arm in `emit-ast`, (3) add test assertions in `test/compiler-test.janet` using `compile-str` helper, (4) run `janet test/compiler-test.janet` then `jpm test`. Tests compare source-to-source output strings, not execution results. Any fn added to `core-bindings` in core.janet must also be added to `core-renames` in compiler.janet. +Compiler tests in `test/compiler-test.janet` (143 lines, 10 test groups). Phase 1: 7 groups, 19 string-output assertions. Phase 2: local classification (3 assertions: shadowing, param shadowing, nested let). Phase 3: compile-and-eval round-trip (9 assertions) + :compile? flag tests (7 assertions). All 317 tests pass. Run with `janet test/compiler-test.janet` or `jpm test`. diff --git a/.dirge/memory/PITFALLS.md b/.dirge/memory/PITFALLS.md index b76934b..625dd33 100644 --- a/.dirge/memory/PITFALLS.md +++ b/.dirge/memory/PITFALLS.md @@ -1,5 +1,5 @@ -Bit operations in Janet (brshift, brushift, band, bor, bxor) use 32-bit signed integers. Values from (hash key) can exceed 32-bit range (>2^31). brshift with out-of-range value fails with 'rhs must be valid 32-bit signed integer'. Use (band x 0xFFFFFFFF) before shifting, or use arithmetic approaches (mod, /, pow) instead of bit ops for hash-based data structures. -§ -`.clj` source files are loaded at init time by Jolt's own reader/evaluator. PersistentVector (17 forms) and PersistentHashMap (12 forms) live in `src/jolt/clojure/lang/persistent_*.clj`. api.janet's `init` calls `load-persistent-structures` which slurps and eval-forms each file, then swaps clojure.core bindings (vec, vector, vector?, hash-map) to the persistent versions. Pass `{:mutable? true}` to skip loading and use Janet-native types. -§ Janet LSP produces false positives on `.janet` files — it doesn't understand Janet syntax (thinks docstring lines are unresolved symbols, doesn't know `declare-project`/`declare-source` macros, etc.). These are pre-existing and should be ignored — they don't affect runtime correctness. +§ +Janet `(parse s)` returns `[symbol, error-position]` for forms — e.g., `(parse "(+ 1 2)")` gives `[+ 1]`, not a parsed tuple. For proper evaluation of special forms, must use the parser pipeline: `parser/new` → `parser/consume` → `parser/eof` → `parser/produce` → `eval`. Without `parser/eof`, `produce` returns nil for simple forms like numbers. Janet's `eval` also doesn't support special forms natively in data structures — `(eval [if true 1 2])` fails with "unknown symbol if". Must parse strings, not eval tuples. +§ +When you need to mutate a local with `set`, use `(var x nil)` not `(def x nil)`. `def` creates constants — `(set ns-name ...)` on a `def` fails with "cannot set constant". This hit us in loader.janet ns-name extraction loop. diff --git a/.dirge/skills/.usage.json b/.dirge/skills/.usage.json index 903384e..6b357ed 100644 --- a/.dirge/skills/.usage.json +++ b/.dirge/skills/.usage.json @@ -1,16 +1,35 @@ { + "jpm-build": { + "created_by": "agent", + "use_count": 0, + "view_count": 11, + "patch_count": 0, + "last_viewed_at": "2026-06-02T17:43:38.327259+00:00", + "created_at": "2026-06-01T20:56:39.144222+00:00", + "state": "active", + "pinned": false + }, "jolt-dev": { "created_by": "agent", - "use_count": 24, - "view_count": 35, - "patch_count": 30, - "last_used_at": "2026-06-02T18:45:23.321430+00:00", - "last_viewed_at": "2026-06-02T18:45:23.310874+00:00", - "last_patched_at": "2026-06-02T17:27:24.746485+00:00", + "use_count": 25, + "view_count": 36, + "patch_count": 31, + "last_used_at": "2026-06-02T19:13:37.620054+00:00", + "last_viewed_at": "2026-06-02T19:13:37.612317+00:00", + "last_patched_at": "2026-06-02T19:14:06.338745+00:00", "created_at": "2026-06-01T21:26:06.614465+00:00", "state": "active", "pinned": false }, + "jolt-compiler": { + "created_by": "agent", + "use_count": 0, + "view_count": 0, + "patch_count": 0, + "created_at": "2026-06-02T17:54:38.690279+00:00", + "state": "active", + "pinned": false + }, "jolt-bootstrap": { "created_by": "agent", "use_count": 9, @@ -22,24 +41,5 @@ "created_at": "2026-06-01T21:49:51.101718+00:00", "state": "active", "pinned": false - }, - "jolt-compiler": { - "created_by": "agent", - "use_count": 0, - "view_count": 0, - "patch_count": 0, - "created_at": "2026-06-02T17:54:38.690279+00:00", - "state": "active", - "pinned": false - }, - "jpm-build": { - "created_by": "agent", - "use_count": 0, - "view_count": 11, - "patch_count": 0, - "last_viewed_at": "2026-06-02T17:43:38.327259+00:00", - "created_at": "2026-06-01T20:56:39.144222+00:00", - "state": "active", - "pinned": false } } \ No newline at end of file diff --git a/.dirge/skills/jolt-dev/SKILL.md b/.dirge/skills/jolt-dev/SKILL.md index 6af2475..fbf049f 100644 --- a/.dirge/skills/jolt-dev/SKILL.md +++ b/.dirge/skills/jolt-dev/SKILL.md @@ -1,6 +1,7 @@ -# jolt-dev - -Jolt development workflow — build, test, special form patterns, Janet gotchas +--- +name: jolt-dev +description: Jolt development workflow — build, test, special form patterns, Janet gotchas +--- # Jolt Development @@ -13,6 +14,29 @@ jpm test # runs all tests janet test/foo.janet # run a single test file from project root ``` +## Compiler Development + +See `jolt-compiler` skill for the Clojure→Janet source-to-source compiler workflow. + +## Janet Eval Pipeline (critical) + +Janet's `(parse s)` does NOT return a parsed form — it returns `[symbol, error-position]`. +For evaluating Janet source strings, use the parser pipeline: + +```janet +(def p (parser/new)) +(parser/consume p source) +(parser/eof p) # REQUIRED — otherwise produce returns nil +(def form (parser/produce p)) +(eval form) +``` + +**Never** try `(eval [if true 1 2])` — Janet's `eval` doesn't recognize special forms in tuple data structures. + +## `var` vs `def` + +When you need to mutate a local with `set`, use `(var x nil)` not `(def x nil)`. `def` creates constants. + ## Special Form Checklist To add a new special form to the evaluator: @@ -28,4 +52,25 @@ The match arm receives `ctx`, `bindings`, and `form` (the full list). Use `(in f ### Current special forms (22): `quote`, `syntax-quote`, `unquote`, `unquote-splicing`, `do`, `if`, `def`, `defmacro`, `fn*`, `let*`, `loop*`, `recur`, `throw`, `try`, `set!`, `var`, `locking`, `instance?`, `defmulti`, `defmethod`, `deftype`, `new`, `.` -## Pers… (truncated, 34945 bytes total) \ No newline at end of file +## Persistent Data Structures + +Located in: +- `src/jolt/clojure/lang/persistent_vector.clj` +- `src/jolt/clojure/lang/persistent_hash_map.clj` + +Loaded at init time by `load-persistent-structures` in `api.janet`. Use `{:mutable? true}` to skip and use Janet-native types. + +### Implementation detail +Simple array-based implementation (node-assoc/node-find/find-key-index), NOT HAMT bit-trie. +HAMT failed because Janet uses 64-bit doubles and bit operations require 32-bit signed ints. + +## Janet Gotchas + +- Bit operations (brshift, brushift, band) use 32-bit signed integers. Hash values can exceed 32-bit range. Use `(band x 0xFFFFFFFF)` before shifting. +- `deftype` creates tables, not structs. `struct?` returns false. +- `(get child :key)` DOES follow table prototype chain — resolved and confirmed working. +- Janet LSP produces many false positives on `.janet` files — safe to ignore. + +## Symbol representation + +Jolt symbols are `{:jolt/type :symbol :ns :name }` as produced by the reader. \ No newline at end of file diff --git a/src/jolt/api.janet b/src/jolt/api.janet index 946fc87..f9860f6 100644 --- a/src/jolt/api.janet +++ b/src/jolt/api.janet @@ -5,10 +5,11 @@ (use ./reader) (use ./evaluator) (use ./core) +(use ./compiler) +(use ./loader) (defn- load-persistent-structures - "Load immutable persistent data structures and swap clojure.core bindings. - Replaces vec, vector, hash-map, hash-set, set with Jolt's persistent versions." + "Load immutable persistent data structures and swap clojure.core bindings." [ctx] (def source (slurp "src/jolt/clojure/lang/persistent_vector.clj")) (var cur source) @@ -24,15 +25,11 @@ (ns-intern core-ns "vector?" (var-get (ns-find pv-ns "vector?"))))) (defn init - "Create a new Jolt evaluation context, optionally with opts. - (init) — empty context with clojure.core loaded - (init opts) — context with opts and clojure.core loaded - - Persistent immutable data structures are loaded by default. - + "Create a new Jolt evaluation context. opts may contain: :namespaces — map of {ns-name → {sym → value, ...}, ...} - :mutable? — if true, use Janet mutable data structures instead" + :mutable? — use Janet mutable data structures instead of persistent + :compile? — enable compilation of Clojure forms to Janet" [&opt opts] (default opts {}) (let [ctx (make-ctx opts) @@ -45,17 +42,40 @@ (defn eval-string "Evaluate a Clojure source string in a Jolt context. - (eval-string ctx s) → value - - Returns the result of evaluating the first form in s." + When :compile? is enabled, compiles to Janet source and evaluates via Janet. + Stateful forms (def, defmacro, ns, deftype) always use the interpreter." [ctx s] - (let [form (parse-string s)] - (eval-form ctx @{} form))) + (let [compile? (get (ctx :env) :compile?) + form (parse-string s)] + (if (and compile? (array? form)) + # Check if this is a stateful form that needs the interpreter + (let [first-form (first form) + head-name (if (and (struct? first-form) (= :symbol (first-form :jolt/type))) + (first-form :name) + nil)] + (if (or (= head-name "def") (= head-name "defmacro") (= head-name "ns") + (= head-name "deftype") (= head-name "defmulti") (= head-name "defmethod") + (= head-name "require") (= head-name "in-ns")) + (eval-form ctx @{} form) + (compile-and-eval form))) + (eval-form ctx @{} form)))) (defn eval-string* - "Evaluate a Clojure source string in a Jolt context. - Like eval-string but with explicit bindings. - (eval-string* ctx s bindings) → value" + "Evaluate a Clojure source string with explicit bindings." [ctx s bindings] (let [form (parse-string s)] (eval-form ctx bindings form))) + +(defn compile-string + "Compile a Clojure source string to Janet source. + Returns the Janet source string." + [s] + (let [form (parse-string s)] + (compile-form form))) + +(defn compile-file + "Compile a .clj file to Janet source and optionally eval it. + When ctx has :compile? enabled, also evaluates the compiled forms. + Returns the namespace name." + [ctx filepath] + (load-ns ctx filepath)) diff --git a/src/jolt/compiler.janet b/src/jolt/compiler.janet index 704439c..4467bce 100644 --- a/src/jolt/compiler.janet +++ b/src/jolt/compiler.janet @@ -2,8 +2,6 @@ # Source-to-source: Clojure forms → Janet source # Two-phase: analyze-form (classify) → emit-ast (generate) -# Mapping from Clojure core names to Janet function names. -# Populated at init time from the core-bindings map. (def- core-renames @{"+" "core-+" "-" "core--" @@ -89,15 +87,10 @@ "mod" "core-mod"}) (defn- literal? - "Check if a form is a self-evaluating literal (not a symbol)." [form] (or (nil? form) (= true form) (= false form) (number? form) (string? form) (keyword? form) (bytes? form) (buffer? form))) -(defn- sym-name-str - [sym-s] - (if (sym-s :ns) (string (sym-s :ns) "/" (sym-s :name)) (sym-s :name))) - (defn- special-form? [name] (or (= name "quote") (= name "syntax-quote") (= name "do") @@ -108,33 +101,26 @@ (= name "defmulti") (= name "defmethod") (= name "locking"))) # ============================================================ -# Analyzer — Clojure form → annotated AST node {:op ...} +# Analyzer # ============================================================ (defn analyze-form - "Analyze a Clojure form and return an AST node with :op key. - Takes bindings (table) for local symbol classification. - Recursively analyzes sub-expressions." [form bindings] (cond - # Literals (literal? form) {:op :const :val form} - # Symbols (and (struct? form) (= :symbol (form :jolt/type))) (let [name (form :name) ns (form :ns)] (if ns {:op :qualified-symbol :ns ns :name name} - # Check local bindings first (let vars, fn params) (if (get bindings name) {:op :local :name name} (if (and (not (special-form? name)) (get core-renames name)) {:op :core-symbol :name name :janet-name (get core-renames name)} {:op :symbol :name name})))) - # Lists/arrays (array? form) (let [first-form (first form) head-name (if (and (struct? first-form) (= :symbol (first-form :jolt/type))) @@ -159,7 +145,6 @@ :name (in form 1) :init (analyze-form (in form 2) bindings)} "fn*" (let [params (in form 1) - # Augment bindings with param names so body refs are :local body-bindings (do (var bb @{}) (loop [[k v] :pairs bindings] (put bb k v)) @@ -177,20 +162,18 @@ (first analyzed-body))}) "let*" (let [bind-vec (in form 1) body-exprs (tuple/slice form 2) - # Analyze binding init values with outer bindings - n-binding-slots (length bind-vec) binding-pairs (do (var pairs @[]) (var i 0) - (while (< i n-binding-slots) - (let [sym-s (in bind-vec i) - name (if (struct? sym-s) (sym-s :name) sym-s) - val-form (if (< (+ i 1) n-binding-slots) (in bind-vec (+ i 1)) nil) - val-ast (if val-form (analyze-form val-form bindings) {:op :const :val nil})] - (array/push pairs {:name name :init val-ast}) - (+= i 2))) + (let [n (length bind-vec)] + (while (< i n) + (let [sym-s (in bind-vec i) + name (if (struct? sym-s) (sym-s :name) sym-s) + val-form (if (< (+ i 1) n) (in bind-vec (+ i 1)) nil) + val-ast (if val-form (analyze-form val-form bindings) {:op :const :val nil})] + (array/push pairs {:name name :init val-ast}) + (+= i 2)))) pairs) - # Augment bindings with let-bound names for body analysis body-bindings (do (var bb @{}) (loop [[k v] :pairs bindings] (put bb k v)) @@ -206,33 +189,29 @@ :statements (array/slice analyzed-body 0 (- n-body 1)) :ret (last analyzed-body)} (first analyzed-body))}) - # Default: function invocation (let [f-ast (analyze-form first-form bindings) args (map |(analyze-form $ bindings) (tuple/slice form 1))] {:op :invoke :fn f-ast :args args})) - # Non-symbol head: function invocation (let [f-ast (analyze-form first-form bindings) args (map |(analyze-form $ bindings) (tuple/slice form 1))] {:op :invoke :fn f-ast :args args}))) - # Tuples (vectors) (tuple? form) (let [items (map |(analyze-form $ bindings) form)] {:op :vector :items items}) - # Structs (maps) (struct? form) {:op :map :form form} - # Fallback {:op :const :val form})) # ============================================================ -# Emitter — AST node → Janet source string +# Emitter — AST → Janet source string # ============================================================ -(defn- emit-const - "Emit a literal constant value." +(var emit-ast nil) + +(defn- emit-const-str [val buf] (cond (nil? val) (buffer/push buf "nil") @@ -242,11 +221,7 @@ (keyword? val) (do (buffer/push buf ":") (buffer/push buf (string val))) (buffer/push buf (string val)))) -# Forward declaration for mutual recursion -(var emit-ast nil) - -(defn- emit-do - [statements ret buf] +(defn- emit-do-str [statements ret buf] (buffer/push buf "(do ") (var i 0) (let [n (length statements)] @@ -254,91 +229,57 @@ (emit-ast (in statements i) buf) (buffer/push buf " ") (++ i))) - (when ret - (emit-ast ret buf)) + (when ret (emit-ast ret buf)) (buffer/push buf ")")) -(defn- emit-if - [test then else buf] +(defn- emit-if-str [test then else buf] (buffer/push buf "(if ") - (emit-ast test buf) - (buffer/push buf " ") + (emit-ast test buf) (buffer/push buf " ") (emit-ast then buf) - (when else - (buffer/push buf " ") - (emit-ast else buf)) + (when else (buffer/push buf " ") (emit-ast else buf)) (buffer/push buf ")")) -(defn- emit-def - [name-sym init buf] - (buffer/push buf "(def ") - (buffer/push buf (name-sym :name)) - (buffer/push buf " ") - (emit-ast init buf) - (buffer/push buf ")")) +(defn- emit-def-str [name-sym init buf] + (buffer/push buf "(def ") (buffer/push buf (name-sym :name)) + (buffer/push buf " ") (emit-ast init buf) (buffer/push buf ")")) -(defn- emit-fn - [params body buf] +(defn- emit-fn-str [params body buf] (buffer/push buf "(fn [") (var i 0) (let [n (length params)] (while (< i n) (let [p (in params i)] (buffer/push buf (if (struct? p) (p :name) (string p)))) - (when (< (+ i 1) n) - (buffer/push buf " ")) + (when (< (+ i 1) n) (buffer/push buf " ")) (++ i))) - (buffer/push buf "] ") - (emit-ast body buf) - (buffer/push buf ")")) + (buffer/push buf "] ") (emit-ast body buf) (buffer/push buf ")")) -(defn- emit-let - [binding-pairs body buf] +(defn- emit-let-str [binding-pairs body buf] (buffer/push buf "(let [") (var i 0) (let [n (length binding-pairs)] (while (< i n) (let [bp (in binding-pairs i)] - (buffer/push buf (bp :name)) - (buffer/push buf " ") + (buffer/push buf (bp :name)) (buffer/push buf " ") (emit-ast (bp :init) buf) (when (< (+ i 1) n) (buffer/push buf " "))) (++ i))) - (buffer/push buf "] ") - (emit-ast body buf) + (buffer/push buf "] ") (emit-ast body buf) (buffer/push buf ")")) + +(defn- emit-invoke-str [f-ast args buf] + (buffer/push buf "(") (emit-ast f-ast buf) + (each arg args (buffer/push buf " ") (emit-ast arg buf)) (buffer/push buf ")")) -(defn- emit-invoke - [f-ast args buf] - (buffer/push buf "(") - (emit-ast f-ast buf) - (each arg args - (buffer/push buf " ") - (emit-ast arg buf)) - (buffer/push buf ")")) +(defn- emit-symbol-str [name buf] (buffer/push buf name)) +(defn- emit-local-str [name buf] (buffer/push buf name)) +(defn- emit-core-symbol-str [janet-name buf] (buffer/push buf janet-name)) -(defn- emit-symbol - [name buf] - (buffer/push buf name)) +(defn- emit-qualified-symbol-str [ns name buf] + (buffer/push buf "(ns-get \"") (buffer/push buf ns) + (buffer/push buf "\" \"") (buffer/push buf name) (buffer/push buf "\")")) -(defn- emit-local - [name buf] - (buffer/push buf name)) - -(defn- emit-core-symbol - [janet-name buf] - (buffer/push buf janet-name)) - -(defn- emit-qualified-symbol - [ns name buf] - (buffer/push buf "(ns-get \"") - (buffer/push buf ns) - (buffer/push buf "\" \"") - (buffer/push buf name) - (buffer/push buf "\")")) - -(defn- emit-vector - [items buf] +(defn- emit-vector-str [items buf] (buffer/push buf "[") (var i 0) (let [n (length items)] @@ -348,35 +289,34 @@ (++ i))) (buffer/push buf "]")) -(defn- emit-map - [form buf] - (buffer/push buf (string form))) +(defn- emit-map-str [form buf] (buffer/push buf (string form))) -(defn- emit-quote - [expr buf] - (buffer/push buf "'") - (emit-ast (analyze-form expr @{}) buf)) +(defn- emit-quote-str [expr buf] + (buffer/push buf "'") (emit-ast (analyze-form expr @{}) buf)) (set emit-ast (fn [ast buf] (match (ast :op) - :const (emit-const (ast :val) buf) - :symbol (emit-symbol (ast :name) buf) - :local (emit-local (ast :name) buf) - :core-symbol (emit-core-symbol (ast :janet-name) buf) - :qualified-symbol (emit-qualified-symbol (ast :ns) (ast :name) buf) - :do (emit-do (ast :statements) (ast :ret) buf) - :if (emit-if (ast :test) (ast :then) (ast :else) buf) - :def (emit-def (ast :name) (ast :init) buf) - :fn (emit-fn (ast :params) (ast :body) buf) - :let (emit-let (ast :binding-pairs) (ast :body) buf) - :invoke (emit-invoke (ast :fn) (ast :args) buf) - :vector (emit-vector (ast :items) buf) - :map (emit-map (ast :form) buf) - :quote (emit-quote (ast :expr) buf) - # Fallback for unknown ops + :const (emit-const-str (ast :val) buf) + :symbol (emit-symbol-str (ast :name) buf) + :local (emit-local-str (ast :name) buf) + :core-symbol (emit-core-symbol-str (ast :janet-name) buf) + :qualified-symbol (emit-qualified-symbol-str (ast :ns) (ast :name) buf) + :do (emit-do-str (ast :statements) (ast :ret) buf) + :if (emit-if-str (ast :test) (ast :then) (ast :else) buf) + :def (emit-def-str (ast :name) (ast :init) buf) + :fn (emit-fn-str (ast :params) (ast :body) buf) + :let (emit-let-str (ast :binding-pairs) (ast :body) buf) + :invoke (emit-invoke-str (ast :fn) (ast :args) buf) + :vector (emit-vector-str (ast :items) buf) + :map (emit-map-str (ast :form) buf) + :quote (emit-quote-str (ast :expr) buf) (buffer/push buf (string "/* unhandled op: " (ast :op) " */"))))) +# ============================================================ +# Public API +# ============================================================ + (defn compile-form "Compile a Clojure form to a Janet source string." [form] @@ -384,3 +324,19 @@ buf @""] (emit-ast ast buf) (string buf))) + +(defn eval-janet-source + "Parse and evaluate a Janet source string. + Uses the proper parser→produce→eval pipeline so special forms work." + [source] + (def p (parser/new)) + (parser/consume p source) + (parser/eof p) + (def form (parser/produce p)) + (eval form)) + +(defn compile-and-eval + "Compile a Clojure form to Janet source and evaluate it. + Returns the result value." + [form] + (eval-janet-source (compile-form form))) diff --git a/src/jolt/loader.janet b/src/jolt/loader.janet new file mode 100644 index 0000000..4a2d09a --- /dev/null +++ b/src/jolt/loader.janet @@ -0,0 +1,80 @@ +# Jolt Loader +# Namespace loading with optional compilation. +# Supports in-memory bytecode caching when :compile? is enabled. + +(use ./reader) +(use ./compiler) +(use ./evaluator) + +(defn load-ns + "Load a Clojure namespace from a .clj file. + When ctx has :compile? enabled, forms are compiled to Janet source, + evaluated via Janet's evaluator, and cached. + + (load-ns ctx filepath) → namespace symbol string" + [ctx filepath] + (let [env (ctx :env) + compile? (get env :compile?) + cache (get env :compiled-cache)] + + (def source (slurp filepath)) + (var ns-name nil) + (var remaining source) + (var forms @[]) + + # Parse all forms + (while (> (length (string/trim remaining)) 0) + (def [form rest] (parse-next remaining)) + (set remaining rest) + (when (not (nil? form)) + (array/push forms form) + # Extract ns name from the first ns form + (when (and (nil? ns-name) + (array? form) + (> (length form) 0) + (and (struct? (first form)) + (= :symbol ((first form) :jolt/type)) + (= "ns" ((first form) :name)))) + (let [name-form (in form 1)] + (set ns-name (if (struct? name-form) (name-form :name) (string name-form))))))) + + (when (nil? ns-name) + (error (string "No ns form found in " filepath))) + + (if compile? + (do + # Compile each form and eval as Janet + (var cached (get cache ns-name)) + (when (nil? cached) + (set cached @[]) + (put cache ns-name cached)) + + (each form forms + (let [janet-src (compile-form form)] + (array/push cached janet-src) + (eval-janet-source janet-src))) + ns-name) + # Interpreter path + (do + (each form forms + (eval-form ctx @{} form)) + ns-name)))) + +(defn compiled? + "Check if a namespace has been compiled and cached." + [ctx ns-name] + (let [cache (get (ctx :env) :compiled-cache)] + (not (nil? (get cache ns-name))))) + +(defn get-compiled-forms + "Get the compiled Janet source forms for a namespace." + [ctx ns-name] + (get (ctx :env) :compiled-cache ns-name)) + +(defn clear-compiled-cache + "Clear the compiled form cache for a namespace or all namespaces." + [ctx &opt ns-name] + (let [cache (get (ctx :env) :compiled-cache)] + (if ns-name + (put cache ns-name nil) + (loop [[k] :pairs cache] (put cache k nil))))) diff --git a/src/jolt/types.janet b/src/jolt/types.janet index dcaa9c9..96899aa 100644 --- a/src/jolt/types.janet +++ b/src/jolt/types.janet @@ -240,9 +240,12 @@ :namespaces — struct of {ns-symbol → {sym → value, ...}, ...}" [&opt opts] (default opts nil) - (let [env @{:namespaces @{} + (let [compile? (if opts (get opts :compile?) false) + env @{:namespaces @{} :class->opts @{} - :current-ns "user"} + :current-ns "user" + :compile? compile? + :compiled-cache @{}} # create the user namespace via a partial context _ (ctx-find-ns {:env env} "user")] # initialize from opts diff --git a/test/compiler-test.janet b/test/compiler-test.janet index 0275f22..034ae05 100644 --- a/test/compiler-test.janet +++ b/test/compiler-test.janet @@ -93,3 +93,51 @@ (print " passed") (print "\nAll compiler Phase 2 tests passed!") + +# ============================================================ +# 9. Compile-and-eval round-trip (Phase 3) +# ============================================================ +(print "9: compile-and-eval...") +(use ../src/jolt/core) # need core fns in scope for eval + +(defn compile-eval-str [s] + (let [form (parse-string s)] + (compile-and-eval form))) + +(assert (= 42 (compile-eval-str "42")) "eval literal") +(assert (= 2 (compile-eval-str "(inc 1)")) "eval inc") +(assert (= 3 (compile-eval-str "(+ 1 2)")) "eval +") +(assert (= 6 (compile-eval-str "(+ (inc 1) (inc 3))")) "eval nested") +(assert (= 2 (compile-eval-str "(do 1 2)")) "eval do") +(assert (= 1 (compile-eval-str "(if true 1 2)")) "eval if true") +(assert (= 2 (compile-eval-str "(if false 1 2)")) "eval if false") +(assert (= 2 (compile-eval-str "(let* [x 1] (inc x))")) "eval let") +(let [f (compile-eval-str "(fn* [x] (inc x))")] + (assert (function? f) "eval fn returns fn") + (assert (= 6 (f 5)) "eval fn works")) +(print " passed") + +# ============================================================ +# 10. Compile flag in context (Phase 3) +# ============================================================ +(print "10: compile flag...") +(use ../src/jolt/api) + +# Without compile flag +(let [ctx (init)] + (assert (= 2 (eval-string ctx "(inc 1)")) "no-compile flag: inc works")) + +# With compile flag: pure expressions use compile-and-eval +(let [ctx (init {:compile? true})] + (assert (= 2 (eval-string ctx "(inc 1)")) "compile flag: inc works") + (assert (= 3 (eval-string ctx "(+ 1 2)")) "compile flag: + works") + (assert (= 6 (eval-string ctx "(+ (inc 1) (inc 3))")) "compile flag: nested works")) + +# With compile flag: stateful forms fall back to interpreter +(let [ctx (init {:compile? true})] + (eval-string ctx "(def foo 99)") + (assert (= 99 (eval-string ctx "foo")) "compile flag: def works")) + +(print " passed") + +(print "\nAll compiler Phase 3 tests passed!")