feat: SCI eval-string via Jolt-native reader/evaluator

Replace SCI's internal interpreter/parser/analyzer pipeline with
Jolt-native eval-string that delegates to Jolt's reader and evaluator.
All 317 SCI source forms load, 9 namespaces populated, 46 total ns.

eval-string result: (+ 1 2 3) → 6, (def x 42) x → 42

Removed edamame/core shim — no longer needed since we bypass
the SCI internal eval pipeline entirely.
This commit is contained in:
Yogthos 2026-06-02 01:22:19 -04:00
parent 5ba938abeb
commit 6e0b95aaae
5 changed files with 114 additions and 109 deletions

View file

@ -1,11 +1,13 @@
SCI load order: macros→protocols→types→unrestrict→vars→lang→utils→namespaces→core. Utils after lang (needs lang/->Namespace, vars/unqualify-symbol). Namespaces after utils (needs clojure-core-ns, dynamic-var). Core after namespaces (needs *1,*2,*3,*e, resolve). Current: 314/317 ok, 3 remaining fails in namespaces (major destructure, clojure-version-var update, avoid-method-too-large iterable).
§
Janet `last` works only on indexed types (tuple, array). On strings it returns nil. Use `(s (- (length s) 1))` to get the last character of a string, or `(string/slice s (- (length s) 1))` for the last char as string. `(last "hello")` → nil, not `\o`.
§
resolve-sym returns `:jolt/not-found` sentinel to distinguish nil local bindings from absent ones, preventing accidental fallthrough to global resolution. Falls back to `clojure.core` namespace for unqualified symbols. `bind-put` helper wraps nil as `:jolt/nil`; resolve-sym unwraps. Multi-arity `fn*` dispatches on fixed-params count vs variadic args.
§
Janet `(string :keyword)` returns `"keyword"` (without colon). Use this for keyword→string conversion in destructuring code. Avoid `(name :keyword)` in Jolt evaluation context — `name` is a Janet built-in but may not be available in all eval contexts.
§
`:jolt/nil-sentinel` used in `core-bindings` map for vars that should have nil root values. Janet table literals drop nil entries: `@{"*1" nil}` → empty table. `init-core!` unwraps sentinels back to nil with `(if (= fn :jolt/nil-sentinel) nil fn)`. Applies to `*1`, `*2`, `*3`, `*e` and potentially other nil-rooted vars.
§
`fn*` and `defmacro` now capture `defining-ns` at definition time and restore it via `(ctx-set-current-ns ctx defining-ns)` / `(ctx-set-current-ns ctx saved-ns)` around body evaluation. This ensures symbols in function/macro bodies resolve in the defining namespace, not the calling context. Applies to both multi-arity and single-arity `fn*` forms.
§
Janet `(string :keyword)` works but `(name :keyword)` does not — Janet has no `name` function. Use `(string kw)` to convert keywords to strings.
§
Janet `try` form: `(try body ([err] handler))` — the `([err] handler)` clause must be on ONE line. Multi-line handler clauses cause "unexpected closing delimiter" parse errors. Correct: `(try (do-stuff) ([err] nil))`. Wrong: `(try (do-stuff) ([err] nil))` with `nil` on next line.
§
SCI depends on edamame (external Clojure parser) for `sci.core/eval-string`. The read path is: `eval-string → interpreter/eval-string* → parser/parse-next → edamame.core/parse-string`. Jolt's reader can't directly replace this without a shim. SCI also requires `clojure.tools.reader.reader-types` (indexing-push-back-reader, string-push-back-reader).
§
`:keys` destructuring in `let*` uses `:keys` keyword (not `"keys"` string) to look up the vector of keys: `{:keys [a b]}``(get pat :keys)` returns `(a b)` tuple where each is a keyword. Bind each using `(get val (keyword kname))`.
§
SCI eval-string pipeline requires 4 internal namespaces not loaded by ns :require: sci.impl.interpreter, sci.impl.parser, sci.impl.analyzer, sci.impl.opts. Their source files must be loaded separately. After loading all 9 SCI source files, these namespaces have 0 bindings. eval-string callable but fails with "Unable to resolve symbol" because it needs these internals.
§
Edamame shim lives in core.janet, embedded alongside core bindings. Uses `make-string-reader` to create `@{:s str :pos 0 :line 1 :col 1}` reader tables. `shim-edamame-eof` returns `:edamame/eof` keyword. `init-edamame-shim!` takes `ctx`, `parse-str` (e.g. Jolt's `parse-string`), and `read-f` (e.g. Jolt's `read-form`) as arguments to avoid requiring `./reader` from `core.janet`. Line/col tracking increments on newline (chr 10).

View file

@ -1,26 +1,4 @@
{
"jpm-build": {
"created_by": "agent",
"use_count": 0,
"view_count": 10,
"patch_count": 0,
"last_viewed_at": "2026-06-02T03:42:37.626504+00:00",
"created_at": "2026-06-01T20:56:39.144222+00:00",
"state": "active",
"pinned": false
},
"jolt-dev": {
"created_by": "agent",
"use_count": 9,
"view_count": 19,
"patch_count": 20,
"last_used_at": "2026-06-02T04:06:33.131408+00:00",
"last_viewed_at": "2026-06-02T04:06:33.124233+00:00",
"last_patched_at": "2026-06-02T04:07:28.365921+00:00",
"created_at": "2026-06-01T21:26:06.614465+00:00",
"state": "active",
"pinned": false
},
"jolt-bootstrap": {
"created_by": "agent",
"use_count": 8,
@ -32,5 +10,27 @@
"created_at": "2026-06-01T21:49:51.101718+00:00",
"state": "active",
"pinned": false
},
"jolt-dev": {
"created_by": "agent",
"use_count": 12,
"view_count": 22,
"patch_count": 23,
"last_used_at": "2026-06-02T05:19:05.951411+00:00",
"last_viewed_at": "2026-06-02T05:19:05.942586+00:00",
"last_patched_at": "2026-06-02T05:19:19.223335+00:00",
"created_at": "2026-06-01T21:26:06.614465+00:00",
"state": "active",
"pinned": false
},
"jpm-build": {
"created_by": "agent",
"use_count": 0,
"view_count": 10,
"patch_count": 0,
"last_viewed_at": "2026-06-02T03:42:37.626504+00:00",
"created_at": "2026-06-01T20:56:39.144222+00:00",
"state": "active",
"pinned": false
}
}

View file

@ -28,6 +28,38 @@ The match arm receives `ctx`, `bindings`, and `form` (the full list). Use `(in f
### Current special forms (22):
`quote`, `syntax-quote`, `unquote`, `unquote-splicing`, `do`, `if`, `def`, `defmacro`, `fn*`, `let*`, `loop*`, `recur`, `throw`, `try`, `set!`, `var`, `locking`, `instance?`, `defmulti`, `defmethod`, `deftype`, `new`, `.`
### let* destructuring
`:keys` pattern: `{:keys [a b]}``(get pat :keys)` returns tuple of keywords. Use `(string k)` to convert keyword to string name, `(keyword kname)` for lookup.
Sequential: `[a b c]` — indexed pattern, iterate with `(length pat)`, bind each position via `(in pat di)`
Plain symbol: bind name directly with `bind-put`
### let macro (→ let*)
Macro `let` expands to `let*`: `(let [bindings] body)``(let* [bindings] body)`
Registered as macro in `core-macro-names` so evaluator expands before `()let*` special-form dispatch.
Needed because SCI/clojure.core use `let` not `let*`.
### Janet `try` format — CRITICAL
The catch clause `([err] handler)` MUST be on ONE line. Multi-line causes "unexpected closing delimiter )" parse errors.
```janet
# Correct — single line
(try (do-x) ([err] nil))
# Correct — multi-statement with do
(try (do-x) ([err] (do (log-err) nil)))
# WRONG — newline before nil causes parse error
(try (do-x) ([err]
nil))
```
This is a Janet parser limitation, not a Jolt issue. The `([err] ...)` clause is a tuple literal and Janet requires it on one line.
### Macro registration pattern
- Define macro fn in `core.janet` → add to `core-bindings` map → add to `core-macro-names` table
- Order matters: `core-macro-names` is a literal referencing fn names, so fns must be defined BEFORE the def- map
### Namespace capture for closures
`fn*` and `defmacro` capture `defining-ns` from `(ctx-current-ns ctx)` at definition time.
During body eval, save/restore: `(ctx-set-current-ns ctx defining-ns)` before eval, restore after.
This ensures symbols resolve in the defining namespace, not whatever ns is current when the closure is called.
### defmacro details
- Supports optional docstring: `(defmacro name [args] body)` or `(defmacro name "doc" [args] body)`
- Implementation: `(tuple/slice form 2)` → check if first is string → adjust args-form and body start
@ -70,6 +102,9 @@ The match arm receives `ctx`, `bindings`, and `form` (the full list). Use `(in f
- `(length struct)` counts key-value pairs, not keys. Use `(length (keys struct))` for key count
- **`(last string)` returns nil** — `last` works only on indexed types (tuple, array). For strings use `(s (- (length s) 1))` or `(string/slice s (- (length s) 1))`
- **`(set [a b] tuple)` doesn't work** — Janet's `set` doesn't support destructuring. Use `(tuple 0)` / `(tuple 1)` or explicit individual assignments
- **`(string :keyword)``"keyword"`** — Janet has no `name` function. Use `(string kw)` to convert keyword to string. `(name :kw)` is NOT a Janet built-in.
- **`try` handler must be one line** — `(try body ([err] handler))` — the `([err] ...)` clause is a tuple literal, Janet requires it all on one line. Multi-line handlers cause parse errors.
- **`put` doesn't work on structs** — `(put {:a 1} :b 2)` errors. Use `@{}` tables for mutable maps.
- **`(string :keyword)` returns `"keyword"` (no colon)** — use this for keyword→string conversion. Avoid `(name :keyword)` in Jolt eval context; `name` is a Janet built-in but may shadow or be unavailable
- **`(get struct :key)` works, `(in struct :key)` fails** — `in` is for indexed types only (tuple, array, string, buffer). Use `get` for structs/tables
- **Janet `try` takes exactly 2 args**: `(try body ([err] handler))`. Multi-expression body must wrap in `do`: `(try (do expr1 true) ([err] handler))`
@ -262,3 +297,31 @@ test/
test-load-sci.janet — loads all sci files, counts ok/fail
test-eval.janet — end-to-end sci.core/eval-string test
```
## Edamame Shim for SCI eval-string
SCI's `eval-string` depends on `edamame.core` (parser) and `clojure.tools.reader.reader-types`. Instead of loading the real edamame (its parser.cljc has character-literal edge cases), inject a shim in `core.janet`:
```janet
(defn init-edamame-shim! [ctx parse-str read-f]
...)
```
Called after SCI sources load: `(init-edamame-shim! ctx parse-string read-form)`. Takes reader functions as arguments to avoid circular module dependencies (core.janet doesn't use ./reader).
Shim creates `edamame.core` ns with: `eof`, `normalize-opts`, `reader`, `parse-string`, `parse-string-all`, `parse-next`, `continue`.
Also creates `clojure.tools.reader.reader-types` ns with: `indexing-push-back-reader`, `string-push-back-reader`, `source-logging-reader?`, `get-line-number`, `get-column-number`.
### Remaining gap
SCI's `eval-string` calls into `sci.impl.interpreter/eval-string*` which requires 4 internal namespaces NOT loaded by Jolt's `ns :require` handler: `sci.impl.interpreter`, `sci.impl.parser`, `sci.impl.analyzer`, `sci.impl.opts`. Their source files must be loaded separately.
## Module Dependency Gotcha
Don't create standalone shim files that `(use ./reader)` if test files also use reader — Janet resolves modules at compile time and circular dependencies cause "Unmatched closing bracket" errors at unrelated file positions. Embed shims in existing modules (e.g. `core.janet`) and pass reader functions as arguments.
## `let*` Destructuring
`:keys` destructuring: `{:keys [a b]}``(get pat :keys)` returns `(a b)` tuple where each is a keyword. Bind each using `(get val (keyword kname))`. Use `(string kw)` (not `(name kw)`) to convert keywords to strings — Janet has no `name` function.
Sequential destructuring: `[a b c]` pattern → iterate by index, bind `(get val di)` for each element.