diff --git a/README.md b/README.md index 1401e6a..fa4adbb 100644 --- a/README.md +++ b/README.md @@ -193,7 +193,7 @@ Jolt targets Clojure semantics but runs on Janet, not the JVM. The notable diver - **Concurrency / STM.** No refs, `dosync`, agents, or `send`; `locking` evaluates its body without real locking. Atoms, volatiles, promises, and delays are supported. - **Futures.** `future` runs its body on a *real* OS thread (Janet's `ev/thread`), so it can use a second core for CPU-bound work — unlike the cooperatively-scheduled `go` blocks. `deref`/`@` parks until the result is ready (with the optional `(deref f timeout-ms timeout-val)` arity); `future?`, `future-done?`, `realized?`, `future-cancel`, and `future-cancelled?` are supported. Two important divergences from the JVM: (1) **snapshot semantics** — Janet threads have separate heaps, so the body and the state it closes over are *copied* to the worker thread and only the return value is copied back; mutating a captured atom does not propagate to the parent (communicate via the return value). (2) **no thread interruption** — Janet OS threads can't be cancelled mid-run, so `future-cancel` marks the *future* cancelled (deref then throws and the predicates flip) but the underlying computation still runs to completion in the background. As on the JVM, a live future thread keeps the process alive until it finishes (the JVM's non-daemon future pool behaves the same). - **core.async.** `clojure.core.async` runs on Janet fibers and channels (`chan`, `go`, `go-loop`, `!`/`!!`, `close!`, `alts!`, `timeout`, `put!`/`take!`, `buffer`/`dropping-buffer`/`sliding-buffer`, and channel transducers via `(chan n xform)`). Because Janet fibers are stackful coroutines, a `go` block is just its body run in a fiber — no CPS/state-machine rewrite — so `!` work *anywhere*, including inside `try`, nested `fn`s, and loops (positions Clojure's `go` macro forbids). Go blocks are cooperatively scheduled on one OS thread, so parking (`…)`). +- **Regex.** Compiled to Janet's PEG engine (Janet has no regex). Supported: capturing groups (`[whole g1 …]`), greedy and lazy quantifiers with backtracking, `(?:…)`, lookahead `(?=…)`/`(?!…)`, alternation, anchors `^ $ \b \B`, character classes, and the `(?i)` flag. Not supported: lookbehind, backreferences (`\1`), named groups (`(?…)`), and Unicode property classes (`\p{Lu}`). - **Arrays.** Java-style arrays map onto Janet's native types: `byte-array` is a Janet buffer (contiguous, C-backed); `object-array`/`int-array`/`double-array`/etc. are Janet arrays. `aget`/`aset`/`alength`/`aclone` work over both. - **Transients.** `transient`/`conj!`/`assoc!`/`dissoc!`/`disj!`/`pop!`/`persistent!` are real mutable scratch collections backed by Janet's native arrays and tables (vectors → arrays, maps/sets → tables), so building a collection with them avoids the per-step copying of the persistent path (notably for maps/sets). `persistent!` freezes back to a persistent value. - **Not implemented.** JVM reflection, `proxy`, and the `clojure.repl`/`clojure.template` namespaces. diff --git a/doc/grammar.ebnf b/doc/grammar.ebnf index d32a3c6..26f450c 100644 --- a/doc/grammar.ebnf +++ b/doc/grammar.ebnf @@ -122,6 +122,10 @@ unquote = "~" , form ; (* (unquote form) *) deref = "@" , form ; (* (deref form) *) metadata = "^" , meta-form , ws , form ; (* attach metadata to form *) meta-form = map | keyword | symbol | string ; +(* Normalized like Clojure: a symbol or string is a type hint -> {:tag ...}; + a keyword -> {keyword true}; a map is used as-is. On a symbol the metadata + rides on the symbol (it stays a bare symbol, so a hint like ^String is + transparent in params/lets/bodies); other targets use a runtime with-meta. *) (* -------------------------------------------------------------------------- Dispatch forms — introduced by "#". diff --git a/doc/tools-deps.md b/doc/tools-deps.md index 96dbe9d..77f1e20 100644 --- a/doc/tools-deps.md +++ b/doc/tools-deps.md @@ -56,6 +56,12 @@ then `.cljc`. Extra roots come from `JOLT_PATH` or `init`'s `:paths` option. together: it resolves the roots and runs the `jolt` binary with them on `JOLT_PATH`. The runtime's only dependency interface is that env var. +`jolt uberscript` bundles a namespace and everything it requires into one +standalone `.clj`. It requires the entry namespace and uses the order in which +the loader finishes loading files — a dependency finishes before the file that +required it, so the order is topological — then concatenates that source. The +baked-in stdlib is excluded (it's part of the runtime, not bundled). + Gotcha worth remembering: the `jolt` CLI's context is built into its image at build time, so `JOLT_PATH` is applied at runtime in `main`, not in `init` (whose env read would be frozen at build). @@ -68,29 +74,18 @@ env read would be frozen at build). - Source only; compiled `.class` files in a git dep are ignored. - git `:git/sha` must be a full SHA (`git fetch` can't resolve a short one). -## Status +## Conformance -- **Loader source roots** — done. `find-ns-file` + `:source-paths`, `.clj`/`.cljc`, - `JOLT_PATH`/`:paths`. Test: `test/integration/deps-loader-test.janet`. -- **Resolve git/local deps via jpm** — done. `deps.janet` + the `jolt-deps` tool. - Test: `test/integration/deps-resolve-test.janet`. -- **Bundling (uberscript)** — done. `jolt uberscript OUT -m NS` requires `NS`, - records the load order the loader emits (a dependency finishes loading before - the file that required it, so it's topological), and concatenates the source - into one `.clj` that runs on a plain `jolt` — no deps, no jpm. `jolt-deps - uberscript` resolves first. Test: `test/integration/uberscript-test.janet`. - (This is the practical form of "compile the used namespaces into the build"; - embedding into a custom binary image is a heavier variant we haven't needed.) -- **Conformance** — harness in place: - `test/integration/deps-conformance-test.janet` resolves real pure-`cljc` git - libs and reports load/run status (network-gated behind `JOLT_CONFORMANCE=1`, so - CI stays offline). First run: - - `medley` — loads and runs. - - `cuerdas` — now loads (it used `\{`, a one-char literal the reader rejected; - fixed). A function (`kebab`) still hits a separate runtime gap — a smaller, - per-function issue rather than a whole-namespace failure. - - `stuartsierra/dependency` — `Long/MAX_VALUE`: JVM interop, so out of scope - (it isn't actually pure-`cljc`). +`test/integration/deps-conformance-test.janet` resolves a few real pure-`cljc` +git libraries and reports whether their namespaces load and a sample call works. +It's network-gated behind `JOLT_CONFORMANCE=1` so CI stays offline. Use it to +check a library against the current interpreter, and to drive fixes for whatever +gap a failure points at (the same loop as the clojure-test-suite battery). A +library fails when it relies on something Jolt doesn't provide — JVM interop, or +a regex feature like Unicode property classes (`\p{…}`). - The loop from here is the same as the clojure-test-suite battery: add libs, - see what breaks, fix interpreter/reader gaps. +## Not yet + +- **Compiling deps into a binary image.** `uberscript` already produces a + standalone `.clj`; baking a project's dependencies directly into a custom + executable image is a heavier variant that isn't implemented. diff --git a/src/jolt/core.janet b/src/jolt/core.janet index ac2a4c7..ebf6b90 100644 --- a/src/jolt/core.janet +++ b/src/jolt/core.janet @@ -1410,8 +1410,12 @@ (defn core-meta [x] "Returns the metadata of x, or nil." - (if (var? x) (var-meta x) - (if (table? x) (or (get x :jolt/meta) (get x :meta)) nil))) + (cond + (var? x) (var-meta x) + # symbols carry reader metadata (type hints etc.) in a :meta field + (and (struct? x) (= :symbol (get x :jolt/type))) (get x :meta) + (table? x) (or (get x :jolt/meta) (get x :meta)) + nil)) (defn core-every-pred [& preds] (fn [& xs] diff --git a/src/jolt/evaluator.janet b/src/jolt/evaluator.janet index c4e3011..d17c6f6 100644 --- a/src/jolt/evaluator.janet +++ b/src/jolt/evaluator.janet @@ -632,10 +632,16 @@ (if (> (length form) 3) (eval-form ctx bindings (in form 3)) nil))) "def" (let [raw-name (in form 1) name-sym (unwrap-meta-name raw-name) - # Check for ^:dynamic metadata - dynamic? (and (array? raw-name) (> (length raw-name) 0) - (sym-name? (first raw-name) "with-meta") - (= :dynamic (last raw-name))) + # Metadata on the name: keyword/type-hint metadata rides on the + # symbol (:meta); a ^{:map} reads as a with-meta form we evaluate. + sym-meta (or (and (struct? name-sym) (get name-sym :meta)) {}) + wm-meta (if (and (array? raw-name) (> (length raw-name) 0) + (sym-name? (first raw-name) "with-meta")) + (let [mv (protect (eval-form ctx bindings (last raw-name)))] + (if (and (mv 0) (or (table? (mv 1)) (struct? (mv 1)))) (mv 1) {})) + {}) + name-meta (merge wm-meta sym-meta) + dynamic? (truthy? (get name-meta :dynamic)) ns-name (ctx-current-ns ctx) ns (ctx-find-ns ctx ns-name) # Create var first (unbound) so self-referencing defs resolve @@ -644,8 +650,9 @@ has-doc (and (> (length form) 3) (string? (in form 2))) val (eval-form ctx bindings (in form (if has-doc 3 2)))] (bind-root v val) - (when has-doc - (put v :meta (merge (or (get v :meta) {}) {:doc (in form 2)}))) + (let [extra (if has-doc (merge name-meta {:doc (in form 2)}) name-meta)] + (when (not (empty? extra)) + (put v :meta (merge (or (get v :meta) {}) extra)))) (when dynamic? (put v :dynamic true)) # def returns the var (Clojure semantics); REPL prints #'ns/name diff --git a/src/jolt/reader.janet b/src/jolt/reader.janet index 65b0b36..45016fd 100644 --- a/src/jolt/reader.janet +++ b/src/jolt/reader.janet @@ -446,11 +446,30 @@ (let [[form final-pos] (read-form s new-pos)] [(array token-sym form) final-pos])) +(defn- meta-form->map + "Normalize a metadata reader form (Clojure semantics): a symbol or string is a + :tag, a keyword is {kw true}. Returns a metadata table, or nil if it isn't one + of those simple shapes (e.g. a map literal — handled via with-meta instead)." + [meta-form] + (cond + (keyword? meta-form) {meta-form true} + (and (struct? meta-form) (= :symbol (meta-form :jolt/type))) {:tag (meta-form :name)} + (string? meta-form) {:tag meta-form} + nil)) + (defn read-meta [s pos] # pos is at ^ (let [[meta-form new-pos] (read-form s (+ pos 1)) - [form new-pos2] (read-form s new-pos)] - [(array (sym "with-meta") form meta-form) new-pos2])) + [form new-pos2] (read-form s new-pos) + m (meta-form->map meta-form)] + (if (and m (struct? form) (= :symbol (form :jolt/type))) + # Attach the metadata to the symbol itself and keep it a bare symbol, so + # type hints (^String x) and ^:dynamic etc. are transparent in every + # position (params, lets, bodies) — the evaluator reads :name and ignores + # :meta. This is what makes type hints "parse and otherwise do nothing". + [(struct ;(kvs form) :meta (merge (or (form :meta) {}) m)) new-pos2] + # Map metadata or non-symbol targets keep the runtime with-meta form. + [(array (sym "with-meta") form meta-form) new-pos2]))) (defn read-until-newline [s pos] (if (or (>= pos (length s)) (= (s pos) 10)) diff --git a/test/spec/metadata-spec.janet b/test/spec/metadata-spec.janet index bdaf1cf..b3df3b5 100644 --- a/test/spec/metadata-spec.janet +++ b/test/spec/metadata-spec.janet @@ -9,3 +9,21 @@ ["vary-meta" "{:a 2}" "(meta (vary-meta (with-meta [1] {:a 1}) update :a inc))"] ["meta reader ^" "{:tag :int}" "(meta ^{:tag :int} [1 2])"] ["with-meta on fn ok" "true" "(fn? (with-meta inc {:a 1}))"]) + +(defspec "metadata / type hints" + # ^Type / ^:kw / ^"str" on a symbol attach as metadata and are otherwise inert: + # the symbol stays a symbol so hints are transparent in every position. + ["type hint on param" "\"hi\"" "(do (defn f [^String s] s) (f \"hi\"))"] + ["type hint, extra params" "[1 2]" "(do (defn g [^String x y] [x y]) (g 1 2))"] + ["type hint in let" "6" "(let [^long x 5] (inc x))"] + ["type hint in body" "2" "(let [s \"ab\"] (count ^String s))"] + ["type hint in destructure" "3" "(let [{:keys [^long a]} {:a 3}] a)"] + ["symbol hint -> :tag" "\"String\"" "(:tag (meta (read-string \"^String x\")))"] + ["keyword hint -> true" "true" "(:foo (meta (read-string \"^:foo x\")))"]) + +(defspec "metadata / def metadata" + ["^:dynamic var binds" "9" "(do (def ^:dynamic *d* 1) (binding [*d* 9] *d*))"] + ["^:private on var" "true" "(do (def ^:private pv 1) (:private (meta (var pv))))"] + ["^Type tag on var" "\"String\"" "(do (def ^String tv \"a\") (:tag (meta (var tv))))"] + ["^{:doc} on var" "\"hi\"" "(do (def ^{:doc \"hi\"} dv 1) (:doc (meta (var dv))))"] + ["(def name doc val) doc" "\"d\"" "(do (def dd \"d\" 5) (:doc (meta (var dd))))"]) diff --git a/test/unit/reader-test.janet b/test/unit/reader-test.janet index d5c6bc7..87a1565 100644 --- a/test/unit/reader-test.janet +++ b/test/unit/reader-test.janet @@ -83,11 +83,19 @@ (assert (deep= @[(sym "deref") (sym "x")] (parse-string "@x")) "deref shorthand") -# Metadata +# Metadata: a keyword/symbol/string hint on a symbol attaches to the symbol's +# :meta and keeps it a bare symbol (so type hints are transparent everywhere). (let [form (parse-string "^:meta x")] - (assert (array? form) "metadata is array") - (assert (deep= @[(sym "with-meta") (sym "x") :meta] form) - "metadata form")) + (assert (and (struct? form) (= :symbol (form :jolt/type))) "meta-hinted symbol stays a symbol") + (assert (= "x" (form :name)) "symbol name preserved") + (assert (= true (get (form :meta) :meta)) "keyword hint -> {kw true}")) +(let [form (parse-string "^String s")] + (assert (= "s" (form :name)) "type-hinted symbol name preserved") + (assert (= "String" (get (form :meta) :tag)) "symbol hint -> {:tag name}")) +# Map metadata on a symbol still uses a runtime with-meta form. +(let [form (parse-string "^{:doc \"d\"} y")] + (assert (and (array? form) (struct? (first form)) (= "with-meta" ((first form) :name))) + "map metadata -> with-meta form")) # Comments (skip to end of line) (assert (= 42 (parse-string "; comment\n42"))