1ded89b47b
* compiler: self-hosted analyzer compiles set literals (#{…})
Stage 1 Task 1. analyzer.clj punted set literals to the interpreter
((form-set? form) (uncompilable "set literal")); now it builds the set-node IR
(already defined in ir.clj) from (form-set-items form), and backend.janet emits
(make-phs e1 e2 …) — each element evaluated then the persistent set built,
mirroring compiler.janet's emit-set-expr and the interpreter's :jolt/set path.
Closes a self-hosted-analyzer vs bootstrap-compiler parity gap: #{…} no longer
forces interpreter fallback on the compile path.
Gate: conformance 262x3 (+4 set-literal cases incl computed elements / empty /
in-let), fixpoint, self-host, sci, suite 3981/66, specs+unit green; core-bench
neutral (A/B). set?/disj-as-fns remain deliberately interpreted (in-sync across
all three lists) — adjudicated in Task 2.
* test: fallback-zero harness — assert non-stateful forms compile (not interpret)
Stage 1 Task 3. self-host-test checks results but not which path ran. This runs
the portable analyzer (backend/analyze-form) on a corpus of non-stateful forms
and asserts NONE raise :jolt/uncompilable — i.e. the self-hosted analyzer
compiled them, not the interpreter fallback. Inverse sanity list confirms a few
intentional-interpret forms (ns/defmacro/require/set?/letfn) still punt, so the
harness can't pass by compiling everything.
29 must-compile (incl set literals from Task 1) + 5 must-punt, 0 failures. As
Stage 1 parity grows, forms move from the punt list into must-compile; when the
fallback set equals the frozen intentional stateful set, the bootstrap is
retireable.
* core: migrate 7 lazy seq fns from the Janet seed to the Clojure overlay (40-lazy)
Finishes a port the prior team started and reverted (bb4a3e0): the 40-lazy.clj
tier moved lazy seq fns Janet→Clojure but regressed the suite to 849 because
lazy-seq's expansion leaked as data in compile mode — that was jolt-r81, since
root-fixed (lazy-seq/lazy-cat moved to 00-syntax). With the wall gone, the port
works. This shrinks the Janet seed toward the north star (self-hosted
clojure-in-clojure on a minimal host bootstrap).
Moved to core/40-lazy.clj (wired as a loaded tier after 30-macros):
distinct keep keep-indexed map-indexed cycle repeat iterate
40-lazy.clj completed to full parity: distinct gains its transducer arity;
keep/keep-indexed/map-indexed already had both arities.
Removed from the Janet seed (core.janet): the 7 core-* fns + their core-bindings
entries, the now-dead td-keep/td-map-indexed transducer helpers (the CLJ versions
carry their own), and the already-dead core-partition-by/core-xml-seq (shadowed by
10-seq/20-coll). Net: core.janet −131 lines.
Deferred (kept in Janet, separate follow-ups): partition-all (a CLJ port via
take/drop realizes a non-minimal element count, tripping the §6.3 laziness
counters + a suite file) and repeatedly (canonical CLJ doesn't validate args, so
the repeatedly.cljc throw cases regress). Both need behavior-matching first.
Gate: conformance 262x3, lazy-infinite 44/44, clojure-test-suite 4004/66 (UP from
3981 — the CLJ versions add coverage, e.g. distinct value-equality), fixpoint,
self-host, sci 422/0, fallback-zero, specs+unit, core-bench all green.
* test: raise clojure-test-suite baseline 3981 -> 4004 (lazy-fn migration coverage)
* core: migrate partition-all to the Clojure overlay (minimal realization)
Resolves the deferred partition-all port (jolt-yo3). The earlier CLJ attempt via
lazy take/drop over-realized vs the Janet pstep, tripping the §6.3 laziness
counter. The collection arities now realize EXACTLY n per chunk with a first/rest
loop and continue from the advanced cursor (no re-drop), so (take 3 (partition-all
2 (map counting (range)))) realizes exactly 6 — matching minimal realization in
both interpret and compile modes. Keeps transducer + [n coll] + [n step coll]
arities. letfn-bound recursion sidesteps the compile-mode multi-arity closure bug
(jolt-zxw), like keep-indexed/map-indexed.
Removed from the Janet seed: core-partition-all + its binding + the now-dead
td-partition-all helper (the CLJ version carries its own transducer arity).
Gate: conformance 262x3, lazy-infinite 44/44 (incl the §6.3 partition-all
counter), clojure-test-suite 4004/66, fixpoint, self-host, specs+unit green.
* core: migrate repeatedly to Clojure + fix char-not-callable / take count validation
Resolves the deferred repeatedly port (jolt-8qx). The blockers were two jolt
leniencies vs Clojure, now fixed (and correct beyond repeatedly):
- A char (a :jolt/type-tagged struct) fell into the struct-as-map branch of both
jolt-call (compile path) and the interpreter's apply dispatch, so (\a) returned
nil instead of throwing. Now only an UNtagged struct (a map literal) — or a
record — is callable as a key lookup; tagged structs fall through to "Cannot
call … as a function". Symbols are still handled (keyword-style get).
- core-take didn't validate its count, letting Janet's >= silently compare an int
to a char/string. It now rejects a non-number n like Clojure.
With those, the canonical CLJ repeatedly matches: (first (repeatedly non-fn)) and
(repeatedly non-number f) throw. Moved repeatedly to core/40-lazy.clj; removed
core-repeatedly + its binding from the seed.
These correctness fixes help broadly: repeatedly.cljc goes clean (19/10 -> 29/0),
and the suite rises 4004 -> 4034 pass / 66 -> 67 clean. Baseline raised.
Gate: conformance 262x3, lazy-infinite 44/44, clojure-test-suite 4034/67,
fixpoint, self-host, sci, fallback-zero, specs+unit green.
* test: document the 4004 -> 4034 baseline raise (partition-all/repeatedly + char/take fixes)
* docs: partition-all letfn is for minimal realization, not jolt-zxw
jolt-zxw (multi-arity arity-param mis-capture in a nested lazy-seq under :compile?)
is no longer reproducible: an arity-direct partition-all now compiles correctly in
the overlay. The original failure was an artifact of the CLJ multi-arity version
coexisting with the Janet core-partition-all ([n & rest]) during migration — the
shadowing confused multi-arity dispatch; removing the Janet version resolved it.
The letfn in partition-all stays purely for minimal realization (jolt-yo3).
* compiler: compile set?/disj as plain fns (close the last Stage-1 fallback gap)
Stage 1 jolt-g3h. set? and disj were special-cased in all three "can't compile"
lists (host_iface special-names, compiler.janet uncompilable-heads, evaluator
special-symbol? + handlers) — but they're pure value-production with callable
core vars (core-set?/core-disj), and those vars are byte-for-byte equivalent to
the evaluator handlers. Removed them from all three lists + dropped the now-dead
evaluator handler arms, so they're ordinary clojure.core fns everywhere: the
analyzer compiles (set? x)/(disj s x) as normal var calls instead of punting to
the interpreter.
Verified identical results in default AND JOLT_MUTABLE builds (no representation
sensitivity — sets are phs in both, unlike vector?/list? which collapse).
With this, the self-hosted analyzer's compile-path fallback set equals the frozen
intentional stateful set (Task 2) — it's now a strict superset of the bootstrap
compiler's compilable surface, so the Janet bootstrap is retireable (Stage 2).
fallback-zero: set?/disj moved to must-compile (31 now), set! into must-punt.
Gate: conformance 267x3 (+5 set?/disj cases), lazy-infinite 44/44, suite 4034/67,
fixpoint, self-host, sci, specs+unit green.
141 lines
7.1 KiB
Text
141 lines
7.1 KiB
Text
# clojure-test-suite conformance: runs the external, cross-dialect
|
|
# clojure-test-suite (jank-lang fork) against Jolt and asserts the number of
|
|
# passing per-function test files stays at/above a baseline. The suite is a git
|
|
# submodule at vendor/clojure-test-suite (CI checks it out via submodules:
|
|
# recursive). The test SKIPS cleanly only if the submodule isn't initialized
|
|
# (run `git submodule update --init`).
|
|
#
|
|
# Each suite file is a `clojure.test` namespace (one per clojure.core/string
|
|
# function). A minimal clojure.test + portability shim (test/support/clojure_test.clj)
|
|
# lets Jolt load them; `when-var-exists` auto-skips fns Jolt doesn't implement.
|
|
#
|
|
# Files are run in a one-shot worker subprocess (test/integration/suite-worker.janet)
|
|
# under a wall-clock deadline. A few suite tests build infinite sequences that an
|
|
# uncompilable/eager path can't truncate and so HANG rather than fail; the
|
|
# deadline contains them — a timed-out file contributes nothing, no skip-list.
|
|
|
|
(def suite-dir "vendor/clojure-test-suite/test/clojure")
|
|
|
|
# Baseline: assertions Jolt currently passes across the suite. Raise as Jolt
|
|
# improves so a regression (previously-passing assertion breaking) is caught.
|
|
# Lowered 3915 -> 3913 when futures landed: `realized?`/realized_qmark.cljc has a
|
|
# `(when-var-exists future ...)` block that was skipped while `future` was
|
|
# unresolved. With futures implemented the block now runs, but it depends on JVM
|
|
# `Thread/sleep` (jolt has no JVM interop) and on `future-cancel` interrupting a
|
|
# running thread (Janet OS threads can't be interrupted), so `(deref (future
|
|
# (sleep 1)))` re-raises the unresolved-`Thread/sleep` error — a documented
|
|
# platform gap, not a regression in any previously-working behavior.
|
|
# Raised 3913 -> 3916 with the staged-bootstrap kernel tier (ns-restore-on-throw
|
|
# + faithful subvec coercion), then 3916 -> 3919 moving juxt/every-pred/some-fn to
|
|
# Clojure (the canonical defs are more correct than the prior Janet ones). Raised
|
|
# 3919 -> 3926 preserving nil map values (jolt-c7h): a nil value is a present key,
|
|
# which several suite tests assert. Runs read 3927 consistently, occasionally 3926
|
|
# when a timeout-prone test (of the 9 that can time out) doesn't finish; floor at
|
|
# the consistent-minus-one 3926.
|
|
# Raised 3971 -> 3981 with Option A full laziness (jolt-fng): transformers return
|
|
# lazy seqs, lazy interleave stops timing out, and the lazy-seq nil-element +
|
|
# non-seqable-input fixes (case/seq/reverse/empty? over nil-first lazy seqs;
|
|
# lazy-from throws on non-seqable like Clojure) recovered + extended the suite.
|
|
# clean files 45 -> 66 (Option A makes seq?/vector? results match Clojure across
|
|
# many cross-dialect files). Stable across runs.
|
|
# Raised 3981 -> 4004 migrating 7 lazy seq fns to the Clojure overlay (40-lazy
|
|
# tier): the canonical CLJ versions add coverage (e.g. distinct value-equality).
|
|
# Raised 4004 -> 4034 / clean 66 -> 67 porting partition-all + repeatedly to the
|
|
# overlay, which required fixing two leniencies (a char is not callable; take
|
|
# validates its count) — correct beyond those fns, so the suite rose broadly.
|
|
(def baseline-pass 4034)
|
|
# A file is "clean" when it ran with zero failures AND zero errors.
|
|
(def baseline-clean-files 67)
|
|
# Per-file wall-clock budget (seconds). Normal files finish in well under 1s, so
|
|
# this normally only fires on genuinely-infinite-sequence hangs. It's an env var
|
|
# (JOLT_SUITE_TIMEOUT) so CI — whose runners are slower than a dev machine — can
|
|
# give slow-but-finite files generous headroom without timing them out (which
|
|
# would drop total-pass below the baseline and flake CI red). Default 6 locally.
|
|
(def per-file-timeout
|
|
(let [e (os/getenv "JOLT_SUITE_TIMEOUT")]
|
|
(or (and e (scan-number e)) 6)))
|
|
|
|
(defn- walk [dir acc]
|
|
(each e (os/dir dir)
|
|
(def p (string dir "/" e))
|
|
(case ((os/stat p) :mode)
|
|
:directory (walk p acc)
|
|
:file (when (and (string/has-suffix? ".cljc" p)
|
|
(not (string/has-suffix? "portability.cljc" p)))
|
|
(array/push acc p))))
|
|
acc)
|
|
|
|
# Run one file in a worker subprocess; return its "pass fail error" stdout, or
|
|
# nil if it exceeded the deadline (hang) or crashed.
|
|
(defn- run-file [path]
|
|
(def proc (os/spawn ["janet" "test/integration/suite-worker.janet" path] :p {:out :pipe}))
|
|
(def out (proc :out))
|
|
(var data nil)
|
|
(def ok
|
|
(try
|
|
(ev/with-deadline per-file-timeout
|
|
(set data (ev/read out 0x10000)) # workers print a single short line
|
|
(os/proc-wait proc)
|
|
true)
|
|
([err] false)))
|
|
(when (not ok)
|
|
(protect (os/proc-kill proc true))
|
|
(protect (ev/with-deadline 2 (os/proc-wait proc))))
|
|
(protect (:close out))
|
|
(if (and ok data) (string data) nil))
|
|
|
|
(defn- parse-counts [s]
|
|
# Find the "@@COUNTS p f e" sentinel line (a test body may have printed other
|
|
# lines to stdout, e.g. with-out-str tests).
|
|
(var result nil)
|
|
(each line (string/split "\n" s)
|
|
(when (string/has-prefix? "@@COUNTS " line)
|
|
(let [parts (string/split " " (string/trim line))]
|
|
(when (= 4 (length parts))
|
|
(set result [(scan-number (parts 1)) (scan-number (parts 2)) (scan-number (parts 3))])))))
|
|
result)
|
|
|
|
(if (not (os/stat suite-dir))
|
|
(print "clojure-test-suite: vendor/clojure-test-suite not initialized — skipped (run: git submodule update --init)")
|
|
(do
|
|
(def progress? (os/getenv "SUITE_PROGRESS"))
|
|
(def files (sort (walk suite-dir @[])))
|
|
(var total-pass 0)
|
|
(var total-fail 0)
|
|
(var total-error 0)
|
|
(var clean-files 0)
|
|
(var ran-files 0)
|
|
(var timeouts 0)
|
|
(def worst @[])
|
|
|
|
(each path files
|
|
(def rel (string/slice path (+ 1 (length suite-dir))))
|
|
(when progress? (eprintf " %s" rel) (eflush))
|
|
(def out (run-file path))
|
|
(def counts (and out (parse-counts out)))
|
|
(cond
|
|
(nil? out) (do (++ timeouts) (when progress? (eprint " TIMEOUT")))
|
|
(nil? counts) (when progress? (eprint " (no counts)"))
|
|
(let [[pn fn* en] counts]
|
|
(++ ran-files)
|
|
(+= total-pass pn)
|
|
(+= total-fail fn*)
|
|
(+= total-error en)
|
|
(when (and (= 0 fn*) (= 0 en) (> pn 0)) (++ clean-files))
|
|
(when (> (+ fn* en) 0) (array/push worst [(+ fn* en) rel pn fn* en])))))
|
|
|
|
(def total (+ total-pass total-fail total-error))
|
|
(printf "\nclojure-test-suite: %d files ran (%d timed out), %d assertions — %d pass / %d fail / %d error"
|
|
ran-files timeouts total total-pass total-fail total-error)
|
|
(printf "\n clean files (0 fail/error, >0 pass): %d" clean-files)
|
|
(sort-by (fn [x] (- (x 0))) worst)
|
|
(when (> (length worst) 0)
|
|
(print " top files by fail+error:")
|
|
(each w (slice worst 0 (min 15 (length worst)))
|
|
(printf " %-40s pass=%d fail=%d err=%d" (w 1) (w 2) (w 3) (w 4))))
|
|
|
|
(assert (>= total-pass baseline-pass)
|
|
(string/format "regression: total-pass %d < baseline %d" total-pass baseline-pass))
|
|
(assert (>= clean-files baseline-clean-files)
|
|
(string/format "regression: clean-files %d < baseline %d" clean-files baseline-clean-files))
|
|
(printf "\nclojure-test-suite: OK (>= %d pass, >= %d clean files)\n" baseline-pass baseline-clean-files)))
|