diff --git a/host/chez/emit.janet b/host/chez/emit.janet index d51e89e..fa1c9ce 100644 --- a/host/chez/emit.janet +++ b/host/chez/emit.janet @@ -382,6 +382,27 @@ (string "(jolt-invoke " (emit fnode) " " (string/join args " ") ")") (string "(" (emit fnode) " " (string/join args " ") ")"))) +# Native-op Scheme procedures that return a genuine Scheme boolean (#t/#f), so an +# :if test built from them needs no jolt-truthy? wrapper (jolt-nkcb). Comparisons +# and `not` are #t/#f; the numeric/collection predicates bottom out in fx=?/>/etc. +(def- bool-returning-ops + {"<" true "<=" true ">" true ">=" true "jolt=" true "jolt-not" true + "jolt-even?" true "jolt-odd?" true "jolt-pos?" true "jolt-neg?" true + "jolt-zero?" true "jolt-empty?" true "jolt-contains?" true}) + +# Does this IR node emit to an expression that yields a Scheme boolean? Used to +# drop the redundant jolt-truthy? on an :if test — sound because jolt-truthy? of +# #t/#f is the identity. Conservative: only a boolean const or an :invoke that +# lowers to a bool-returning native op (any other shape keeps the wrapper). +(defn- returns-scheme-bool? [node] + (def node (nn node)) + (cond + (and (= :const (get node :op)) (boolean? (get node :val))) true + (= :invoke (get node :op)) + (let [nop (native-op (nn (get node :fn)) (length (vv (get node :args))))] + (truthy? (and nop (get bool-returning-ops nop)))) + false)) + (set emit (fn emit [node] (def node (nn node)) (case (get node :op) @@ -400,8 +421,10 @@ (string "(var-deref " (chez-str-lit (get node :ns)) " " (chez-str-lit (get node :name)) ")"))) :host (errorf "emit: unsupported host ref `%s` (no host interop on Chez yet)" (get node :name)) - :if (string "(if (jolt-truthy? " (emit (get node :test)) ") " - (emit (get node :then)) " " (emit (get node :else)) ")") + :if (let [test (get node :test) + t (if (returns-scheme-bool? test) (emit test) + (string "(jolt-truthy? " (emit test) ")"))] + (string "(if " t " " (emit (get node :then)) " " (emit (get node :else)) ")")) :do (string "(begin " (string/join (map emit (vv (get node :statements))) " ") (if (empty? (vv (get node :statements))) "" " ") diff --git a/spike/chez/RESULTS.md b/spike/chez/RESULTS.md index 5aece92..4ab415d 100644 --- a/spike/chez/RESULTS.md +++ b/spike/chez/RESULTS.md @@ -84,6 +84,28 @@ is clearly worse: ~2.5x Janet's fixed footprint. Trades RAM for speed. - **Size: fine** (~1.3-1.8x base, ~2-3x full; single-digit MB). - **Memory: the cost** (~2.5x fixed baseline). +## Phase 1 — real-pipeline measurement (2026-06-18) + +The numbers above are hand-translated Scheme (the substrate ceiling). Phase 1 +(jolt-cf1q.2) ran the SAME benches end to end through the real pipeline (Clojure +source -> Janet-hosted analyzer -> IR -> Scheme emitter -> Chez compile), timed +in-process (`test/chez/bench-pipeline.janet`, Chez startup excluded): + +| bench | real pipeline | ceiling (this run) | gap = Phase 4 lever | +|---------------------|---------------|---------------------------|---------------------| +| fib 30 (flonum) | 14.4 ms | 7.1 ms hand-flonum | 2.0x dispatch/var | +| fib 30 (vs fixnum) | 14.4 ms | 5.2 ms fixnum | all-double model | +| mandelbrot 200 | 87.3 ms | 98.1 ms generic-flonum | AT/below ceiling | +| mandelbrot 200 typed| 87.3 ms | 13.4 ms typed fl*/fx* | typed emit (Phase 4)| + +Findings: (1) **compile-only is total** for the compute subset — every form +emits, no interpreter fallback (Chez has none). (2) Mandelbrot through the real +pipeline runs AT the generic-flonum ceiling (87 vs 98 ms) — the substrate ceiling +is reached end-to-end. (3) The fib residual is jolt's all-double number model +(the spike's 5.2 ms fib is fixnum); closing it to the 13.4 ms / fixnum ceiling is +the typed fl*/fx* emission Phase 4 owns. Eliding the redundant `jolt-truthy?` +wrapper on boolean-test `if`s (jolt-nkcb) cut fib 24.0 -> 14.4 ms. + ## NOT yet measured (needs the RT port — the real project, not a spike) - collections / binary-trees: these hit persistent collections + GC. Chez's GC diff --git a/test/chez/README.md b/test/chez/README.md index 744a8d4..4f226b6 100644 --- a/test/chez/README.md +++ b/test/chez/README.md @@ -196,6 +196,16 @@ class names, eval-order, with-open — all deferred Phase-2 / dynamic-var gaps). lenient nil-fill was non-Clojure and inconsistent with the seed's own `assoc`. The 4 `assoc! odd args` spec rows became 3 `:throws` + 1 even-args positive; corpus.edn regenerated. The Chez host already threw, so this only realigns the corpus contract. +- Phase 1 close-out (jolt-nkcb): truthy? elision + end-to-end compute bench. `emit.janet` + now drops the redundant `jolt-truthy?` wrapper on an `:if` test that provably yields a + Scheme boolean (a native comparison/`not`, or a boolean const) — sound because + `jolt-truthy?` of `#t`/`#f` is identity; the hot fib/mandelbrot tests are all + comparisons, so this is a direct ceiling lever (fib 30 end-to-end 24.0 -> 14.4 ms). + `bench-pipeline.janet` (JOLT_CHEZ_BENCH=1, opt-in) times fib(30) + mandelbrot(200) + through the real pipeline vs the spike ceiling: mandelbrot 87 ms runs AT the + generic-flonum ceiling (98 ms); fib is 2x its hand-flonum ceiling, the residual being + jolt's all-double number model (typed fl*/fx* emission = Phase 4). Compile-only is + total for the compute subset (every form emits; Chez has no interpreter fallback). The remaining buckets are the punch-list the next increments chase: ~361 emit-fail (genuine host interop — qualified Java/Janet refs, runtime `defmacro`/`eval`, out of diff --git a/test/chez/bench-pipeline.janet b/test/chez/bench-pipeline.janet new file mode 100644 index 0000000..9f5dac2 --- /dev/null +++ b/test/chez/bench-pipeline.janet @@ -0,0 +1,102 @@ +# Phase 1 (jolt-cf1q.2) close-out bench — the compute benches run END TO END +# through the REAL pipeline (Clojure source -> Janet-hosted analyzer -> IR -> +# Scheme emitter -> Chez compile -> run), timed in-process (Chez startup +# excluded), and reported against the spike ceiling (spike/chez/RESULTS.md). +# +# This is the Phase 1 gate evidence: (1) compile-only is TOTAL for the compute +# subset — every form emits, no interpreter fallback (Chez has none); (2) the +# emitted code runs at ~the substrate ceiling, with the residual gap being +# exactly the typed fl*/fx* emission that Phase 4 owns. +# +# JOLT_CHEZ_BENCH=1 janet test/chez/bench-pipeline.janet +# Opt-in (like core-bench); skipped in the normal gate. +(import ../../src/jolt/backend :as backend) +(import ../../src/jolt/reader :as r) +(import ../../host/chez/driver :as d) +(import ../../host/chez/emit :as emit) + +(unless (os/getenv "JOLT_CHEZ_BENCH") + (print "skip: set JOLT_CHEZ_BENCH=1 to run the Chez pipeline bench") + (os/exit 0)) +(unless (d/chez-available?) + (print "skip: chez not on PATH") + (os/exit 0)) + +(def ctx (d/make-ctx)) + +# Emit a top-level program (one or more defns) through the real pipeline. Returns +# the concatenated Scheme for every form — each form must emit (compile-only is +# total) or this throws, which is itself the totality check. +(defn emit-program [src] + (def forms (map first (r/parse-all-positioned src))) + (string/join (map (fn [f] (emit/emit (backend/analyze-form ctx f))) forms) "\n")) + +(def fib-src "(defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))") + +# mandelbrot kernel: loop/recur + let + or-expansion + cross-var call, all flonum. +(def mandel-src `` +(defn count-point [cr ci cap] + (loop [i 0 zr 0.0 zi 0.0] + (if (or (>= i cap) (> (+ (* zr zr) (* zi zi)) 4.0)) + i + (recur (inc i) + (+ (- (* zr zr) (* zi zi)) cr) + (+ (* 2.0 (* zr zi)) ci))))) +(defn run [n] + (let [cap 200 nd (* 1.0 n)] + (loop [y 0 acc 0] + (if (>= y n) acc + (let [ci (- (* 2.0 (/ (* 1.0 y) nd)) 1.0) + row (loop [x 0 a 0] + (if (>= x n) a + (let [cr (- (* 3.5 (/ (* 1.0 x) nd)) 2.5)] + (recur (inc x) (+ a (count-point cr ci cap))))))] + (recur (inc y) (+ acc row))))))) +``) + +(def fib-scm (emit-program fib-src)) +(def mandel-scm (emit-program mandel-src)) +(print "compile-only total: fib + mandelbrot emitted with no fallback") + +# One Chez program: load the RT, the emitted defns, a hand-written FLONUM fib +# reference (jolt's realistic ceiling given the all-double model), then time each +# end-to-end value (warm up first; exclude Chez startup via a monotonic clock). +(def prog + (string + "(import (chezscheme))\n(load \"host/chez/rt.ss\")\n" + fib-scm "\n" mandel-scm "\n" + "(define fib (var-deref \"user\" \"fib\"))\n" + "(define mrun (var-deref \"user\" \"run\"))\n" + # hand flonum fib — the substrate ceiling for jolt's number model + "(define (ffib n) (if (fl< n 2.0) n (fl+ (ffib (fl- n 1.0)) (ffib (fl- n 2.0)))))\n" + "(define (now-ns) (let ((t (current-time 'time-monotonic))) (+ (* (time-second t) 1000000000) (time-nanosecond t))))\n" + "(define (timed thunk) (let* ((t0 (now-ns)) (r (thunk)) (ms (/ (- (now-ns) t0) 1000000.0))) (cons r ms)))\n" + "(fib 24)(ffib 24.0)(mrun 30)\n" # warm up + "(let ((a (timed (lambda () (fib 30))))\n" + " (b (timed (lambda () (ffib 30.0))))\n" + " (c (timed (lambda () (mrun 200)))))\n" + " (printf \"~a ~a ~a ~a ~a ~a\\n\"\n" + " (jolt-pr-str (car a)) (exact->inexact (cdr a))\n" + " (jolt-pr-str (car b)) (exact->inexact (cdr b))\n" + " (jolt-pr-str (car c)) (exact->inexact (cdr c))))")) + +(def path (string "/tmp/chez-bench-pipeline-" (os/getpid) ".ss")) +(spit path prog) +(def proc (os/spawn ["chez" "--script" path] :p {:out :pipe :err :pipe})) +(def out (string/trim (string (ev/read (proc :out) 0x100000)))) +(def err (string/trim (string (or (ev/read (proc :err) 0x100000) "")))) +(def code (os/proc-wait proc)) +(unless (= code 0) (printf "BENCH FAILED (code %d): %s" code err) (os/exit 1)) + +(def p (string/split " " out)) +(defn num [i] (scan-number (get p i "0"))) +(printf "\nReal-pipeline compute benches (Chez startup excluded):\n") +(printf " fib 30 (jolt, flonum) = %s in %6.2f ms" (get p 0) (num 1)) +(printf " fib 30 (hand flonum ceiling) = %6.2f ms <- jolt's number-model ceiling" (num 3)) +(printf " fib 30 (spike fixnum ceiling)= 5.20 ms <- Phase 4 fl/fx typed-emit target") +(printf " mandelbrot 200 (jolt) = %s in %6.2f ms" (get p 4) (num 5)) +(printf " mandelbrot 200 (spike generic)= 98.10 ms <- generic-flonum ceiling") +(printf " mandelbrot 200 (spike typed) = 13.40 ms <- Phase 4 fl/fx typed-emit target") +(def fib-overhead (if (> (num 3) 0) (/ (num 1) (num 3)) 0)) +(printf "\n jolt fib is %.2fx the hand-flonum ceiling (residual = truthy/dispatch; typed-emit closes the fixnum gap)." fib-overhead) +(os/exit 0) diff --git a/test/chez/emit-test.janet b/test/chez/emit-test.janet index bb9fe37..6106a59 100644 --- a/test/chez/emit-test.janet +++ b/test/chez/emit-test.janet @@ -76,6 +76,24 @@ (ok "mandelbrot run(40) parity" (and (= code 0) (= out (oracle src))) (string "chez=" out " janet=" (oracle src) " | " err))) +# 3a2) truthy? elision (jolt-nkcb): an :if test that provably yields a Scheme +# boolean (native comparison/not, or a boolean const) needs no jolt-truthy? +# wrapper — (if (jolt-truthy? (< a b)) …) === (if (< a b) …). Sound because +# jolt-truthy? of #t/#f is identity. The hot fib/mandelbrot tests are all +# comparisons, so this is a direct ceiling lever. Inspect the emitted Scheme. +(defn- emit-scm [src] (emit/emit (backend/analyze-form ctx (in (r/parse-next src) 0)))) +(each [label src wrapped?] [["if < elides wrapper" "(fn [n] (if (< n 2) 1 2))" false] + ["if > elides wrapper" "(fn [n] (if (> n 2) 1 2))" false] + ["if = elides wrapper" "(fn [n] (if (= n 2) 1 2))" false] + ["if <= elides wrapper" "(fn [n] (if (<= n 2) 1 2))" false] + ["if not elides wrapper" "(fn [x] (if (not x) 1 2))" false] + ["if true const elides" "(fn [] (if true 1 2))" false] + ["if local keeps wrapper" "(fn [x] (if x 1 2))" true] + ["if + keeps wrapper" "(fn [n] (if (+ n 1) 1 2))" true]] + (let [scm (emit-scm src) has (truthy? (string/find "jolt-truthy?" scm))] + (ok (string "truthy elision: " label) (= has wrapped?) + (string "wrapped?=" has " want " wrapped? " | " scm)))) + # 3b) regressions found via the corpus probe: # - loop binds SEQUENTIALLY (Scheme named-let is parallel); b must see a. # - #(...) shorthand gensyms params with a trailing `#` (invalid in Scheme). @@ -642,8 +660,10 @@ # 4) perf signal: emitted fib(30) in-Scheme timing (excludes Chez startup), to -# track against the spike ceiling (hand-Scheme fib ~5ms). Informational — the -# jolt-truthy? wrapper (~3x) and flonum modeling are known Phase-4 levers. +# track against the spike ceiling (hand-Scheme fixnum fib ~5ms). Informational +# — the truthy? wrapper is now elided (jolt-nkcb); the residual gap is jolt's +# all-flonum number model vs the spike's fixnum fib (typed fl*/fx* = Phase 4). +# The full timed bench (fib + mandelbrot vs ceilings) is bench-pipeline.janet. (let [fib-ir (backend/analyze-form ctx (in (r/parse-next "(defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))") 0)) fib-scm (emit/emit fib-ir) timed (string "(import (chezscheme))\n(load \"host/chez/rt.ss\")\n"