Perf round 1: self-call, keyword interning, fast record field reads (#221)

* Make the benchmark harness build optimized binaries on Chez

bench/run.sh was Janet-era: it invoked a 'jolt' binary and set
JOLT_DIRECT_LINK/JOLT_WHOLE_PROGRAM, none of which exist on Chez, where
'joltc run -m' runs fully unoptimized (direct-link and inline default off). So
the suite was measuring jolt's unoptimized path.

run.sh now compiles each benchmark to an optimized AOT binary (joltc build
--direct-link --opt) and times it against JVM Clojure on the same portable
source, auto-detecting the Chez kernel dev files like build-smoke.sh. Adds
bench/deps.edn so joltc resolves the namespaces, NO_JVM to skip the reference.

mandelbrot.clj dropped its jolt.png require so the JVM reference can run it; the
picture demo moved to mandelbrot_png.clj (jolt-only). README scorecard refreshed
with current Chez numbers and the two-regime read (compute ~8-10x substrate floor;
dispatch/alloc ~120-330x architectural gaps the passes don't touch). Stale
'jolt -m' header lines point at bench/run.sh.

* Emit direct self-calls for named-fn self-recursion

A self-recursive call to a named fn compiled to (jolt-invoke fib ...) instead of
a direct (fib ...): emit-invoke handled a :local callee only when it was NOT a
known proc, so a :local that IS in *known-procs* (the letrec-bound self-name) fell
through to the :else jolt-invoke branch. Now a :local known proc emits a direct
Scheme call — no jolt-invoke, no per-call arg-list consing; case-lambda handles
arity.

fib 30: 63.3ms -> 4.7ms (faster than JVM Clojure's 7.1ms; was 9x slower). The win
is on every self-recursive non-loop fn, including the compiler's own. No semantic
change — selfhost holds, make test green, shakesmoke/buildsmoke byte-identical.

Re-mint (backend is seed). Corpus rows pin self-recursion across fixed/multi/
variadic arities.

* Intern no-ns keywords without per-call allocation

(keyword #f name) built a fresh combined-key string (string-append) on every
call just to do the intern-table lookup — ~80 bytes of garbage per (:kw x), map
literal, keyword arg, etc. A no-ns keyword now interns in a table keyed by the
name string directly, so a lookup of an already-interned keyword is one
hashtable-ref with no allocation. The ns table keeps the combined key; both share
the keyword-t khash (equal-hash of the combined key) so hash values are unchanged.

Small time win on its own (the field-read dispatch dominates hot record code —
see jolt-unx4) but removes per-call keyword allocation everywhere. Runtime .ss,
no re-mint; identity/=/hash unchanged, make test green.

* Fast record field reads: single eq? scan, skip the get-arm walk

(:field rec) / (get rec :field) lowers to (jolt-get rec kw), which walked the
get-arm list to reach the jrec arm, then did jrec-has? + jrec-lookup — TWO linear
scans, each comparing keys through the generic jolt=2 equality dispatcher. Field
keys are interned keywords, so:

- jrec-key=? compares a keyword query by eq? (jolt=2 only for non-keyword keys),
- jrec-ref does ONE scan (vs has?+lookup) and runs a deftype's ILookup valAt only
  when the field is genuinely absent (present-nil still returns nil, not default),
- jolt-get-dispatch checks jrec? first, skipping the get-arm walk for the hottest
  get target. jrec-lookup/jrec-has? (used by =, contains?, etc.) get the fast
  compare too.

binary-trees 135x->18.9x, dispatch 121x->26.4x, mono-dispatch 327x->108x vs JVM.
Runtime .ss (collections.ss + records.ss), no re-mint; make test + shakesmoke +
buildsmoke green, record get/assoc/keys/=/count semantics unchanged.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>

host/chez/collections.ss

@@ -239,11 +239,16 @@
         ((string? coll) (let ((i (->idx k)))
                           (if (and (fixnum? i) (fx>=? i 0) (fx<? i (string-length coll))) (string-ref coll i) d)))
         (else d)))
+;; jrec? / jrec-ref live in records.ss (loaded later); these are forward references
+;; resolved at call time. A record field read is the hottest get, so check it first
+;; and skip the get-arm walk.
 (define (jolt-get-dispatch coll k d)
+  (if (jrec? coll)
+      (jrec-ref coll k d)
       (let loop ((as jolt-get-arms))
         (cond ((null? as) (jolt-get-base coll k d))
               (((caar as) coll) ((cdar as) coll k d))
-          (else (loop (cdr as))))))
+              (else (loop (cdr as)))))))
 (define jolt-get
   (case-lambda
     ((coll k) (jolt-get-dispatch coll k jolt-nil))

host/chez/records.ss

@@ -16,16 +16,32 @@
 (define-record-type jrec (fields tag pairs) (nongenerative chez-jrec-v1))
 (define jolt-deftype-kw (keyword "jolt" "deftype"))
 
+;; Field keys are interned keywords, so a keyword query compares by eq? (the hot
+;; (:field rec) / (get rec :field) case); any other key type falls back to jolt=2.
+(define (jrec-key=? field-key k)
+  (if (keyword? k) (eq? field-key k) (jolt=2 field-key k)))
 (define (jrec-lookup r k d)
-  (if (jolt=2 k jolt-deftype-kw)
+  (if (eq? k jolt-deftype-kw)
       (jrec-tag r)
       (let loop ((ps (jrec-pairs r)))
         (cond ((null? ps) d)
-              ((jolt=2 (caar ps) k) (cdar ps))
+              ((jrec-key=? (caar ps) k) (cdar ps))
               (else (loop (cdr ps)))))))
 (define (jrec-has? r k)
   (let loop ((ps (jrec-pairs r)))
-    (cond ((null? ps) #f) ((jolt=2 (caar ps) k) #t) (else (loop (cdr ps))))))
+    (cond ((null? ps) #f) ((jrec-key=? (caar ps) k) #t) (else (loop (cdr ps))))))
+;; The get path: one scan (vs jrec-has? + jrec-lookup = two); a deftype's ILookup
+;; valAt runs only when the field is genuinely missing.
+(define (jrec-ref coll k d)
+  (if (eq? k jolt-deftype-kw)
+      (jrec-tag coll)
+      (let loop ((ps (jrec-pairs coll)))
+        (cond ((null? ps)
+               (cond ((find-method-any-protocol (jrec-tag coll) "valAt")
+                      => (lambda (m) (jolt-invoke m coll k d)))
+                     (else d)))
+              ((jrec-key=? (caar ps) k) (cdar ps))
+              (else (loop (cdr ps)))))))
 ;; mutate a deftype's mutable field in place: the pairs are runtime cons cells,
 ;; so set-cdr! updates the field. (set! field v) inside a method
 ;; lowers to this; returns v, as set! does.
@@ -77,12 +93,10 @@
 ;; compiled to (get inst :field), never recurse); a NON-field key on a deftype that
 ;; implements clojure.lang.ILookup routes to its valAt (core.match's pattern types
 ;; compute ::tag in valAt), else the default.
-(register-get-arm! jrec?
-  (lambda (coll k d)
-    (cond ((jrec-has? coll k) (jrec-lookup coll k d))
-          ((find-method-any-protocol (jrec-tag coll) "valAt")
-           => (lambda (m) (jolt-invoke m coll k d)))
-          (else d))))
+;; jrec is the hottest get target (every record field read); jolt-get-dispatch
+;; (collections.ss) checks jrec? directly and calls jrec-ref, skipping the get-arm
+;; walk. This registration is the equivalent fallback for any other caller.
+(register-get-arm! jrec? jrec-ref)
 ;; A jrec is a defrecord (map of fields) by default, BUT a deftype that
 ;; implements a clojure.lang collection interface carries the op as an inline
 ;; method — prefer that method, else fall back to the field/map behavior. (jrec-cl

host/chez/values.ss

@@ -23,14 +23,25 @@
 ;; --- keywords: interned so identity works; optional namespace ----------------
 (define-record-type keyword-t (fields ns name khash) (nongenerative keyword-v1))
 (define keyword-table (make-hashtable string-hash string=?))
+;; The common no-ns keyword is interned in a table keyed by NAME directly, so a
+;; lookup of an already-interned :kw (the hot case — every (:kw x), map literal,
+;; keyword arg) is one hashtable-ref with NO allocation. The ns table keeps the
+;; combined key. Both share the keyword-t khash (equal-hash of the combined key),
+;; so hash values are unchanged.
+(define keyword-table-bare (make-hashtable string-hash string=?))
 ;; NUL separator can't occur in a keyword ns/name, so the intern key is
 ;; unambiguous (a "/" separator would collide ns="a" name="b/c" with ns="a/b").
 (define (keyword-intern-key ns name) (string-append (or ns "") "\x0;" name))
 (define (keyword ns name)
+  (if ns
       (let ((k (keyword-intern-key ns name)))
         (or (hashtable-ref keyword-table k #f)
             (let ((kw (make-keyword-t ns name (equal-hash k))))
               (hashtable-set! keyword-table k kw)
+              kw)))
+      (or (hashtable-ref keyword-table-bare name #f)
+          (let ((kw (make-keyword-t #f name (equal-hash (keyword-intern-key #f name)))))
+            (hashtable-set! keyword-table-bare name kw)
             kw))))
 (define (keyword? x) (keyword-t? x))
 

jolt-core/jolt/backend_scheme.clj

@@ -520,9 +520,15 @@
                          (if (empty? as) "" (str " " (str/join " " as))) ")")))
       (= :host (:op fnode))
       (throw (ex-info (str "emit: unsupported host call `" (:name fnode) "`") {}))
-      ;; a :local callee that isn't a known procedure -> dynamic IFn dispatch.
-      (and (= :local (:op fnode)) (not (*known-procs* (munge-name (:name fnode)))))
-      (invoke)
+      ;; a :local callee: a known procedure (the letrec-bound self-name of a named
+      ;; fn — i.e. self-recursion) is a real Scheme proc, so call it directly with
+      ;; no jolt-invoke / arg consing; case-lambda handles arity. Any other local
+      ;; holds an arbitrary IFn -> dynamic dispatch.
+      (= :local (:op fnode))
+      (if (*known-procs* (munge-name (:name fnode)))
+        (order-args (fn [as] (str "(" (munge-name (:name fnode))
+                                  (if (seq as) (str " " (str/join " " as)) "") ")")))
+        (invoke))
       ;; closed-world direct call: the callee var is an app fn def already emitted
       ;; with a Scheme binding — apply it directly, no var lookup, no jolt-invoke.
       ;; Only fn-valued defs qualify; a non-fn invokable value (a map/set/keyword

test/chez/corpus.edn

@@ -284,6 +284,10 @@
   {:suite "functions / definition" :label "multi-arity" :expected "[1 5]" :actual "(do (defn f ([x] x) ([x y] (+ x y))) [(f 1) (f 2 3)])"}
   {:suite "functions / definition" :label "variadic" :expected "[1 2 3]" :actual "(do (defn f [& xs] xs) (f 1 2 3))"}
   {:suite "functions / definition" :label "variadic with fixed" :expected "[1 [2 3]]" :actual "(do (defn f [a & xs] [a xs]) (f 1 2 3))"}
+  {:suite "functions / definition" :label "named fn-literal self-recursion (direct self-call)" :expected "3628800" :actual "((fn fact [n] (if (< n 2) 1 (* n (fact (dec n))))) 10)"}
+  {:suite "functions / definition" :label "defn self-recursion" :expected "832040" :actual "(do (defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2))))) (fib 30))"}
+  {:suite "functions / definition" :label "multi-arity self-recursion" :expected "15" :actual "(do (defn sum ([xs] (sum xs 0)) ([xs a] (if (seq xs) (sum (rest xs) (+ a (first xs))) a))) (sum [1 2 3 4 5]))"}
+  {:suite "functions / definition" :label "variadic self-recursion" :expected "10" :actual "(do (defn g [& xs] (if (next xs) (apply g (cons (+ (first xs) (second xs)) (nnext xs))) (first xs))) (g 1 2 3 4))"}
   {:suite "functions / definition" :label "closure captures" :expected "8" :actual "(do (defn adder [n] (fn [x] (+ x n))) ((adder 5) 3))"}
   {:suite "functions / definition" :label "recursion" :expected "120" :actual "(do (defn fact [n] (if (< n 2) 1 (* n (fact (dec n))))) (fact 5))"}
   {:suite "functions / definition" :label "named fn self-ref" :expected "120" :actual "((fn fact [n] (if (< n 2) 1 (* n (fact (dec n))))) 5)"}