eacfa04e5b
* Make the benchmark harness build optimized binaries on Chez bench/run.sh was Janet-era: it invoked a 'jolt' binary and set JOLT_DIRECT_LINK/JOLT_WHOLE_PROGRAM, none of which exist on Chez, where 'joltc run -m' runs fully unoptimized (direct-link and inline default off). So the suite was measuring jolt's unoptimized path. run.sh now compiles each benchmark to an optimized AOT binary (joltc build --direct-link --opt) and times it against JVM Clojure on the same portable source, auto-detecting the Chez kernel dev files like build-smoke.sh. Adds bench/deps.edn so joltc resolves the namespaces, NO_JVM to skip the reference. mandelbrot.clj dropped its jolt.png require so the JVM reference can run it; the picture demo moved to mandelbrot_png.clj (jolt-only). README scorecard refreshed with current Chez numbers and the two-regime read (compute ~8-10x substrate floor; dispatch/alloc ~120-330x architectural gaps the passes don't touch). Stale 'jolt -m' header lines point at bench/run.sh. * Emit direct self-calls for named-fn self-recursion A self-recursive call to a named fn compiled to (jolt-invoke fib ...) instead of a direct (fib ...): emit-invoke handled a :local callee only when it was NOT a known proc, so a :local that IS in *known-procs* (the letrec-bound self-name) fell through to the :else jolt-invoke branch. Now a :local known proc emits a direct Scheme call — no jolt-invoke, no per-call arg-list consing; case-lambda handles arity. fib 30: 63.3ms -> 4.7ms (faster than JVM Clojure's 7.1ms; was 9x slower). The win is on every self-recursive non-loop fn, including the compiler's own. No semantic change — selfhost holds, make test green, shakesmoke/buildsmoke byte-identical. Re-mint (backend is seed). Corpus rows pin self-recursion across fixed/multi/ variadic arities. * Intern no-ns keywords without per-call allocation (keyword #f name) built a fresh combined-key string (string-append) on every call just to do the intern-table lookup — ~80 bytes of garbage per (:kw x), map literal, keyword arg, etc. A no-ns keyword now interns in a table keyed by the name string directly, so a lookup of an already-interned keyword is one hashtable-ref with no allocation. The ns table keeps the combined key; both share the keyword-t khash (equal-hash of the combined key) so hash values are unchanged. Small time win on its own (the field-read dispatch dominates hot record code — see jolt-unx4) but removes per-call keyword allocation everywhere. Runtime .ss, no re-mint; identity/=/hash unchanged, make test green. * Fast record field reads: single eq? scan, skip the get-arm walk (:field rec) / (get rec :field) lowers to (jolt-get rec kw), which walked the get-arm list to reach the jrec arm, then did jrec-has? + jrec-lookup — TWO linear scans, each comparing keys through the generic jolt=2 equality dispatcher. Field keys are interned keywords, so: - jrec-key=? compares a keyword query by eq? (jolt=2 only for non-keyword keys), - jrec-ref does ONE scan (vs has?+lookup) and runs a deftype's ILookup valAt only when the field is genuinely absent (present-nil still returns nil, not default), - jolt-get-dispatch checks jrec? first, skipping the get-arm walk for the hottest get target. jrec-lookup/jrec-has? (used by =, contains?, etc.) get the fast compare too. binary-trees 135x->18.9x, dispatch 121x->26.4x, mono-dispatch 327x->108x vs JVM. Runtime .ss (collections.ss + records.ss), no re-mint; make test + shakesmoke + buildsmoke green, record get/assoc/keys/=/count semantics unchanged. --------- Co-authored-by: Yogthos <yogthos@gmail.com> |
||
|---|---|---|
| .github/workflows | ||
| bench | ||
| bin | ||
| docs | ||
| host/chez | ||
| img | ||
| jolt-core | ||
| stdlib | ||
| test | ||
| tools | ||
| vendor | ||
| .DS_Store | ||
| .gitignore | ||
| .gitmodules | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
Jolt
A Clojure implementation on Chez Scheme.
Jolt reads Clojure source, analyzes it to a host-neutral IR, emits Scheme, and
runs it on Chez. The compiler is self-hosted: it is written in Clojure
(jolt-core/) and compiles itself. It ships a Clojure-compatible standard library.
Requirements
Only Chez Scheme (the gate invokes it as
chez). The conformance gate additionally uses Clojure on the JVM as an oracle,
but running jolt does not.
Build
There is no build step. The bootstrap seed (host/chez/seed/{prelude,image}.ss)
is checked in, so a fresh clone runs immediately:
git clone --recurse-submodules https://github.com/jolt-lang/jolt.git
cd jolt
bin/joltc -e '(+ 1 2)' # => 3
After changing a compiler source — the reader (host/chez/reader.ss), the
analyzer/IR/backend (jolt-core/jolt/*.clj), or the clojure.core overlay
(jolt-core/clojure/core/*.clj) — re-mint the seed:
make remint # iterates host/chez/bootstrap.ss to a byte-fixpoint
Run
bin/joltc -e EXPR # evaluate a Clojure expression and print the result
$ bin/joltc -e '(->> (range 10) (filter even?) (map (fn [x] (* x x))) (reduce +))'
120
$ bin/joltc -e '(/ 1 2)'
1/2
Compile a binary
bin/joltc build ahead-of-time compiles a project into a single self-contained
executable — the runtime, clojure.core, the standard library, the app, and its
deps.edn dependencies are linked in, so the result needs no Chez install, no
JVM, and no source on disk to run.
bin/joltc build -m myapp.core -o myapp # compile myapp.core's -main into ./myapp
./myapp arg1 arg2 # runs anywhere; args reach -main
Modes trade dynamism for speed: the default (release) build uses the proven code
generator; --opt also runs the inference + inlining + scalar-replacement passes
over the closed-world program; --dev is unoptimized.
Two opt-in closed-world flags cut dispatch cost and binary size:
bin/joltc build -m myapp.core --direct-link # app->app calls bind directly (no var lookup)
bin/joltc build -m myapp.core --tree-shake # ship only code reachable from -main
--tree-shake walks the call graph across your app, its libraries, and
clojure.core, drops everything unreachable from -main (and the compiler itself
when the app never evals), and typically removes 1–2 MB. It stays sound by bailing
out — keeping everything, and reporting which library is responsible — when reachable
code resolves vars by name at runtime (eval/resolve/ns-resolve/…). See
docs/tools-deps.md and docs/rfc/0007.
This needs Chez's kernel development files (libkernel.a, scheme.h) and a C
compiler. They come with a from-source Chez install; a distro chezscheme
package ships only the runtime, so build won't link a binary there.
RFC 0007 (docs/rfc/) covers the design and the three-mode model.
Architecture
A small Chez runtime (host/chez/*.ss: value model, persistent collections, seqs,
vars/namespaces, host interop) hosts a portable Clojure overlay split across two
source roots by when they load:
jolt-core/is baked into the seed — the compiler (jolt-core/jolt/: reader/analyzer/IR/backend, plusjolt.main/jolt.deps) andclojure.corein dependency-ordered tiers (jolt-core/clojure/core/NN-*.clj). Changing anything here means re-minting the seed.stdlib/loads lazily at runtime off the source roots — the rest of the standard library (clojure.string/set/walk/edn/pprint/…) plus thejolt.ffihost library. Editing these needs no re-mint.
bin/joltc loads the checked-in seed and the spine, then compiles and evaluates on
Chez (read → analyze → IR → emit → eval). host/chez/bootstrap.ss rebuilds that
seed from source on pure Chez; the build is a self-hosting fixpoint (a rebuild
reproduces the checked-in seed byte-for-byte).
Differences from Clojure
Jolt targets Clojure semantics but runs on Chez, not the JVM. Most portable
Clojure runs unchanged — persistent collections (32-way-trie vectors, HAMT
maps/sets), the numeric tower (exact integers, bignums, ratios, doubles), lazy
and infinite sequences, transducers, destructuring, multimethods with
hierarchies, protocols/records (deftype/defrecord/reify/extend-protocol),
metadata, namespaces, atoms, future/promise/agent/pmap,
clojure.core.async, runtime eval/load-string/defmacro, and the full
reader (#(), #_, #?, tagged literals, #"…") all behave as on the JVM.
= is category-aware ((= 3 3.0) ⇒ false) and == is value-equality, as in
Clojure. The genuine divergences:
- No JVM, no Java interop. No reflection, no
gen-class/proxy. Interop syntax (Class.,Class/static,.method) resolves only against a shimmed subset of thejava.*standard library; a class token is a name, not a loaded class. See docs/host-interop.md. To call C libraries directly, use thejolt.ffiforeign-function interface (how the db and http-client libraries bind SQLite/libpq and sockets/OpenSSL/zlib). - No
BigDecimal.decimal?is always false and there is noMliteral; the rest of the numeric tower matches the JVM. - No STM. No
ref/dosync/alter/commute— coordinated shared state uses atoms (per-atom mutex, JVM-style CAS). The concurrency primitives above are otherwise present and run on a shared heap. - Regex engine. Patterns compile through
irregex (vendored), not
java.util.regex; common patterns work, Java-specific features can differ. - Coverage.
clojure.coreis implemented function by function against the JVM-sourced conformance corpus — broad but not total; a namespace can load with most functions working and a few not yet implemented.
Test
make test # the full gate
make corpus # conformance corpus vs the JVM-sourced spec
make unit # host-specific unit cases
make selfhost # bootstrap fixpoint (rebuild == checked-in seed)
make smoke # bin/joltc CLI smoke
make sci # load borkdude/sci's source through joltc (compat stress)
make ffi # HTTP-server GC-safety + http-client temp paths
make transient # transient mutation + linear-time builds
make certify # JVM oracle (skips if clojure is absent)
The conformance corpus (test/chez/corpus.edn) is a host-neutral language spec
whose expected values are sourced from reference JVM Clojure. See
test/conformance/SPEC.md.