Commit graph

14 commits

Author SHA1 Message Date
Dmitri Sotnikov
d3194aae59
Compiler research (#10)
adds self-hosted compiler is functionally:
 
- The default compile path is the portable pipeline using jolt.analyzer (Clojure) → host-neutral IR → backend.janet.
- The analyzer is itself Clojure, compiled by jolt for true self-hosting.
- bootstrap-fixpoint passes (stage1 == stage2 == stage3): rebuilding the compiler on its own output.
- clojure.core is now self-hosted in the overlay.
- Stateful forms (defmacro/ns/deftype/defmulti/require/in-ns) are interpreted by design.
2026-06-09 07:30:25 +08:00
Yogthos
41f78e218f destructuring compliance, deps.edn comments, clojure.java.io shim
Destructuring (drove by trying to load Selmer):
- defmacro params now destructure like fn (parse-params + destructure-bind), so
  [& [a & more :as all]] and {:keys ...} work in macro arglists.
- map-destructuring a sequential value treats it as keyword args (alternating
  k/v, or a trailing map) — the [& {:keys [...]}] form, in fns and macros.
- :keys/:syms accept namespaced symbols (x/y): looks up the namespaced key,
  binds the bare local.

deps.edn: read with Jolt's reader instead of Janet's parse, so EDN ';' line
comments are handled (Janet treats ';' as splice).

clojure.java.io: the shim was at clojure/java_io.clj (ns clojure.java-io, never
the clojure.java.io that code requires) and used bare-qualified Janet calls that
the janet.* bridge no longer resolves. Moved to clojure/java/io.clj and rewritten
on janet.file/os: file/reader/writer/resource/copy/delete-file/make-parents.

Specs in destructuring-spec; destructuring note in the ebnf.
2026-06-06 01:01:04 -04:00
Yogthos
761a2b8f72 support type hints; make metadata coherent
The reader expanded ^X form to (with-meta form X), which evaluated the tag (so
^String errored 'Unable to resolve symbol: String') and, as a param, was no
longer a bare symbol (so the arg bound to nil). Now a keyword/symbol/string hint
on a symbol attaches to the symbol's :meta and the symbol stays bare, so type
hints are transparent in params, lets, and bodies. Map metadata still uses a
runtime with-meta form.

meta now returns a symbol's :meta, and def applies the name's metadata
(^:dynamic, ^:private, ^Type tag, ^{:doc}) to the var, so (meta (var x)) is
consistent. Specs in metadata-spec; grammar note in the ebnf.

README notes regex \p{...} as unsupported (separate from this).
2026-06-06 00:36:10 -04:00
Yogthos
d724aa7069 reader: one-char literals for non-symbol chars (\{ \( \, \%)
read-char read the char name with symbol-char?, so a literal whose char isn't a
symbol char (\{, \(, \), \,, \%, …) came back as an empty name and errored
'Unsupported character'. Now a single non-symbol char after \ is taken as a
one-character literal of that char. Surfaced by funcool/cuerdas (uses \{), which
now loads. Spec cases added.
2026-06-06 00:10:17 -04:00
Yogthos
f3f2c63ee7 bake the .clj stdlib into the binary
The Clojure stdlib namespaces (clojure.string/set/walk/edn/zip, jolt.http/
interop/shell/nrepl) were loaded cwd-relative, so the built binary couldn't
load them outside the repo. Now stdlib_embed slurps them all at build time into
a {ns -> source} map frozen in the image, and the loader falls back to it when a
namespace isn't found on disk. The artifact is self-contained — clojure.string
works from any directory.

Drops the one-off nREPL source embed (jolt.nrepl is covered by this now). The
baked-in stdlib is excluded from uberscript bundles since it's part of the
runtime. clojure.core is unchanged (Janet, auto-referred).

Test: embedded-stdlib-test requires from the embedded copy with FS roots cleared.
2026-06-05 23:51:03 -04:00
Yogthos
30c4bb0dc2 deps phases 3-4: uberscript bundling + conformance harness
Phase 3 — bundling. `jolt uberscript OUT -m NS` requires NS, uses the loader's
recorded load order (a dep finishes loading before its requirer, so it's
topological) to concatenate the used namespaces into one .clj that runs on a
plain jolt with no deps/jpm. `jolt-deps uberscript` resolves first. The loader
records loaded file paths when ctx env has :loaded-files.

Phase 4 — conformance. deps-conformance-test resolves real pure-cljc git libs
and reports load/run status, gated behind JOLT_CONFORMANCE so CI stays offline.
First run: medley works; cuerdas hits a reader gap (filed); stuartsierra/
dependency uses Long/MAX_VALUE (JVM interop, out of scope).

Tests: uberscript-test, deps-conformance-test (skips without the flag).
2026-06-05 23:36:40 -04:00
Yogthos
ed9d5fc998 cli: babashka-style flags (--version, --eval, --main, nrepl-server)
Add --version/version, -e/--eval, -f/--file, -m/--main (require NS and apply
its -main to *command-line-args*), and nrepl-server/--nrepl-server taking an
optional [host:]port (nrepl stays an alias). repl/help/-h round it out.

Also rewrite doc/tools-deps.md as design notes for the now-implemented deps
support (it still read as pre-implementation research), and note the new flags
in the README. Test: cli-test runs the flags from source.
2026-06-05 23:20:29 -04:00
Yogthos
91df7233b3 docs: add building-and-deps guide, link from README
Move build details, JOLT_PATH/namespace resolution, and the jolt-deps/deps.edn
usage into doc/building-and-deps.md instead of growing the README; README Build
now links to it.
2026-06-05 23:11:16 -04:00
Yogthos
4d8494d620 deps phase 2: jolt-deps tool resolves deps.edn git/local deps
A separate jolt-deps executable (mirroring jpm beside janet) resolves a
deps.edn into source roots and runs jolt with them on JOLT_PATH; the runtime
stays deps-agnostic.

- src/jolt/deps.janet: read deps.edn (Janet parses it directly), resolve
  :git/* and :local/root via jpm/pm's download-bundle (git fetch + cache, no
  build), recurse for transitive deps, return de-duped roots. jpm loaded
  lazily so it's never embedded. Roots cached on a deps.edn hash.
- src/jolt/deps_cli.janet + project.janet: the jolt-deps tool — path/run/repl/-e.
- main: apply JOLT_PATH at runtime (the CLI ctx is frozen into the image at
  build, so init's env read wouldn't see it).

Verified end to end against a local dep and a real git dep (medley). git only,
pure clj/cljc. Test: deps-resolve-test (local deps, no network).
2026-06-05 23:05:47 -04:00
Yogthos
6ea43fb623 deps phase 1: loader searches multiple source roots
The loader resolved a namespace against one hardcoded root (src/jolt, .clj
only). Now it searches an ordered list (ctx env :source-paths, stdlib first)
and tries .clj then .cljc. init adds roots from the :paths opt and JOLT_PATH.
This is the foundation for loading deps.edn libraries; verified loading medley
from an added root. No change to stdlib loading (3913 suite baseline held).
2026-06-05 22:53:20 -04:00
Yogthos
47bcc743b4 Narrow deps.edn research to git-only on jpm
Drop Maven. Verified the piggyback: jpm's download-bundle clones a git dep
into its cache without building (the build half, bundle-install, needs a
project.janet and is separable). Plan is git resolution via jpm/pm, a loader
that searches the clones' src dirs, and compiling used namespaces into the
image at build. Pure clj/cljc only.
2026-06-05 22:42:08 -04:00
Yogthos
5e20ff21ff Research notes: loading Clojure libs via deps.edn
Findings and a phased plan for consuming deps.edn so projects can require
real Clojure libraries. Jolt already reads EDN, jars ship .clj/.cljc source,
and a real lib (medley) loads and runs. The gaps are a single-rooted loader
(needs a classpath + .cljc) and dependency resolution. Recommends reusing
'clojure -Spath' to start, then native git-dep resolution; jpm stays the
Janet build tool alongside this. See doc/tools-deps.md.
2026-06-05 22:28:22 -04:00
Yogthos
858c7fed14 docs: bring EBNF grammar and project docs up to date
- grammar.ebnf: rewrite the number rule to cover the literal syntaxes the reader
  now accepts — 0x/0X hex, N (bigint) / M (bigdec) suffixes, ratios a/b, radixed
  integers (NrXXX, base 2..36), exponents, and the ##Inf/##-Inf/##NaN symbolic
  floats — noting Jolt reads them as plain Janet numbers.
- README: Numbers bullet notes the literal syntaxes read; conformance count.
- reader-syntax-spec: drop the stale 'ratio not supported' case; add coverage
  for hex-uppercase/N/M/ratio/radix/exponent/##Inf/##NaN.
- PLAN.md: refresh the stale Current State snapshot for the 3-layer test
  structure (spec/integration/unit), ~3,920 suite assertions, 218/218
  conformance, current source size.

jpm test green.
2026-06-05 14:40:47 -04:00
Yogthos
a6491d025c docs: add EBNF grammar for the reader syntax (doc/grammar.ebnf)
Specify the surface syntax Jolt's reader accepts as an EBNF grammar — the
syntactic half of the contract (behavioural half is test/spec/). Grounded in
src/jolt/reader.janet and verified against the running reader: whitespace
(comma included), comments, #_ discard, scalars (nil/bool/number incl 0x hex &
floats/string/char incl \uNNNN \oNNN & named/keyword incl ::auto/symbol),
collections, reader macros (quote/syntax-quote/unquote/~@/deref/metadata), and
dispatch (#{}, #(), #', #"regex", #?/#?@, tagged #inst/#uuid). Jolt-vs-Clojure
deviations noted inline (no ratios/radix/BigInt literals; PEG regex limits).
Referenced from README.
2026-06-05 01:34:33 -04:00