cli: babashka-style flags (--version, --eval, --main, nrepl-server)
Add --version/version, -e/--eval, -f/--file, -m/--main (require NS and apply its -main to *command-line-args*), and nrepl-server/--nrepl-server taking an optional [host:]port (nrepl stays an alias). repl/help/-h round it out. Also rewrite doc/tools-deps.md as design notes for the now-implemented deps support (it still read as pre-implementation research), and note the new flags in the README. Test: cli-test runs the flags from source.
This commit is contained in:
Yogthos
parent
f5205e8c0b
commit
ed9d5fc998
4 changed files with 160 additions and 139 deletions
|
|
@ -1,140 +1,82 @@
|
|||
# Loading Clojure libraries via deps.edn (git deps, on jpm)
|
||||
# deps.edn support — design notes
|
||||
|
||||
Research notes for letting a Jolt project pull pure-Clojure libraries through a
|
||||
`deps.edn` and `(require ...)` them. Scope is deliberately narrow:
|
||||
How Jolt loads pure-Clojure libraries from a `deps.edn`, and why it's built the
|
||||
way it is. For how to *use* it, see [building-and-deps.md](building-and-deps.md).
|
||||
|
||||
- **git deps only** — no Maven/`~/.m2` resolution.
|
||||
- **pure `clj`/`cljc`** — anything needing the JVM (Java interop, host classes)
|
||||
won't load or run, and that's expected.
|
||||
- **no classpath machinery** — we just need `require` to find a dep's namespaces
|
||||
at dev time (REPL) so they can be compiled into the image at build time.
|
||||
- **piggyback on jpm** — reuse jpm's existing git fetch + cache; don't write a
|
||||
package manager.
|
||||
Scope, decided up front:
|
||||
|
||||
Nothing here is implemented yet. Everything below was verified against the
|
||||
installed jpm and a real lib (medley).
|
||||
- **git + local deps only** — no Maven/`~/.m2` resolution.
|
||||
- **pure `clj`/`cljc`** — anything needing the JVM won't load or run; expected.
|
||||
- **no classpath abstraction** — `require` just needs to find a dep's namespaces;
|
||||
"the classpath" is an ordered list of source directories.
|
||||
- **piggyback on jpm** — reuse jpm's git fetch + cache; don't write a package
|
||||
manager.
|
||||
- **separate tool** — resolution lives in `jolt-deps`, beside the runtime, the
|
||||
way `jpm` sits beside `janet`. The `jolt` runtime knows nothing about deps.edn.
|
||||
|
||||
## How jpm handles dependencies today
|
||||
## How jpm handles dependencies
|
||||
|
||||
jpm's package code lives in `/opt/homebrew/lib/janet/jpm/pm.janet`. The relevant
|
||||
pieces:
|
||||
jpm's package code (`jpm/pm.janet`) splits into a fetch half and a build half,
|
||||
and we use only the first:
|
||||
|
||||
- **`resolve-bundle`** normalizes a dep spec to `{:url :tag :type :shallow}`. It
|
||||
accepts a table (`:url`/`:repo`, and `:tag`/`:sha`/`:commit`/`:ref`) or a
|
||||
`"url::type::tag"` string. So a deps.edn `{:git/url … :git/sha …}` maps onto it
|
||||
directly.
|
||||
- **`download-bundle url :git tag shallow`** clones into a cache dir and returns
|
||||
the path. The cache is `(find-cache)` = `<modpath>/.cache`, and the per-dep
|
||||
directory is a deterministic id: `git_<tag>_<sanitized-url>`. Under the hood
|
||||
`download-git-bundle` does `git init` + `remote add origin` + fetch + reset to
|
||||
the tag/sha (plus submodules). Pure git — **no build step**.
|
||||
- **`bundle-install`** is the part we *don't* want: after downloading it does
|
||||
`(require-jpm "./project.janet")` and runs build/install rules. A Clojure lib
|
||||
has no `project.janet`, so this would fail. The clone/cache half
|
||||
(`download-bundle`) is cleanly separable from this build half.
|
||||
- **`resolve-bundle`** normalizes a dep spec to `{:url :tag :type :shallow}`,
|
||||
accepting `:url`/`:repo` + `:tag`/`:sha`/`:commit`/`:ref`. A deps.edn
|
||||
`{:git/url … :git/sha …}` maps straight onto it.
|
||||
- **`download-bundle url :git tag shallow`** clones into a content-addressed cache
|
||||
(`<modpath>/.cache/git_<tag>_<sanitized-url>`) and returns the path —
|
||||
`git init` + `remote add` + fetch + reset, plus submodules. No build step.
|
||||
- **`bundle-install`** is the half we skip: it then runs `project.janet` build
|
||||
rules, which a Clojure lib doesn't have. It's cleanly separable from the clone.
|
||||
|
||||
So jpm already gives us git resolution + a content-addressed cache for free; we
|
||||
just skip its build phase.
|
||||
So jpm gives us git resolution and a cache for free; calling `download-bundle`
|
||||
needs `jpm/config/load-default` first (it sets `gitpath` and the cache dyns).
|
||||
|
||||
### Verified
|
||||
## How it works
|
||||
|
||||
```janet
|
||||
(import jpm/config :as cfg)
|
||||
(import jpm/pm :as pm)
|
||||
(cfg/load-default) ; sets gitpath, cache defaults, etc.
|
||||
(setdyn :modpath "<project>/jpm_tree") ; where the cache lives
|
||||
(def s (pm/resolve-bundle {:url "https://github.com/weavejester/medley"
|
||||
:tag "1.0.0" :shallow true}))
|
||||
(pm/download-bundle (s :url) (s :type) (s :tag) (s :shallow))
|
||||
;; => "<modpath>/.cache/git_1.0.0_https___github.com_weavejester_medley"
|
||||
;; with source at <that>/src/medley/core.cljc
|
||||
```
|
||||
`src/jolt/deps.janet` reads `deps.edn` (Janet parses it directly — EDN and Janet
|
||||
syntax overlap for the `:deps`/`:paths` subset), then walks `:deps`:
|
||||
|
||||
`cfg/load-default` is required — calling `download-bundle` without jpm's config
|
||||
dyns set fails in its `shell` helper.
|
||||
- `:git/url` (+ `:git/sha` or `:git/tag`) → `resolve-bundle` + `download-bundle`
|
||||
into `jpm_tree/.cache`;
|
||||
- `:local/root` → the path as-is;
|
||||
- `:mvn/*` and anything else → ignored.
|
||||
|
||||
And the source loads and runs in Jolt: `(medley.core/abs -5)` → `5`,
|
||||
`(medley.core/find-first odd? [2 4 5])` → `5` (coverage is per-function; a few
|
||||
hit interpreter gaps, which is fine).
|
||||
Each resolved dependency contributes its own `:paths` (default `["src"]`) as
|
||||
source roots, and we recurse into its `deps.edn` for transitive deps. The result
|
||||
is a de-duplicated, ordered list of directories. `resolve-deps-cached` memoizes
|
||||
that list in the tree keyed on a hash of `deps.edn`, so an unchanged file doesn't
|
||||
re-fetch. jpm is loaded lazily (`require`, not `import`) so it's pulled in only
|
||||
when resolving — never embedded in a built binary.
|
||||
|
||||
## The shape
|
||||
The loader (`evaluator.janet/find-ns-file`) resolves a namespace by searching the
|
||||
context's `:source-paths` in order (the stdlib `src/jolt` first), trying `<ns>.clj`
|
||||
then `<ns>.cljc`. Extra roots come from `JOLT_PATH` or `init`'s `:paths` option.
|
||||
|
||||
1. **Resolve.** Read `deps.edn`, take the `:deps` whose specs are git
|
||||
(`:git/url` + `:git/sha`/`:git/tag`). For each, `resolve-bundle` +
|
||||
`download-bundle` into the project's `jpm_tree/.cache`. Read each cloned
|
||||
dep's own `deps.edn` and recurse for transitive git deps. (`:local/root`
|
||||
deps are even simpler — just a path, no fetch.)
|
||||
2. **Collect source roots.** A dep's source dirs are its deps.edn `:paths`
|
||||
(default `["src"]`), so a root is `<clone-dir>/<path>`. The result is just an
|
||||
ordered list of directories — not a classpath abstraction, a list of roots.
|
||||
3. **Teach the loader the roots.** `evaluator.janet/ns->path` currently hardcodes
|
||||
`src/jolt/<ns>.clj`. Generalize `maybe-require-ns` to, after the stdlib path,
|
||||
search each dep root for `<ns>.clj` then `<ns>.cljc`. `src/jolt/` stays first
|
||||
so the stdlib always wins. (Two small changes: a root list in the ctx, and
|
||||
trying `.cljc`.)
|
||||
4. **Dev vs build.**
|
||||
- *Dev:* `jolt-deps` resolves `deps.edn` (cached on a hash of it, so it's a
|
||||
no-op when unchanged) and runs jolt with the roots on `JOLT_PATH`;
|
||||
`(require '[medley.core])` then just works.
|
||||
- *Build:* the dep namespaces a project actually uses get compiled into the
|
||||
image the same way the embedded `jolt.nrepl` source is today — load them at
|
||||
build time so the shipped binary needs neither the deps nor jpm.
|
||||
`jolt-deps` (`src/jolt/deps_cli.janet`, its own `declare-executable`) ties it
|
||||
together: it resolves the roots and runs the `jolt` binary with them on
|
||||
`JOLT_PATH`. The runtime's only dependency interface is that env var.
|
||||
|
||||
## Why this fits "no classpath"
|
||||
|
||||
We never construct a Java-style classpath or deal with jar extraction, ordering
|
||||
semantics, or version conflict resolution. Resolution is a tree walk over git
|
||||
deps; "the classpath" is just the list of `src` dirs of the clones. The loader
|
||||
already does path-based namespace lookup — we widen it from one root to a few.
|
||||
|
||||
## A separate tool, like jpm beside janet
|
||||
|
||||
The jolt *runtime* knows nothing about deps.edn — exactly as the `janet` binary
|
||||
knows nothing about jpm. Resolution lives in a separate `jolt-deps` tool (its own
|
||||
`declare-executable`), and the runtime's only interface is the source roots in
|
||||
`JOLT_PATH`:
|
||||
|
||||
```
|
||||
jolt-deps path # print the resolved roots, ':'-joined
|
||||
jolt-deps run FILE [args] # resolve, then run `jolt FILE …` with JOLT_PATH set
|
||||
jolt-deps repl # resolve, then start a jolt REPL with JOLT_PATH set
|
||||
jolt-deps -e EXPR [args] # resolve, then `jolt -e …`
|
||||
```
|
||||
|
||||
It **reuses `jpm/pm`** (`resolve-bundle` + `download-bundle`, after
|
||||
`jpm/config/load-default`) for the git fetch/cache rather than shelling `git` —
|
||||
less code, shares jpm's cache. jpm is loaded lazily (`require`, not `import`), so
|
||||
it's pulled in only when resolving (dev time), never embedded in a binary. The
|
||||
internal-API risk is acceptable since it's dev-time only.
|
||||
|
||||
One gotcha worth recording: `jolt`'s context is built into its image at build
|
||||
time, so `JOLT_PATH` is read at runtime in `main` (not in `init`, whose env read
|
||||
would be frozen at build).
|
||||
Gotcha worth remembering: the `jolt` CLI's context is built into its image at
|
||||
build time, so `JOLT_PATH` is applied at runtime in `main`, not in `init` (whose
|
||||
env read would be frozen at build).
|
||||
|
||||
## Limitations
|
||||
|
||||
- Pure `clj`/`cljc` only. JVM interop, host classes, and unimplemented
|
||||
`clojure.core` corners fail — expected, not a goal.
|
||||
- Per-function coverage: a namespace can load with most functions working and a
|
||||
few not.
|
||||
- Source only; compiled `.class` files (if any in a git dep) are ignored.
|
||||
- Pure `clj`/`cljc` only — JVM interop, host classes, and unimplemented
|
||||
`clojure.core` corners fail. Coverage is per-function: a namespace can load with
|
||||
most functions working and a few not.
|
||||
- Source only; compiled `.class` files in a git dep are ignored.
|
||||
- git `:git/sha` must be a full SHA (`git fetch` can't resolve a short one).
|
||||
|
||||
## Plan
|
||||
## Status
|
||||
|
||||
1. **Loader roots.** *(done)* `maybe-require-ns` now searches an ordered root
|
||||
list (`ctx.env :source-paths`, stdlib first) trying `.clj` then `.cljc`;
|
||||
`init` adds roots from the `:paths` opt and `JOLT_PATH` (colon-separated).
|
||||
Verified loading a real lib (medley) from an added root. See
|
||||
`test/integration/deps-loader-test.janet`.
|
||||
2. **Resolve git deps via jpm.** *(done)* `src/jolt/deps.janet` reads `deps.edn`,
|
||||
resolves `:git/*` + `:local/root` through `jpm/pm` into `jpm_tree/.cache`,
|
||||
recurses for transitive deps, and returns the roots (cached on a deps.edn
|
||||
hash). The separate `jolt-deps` tool (`src/jolt/deps_cli.janet`,
|
||||
`declare-executable`) exposes `path`/`run`/`repl`/`-e`. See
|
||||
`test/integration/deps-resolve-test.janet`.
|
||||
3. **Build-time compile-in.** Fold the used dep namespaces into the image at
|
||||
build (as with embedded `jolt.nrepl`), so a built artifact needs neither the
|
||||
deps nor jpm. *(not started)*
|
||||
4. **Conformance.** Pull a few popular pure-`cljc` git libs, see what loads/runs,
|
||||
and drive interpreter gaps from the failures — same loop as the
|
||||
clojure-test-suite battery.
|
||||
- **Loader source roots** — done. `find-ns-file` + `:source-paths`, `.clj`/`.cljc`,
|
||||
`JOLT_PATH`/`:paths`. Test: `test/integration/deps-loader-test.janet`.
|
||||
- **Resolve git/local deps via jpm** — done. `deps.janet` + the `jolt-deps` tool.
|
||||
Test: `test/integration/deps-resolve-test.janet`.
|
||||
- **Build-time compile-in** — not started. Fold the dep namespaces a project uses
|
||||
into the image at build (as with the embedded `jolt.nrepl` source), so a built
|
||||
artifact needs neither the deps nor jpm.
|
||||
- **Conformance** — not started. Pull popular pure-`cljc` git libs, see what
|
||||
loads/runs, and drive interpreter gaps from the failures — the same loop as the
|
||||
clojure-test-suite battery.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue