Direct-link + whole-program by default for program runs; open for the REPL

Running a program is a closed world — every namespace is required, then it runs
to completion — so make it direct-link by default (inlining, record shapes, the
inference's specialization), and for a -m/-M entry auto-enable the whole-program
cross-namespace inference pass. A decomposed multi-namespace program was ~3.7x
slower than the same code in one namespace purely because per-namespace
inference can't see a caller in a not-yet-loaded namespace; this closes that for
the common case with no flags and no hints.

Interactive modes (repl, -e, nrepl-server) stay indirect/open — they have to let
you redefine vars, which direct-linking seals against. Opt-outs:
JOLT_NO_DIRECT_LINK forces the open path even for a program run (hot-reload,
runtime redefinition); JOLT_NO_WHOLE_PROGRAM keeps direct-linking but per-ns;
JOLT_DIRECT_LINK / JOLT_WHOLE_PROGRAM still force-on. Namespaces required inside
-main (after the batch pass) fall back to per-ns inference.

The success checker (RFC 0006) rides on the inference for free, but a casual
program run shouldn't spam type warnings just because it now direct-links, so its
default-on is suppressed when direct-linking was auto-enabled (:direct-link-auto?);
an explicit JOLT_DIRECT_LINK or JOLT_TYPE_CHECK still turns it on. whole-program-
test and devirt-test opt their per-ns baseline out of the new auto-default.

Docs: RFC 0005 gains 'Compilation modes and defaults' + 'Cross-namespace
inference'; RFC 0004 documents cross-ns/param hints; self-hosting-compiler and
--help updated. Full gate green.
This commit is contained in:
Yogthos 2026-06-14 15:44:01 -04:00
parent 230a0c2160
commit 6abfd660ce
8 changed files with 159 additions and 26 deletions

View file

@ -81,6 +81,33 @@ The same machinery covers both `(:k m)` and `(get m :k [default])` when the key
is a constant keyword. A `get` with a variable, numeric, or string key falls
through to `core-get` unchanged.
## Record hints across namespaces, and as inference seeds
A `^RecordType` hint does two things beyond dropping the lookup guard.
**It carries the specific type, not just "a struct".** The guard-skip only needs
to know the value is raw-get-safe (`:struct`), but the structural inference (RFC
0005) wants the actual record type so a field read gets the field's type —
`(:origin ray)` on a `^Ray ray` is a `Vec3`, not `:any`. A record hint on a
parameter is resolved to the record's constructor key and used to **seed the
inference's parameter type**. That is what keeps a record parameter's reads typed
across a namespace boundary *without* whole-program inference (RFC 0005,
"Cross-namespace inference") — the open-world counterpart to the whole-program
pass. Hinting only the public entry point is not enough; the hint has to be on
the function where the hot reads actually happen.
**It resolves across namespaces.** A hint may name a record defined in another
namespace, in either spelling — `^Vec3` where the type is `:refer`-ed, or
`^v/Vec3` where the namespace is `:as`-aliased. Resolution (`record-ctor-key` in
`src/jolt/host_iface.janet`, backed by `record-hint-ctor-key` in
`src/jolt/evaluator.janet`) runs against the *compile* namespace and maps the
type to its home constructor key through a constructor-value index — keyed by the
constructor value, not a var's namespace, so a `:refer`-interned var (whose
namespace is the referring one) still resolves home. The reader keeps a tag's
namespace qualifier (`^v/Vec3``"v/Vec3"`, not `"Vec3"`) so the aliased
spelling has something to resolve. Both `defrecord` field hints and function
parameter hints use this resolution.
## Soundness and the checked mode
An accurate hint is correctness-preserving by construction: for a struct or

View file

@ -163,6 +163,63 @@ wrong specialization is therefore impossible. The inter-procedural part keeps
the closed-world (optimization-mode) assumption already adopted, which is sound
under whole-program / source-distribution compilation.
## Compilation modes and defaults
Direct-linking — and the inference and specialization it enables — is the
**default for running a program** and stays **off for interactive work**, chosen
by the CLI run mode rather than a global opt-in flag:
| mode | linking | whole-program |
|---|---|---|
| `-m` / `-M NS` (program entry) | direct (default) | **auto** (closed world) |
| `FILE` / `-f` / stdin (`-`) | direct (default) | no (per-namespace) |
| `repl`, `-e`, `nrepl-server` | indirect / open | no |
A program run is a closed world — every namespace is required, then the code
runs to completion — so it direct-links: user code gets inlining, record shapes,
and the inference's specialization. A `-m` / `-M` entry is the exact point where
all requires are done and `-main` is about to run, so the whole-program
cross-namespace pass (below) runs there automatically. Interactive modes stay
open: a REPL, `-e`, and the nREPL server must let you redefine vars — which
direct-linking seals against — so they keep the indirect, live-deref path.
Env overrides, all winning over the mode default:
- `JOLT_NO_DIRECT_LINK=1` — force the open/indirect path even for a program run
(runtime redefinition, hot-reload, self-modifying code).
- `JOLT_NO_WHOLE_PROGRAM=1` — keep direct-linking but skip the whole-program
pass (per-namespace inference only).
- `JOLT_DIRECT_LINK=1` — force direct-linking on even in an interactive mode.
- `JOLT_WHOLE_PROGRAM=1` — force the whole-program pass on in any direct-linked
mode.
- `JOLT_NO_SHAPE=1` — disable the record/shape representation under direct-linking.
What direct-linking gives up is what Clojure's `:direct-linking` and jank's
`-Odirect-call` give up: a direct call embeds its callee, so redefining the
callee is not seen by already-compiled callers. Whole-program additionally
const-links stable vars (data defs, record types, `^:redef`), extending the same
trade. That is why the interactive modes stay open and the opt-outs exist.
### Cross-namespace inference
Per-namespace inference (a `FILE` run, or any namespace under
`JOLT_NO_WHOLE_PROGRAM`) types a function's parameters from the call sites it can
see **within that namespace**. A function whose record parameter is supplied by a
caller in *another* namespace is left `:any`, its field reads keep the guard, and
the values derived from it widen — so a decomposed program is markedly slower
than the same code in one namespace (measured at ~3.7× on the ray tracer split
across five namespaces). The information exists in the program; per-namespace
compilation just can't see a caller in a not-yet-loaded namespace. Two ways to
supply it:
1. **Whole-program** (auto for `-m` / `-M`) runs one closed-world inference
fixpoint over every loaded namespace before `-main`, typing each parameter
from its call sites wherever they live. Namespaces required later (inside
`-main`) fall back to per-namespace inference.
2. **Parameter type hints** (`^RecordType`, RFC 0004) declare the type directly,
so it also works in the open world — REPL, library code that must be fast for
any caller, and hot-reloading servers — where the world cannot be closed.
## Relationship to Hindley-Milner and soft typing
This is HM-shaped with two deliberate departures, which is the textbook