Source locations: reader positions, error locations, native stack traces (#218)

* Reader records source line/column on list forms

The reader stamps 1-based :line/:column metadata on every list form (plus
:file when load-jolt-file is reading a file), and jolt.host/form-position
reads it back so the analyzer's :pos scaffold finally gets real data. A
left-to-right cursor counts newlines over the delta between successive forms,
so it stays O(n). Vector/map/set literals are untouched (their metadata is a
runtime value the analyzer would have to wrap in with-meta); empty () can't
carry meta. ^meta now merges onto the position keys instead of clobbering them.

Re-mint is byte-identical (the backend doesn't emit :pos), so this is a pure
scaffold for the error-location work that follows.

* Report source location on uncaught errors

Each top-level form records its source position (thread-local) before it
compiles+evals, and cli.ss jolt-report-uncaught appends 'at file:line:col'
when an error propagates out. Covers joltc -e, joltc run <file>, and
load-string — every interpreted path. Top-level granularity, one set per
form; deeper frames come from the Phase 2 frame walk.

Runtime .ss only, no re-mint.

* Clojure stack traces via source registry + native frame walk

A direct-link build emits (jolt-register-source! short-name ns name file line)
once per fn def — at definition time, so zero per-call cost. On an uncaught
error the reporter walks Chez's native continuation frames (jolt-throw captures
the live continuation via call/cc; host conditions carry their own
&continuation), maps each frame's procedure name through the registry, and
prints a Clojure backtrace 'ns/name (file:line)'. Wired into both the cli and a
built binary's launcher.

Frames are keyed by the short munged fn name Chez actually reports (emit-fn's
letrec self-binding), not jv$ns$name; a cross-namespace collision degrades to
the bare frame name rather than a wrong attribution. The analyzer carries the
original form's position through defn macroexpansion onto the def node.

Calling a non-fn now throws a catchable ClassCastException (via jolt-throw)
naming the operator, instead of a raw Chez error.

Caveats (documented in source-registry.ss): names map only in direct-link/AOT
closed-world builds — the open-world -e/repl/run path falls back to the
top-level location; and pervasive TCO erases tail-call frames, so a mapped
trace shows only the non-tail spine. JOLT_DEBUG_FRAMES dumps raw frame names.

Re-mint (analyzer + backend); prelude byte-identical (direct-link off during
mint). Corpus rows certified, build-smoke asserts the trace.

* Propagate source position through macroexpansion

hc-expand-1 now carries the macro call form's :line/:column onto the top of a
list expansion that has none of its own (merged under any meta the macro set),
so errors and stack traces in macro-generated code point at the call site —
Clojure parity. The analyze recursion re-expands inner macros, so each level's
top form picks it up, matching the reference compiler. (meta (macroexpand-1
'(when x y))) now reports the call-site line.

A direct-link fn defined through a user macro (build-app's defguarded) registers
with a real line, so build-smoke's trace assertion covers macro-defined fns.

Runtime .ss (host-contract.ss) — no re-mint; selfhost holds.

Phase 3's optional items are deferred: :line-in-ex-data has no clean consumer
(it would pollute ex-data, break = and printing, and positions already surface
via the trace + top-level location), and Chez source-object emission is a large
backend change the jv$-name registry already sidesteps.

* Review fixes: registration key, thread-locals, debug flag timing

- Register a fn under the name Chez actually reports for its frame, not the def
  name: a named fn literal whose name differs from the def (def foo (fn bar …))
  is framed as 'bar', and an anonymous fn def (def foo (fn …)) as jv$ns$foo.
  Both previously registered under the def name and so never appeared in traces.
- rdr-source-file / rdr-pos-cursor are thread parameters, so concurrent compiles
  (futures, core.async) don't clobber each other's file/line attribution.
- Read JOLT_DEBUG_FRAMES at call time: a built binary evaluates top-level forms
  at heap-build time, where a load-time getenv is always unset.

Re-mint (backend + reader); prelude byte-identical, selfhost holds.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
This commit is contained in:
Dmitri Sotnikov 2026-06-26 02:14:34 +00:00 committed by GitHub
parent bdf436e242
commit 8180c85393
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
18 changed files with 449 additions and 122 deletions

View file

@ -12,6 +12,10 @@
(defmethod greet :soft [_] "greet:soft")
(defn -main [& args]
;; --boom: throw through a two-deep call chain so build-smoke can assert the
;; native stack trace. Off the normal path, so default output is unchanged.
(when (= (first args) "--boom")
(util/mid-boom "not-a-number"))
;; the resource is baked into the binary (deps.edn :jolt/build :embed), so this
;; resolves with no resources/ dir on disk, run from any cwd.
(println (slurp (io/resource "greeting.txt")))

View file

@ -4,6 +4,19 @@
(defn shout [s]
(str/upper-case (str s "!")))
;; A two-deep non-tail call chain that throws — exercises native stack traces in a
;; direct-link build (build-smoke runs -main with a --boom sentinel arg). deep-boom
;; is defined through a USER macro: its source registration only gets a real line
;; if the reader position survives macroexpansion (so the trace frame maps).
(defmacro defguarded [name args & body]
`(defn ~name ~args (assert (number? ~(first args)) "needs a number") ~@body))
(defguarded deep-boom [x]
(* x 2))
(defn mid-boom [x]
(inc (deep-boom x)))
(defmacro twice [x]
`(do ~x ~x))

View file

@ -611,6 +611,8 @@
{:suite "host-interop / class tokens & readers" :label "indexOf int needle is a char code" :expected "1" :actual "(.indexOf \"a=b\" 61)"}
{:suite "host-interop / exception + HashMap shims" :label "getMessage on a thrown string" :expected "\"class java.lang.String cannot be cast to class java.lang.Throwable (java.lang.String and java.lang.Throwable are in module java.base of loader 'bootstrap')\"" :actual "(try (throw \"boom\") (catch Throwable e (.getMessage e)))"}
{:suite "host-interop / exception + HashMap shims" :label "getMessage on ex-info" :expected "\"bad\"" :actual "(try (throw (ex-info \"bad\" {})) (catch Throwable e (.getMessage e)))"}
{:suite "host-interop / exception + HashMap shims" :label "calling a non-fn throws ClassCastException" :expected ":ccx" :actual "(try (1 2) (catch ClassCastException _ :ccx))"}
{:suite "host-interop / exception + HashMap shims" :label "non-fn cast is a RuntimeException too" :expected ":rt" :actual "(try ((identity 5)) (catch RuntimeException _ :rt))"}
{:suite "host-interop / exception + HashMap shims" :label "HashMap get" :expected "2" :actual "(let [m (HashMap. {:a 1 :b 2})] (.get m :b))"}
{:suite "host-interop / exception + HashMap shims" :label "HashMap put + size" :expected "2" :actual "(let [m (HashMap. {})] (.put m :x 1) (.put m :y 2) (.size m))"}
{:suite "host-interop / reader-feature toggle" :label "features default to jolt+default" :expected "true" :actual "(contains? (set (__reader-features)) \"jolt\")"}

View file

@ -331,6 +331,17 @@
{:suite "reader" :expr "(nil? (read-string \" , ,\"))" :expected "true"}
{:suite "reader" :expr "(:tag (meta (read-string \"^String x\")))" :expected "String"}
{:suite "reader" :expr "(:foo (meta (read-string \"^:foo x\")))" :expected "true"}
{:suite "reader" :expr "(:line (meta (read-string \"(foo bar)\")))" :expected "1"}
{:suite "reader" :expr "(:column (meta (read-string \"(foo bar)\")))" :expected "1"}
{:suite "reader" :expr "(:line (meta (read-string \"\\n\\n (x)\")))" :expected "3"}
{:suite "reader" :expr "(:column (meta (read-string \"\\n\\n (x)\")))" :expected "3"}
{:suite "reader" :expr "(nil? (meta (read-string \"[1 2]\")))" :expected "true"}
{:suite "reader" :expr "(nil? (meta (read-string \"()\")))" :expected "true"}
{:suite "reader" :expr "(:foo (meta (read-string \"^:foo (x y)\")))" :expected "true"}
{:suite "reader" :expr "(:line (meta (read-string \"^:foo (x y)\")))" :expected "1"}
{:suite "reader" :expr "(:line (meta (macroexpand-1 (read-string \"(when x y)\"))))" :expected "1"}
{:suite "reader" :expr "(:line (meta (macroexpand-1 (read-string \"\\n\\n(when x y)\"))))" :expected "3"}
{:suite "reader" :expr "(:column (meta (macroexpand-1 (read-string \"(when x y)\"))))" :expected "1"}
{:suite "reader" :expr "(= 42 (with-in-str \"42\" (read)))" :expected "true"}
{:suite "reader" :expr "(= (quote (+ 1 2)) (with-in-str \"(+ 1 2)\" (read)))" :expected "true"}
{:suite "reader" :expr "(with-in-str \"1 2\" [(read) (read)])" :expected "[1 2]"}