Source locations: reader positions, error locations, native stack traces (#218)

* Reader records source line/column on list forms

The reader stamps 1-based :line/:column metadata on every list form (plus
:file when load-jolt-file is reading a file), and jolt.host/form-position
reads it back so the analyzer's :pos scaffold finally gets real data. A
left-to-right cursor counts newlines over the delta between successive forms,
so it stays O(n). Vector/map/set literals are untouched (their metadata is a
runtime value the analyzer would have to wrap in with-meta); empty () can't
carry meta. ^meta now merges onto the position keys instead of clobbering them.

Re-mint is byte-identical (the backend doesn't emit :pos), so this is a pure
scaffold for the error-location work that follows.

* Report source location on uncaught errors

Each top-level form records its source position (thread-local) before it
compiles+evals, and cli.ss jolt-report-uncaught appends 'at file:line:col'
when an error propagates out. Covers joltc -e, joltc run <file>, and
load-string — every interpreted path. Top-level granularity, one set per
form; deeper frames come from the Phase 2 frame walk.

Runtime .ss only, no re-mint.

* Clojure stack traces via source registry + native frame walk

A direct-link build emits (jolt-register-source! short-name ns name file line)
once per fn def — at definition time, so zero per-call cost. On an uncaught
error the reporter walks Chez's native continuation frames (jolt-throw captures
the live continuation via call/cc; host conditions carry their own
&continuation), maps each frame's procedure name through the registry, and
prints a Clojure backtrace 'ns/name (file:line)'. Wired into both the cli and a
built binary's launcher.

Frames are keyed by the short munged fn name Chez actually reports (emit-fn's
letrec self-binding), not jv$ns$name; a cross-namespace collision degrades to
the bare frame name rather than a wrong attribution. The analyzer carries the
original form's position through defn macroexpansion onto the def node.

Calling a non-fn now throws a catchable ClassCastException (via jolt-throw)
naming the operator, instead of a raw Chez error.

Caveats (documented in source-registry.ss): names map only in direct-link/AOT
closed-world builds — the open-world -e/repl/run path falls back to the
top-level location; and pervasive TCO erases tail-call frames, so a mapped
trace shows only the non-tail spine. JOLT_DEBUG_FRAMES dumps raw frame names.

Re-mint (analyzer + backend); prelude byte-identical (direct-link off during
mint). Corpus rows certified, build-smoke asserts the trace.

* Propagate source position through macroexpansion

hc-expand-1 now carries the macro call form's :line/:column onto the top of a
list expansion that has none of its own (merged under any meta the macro set),
so errors and stack traces in macro-generated code point at the call site —
Clojure parity. The analyze recursion re-expands inner macros, so each level's
top form picks it up, matching the reference compiler. (meta (macroexpand-1
'(when x y))) now reports the call-site line.

A direct-link fn defined through a user macro (build-app's defguarded) registers
with a real line, so build-smoke's trace assertion covers macro-defined fns.

Runtime .ss (host-contract.ss) — no re-mint; selfhost holds.

Phase 3's optional items are deferred: :line-in-ex-data has no clean consumer
(it would pollute ex-data, break = and printing, and positions already surface
via the trace + top-level location), and Chez source-object emission is a large
backend change the jv$-name registry already sidesteps.

* Review fixes: registration key, thread-locals, debug flag timing

- Register a fn under the name Chez actually reports for its frame, not the def
  name: a named fn literal whose name differs from the def (def foo (fn bar …))
  is framed as 'bar', and an anonymous fn def (def foo (fn …)) as jv$ns$foo.
  Both previously registered under the def name and so never appeared in traces.
- rdr-source-file / rdr-pos-cursor are thread parameters, so concurrent compiles
  (futures, core.async) don't clobber each other's file/line attribution.
- Read JOLT_DEBUG_FRAMES at call time: a built binary evaluates top-level forms
  at heap-build time, where a load-time getenv is always unset.

Re-mint (backend + reader); prelude byte-identical, selfhost holds.

---------

Co-authored-by: Yogthos <yogthos@gmail.com>
This commit is contained in:
Dmitri Sotnikov 2026-06-26 02:14:34 +00:00 committed by GitHub
parent bdf436e242
commit 8180c85393
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
18 changed files with 449 additions and 122 deletions

View file

@ -613,7 +613,12 @@
;; macro `def`/`and`/`or` (clojure.spec.alpha) keeps the special form `def`.
(and (form-sym? head) (not shadowed)
(not (contains? handled hname)) (form-macro? ctx head))
(analyze ctx (form-expand-1 ctx form) env)
;; defn/defn- expand to (def name (fn …)); carry the ORIGINAL form's
;; source offset onto the resulting def, since the macro builds a fresh
;; (def …) with no metadata. So the back end can register fn defs.
(let [node (analyze ctx (form-expand-1 ctx form) env)
p (form-position form)]
(if (and p (= :def (:op node))) (assoc node :pos p) node))
;; jolt.ffi/__cfn — the foreign-function special form (always emitted
;; fully-qualified by the jolt.ffi/foreign-fn macro, so aliases resolve).
(and (form-sym? head) (= "jolt.ffi" (form-sym-ns head))
@ -628,7 +633,11 @@
;; `if` does not change the meaning of (if …) in operator position, per
;; spec §3 and the reference. No (not shadowed) guard here.
(and hname (contains? handled hname))
(analyze-special ctx hname items env)
;; stamp the form's source offset onto a top-level def so the back end
;; can register it (jv$ns$name -> source) for native stack traces.
(let [node (analyze-special ctx hname items env)
p (form-position form)]
(if (and p (= :def (:op node))) (assoc node :pos p) node))
(and hname (not shadowed) (method-head? hname))
(analyze-host-call ctx hname items env)
;; (Class. args*) — trailing-dot constructor sugar.

View file

@ -674,14 +674,29 @@
(str "(begin " (str/join " " (map emit-top-form (:statements node)))
(if (empty? (:statements node)) "" " ") (emit-top-form (:ret node)) ")")
(and (= :def (:op node)) (not (:no-init node)) (not (dl-opt-out? (:meta node))))
(let [ns (:ns node) nm (:name node) b (dl-name ns nm)]
(let [ns (:ns node) nm (:name node) b (dl-name ns nm)
fn? (= :fn (:op (:init node)))
;; A fn def gets a source-registry entry so a native backtrace can map its
;; frame to ns/name (file:line). Chez names the frame by whatever emit-fn
;; binds the lambda to: a NAMED fn (defn, or (fn foo …)) gets a letrec
;; self-binding = munge-name of the fn's own name; an ANONYMOUS fn def has
;; no letrec, so the lambda sits directly under (define jv$ns$name …) and
;; takes that name. Register under whichever Chez will report.
pos (:pos node)
frame-name (when fn?
(if-let [fnm (:name (:init node))] (munge-name fnm) b))
reg (when (and fn? pos)
(str " (jolt-register-source! " (chez-str-lit frame-name) " "
(chez-str-lit ns) " " (chez-str-lit nm) " "
(if (get pos :file) (chez-str-lit (get pos :file)) "jolt-nil") " "
(or (get pos :line) 0) ")"))]
;; register before emitting the init so a self-referential body direct-links.
(swap! direct-link-defined conj (dl-fqn ns nm))
(when (= :fn (:op (:init node))) (swap! direct-link-fns conj (dl-fqn ns nm)))
(when fn? (swap! direct-link-fns conj (dl-fqn ns nm)))
(let [init (emit (:init node))]
(if (jmeta-nonempty? (:meta node))
(str "(begin (define " b " " init ") (def-var-with-meta! "
(chez-str-lit ns) " " (chez-str-lit nm) " " b " " (emit-def-meta node) "))")
(chez-str-lit ns) " " (chez-str-lit nm) " " b " " (emit-def-meta node) ")" (or reg "") ")")
(str "(begin (define " b " " init ") (def-var! "
(chez-str-lit ns) " " (chez-str-lit nm) " " b "))"))))
(chez-str-lit ns) " " (chez-str-lit nm) " " b ")" (or reg "") ")"))))
:else (emit node)))