# §2 The Reader (Lexical Syntax)

**Status**: token grammar drafted; reader-macro catalog complete with
normative entries; #inst and literal-collapse divergences resolved.
Conformance: jolt `reader-forms-spec` + `reader-syntax-spec` (granularity
model: jank's per-construct corpus, 62 files under
`test/jank/{reader-macro,syntax-quote}` — adapted rows cited per entry).

The reader maps a stream of characters to *forms* (data). Reading is
independent of evaluation: every form the reader produces is a value of the
language (§4), and `read-string` exposes the reader as a function. Evaluation
of forms is §1's concern; only `quote`-family reader macros reference it here.

## 2.1 Tokens

Whitespace is space, tab, newline, return, **and comma** (`,` is whitespace —
S1). A `;` begins a comment to end of line (S2). Tokens:

```
form        := literal | symbol | keyword | list | vector | map | set
             | reader-macro-form
list        := '(' form* ')'
vector      := '[' form* ']'
map         := '{' (form form)* '}'
literal     := nil | boolean | number | string | character
nil         := 'nil'        boolean := 'true' | 'false'
```

- S3. A map literal MUST contain an even number of forms; duplicate keys
  MUST be an error at read time.
- S4. A set literal (`#{…}`, §2.3) with duplicate elements MUST be an error
  at read time.

### Numbers

```
integer  := ['+'|'-'] (digits | '0' [xX] hexdigits | '0' octdigits | radixR digits)
float    := ['+'|'-'] digits '.' digits? exponent? | ['+'|'-'] digits exponent
ratio    := ['+'|'-'] digits '/' digits            ; host-numeric-tower (§4 note)
exponent := [eE] ['+'|'-'] digits
```

- S5. Trailing `N` (BigInt) and `M` (BigDecimal) suffixes are part of the
  grammar; their value semantics are the §4 numeric-tower question.
  Implementations without those towers SHOULD read them as the nearest
  numeric type and MUST document the choice.

### Symbols and keywords

```
symbol   := name | ns '/' name        ; '/' alone names the division fn
keyword  := ':' name | ':' ns '/' name | '::' name | '::' alias '/' name
```

- S6. Symbol constituent characters: alphanumerics and `* + ! - _ ' ? < > =
  . $ & %` (with `%` and `&` further constrained inside `#()`); a symbol
  MUST NOT begin with a digit; `.` and `/` have positional restrictions.
- S7. `::kw` MUST resolve to the current namespace at *read* time
  (`::k` in ns `user` reads as `:user/k`); `::alias/k` resolves the alias or
  MUST be a read error if the alias does not exist.

### Strings and characters

- S8. Strings are `"…"` with escapes `\" \\ \n \t \r \b \f \uNNNN \oNNN`.
- S9. Character literals: `\c`, the named set `\newline \space \tab
  \return \backspace \formfeed`, unicode `\uNNNN`, octal `\oNNN`.

**Conformance** (2.1): jolt `reader-syntax-spec` "dispatch & sugar";
clojure-test-suite reader files; jank `form/*` literal dirs. S3/S4 duplicate
checks → UNVERIFIED (rows to add).

## 2.2 Quote-family reader macros

| Sugar | Reads as | |
|---|---|---|
| `'form` | `(quote form)` | S10 |
| `@form` | `(clojure.core/deref form)` | S11 |
| `^meta form` | form with metadata attached (see below) | S12 |
| `#'sym` | `(var sym)` | S13 |
| `` `form `` | syntax-quote (§2.4) | |
| `~form`, `~@form` | unquote / unquote-splicing — only within syntax-quote (S14: MUST error outside) | |

- S11. `@form` reads as `(clojure.core/deref form)` — the operator is the
  fully-qualified `clojure.core/deref`, not a bare `deref`, so `@x` still
  dereferences in a namespace that excludes and rebinds `deref`
  (`(ns … (:refer-clojure :exclude [deref]))`), matching Clojure.
- S12a. `^:kw form` ≡ `^{:kw true} form`; `^Sym form` ≡ `^{:tag Sym} form`;
  `^"str"` ≡ `^{:tag "str"} form`. Multiple `^` stack, rightmost innermost,
  merged left-over-right.
- S12b. Type hints are semantically transparent: a hint MUST NOT change a
  program's result. Hints parse in every position they do in Clojure (params,
  `let` bindings, `def` names, return position, arbitrary forms) and are
  otherwise inert. As a non-normative optimization, jolt recognizes two hints
  on a local as an assertion that a constant-keyword lookup may skip its
  runtime representation guard: `^:struct` (a plain struct/record map) and
  `^Name` where `Name` is a `defrecord`/`deftype`. The assertion is the
  programmer's (an inaccurate hint yields a wrong lookup, like a wrong Clojure
  `^String`); `JOLT_CHECK_HINTS=1` turns a violated hint into an error at no
  cost to unchecked builds. See RFC 0004.
- S13a. `#'ns/sym` MUST denote the same var as `(var ns/sym)`:
  `(= (var clojure.core/str) #'clojure.core/str)` is true.

**Conformance**: jolt `reader-forms-spec` "var-quote #'", "metadata ^",
"syntax-quote"; jank `var-quote/pass-qualified.jank`, `metadata/*`.

## 2.3 Dispatch (`#`) reader macros

| Form | Meaning | Entry |
|---|---|---|
| `#{…}` | set literal | S4 above |
| `#"…"` | regex literal — reads to a regex value; escaping is regex-level, not string-level (single `\d`) | S15 |
| `#(…)` | anonymous fn | S16 below |
| `#_form` | discard | S17 below |
| `#?(…)` / `#?@(…)` | reader conditional (+splicing) | S18 below |
| `##Inf ##-Inf ##NaN` | symbolic floats | S19 |
| `#tag form` | tagged literal | S20 below |
| `#! …` | shebang comment line (implementations SHOULD accept) | |

### S16 — anonymous function `#(…)`

- `#(body)` reads as `(fn [args…] (body))` with parameters derived from the
  `%`-symbols appearing in body: `%`≡`%1`, `%n` positional, `%&` the rest
  parameter. Arity = highest `%n` mentioned (plus rest if `%&`).
- The `%`-symbols are collected from the WHOLE body, recursing through every
  nested form including vector, map and set literals — `#(assoc {} :k %)`,
  `#(hash-set % %2)` and `#(get {:t %} :t)` all see their `%`s. (A reader that
  scanned only call forms would miscompile `#(identity {:text %})` as a 0-arg fn.)
- The synthesized parameters are auto-gensyms (their names carry the `#` suffix,
  like Clojure's `p1__N#`), so an `#()` written inside a syntax-quote survives:
  the params are mapped consistently and left unqualified rather than being
  qualified to the current namespace (a qualified symbol is not a valid
  parameter). E.g. ``  `(map #(inc %) xs) `` expands correctly inside a macro.
- `#()` literals MUST NOT nest.

```clojure
(#(+ %1 %2) 1 2)            ;=> 3
(apply #(apply + %&) [1 2 3]) ;=> 6
(map #(* % %) [1 2])        ;=> (1 4)
```

### S17 — discard `#_`

- `#_form` reads and discards the next form entirely (it is never evaluated).
- Discards compose: `#_ #_ a b` discards two following forms.
- `#_` inside collection literals removes the element: `[1 #_2 3]` ⇒ `[1 3]`.

### S18 — reader conditionals

- `#?(:feat₁ f₁ :feat₂ f₂ …)` reads as the form of the first feature key the
  platform satisfies, else nothing. `:default` matches any platform.
  `#?@(…)` splices a sequential form into the surrounding context.
- Feature keys are implementation-defined; each implementation MUST document
  its feature set, and SHOULD follow the portable convention *own dialect key
  + `:default`*. Matching MUST be by **clause order** — the first clause whose
  key the platform satisfies wins (`#?(:default 5 :clj 6)` is `5` everywhere)
  — not by key priority. Implementations SHOULD provide a per-loading-context
  compatibility override for foreign-dialect libraries. (jolt:
  `#{:jolt :default}`, opt-in via `reader-features-set!`/`JOLT_FEATURES`;
  decision + A/B data in RFC 0002 — inheriting `:clj` cost 146 suite
  assertions and 38 errors.)
- Reader conditionals MUST be an error outside `.cljc`-style reading unless
  the implementation documents otherwise.

### S19 — symbolic values

`##Inf`, `##-Inf`, `##NaN` read as the IEEE-754 values. `(= ##NaN ##NaN)` is
false; `(NaN? ##NaN)` is true.

### S20 — tagged literals

- `#tag form`: the reader resolves `tag` in the data-reader table and MUST
  apply the reader function to the *read* form, yielding its result as the
  read value. An unknown tag MUST be a read error (jank
  `fail-unsupported-tag`).
- Built-in tags every implementation MUST provide: `#uuid "…"` → a UUID
  value (§9 `parse-uuid` semantics — round-trips through printing), and
  `#inst "…"` → an instant value: RFC3339 with partial-timestamp defaults
  (`#inst "2020"` ≡ `#inst "2020-01-01T00:00:00.000-00:00"`), equality by
  instant (offset-normalized), `inst?`/`inst-ms` (epoch milliseconds), printed
  canonically as `#inst "yyyy-MM-ddThh:mm:ss.fff-00:00"` and round-tripping. A
  malformed timestamp MUST be an error.

**Conformance** (2.3): jolt `reader-forms-spec` "#() (% %N %&)" + new rows
(symbolic values, stacked discard, conditionals); `uuid-spec` reader-literal
group; jank `reader-macro/{function,regex,uuid,symbolic-value}/*`,
`fail-unsupported-tag.jank`.

## 2.4 Syntax-quote

Syntax-quote (`` ` ``) is read-level template construction with namespace
resolution:

- S21. Inside syntax-quote, an unqualified symbol that resolves in
  `clojure.core` MUST be qualified to `clojure.core/sym`; a symbol resolving
  through a namespace alias MUST be qualified to the aliased namespace; an
  unresolved symbol MUST be qualified to the current namespace. Special-form
  names stay bare.
- S22. `sym#` generates a fresh symbol, stable *within one syntax-quote
  template* (all `sym#` in the same template denote the same generated
  symbol; distinct templates generate distinct symbols).
- S23. `~form` inserts the value of `form`; `~@form` splices a sequential
  value; `~'sym` is the idiom for an intentionally-unqualified symbol.
- S24. Syntax-quote distributes through collection literals (vectors, maps,
  sets) — qualification and unquoting apply inside them.
- S25. A syntax-quoted self-evaluating literal is the literal, collapsed at
  read time — so nested/adjacent backticks over literals are inert:
  `(= "meow" ```"meow")` is true. General nested syntax-quote over symbols
  and collections expands recursively (quasiquote semantics) — that general
  case remains UNVERIFIED pending dedicated conformance rows.

**Conformance**: jolt `reader-forms-spec` "syntax-quote" (gensym, unquote,
splice) + conformance "syntax-quote fully-qualifies"; jank
`syntax-quote/{pass-gensym,pass-namespace-resolution,pass-resolve-alias,
unquote,unquote-splice}/*`. S25 → UNVERIFIED.

## 2.5 What the reader is not

The reader performs **no macroexpansion and no evaluation** (tagged-literal
reader functions are the deliberate exception, S20). Forms read identically
whether or not they will be evaluated; `read-string` of any printable value
`v` followed by evaluation yields a value equal to `v` for the
self-evaluating types (§4 print/read round-trip contract).