Regex: accept Java-compatible char-class dash and (X+)* quantifier

irregex rejected two patterns the JVM accepts, which blocked library loads:

- [\w-_] errored with bad char-set because a - after a shorthand class was
  read as a range start. Java reads it as a literal hyphen. Preprocess the
  pattern to escape such a dash.
- (X+)* errored with duplicate repetition because sre-repeater? recurses
  through submatch, treating a quantified group like a dangling a**. Override
  it to a bare leading * / + check, matching the JVM (which only rejects the
  dangling case).

Both in regex.ss (runtime). Unblocks cuerdas (was load-fail, now 292 passing)
and aws-api config-test. Also documents the host/chez/java source-layering rule
in host-interop.md.

jolt-l8so
This commit is contained in:
Yogthos 2026-06-26 17:35:08 -04:00
parent 687dc60af6
commit 8a877662dc
3 changed files with 74 additions and 1 deletions

View file

@ -19,6 +19,36 @@ reflection and no class hierarchy. `(class x)` returns the JVM class name for th
scalar/collection types Clojure programs compare against (`"java.lang.Long"`, scalar/collection types Clojure programs compare against (`"java.lang.Long"`,
`"java.lang.String"`, and so on). `"java.lang.String"`, and so on).
## Source layering: JVM-specific code lives in the java layer
Keep anything JVM-specific in `host/chez/java/`. The rest of the runtime stays
JVM-free, and the compiler in `jolt-core/` is JVM-free by construction.
- `host/chez/java/` holds the JVM model: the `java.*` mirrors, the class tokens
and class hierarchy, `(class x)`/`(type x)`/`instance?`, exception classes, the
interop dispatch for `.method`/`Class/static`/`(Class.)`. If a value or name
only means something because the JVM has it, it belongs here.
- The rest of `host/chez/` is the host-neutral runtime — the value model
(`values.ss`, `collections.ss`, `seq.ss`), reader, vars, multimethods, meta. It
speaks jolt's own taxonomy (`:string`, `:vector`, `:jolt/inst`), never JVM class
names.
- `jolt-core/` (the Clojure compiler + `clojure.core` overlay) emits and reasons
in that taxonomy only. The JVM mapping happens *after*, in the java layer.
The worked example is `type`. The core layer (`natives-meta.ss`) computes the
keyword taxonomy and binds it as `__type-tag` — that's what `print-method` and the
reader dispatch on, with no JVM in scope. The java layer (`java/host-class.ss`)
then rebinds the public `clojure.core/type` to Clojure's `(or (:type meta) (class
x))`, mapping `:jolt/inst` → `java.util.Date` and so on, right next to `(class
…)`. So the compiler keeps emitting `:jolt/inst`; the java layer remaps it.
When you add interop behaviour, prefer registering it through the generic hooks a
java-layer file already uses — `register-class-arm!` for `(class x)`,
`register-instance-check-arm!` for `instance?`, `register-eq-arm!` for value
equality — rather than threading a JVM concept back into a host-neutral file. A
new `java.*` shim is a new file under `host/chez/java/` loaded from `rt.ss`, not a
branch added to `collections.ss` or `seq.ss`.
## What's shimmed ## What's shimmed
This is the surface today, not the whole JVM. Methods not listed generally This is the surface today, not the whole JVM. Methods not listed generally

View file

@ -33,6 +33,14 @@
(apply %chez-error args))) (apply %chez-error args)))
(load "vendor/irregex/irregex.scm") (load "vendor/irregex/irregex.scm")
;; irregex rejects a quantifier applied to anything that already contains one —
;; including a GROUP like (a+)* — because sre-repeater? recurses through submatch.
;; Java only rejects a DANGLING double quantifier (a**); it allows a quantifier on
;; a group whose body is quantified. Restrict the check to a bare leading * / + so
;; a** still errors but (a+)* parses (cuerdas's format tokenizer needs this).
(set! sre-repeater?
(lambda (sre) (and (pair? sre) (memq (car sre) '(* +)) #t)))
;; Unicode property classes \p{...}: irregex's string syntax has no ;; Unicode property classes \p{...}: irregex's string syntax has no
;; \p{...}, so translate a fixed set of property names ;; \p{...}, so translate a fixed set of property names
;; to ASCII char classes before compiling. ASCII-only — \p{L} would need ;; to ASCII char classes before compiling. ASCII-only — \p{L} would need
@ -92,6 +100,36 @@
(write-char c out) (loop (fx+ i 1) #f)) (write-char c out) (loop (fx+ i 1) #f))
(else (write-char c out) (loop (fx+ i 1) in-class)))))))) (else (write-char c out) (loop (fx+ i 1) in-class))))))))
;; Inside a [...] class, irregex reads a '-' that follows a shorthand class
;; (\w \d \s \W \D \S) as the start of a range and errors ("bad char-set"); Java
;; reads it as a literal hyphen (a shorthand can't be a range endpoint). Escape
;; such a '-' to \- so the class parses. Only a '-' right after a shorthand and
;; not the class terminator is touched; a '-' after a plain char (a real range
;; like [a-z]) is left alone.
(define (escape-class-shorthand-dash src)
(let ((len (string-length src)) (out (open-output-string)))
(let loop ((i 0) (in-class #f) (after-shorthand #f))
(if (fx>=? i len)
(get-output-string out)
(let ((c (string-ref src i)))
(cond
;; an escape pair: \w-style shorthand sets after-shorthand inside a class
((and (char=? c #\\) (fx<? (fx+ i 1) len))
(let ((n (string-ref src (fx+ i 1))))
(write-char c out) (write-char n out)
(loop (fx+ i 2) in-class
(and in-class (memv n '(#\w #\d #\s #\W #\D #\S)) #t))))
((and (not in-class) (char=? c #\[))
(write-char c out) (loop (fx+ i 1) #t #f))
((and in-class (char=? c #\]))
(write-char c out) (loop (fx+ i 1) #f #f))
;; the case Java reads as a literal hyphen
((and in-class after-shorthand (char=? c #\-)
(fx<? (fx+ i 1) len) (not (char=? (string-ref src (fx+ i 1)) #\])))
(write-char #\\ out) (write-char #\- out)
(loop (fx+ i 1) in-class #f))
(else (write-char c out) (loop (fx+ i 1) in-class #f))))))))
;; Java/Clojure inline flags: a leading (?imsx…) group sets a flag over the whole ;; Java/Clojure inline flags: a leading (?imsx…) group sets a flag over the whole
;; pattern. irregex has the same semantics but as constructor OPTIONS, not inline ;; pattern. irregex has the same semantics but as constructor OPTIONS, not inline
;; syntax (it rejects (?s)/(?s:…)), so peel any leading flag groups off the source ;; syntax (it rejects (?s)/(?s:…)), so peel any leading flag groups off the source
@ -123,7 +161,8 @@
(define-record-type regex-t (fields source irx) (nongenerative jolt-regex-v1)) (define-record-type regex-t (fields source irx) (nongenerative jolt-regex-v1))
(define (jolt-regex source) (define (jolt-regex source)
(let-values (((opts pat) (regex-parse-flags source))) (let-values (((opts pat) (regex-parse-flags source)))
(make-regex-t source (apply irregex (translate-prop-classes pat) opts)))) (make-regex-t source
(apply irregex (translate-prop-classes (escape-class-shorthand-dash pat)) opts))))
(define (jolt-regex? x) (regex-t? x)) (define (jolt-regex? x) (regex-t? x))
(define (jolt-re-pattern x) (if (regex-t? x) x (jolt-regex x))) (define (jolt-re-pattern x) (if (regex-t? x) x (jolt-regex x)))

View file

@ -1823,6 +1823,10 @@
{:suite "regex / re-seq" :label "all matches" :expected "[\"1\" \"22\" \"333\"]" :actual "(re-seq #\"\\d+\" \"a1b22c333\")"} {:suite "regex / re-seq" :label "all matches" :expected "[\"1\" \"22\" \"333\"]" :actual "(re-seq #\"\\d+\" \"a1b22c333\")"}
{:suite "regex / re-seq" :label "empty when none" :expected "nil" :actual "(seq (re-seq #\"z\" \"abc\"))"} {:suite "regex / re-seq" :label "empty when none" :expected "nil" :actual "(seq (re-seq #\"z\" \"abc\"))"}
{:suite "regex / re-seq" :label "words" :expected "[\"foo\" \"bar\"]" :actual "(re-seq #\"\\w+\" \"foo bar\")"} {:suite "regex / re-seq" :label "words" :expected "[\"foo\" \"bar\"]" :actual "(re-seq #\"\\w+\" \"foo bar\")"}
{:suite "regex / char-class dash" :label "dash after \\w is literal" :expected "\"a_b-c\"" :actual "(re-matches #\"[\\w-_]+\" \"a_b-c\")"}
{:suite "regex / char-class dash" :label "dash + escaped dot in class" :expected "\"a.b_c-d\"" :actual "(re-matches #\"[\\w-_\\.]+\" \"a.b_c-d\")"}
{:suite "regex / char-class dash" :label "trailing dash in class" :expected "[\"a-b\" \"c-d\"]" :actual "(vec (re-seq #\"[\\w-]+\" \"a-b c-d\"))"}
{:suite "regex / nested quantifier" :label "(X+)* parses and matches" :expected "true" :actual "(boolean (re-matches #\"([^%]+)*(%(d))?\" \"abc\"))"}
{:suite "regex / re-pattern & string ops" :label "re-pattern build" :expected "\"hi\"" :actual "(re-find (re-pattern \"\\\\w+\") \"hi!\")"} {:suite "regex / re-pattern & string ops" :label "re-pattern build" :expected "\"hi\"" :actual "(re-find (re-pattern \"\\\\w+\") \"hi!\")"}
{:suite "regex / re-pattern & string ops" :label "re-pattern is regex?" :expected "true" :actual "(regex? (re-pattern \"a\"))"} {:suite "regex / re-pattern & string ops" :label "re-pattern is regex?" :expected "true" :actual "(regex? (re-pattern \"a\"))"}
{:suite "regex / re-pattern & string ops" :label "split on regex" :expected "[\"a\" \"b\" \"c\"]" :actual "(do (require '[clojure.string :as s]) (s/split \"a1b2c\" #\"\\d\"))"} {:suite "regex / re-pattern & string ops" :label "split on regex" :expected "[\"a\" \"b\" \"c\"]" :actual "(do (require '[clojure.string :as s]) (s/split \"a1b2c\" #\"\\d\"))"}