Regex: accept Java-compatible char-class dash and (X+)* quantifier
irregex rejected two patterns the JVM accepts, which blocked library loads: - [\w-_] errored with bad char-set because a - after a shorthand class was read as a range start. Java reads it as a literal hyphen. Preprocess the pattern to escape such a dash. - (X+)* errored with duplicate repetition because sre-repeater? recurses through submatch, treating a quantified group like a dangling a**. Override it to a bare leading * / + check, matching the JVM (which only rejects the dangling case). Both in regex.ss (runtime). Unblocks cuerdas (was load-fail, now 292 passing) and aws-api config-test. Also documents the host/chez/java source-layering rule in host-interop.md. jolt-l8so
This commit is contained in:
parent
687dc60af6
commit
8a877662dc
3 changed files with 74 additions and 1 deletions
|
|
@ -19,6 +19,36 @@ reflection and no class hierarchy. `(class x)` returns the JVM class name for th
|
|||
scalar/collection types Clojure programs compare against (`"java.lang.Long"`,
|
||||
`"java.lang.String"`, and so on).
|
||||
|
||||
## Source layering: JVM-specific code lives in the java layer
|
||||
|
||||
Keep anything JVM-specific in `host/chez/java/`. The rest of the runtime stays
|
||||
JVM-free, and the compiler in `jolt-core/` is JVM-free by construction.
|
||||
|
||||
- `host/chez/java/` holds the JVM model: the `java.*` mirrors, the class tokens
|
||||
and class hierarchy, `(class x)`/`(type x)`/`instance?`, exception classes, the
|
||||
interop dispatch for `.method`/`Class/static`/`(Class.)`. If a value or name
|
||||
only means something because the JVM has it, it belongs here.
|
||||
- The rest of `host/chez/` is the host-neutral runtime — the value model
|
||||
(`values.ss`, `collections.ss`, `seq.ss`), reader, vars, multimethods, meta. It
|
||||
speaks jolt's own taxonomy (`:string`, `:vector`, `:jolt/inst`), never JVM class
|
||||
names.
|
||||
- `jolt-core/` (the Clojure compiler + `clojure.core` overlay) emits and reasons
|
||||
in that taxonomy only. The JVM mapping happens *after*, in the java layer.
|
||||
|
||||
The worked example is `type`. The core layer (`natives-meta.ss`) computes the
|
||||
keyword taxonomy and binds it as `__type-tag` — that's what `print-method` and the
|
||||
reader dispatch on, with no JVM in scope. The java layer (`java/host-class.ss`)
|
||||
then rebinds the public `clojure.core/type` to Clojure's `(or (:type meta) (class
|
||||
x))`, mapping `:jolt/inst` → `java.util.Date` and so on, right next to `(class
|
||||
…)`. So the compiler keeps emitting `:jolt/inst`; the java layer remaps it.
|
||||
|
||||
When you add interop behaviour, prefer registering it through the generic hooks a
|
||||
java-layer file already uses — `register-class-arm!` for `(class x)`,
|
||||
`register-instance-check-arm!` for `instance?`, `register-eq-arm!` for value
|
||||
equality — rather than threading a JVM concept back into a host-neutral file. A
|
||||
new `java.*` shim is a new file under `host/chez/java/` loaded from `rt.ss`, not a
|
||||
branch added to `collections.ss` or `seq.ss`.
|
||||
|
||||
## What's shimmed
|
||||
|
||||
This is the surface today, not the whole JVM. Methods not listed generally
|
||||
|
|
|
|||
|
|
@ -33,6 +33,14 @@
|
|||
(apply %chez-error args)))
|
||||
(load "vendor/irregex/irregex.scm")
|
||||
|
||||
;; irregex rejects a quantifier applied to anything that already contains one —
|
||||
;; including a GROUP like (a+)* — because sre-repeater? recurses through submatch.
|
||||
;; Java only rejects a DANGLING double quantifier (a**); it allows a quantifier on
|
||||
;; a group whose body is quantified. Restrict the check to a bare leading * / + so
|
||||
;; a** still errors but (a+)* parses (cuerdas's format tokenizer needs this).
|
||||
(set! sre-repeater?
|
||||
(lambda (sre) (and (pair? sre) (memq (car sre) '(* +)) #t)))
|
||||
|
||||
;; Unicode property classes \p{...}: irregex's string syntax has no
|
||||
;; \p{...}, so translate a fixed set of property names
|
||||
;; to ASCII char classes before compiling. ASCII-only — \p{L} would need
|
||||
|
|
@ -92,6 +100,36 @@
|
|||
(write-char c out) (loop (fx+ i 1) #f))
|
||||
(else (write-char c out) (loop (fx+ i 1) in-class))))))))
|
||||
|
||||
;; Inside a [...] class, irregex reads a '-' that follows a shorthand class
|
||||
;; (\w \d \s \W \D \S) as the start of a range and errors ("bad char-set"); Java
|
||||
;; reads it as a literal hyphen (a shorthand can't be a range endpoint). Escape
|
||||
;; such a '-' to \- so the class parses. Only a '-' right after a shorthand and
|
||||
;; not the class terminator is touched; a '-' after a plain char (a real range
|
||||
;; like [a-z]) is left alone.
|
||||
(define (escape-class-shorthand-dash src)
|
||||
(let ((len (string-length src)) (out (open-output-string)))
|
||||
(let loop ((i 0) (in-class #f) (after-shorthand #f))
|
||||
(if (fx>=? i len)
|
||||
(get-output-string out)
|
||||
(let ((c (string-ref src i)))
|
||||
(cond
|
||||
;; an escape pair: \w-style shorthand sets after-shorthand inside a class
|
||||
((and (char=? c #\\) (fx<? (fx+ i 1) len))
|
||||
(let ((n (string-ref src (fx+ i 1))))
|
||||
(write-char c out) (write-char n out)
|
||||
(loop (fx+ i 2) in-class
|
||||
(and in-class (memv n '(#\w #\d #\s #\W #\D #\S)) #t))))
|
||||
((and (not in-class) (char=? c #\[))
|
||||
(write-char c out) (loop (fx+ i 1) #t #f))
|
||||
((and in-class (char=? c #\]))
|
||||
(write-char c out) (loop (fx+ i 1) #f #f))
|
||||
;; the case Java reads as a literal hyphen
|
||||
((and in-class after-shorthand (char=? c #\-)
|
||||
(fx<? (fx+ i 1) len) (not (char=? (string-ref src (fx+ i 1)) #\])))
|
||||
(write-char #\\ out) (write-char #\- out)
|
||||
(loop (fx+ i 1) in-class #f))
|
||||
(else (write-char c out) (loop (fx+ i 1) in-class #f))))))))
|
||||
|
||||
;; Java/Clojure inline flags: a leading (?imsx…) group sets a flag over the whole
|
||||
;; pattern. irregex has the same semantics but as constructor OPTIONS, not inline
|
||||
;; syntax (it rejects (?s)/(?s:…)), so peel any leading flag groups off the source
|
||||
|
|
@ -123,7 +161,8 @@
|
|||
(define-record-type regex-t (fields source irx) (nongenerative jolt-regex-v1))
|
||||
(define (jolt-regex source)
|
||||
(let-values (((opts pat) (regex-parse-flags source)))
|
||||
(make-regex-t source (apply irregex (translate-prop-classes pat) opts))))
|
||||
(make-regex-t source
|
||||
(apply irregex (translate-prop-classes (escape-class-shorthand-dash pat)) opts))))
|
||||
(define (jolt-regex? x) (regex-t? x))
|
||||
(define (jolt-re-pattern x) (if (regex-t? x) x (jolt-regex x)))
|
||||
|
||||
|
|
|
|||
|
|
@ -1823,6 +1823,10 @@
|
|||
{:suite "regex / re-seq" :label "all matches" :expected "[\"1\" \"22\" \"333\"]" :actual "(re-seq #\"\\d+\" \"a1b22c333\")"}
|
||||
{:suite "regex / re-seq" :label "empty when none" :expected "nil" :actual "(seq (re-seq #\"z\" \"abc\"))"}
|
||||
{:suite "regex / re-seq" :label "words" :expected "[\"foo\" \"bar\"]" :actual "(re-seq #\"\\w+\" \"foo bar\")"}
|
||||
{:suite "regex / char-class dash" :label "dash after \\w is literal" :expected "\"a_b-c\"" :actual "(re-matches #\"[\\w-_]+\" \"a_b-c\")"}
|
||||
{:suite "regex / char-class dash" :label "dash + escaped dot in class" :expected "\"a.b_c-d\"" :actual "(re-matches #\"[\\w-_\\.]+\" \"a.b_c-d\")"}
|
||||
{:suite "regex / char-class dash" :label "trailing dash in class" :expected "[\"a-b\" \"c-d\"]" :actual "(vec (re-seq #\"[\\w-]+\" \"a-b c-d\"))"}
|
||||
{:suite "regex / nested quantifier" :label "(X+)* parses and matches" :expected "true" :actual "(boolean (re-matches #\"([^%]+)*(%(d))?\" \"abc\"))"}
|
||||
{:suite "regex / re-pattern & string ops" :label "re-pattern build" :expected "\"hi\"" :actual "(re-find (re-pattern \"\\\\w+\") \"hi!\")"}
|
||||
{:suite "regex / re-pattern & string ops" :label "re-pattern is regex?" :expected "true" :actual "(regex? (re-pattern \"a\"))"}
|
||||
{:suite "regex / re-pattern & string ops" :label "split on regex" :expected "[\"a\" \"b\" \"c\"]" :actual "(do (require '[clojure.string :as s]) (s/split \"a1b2c\" #\"\\d\"))"}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue