Regex: accept Java-compatible char-class dash and (X+)* quantifier
irregex rejected two patterns the JVM accepts, which blocked library loads: - [\w-_] errored with bad char-set because a - after a shorthand class was read as a range start. Java reads it as a literal hyphen. Preprocess the pattern to escape such a dash. - (X+)* errored with duplicate repetition because sre-repeater? recurses through submatch, treating a quantified group like a dangling a**. Override it to a bare leading * / + check, matching the JVM (which only rejects the dangling case). Both in regex.ss (runtime). Unblocks cuerdas (was load-fail, now 292 passing) and aws-api config-test. Also documents the host/chez/java source-layering rule in host-interop.md. jolt-l8so
This commit is contained in:
parent
687dc60af6
commit
8a877662dc
3 changed files with 74 additions and 1 deletions
|
|
@ -19,6 +19,36 @@ reflection and no class hierarchy. `(class x)` returns the JVM class name for th
|
||||||
scalar/collection types Clojure programs compare against (`"java.lang.Long"`,
|
scalar/collection types Clojure programs compare against (`"java.lang.Long"`,
|
||||||
`"java.lang.String"`, and so on).
|
`"java.lang.String"`, and so on).
|
||||||
|
|
||||||
|
## Source layering: JVM-specific code lives in the java layer
|
||||||
|
|
||||||
|
Keep anything JVM-specific in `host/chez/java/`. The rest of the runtime stays
|
||||||
|
JVM-free, and the compiler in `jolt-core/` is JVM-free by construction.
|
||||||
|
|
||||||
|
- `host/chez/java/` holds the JVM model: the `java.*` mirrors, the class tokens
|
||||||
|
and class hierarchy, `(class x)`/`(type x)`/`instance?`, exception classes, the
|
||||||
|
interop dispatch for `.method`/`Class/static`/`(Class.)`. If a value or name
|
||||||
|
only means something because the JVM has it, it belongs here.
|
||||||
|
- The rest of `host/chez/` is the host-neutral runtime — the value model
|
||||||
|
(`values.ss`, `collections.ss`, `seq.ss`), reader, vars, multimethods, meta. It
|
||||||
|
speaks jolt's own taxonomy (`:string`, `:vector`, `:jolt/inst`), never JVM class
|
||||||
|
names.
|
||||||
|
- `jolt-core/` (the Clojure compiler + `clojure.core` overlay) emits and reasons
|
||||||
|
in that taxonomy only. The JVM mapping happens *after*, in the java layer.
|
||||||
|
|
||||||
|
The worked example is `type`. The core layer (`natives-meta.ss`) computes the
|
||||||
|
keyword taxonomy and binds it as `__type-tag` — that's what `print-method` and the
|
||||||
|
reader dispatch on, with no JVM in scope. The java layer (`java/host-class.ss`)
|
||||||
|
then rebinds the public `clojure.core/type` to Clojure's `(or (:type meta) (class
|
||||||
|
x))`, mapping `:jolt/inst` → `java.util.Date` and so on, right next to `(class
|
||||||
|
…)`. So the compiler keeps emitting `:jolt/inst`; the java layer remaps it.
|
||||||
|
|
||||||
|
When you add interop behaviour, prefer registering it through the generic hooks a
|
||||||
|
java-layer file already uses — `register-class-arm!` for `(class x)`,
|
||||||
|
`register-instance-check-arm!` for `instance?`, `register-eq-arm!` for value
|
||||||
|
equality — rather than threading a JVM concept back into a host-neutral file. A
|
||||||
|
new `java.*` shim is a new file under `host/chez/java/` loaded from `rt.ss`, not a
|
||||||
|
branch added to `collections.ss` or `seq.ss`.
|
||||||
|
|
||||||
## What's shimmed
|
## What's shimmed
|
||||||
|
|
||||||
This is the surface today, not the whole JVM. Methods not listed generally
|
This is the surface today, not the whole JVM. Methods not listed generally
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,14 @@
|
||||||
(apply %chez-error args)))
|
(apply %chez-error args)))
|
||||||
(load "vendor/irregex/irregex.scm")
|
(load "vendor/irregex/irregex.scm")
|
||||||
|
|
||||||
|
;; irregex rejects a quantifier applied to anything that already contains one —
|
||||||
|
;; including a GROUP like (a+)* — because sre-repeater? recurses through submatch.
|
||||||
|
;; Java only rejects a DANGLING double quantifier (a**); it allows a quantifier on
|
||||||
|
;; a group whose body is quantified. Restrict the check to a bare leading * / + so
|
||||||
|
;; a** still errors but (a+)* parses (cuerdas's format tokenizer needs this).
|
||||||
|
(set! sre-repeater?
|
||||||
|
(lambda (sre) (and (pair? sre) (memq (car sre) '(* +)) #t)))
|
||||||
|
|
||||||
;; Unicode property classes \p{...}: irregex's string syntax has no
|
;; Unicode property classes \p{...}: irregex's string syntax has no
|
||||||
;; \p{...}, so translate a fixed set of property names
|
;; \p{...}, so translate a fixed set of property names
|
||||||
;; to ASCII char classes before compiling. ASCII-only — \p{L} would need
|
;; to ASCII char classes before compiling. ASCII-only — \p{L} would need
|
||||||
|
|
@ -92,6 +100,36 @@
|
||||||
(write-char c out) (loop (fx+ i 1) #f))
|
(write-char c out) (loop (fx+ i 1) #f))
|
||||||
(else (write-char c out) (loop (fx+ i 1) in-class))))))))
|
(else (write-char c out) (loop (fx+ i 1) in-class))))))))
|
||||||
|
|
||||||
|
;; Inside a [...] class, irregex reads a '-' that follows a shorthand class
|
||||||
|
;; (\w \d \s \W \D \S) as the start of a range and errors ("bad char-set"); Java
|
||||||
|
;; reads it as a literal hyphen (a shorthand can't be a range endpoint). Escape
|
||||||
|
;; such a '-' to \- so the class parses. Only a '-' right after a shorthand and
|
||||||
|
;; not the class terminator is touched; a '-' after a plain char (a real range
|
||||||
|
;; like [a-z]) is left alone.
|
||||||
|
(define (escape-class-shorthand-dash src)
|
||||||
|
(let ((len (string-length src)) (out (open-output-string)))
|
||||||
|
(let loop ((i 0) (in-class #f) (after-shorthand #f))
|
||||||
|
(if (fx>=? i len)
|
||||||
|
(get-output-string out)
|
||||||
|
(let ((c (string-ref src i)))
|
||||||
|
(cond
|
||||||
|
;; an escape pair: \w-style shorthand sets after-shorthand inside a class
|
||||||
|
((and (char=? c #\\) (fx<? (fx+ i 1) len))
|
||||||
|
(let ((n (string-ref src (fx+ i 1))))
|
||||||
|
(write-char c out) (write-char n out)
|
||||||
|
(loop (fx+ i 2) in-class
|
||||||
|
(and in-class (memv n '(#\w #\d #\s #\W #\D #\S)) #t))))
|
||||||
|
((and (not in-class) (char=? c #\[))
|
||||||
|
(write-char c out) (loop (fx+ i 1) #t #f))
|
||||||
|
((and in-class (char=? c #\]))
|
||||||
|
(write-char c out) (loop (fx+ i 1) #f #f))
|
||||||
|
;; the case Java reads as a literal hyphen
|
||||||
|
((and in-class after-shorthand (char=? c #\-)
|
||||||
|
(fx<? (fx+ i 1) len) (not (char=? (string-ref src (fx+ i 1)) #\])))
|
||||||
|
(write-char #\\ out) (write-char #\- out)
|
||||||
|
(loop (fx+ i 1) in-class #f))
|
||||||
|
(else (write-char c out) (loop (fx+ i 1) in-class #f))))))))
|
||||||
|
|
||||||
;; Java/Clojure inline flags: a leading (?imsx…) group sets a flag over the whole
|
;; Java/Clojure inline flags: a leading (?imsx…) group sets a flag over the whole
|
||||||
;; pattern. irregex has the same semantics but as constructor OPTIONS, not inline
|
;; pattern. irregex has the same semantics but as constructor OPTIONS, not inline
|
||||||
;; syntax (it rejects (?s)/(?s:…)), so peel any leading flag groups off the source
|
;; syntax (it rejects (?s)/(?s:…)), so peel any leading flag groups off the source
|
||||||
|
|
@ -123,7 +161,8 @@
|
||||||
(define-record-type regex-t (fields source irx) (nongenerative jolt-regex-v1))
|
(define-record-type regex-t (fields source irx) (nongenerative jolt-regex-v1))
|
||||||
(define (jolt-regex source)
|
(define (jolt-regex source)
|
||||||
(let-values (((opts pat) (regex-parse-flags source)))
|
(let-values (((opts pat) (regex-parse-flags source)))
|
||||||
(make-regex-t source (apply irregex (translate-prop-classes pat) opts))))
|
(make-regex-t source
|
||||||
|
(apply irregex (translate-prop-classes (escape-class-shorthand-dash pat)) opts))))
|
||||||
(define (jolt-regex? x) (regex-t? x))
|
(define (jolt-regex? x) (regex-t? x))
|
||||||
(define (jolt-re-pattern x) (if (regex-t? x) x (jolt-regex x)))
|
(define (jolt-re-pattern x) (if (regex-t? x) x (jolt-regex x)))
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1823,6 +1823,10 @@
|
||||||
{:suite "regex / re-seq" :label "all matches" :expected "[\"1\" \"22\" \"333\"]" :actual "(re-seq #\"\\d+\" \"a1b22c333\")"}
|
{:suite "regex / re-seq" :label "all matches" :expected "[\"1\" \"22\" \"333\"]" :actual "(re-seq #\"\\d+\" \"a1b22c333\")"}
|
||||||
{:suite "regex / re-seq" :label "empty when none" :expected "nil" :actual "(seq (re-seq #\"z\" \"abc\"))"}
|
{:suite "regex / re-seq" :label "empty when none" :expected "nil" :actual "(seq (re-seq #\"z\" \"abc\"))"}
|
||||||
{:suite "regex / re-seq" :label "words" :expected "[\"foo\" \"bar\"]" :actual "(re-seq #\"\\w+\" \"foo bar\")"}
|
{:suite "regex / re-seq" :label "words" :expected "[\"foo\" \"bar\"]" :actual "(re-seq #\"\\w+\" \"foo bar\")"}
|
||||||
|
{:suite "regex / char-class dash" :label "dash after \\w is literal" :expected "\"a_b-c\"" :actual "(re-matches #\"[\\w-_]+\" \"a_b-c\")"}
|
||||||
|
{:suite "regex / char-class dash" :label "dash + escaped dot in class" :expected "\"a.b_c-d\"" :actual "(re-matches #\"[\\w-_\\.]+\" \"a.b_c-d\")"}
|
||||||
|
{:suite "regex / char-class dash" :label "trailing dash in class" :expected "[\"a-b\" \"c-d\"]" :actual "(vec (re-seq #\"[\\w-]+\" \"a-b c-d\"))"}
|
||||||
|
{:suite "regex / nested quantifier" :label "(X+)* parses and matches" :expected "true" :actual "(boolean (re-matches #\"([^%]+)*(%(d))?\" \"abc\"))"}
|
||||||
{:suite "regex / re-pattern & string ops" :label "re-pattern build" :expected "\"hi\"" :actual "(re-find (re-pattern \"\\\\w+\") \"hi!\")"}
|
{:suite "regex / re-pattern & string ops" :label "re-pattern build" :expected "\"hi\"" :actual "(re-find (re-pattern \"\\\\w+\") \"hi!\")"}
|
||||||
{:suite "regex / re-pattern & string ops" :label "re-pattern is regex?" :expected "true" :actual "(regex? (re-pattern \"a\"))"}
|
{:suite "regex / re-pattern & string ops" :label "re-pattern is regex?" :expected "true" :actual "(regex? (re-pattern \"a\"))"}
|
||||||
{:suite "regex / re-pattern & string ops" :label "split on regex" :expected "[\"a\" \"b\" \"c\"]" :actual "(do (require '[clojure.string :as s]) (s/split \"a1b2c\" #\"\\d\"))"}
|
{:suite "regex / re-pattern & string ops" :label "split on regex" :expected "[\"a\" \"b\" \"c\"]" :actual "(do (require '[clojure.string :as s]) (s/split \"a1b2c\" #\"\\d\"))"}
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue