I need some help to complete my idea about regexes.

Introduction

There was a question about better syntax for regexes on SE, but I don't think I'd use the fluent syntax. It's surely nice for newbies, but in case of a complicated regex, you replace a line of gibberish by a whole page of slightly better gibberish. I like the approach by Martin Fowler, where a regex gets composed of smaller pieces. His solution is readable, but hand-made; he proposes a smart way to build a complicated regex instead of a class supporting it.

I'm trying to make it to a class using something like (see his example first)

final MyPattern pattern = MyPattern.builder()
.caseInsensitive()
.define("numberOfPoints", "\\d+")
.define("numberOfNights", "\\d+")
.define("hotelName", ".*")
.define(' ', "\\s+")
.build("score `numberOfPoints` for `numberOfNights` nights? at `hotelName`");

MyMatcher m = pattern.matcher("Score 400 FOR 2 nights at Minas Tirith Airport");
System.out.println(m.group("numberOfPoints")); // prints 400

where fluent syntax is used for combining regexes extended as follows:

• define named patterns and use them by enclosing in backticks
  • `name` creates a named group
    • mnemonics: shell captures the result of the command enclosed in backticks
  • `:name` creates a non-capturing group
    • mnemonics: similar to (?:...)
  • `-name` creates a backreference
    • mnemonics: the dash connects it to the previous occurrence
• redefine individual characters and use it everywhere unless quoted
  • here only some characters (e.g., ~ @#%") are allowed
    • redefining + or ( would be extremely confusing, so it's not allowed
    • redefining space to mean any spacing is very natural in the example above
    • redefining a character could make the pattern more compact, which is good unless overused
    • e.g., using something like define('#', "\\\\") for matching backslashes could make the pattern much readable
• redefine some quoted sequences like \s or \w
  • the standard definitions are not Unicode conform
  • sometimes you might have you own idea what a word or space is

The named patterns serves as a sort of local variables helping to decompose a complicated expression into small and easy to understand pieces. A proper naming pattern makes often a comment unnecessary.

Questions

The above shouldn't be hard to implement (I did already most of it) and could be really useful, I hope. Do you think so?

However, I'm not sure how it should behave inside of brackets, sometimes it's meaningful to use the definitions and sometimes not, e.g. in

.define(' ', "\\s")            // a blank character
.define('~', "/\**[^*]+\*/")   // an inline comment (simplified)
.define("something", "[ ~\\d]")

expanding the space to \s makes sense, but expanding the tilde doesn't. Maybe there should be a separate syntax to define own character classes somehow?

Can you think of some examples where the named pattern are very useful or not useful at all? I'd need some border cases and some ideas for improvement.

Reaction to tchrist's answer

Comments to his objections

1. Lack of multiline pattern strings.
   • There are no multiline strings in Java, which I'd like to change, but can not.
2. Freedom from insanely onerous and error-prone double-backslashing...
   • This is again something I can't do, I can only offer a workaround, s. below.
3. Lack of compile-time exceptions on invalid regex literals, and lack of compile-time caching of correctly compiled regex literals.
   • As regexes are just a part of the standard library and not of the language itself, there's nothing what can done here.
4. No debugging or profiling facilities.
   • I can do nothing here.
5. Lack of compliance with UTS#18.
   • This can be easily solved by redefining the corresponding patterns as I proposed. It's not perfect, since in debugger you'll see the blowed up replacements.

I looks like you don't like Java. I'd be happy to see some syntax improvements there, but there's nothing I can do about it. I'm looking for something working with current Java.

RFC 5322

Your example can be easily written using my syntax:

final MyPattern pattern = MyPattern.builder()
.define(" ", "") // ignore spaces
.useForBackslash('#') // (1): see (2)
.define("address",         "`mailbox` | `group`")
.define("WSP",             "[\u0020\u0009]")
.define("DQUOTE",          "\"")
.define("CRLF",            "\r\n")
.define("DIGIT",           "[0-9]")
.define("ALPHA",           "[A-Za-z]")
.define("NO_WS_CTL",       "[\u0001-\u0008\u000b\u000c\u000e-\u001f\u007f]") // No whitespace control
...
.define("domain_literal",  "`CFWS`? #[ (?: `FWS`? `dcontent`)* `FWS`? #] `CFWS1?") // (2): see (1)
...
.define("group",           "`display_name` : (?:`mailbox_list` | `CFWS`)? ; `CFWS`?")
.define("angle_addr",      "`CFWS`? < `addr_spec` `CFWS`?")
.define("name_addr",       "`display_name`? `angle_addr`")
.define("mailbox",         "`name_addr` | `addr_spec`")
.define("address",         "`mailbox` | `group`")
.build("`address`");

Disadvantages

While rewriting your example I encountered the following issues:

• As there are no \xdd escape sequences \udddd must be used
• Using another character instead of backslash is a bit strange
• As I prefer to write it bottom-up, I had to take your lines reverted
• Without much idea what it does, I except myself having done some errors

On the bright side: - Ignoring spaces is no problem - Comments are no problem - The readability is good

And most important: It's plain Java and uses the existing regex-engine as is.