Parsing wikiText with regex in Java

Question

Given a wikiText string such as:

{{ValueDescription
    |key=highway
    |value=secondary
    |image=Image:Meyenburg-L134.jpg
    |description=A highway linking large towns.
    |onNode=no
    |onWay=yes
    |onArea=no
    |combination=
    * {{Tag|name}}
    * {{Tag|ref}}
    |implies=
    * {{Tag|motorcar||yes}}
    }}

I'd like to parse templates ValueDescription and Tag in Java/Groovy. I tried with with regex /\{\{\s*Tag(.+)\}\}/ and it's fine (it returns |name |ref and |motorcar||yes), but /\{\{\s*ValueDescription(.+)\}\}/ doesn't work (it should return all the text above).

The expected output

Is there a way to skip nested templates in the regex?

Ideally I would rather use a simple wikiText 2 xml tool, but I couldn't find anything like that.

Thanks! Mulone

Can you please provide some sample outputs you are expecting from above input? — anubhava
– anubhava, Commented Jun 3, 2011 at 13:37

Community · Accepted Answer · 2017-05-23 12:26:58Z

4

Arbitrarily nested tags won't work since that's makes the grammar non-regular. You need something capable of dealing with a context-free grammar. ANTLR is a fine option.

edited May 23, 2017 at 12:26

CommunityBot

11 silver badge

answered Jun 3, 2011 at 13:31

Mike Samuel

121k30 gold badges230 silver badges255 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

anubhava · Accepted Answer · 2011-06-03 15:31:15Z

2

Create your regex pattern using Pattern.DOTALL option like this:

Pattern p = Pattern.compile("\\{\\{\\s*ValueDescription(.+)\\}\\}", Pattern.DOTALL);

Sample Code:

Pattern p=Pattern.compile("\\{\\{\\s*ValueDescription(.+)\\}\\}",Pattern.DOTALL);
Matcher m=p.matcher(str);
while (m.find())
   System.out.println("Matched: [" + m.group(1) + ']');

OUTPUT

Matched: [
|key=highway
|value=secondary
|image=Image:Meyenburg-L134.jpg
|description=A highway linking large towns.
|onNode=no
|onWay=yes
|onArea=no
|combination=
* {{Tag|name}}
* {{Tag|ref}}
|implies=
* {{Tag|motorcar||yes}}
]

Update

Assuming closing }} appears on a separate line for {{ValueDescription following pattern will work to capture multiple ValueDescription:

Pattern p = Pattern.compile("\\{\\{\\s*ValueDescription(.+?)\n\\}\\}", Pattern.DOTALL);

edited Jun 3, 2011 at 15:31

answered Jun 3, 2011 at 14:29

anubhava

790k67 gold badges603 silver badges671 bronze badges

4 Comments

Mulone Over a year ago

this works but if there's another '''{{ValueDescription''' block it won't stop.

anubhava Over a year ago

@Mulone: Assuming closing }} appears on a separate line for {{ValueDescription following pattern will work to capture multiple ValueDescription: Pattern p = Pattern.compile("\\{\\{\\s*ValueDescription(.+?)\n\\}\\}", Pattern.DOTALL);

Mulone Over a year ago

I don't think that that assumption is valid when reading wikitext. Is there a way to make it robust?

anubhava Over a year ago

@Mulone: Regular expressions do have limitations here, you need to have some type of pattern to match. Closing }} must be either on a separate line or be followed by some other character that we can use in the pattern above. For validating/matching a non-regular text you will eventually need a parser utility or will need to write your own parser.

Collectives™ on Stack Overflow

Parsing wikiText with regex in Java

2 Answers 2

Comments

Sample Code:

OUTPUT

Update

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Sample Code:

OUTPUT

Update

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related