Tokens stick when splitting a String in Java?

Question

I'm trying to split a test String, "1 + 2 = 3 += 4 + --5" into its components without relying on spaces. I want the end result to be { 1, +, 2, =, 3, +=, 4, +, --, 5 } however some tokens seem to stick together. I wrote the following Regex to split the String:

"(?<=(\\.)|(\\w))\\s*(?=[O])|(?<=[O])\\s*(?=(\\.)|(\\w))"

and then used the ReplaceAll function to replace "O" with the following, which are my operators that I want to split on:

"(\\\\+)|(\\\\=)|(\\\\+=)|(\\\\-)"

However when applying this regex to splitting the String I provided as an example, I get the following result: { 1, +, 2, =, 3, +=, 4, +--, 5 }. Why do the minuses stick to the plus in the 2nd to last token? Is there anyway to fix this and make the split tokens appear as { 1, +, 2, =, 3, +=, 4, +, --, 5 }?

I'd urge against parsing with a regex.

Dave Newton
– Dave Newton

2015-04-19 01:25:52 +00:00
Commented Apr 19, 2015 at 1:25 — Dave Newton
– Dave Newton, Commented Apr 19, 2015 at 1:25

Avinash Raj · Accepted Answer · 2015-04-19 01:24:48Z

1

You could do matching instead of splitting.

String a = "1 + 2 = 3 += 4 +--5";
Matcher m = Pattern.compile("\\d+|[^\\w\\s]+").matcher(a);
ArrayList<String> list = new ArrayList<String>();
while (m.find()) {
    list.add(m.group());
}
System.out.println(list);

Output:

[1, +, 2, =, 3, +=, 4, +--, 5]

edited Apr 19, 2015 at 1:24

answered Apr 19, 2015 at 1:14

Avinash Raj

175k32 gold badges247 silver badges289 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Thomas Paine Over a year ago

Sorry I made a mistake in my code. But this doesn't seem to work if the input string is "1 + 2 = 3 += 4 +--5". Is there a regex that does not rely on spaces?

score 0 · Accepted Answer · 2015-04-19 01:47:31Z

0

Try this:

String input = "1 + 2 = 3 += 4 + --5";
//StringTokenizer stringTokenizer = new StringTokenizer(input, " ");
StringTokenizer stringTokenizer = new StringTokenizer(input, "1234567890", true);

StringBuilder builder = new StringBuilder("[");

while (stringTokenizer.hasMoreElements()) {
  //builder.append(stringTokenizer.nextElement());
  builder.append(stringTokenizer.nextElement().toString().trim());
  builder.append(stringTokenizer.hasMoreTokens() ? "," : "]");
}
System.out.printf("Using the java.util.StringTokenizer: %s%n", builder);

OUTPUT:

Using the java.util.StringTokenizer: [1, +, 2, =, 3, +=, 4, +, --5]

edited Apr 19, 2015 at 1:47

answered Apr 19, 2015 at 1:17

user4695271

2 Comments

Thomas Paine Over a year ago

This works but it relies on spaces... Any way without spaces?

user4695271 Over a year ago

Use all the numbers as delim and pass true as returnDelims. That will make the trick also; you can have spaces or not

Collectives™ on Stack Overflow

Tokens stick when splitting a String in Java?

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related