Use RegEx in Python to extract URL and optional query string from web server log data

Question

Disclosure: very much a regex newbie, so I'm trying to tweak some example code I found which parses web server log data into named groups. The snippet of my modified regex thus far that deals with the URL and query string groups:

(?P<url>.+)(?P<querystr>\?.*)

This works just fine when the string against which it's applied actually does have a query string on the URL (each group gets the expected bit of the string) but fails to match if there is none. So I tried adding a '?' after the "querystr" group to indicate that it was optional, i.e. (?P<querystr>\?.*)? ... if there's no query string then it works as expected (nothing is extracted into querystr), but when there is one, it is still extracted as part of url rather than separately into querystr.

What's the best way to identify optional groups (assuming that's even the right approach in this case)? Thanks in advance.

Use ^(?P<url>.+?)(?P<querystr>\?.*)?$ or ^(?P<url>[^?]+)(?P<querystr>\?.*)?$ — Wiktor Stribiżew
– Wiktor Stribiżew, Commented Jul 19, 2021 at 20:10
Replacing the '.' in the url group with [^?] along with adding the '?' after the querystr group seemed to do the trick. Thanks! — ShawnC
– ShawnC, Commented Jul 19, 2021 at 20:59

Wiktor Stribiżew · Accepted Answer · 2021-07-19 21:07:34Z

1

You can use

^(?P<url>[^?]+)(?P<querystr>\?.*)?$

Details

^ - start of string
(?P<url>[^?]+) - Group "url": any one or more chars other than ?
(?P<querystr>\?.*)? - an optional Group "querystr": a ? char and then any zero or more chars other than line break chars as many as possible
$ - end of string.

See the regex demo.

answered Jul 19, 2021 at 21:07

Wiktor Stribiżew

631k41 gold badges502 silver badges632 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Use RegEx in Python to extract URL and optional query string from web server log data

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related