Parsing Apache Log using Regex

Question

I want to get the following:-

Input

GET /1.1/friendships/list.json?user_id=123 HTTP/1.1
GET /1.1/friendships/list.json HTTP/1.1
GET /1.1/users/show.json?include_entities=1&user_id=321 HTTP/1.1
GET /1.1/friendships/list.json?user_id=234 HTTP/1.1
GET /1.1/friendships/create.json HTTP/1.1

Output

/1.1/friendships/list.json
/1.1/friendships/list.json
/1.1/users/show.json
/1.1/friendships/list.json
/1.1/friendships/create.json

I have been able to match till the question mark character. I want to match a character that is either a question mark or a blank space. Here is what I have so far.

([A-Z])+ (\S)+[\?]

r = re.compile(r'\w+ ([/\w\.]+)') and r.match(<string>).groups(1) — TheChetan
– TheChetan, Commented Aug 23, 2017 at 4:09

Jan · Accepted Answer · 2017-08-23 06:57:24Z

The following expression accepts GET and POST:

^(?:GET|POST)\s+([^?\n\r]+).*$

Broken down, this says

^               # start of line
(?:GET|POST)\s+ # GET or POST literally, at least one whitespace
([^?\s]+)       # not a question mark nor whitespace characters
.*              # 0+ chars afterwards
$               # end of line

This needs to be replaced by \1, see a demo on regex101.com and mind the MULTILINE flag.

In Python, this would be:

import re

string = """
GET /1.1/friendships/list.json?user_id=123 HTTP/1.1
GET /1.1/friendships/list.json HTTP/1.1
GET /1.1/users/show.json?include_entities=1&user_id=321 HTTP/1.1
GET /1.1/friendships/list.json?user_id=234 HTTP/1.1
GET /1.1/friendships/create.json HTTP/1.1
POST /some/other/url/here
"""

rx = re.compile(r'^(?:GET|POST)\s+([^?\s]+).*$', re.M)
matches = rx.findall(string)
print(matches)
# ['/1.1/friendships/list.json', '/1.1/friendships/list.json', '/1.1/users/show.json', '/1.1/friendships/list.json', '/1.1/friendships/create.json', '/some/other/url/here']

CinCout · Accepted Answer · 2017-08-23 04:09:55Z

0

This should do:

GET\s*(\S*?[\?\s])

Demo

The idea is to search for ? or (space) in a non-greedy (aka lazy) approach (denoted by *?). Group 1 then has the required captured text.

answered Aug 23, 2017 at 4:09

CinCout

9,62916 gold badges55 silver badges74 bronze badges

Collectives™ on Stack Overflow

Parsing Apache Log using Regex

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related