Python parse web access logs

Question

I'm trying to parse a specific data from access logs, the logs format vary, it could be from nginx or apache. I need to get the following data:

Remote host IP
Request datetime
Request type {GET|POST|PUT|..etc}
Request Path {/main/index.html etc}
HTTP Version {HTTP 1.1 | HTTP 1.0}
HTTP Response code {200|400|403 ...etc}

I tried using split but it does not always work because of the log formats not always the same:

sample = """
::1 - - [03/Jan/2018:21:28:49 +0100] "GET /moodle/course/view.php?id=19 HTTP/1.1" 200 78325 "http://localhost/moodle/login/index.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0"
83.198.250.175 - - [22/Mar/2009:07:40:06 +0100] "GET /style.css HTTP/1.1" 200 1692 "http://www.example.org/" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Wanadoo 6.7; Orange 8.0)" "-"
212.31.110.34 0.597 - [16/May/2018:12:30:44 +0000] safefin.example.com "GET / HTTP/1.1" 200 18193 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36"
151.227.152.48 - - [02/Jul/2014:14:35:55 +0100] "GET /css/main.css HTTP/1.1" 200 4658 "http://example.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
109.169.248.247 - - [12/Dec/2015:18:25:11 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://example.net/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
80.91.33.133 - - [17/May/2015:08:05:24 +0000] "GET /downloads/product_1 HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.17)"
217.168.17.5 - - [17/May/2015:08:05:34 +0000] "GET /downloads/product_1 HTTP/1.1" 200 490 "-" "Debian APT-HTTP/1.3 (0.8.10.3)"
192.168.0.11 - - [27/Jun/2016:18:36:14 -0500] "GET / HTTP/1.1" 302 - "-" "Mozilla/5.0 (Linux; Android 5.1.1; SM-N910T Build/LMY47X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.81 Mobile Safari/537.36"
51.68.152.26 - - [09/Apr/2019:01:37:30 +0400] "GET / HTTP/1.1" 302 0 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
71.169.154.24 - - [01/Mar/2015:20:58:55 -0500] "GET /BarHarborcemeteries/Burns-RichardsonCemeteryimages/general%20view%20(2008).jpg HTTP/1.1" 200 165457 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/600.3.18 (KHTML, like Gecko) Version/7.1.3 Safari/537.85.12"
94.90.115.82 - - [02/Apr/2012:04:56:17 +0900] "GET /manager/html HTTP/1.1" 404 77 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0"
172.20.32.1 - - [25/Feb/2015:10:42:29 +0300] "PUT /putfile?partNumber=5&uploadId=2/fFEtO5aTFYNO7tjxbbmw6QkGOmeeOFt HTTP/1.1" 200 - "-" "-"
172.20.32.1 - - [25/Feb/2015:10:42:32 +0300] "POST /putfile?uploadId=2/fFEtO5aTFYNO7tjxbbmw6QkGOmeeOFt HTTP/1.1" 200 279 "-" "-"
172.20.32.1 - - [25/Feb/2015:10:43:04 +0300] "DELETE /putfile HTTP/1.1" 400 81 "-" "-"
172.20.32.1 - - [25/Feb/2015:10:43:04 +0300] "DELETE /putfile HTTP/1.1" 204 - "-" "-"
172.20.32.1 - - [25/Feb/2015:10:41:02 +0300] "POST /putfile?uploads HTTP/1.1" 200 242 "-" "-"
151.227.152.48 - - [02/Jul/2014:14:35:56 +0100] "GET /img/Customers/Absolute-Steel-Framing.gif HTTP/1.1" 200 10123 "http://example.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
159.226.202.17 - - [31/Aug/2010:23:45:30 +0100] "GET / HTTP/1.1" 403 323 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; iCafeMedia; .NET CLR 2.0.50727; CIBA)"
65.55.3.169 - - [01/Sep/2010:08:03:47 +0100] "GET /robots.txt HTTP/1.1" 403 272 "-" "msnbot/2.0b (+http://search.example.com/msnbot.htm)._"
66.187.104.20 - - [24/Apr/2009:19:15:52 +1100] "GET /misc/arrow-desc.png HTTP/1.1" 404 217
77.35.168.108 - - [28/Apr/2009:10:38:09 +1100] "GET / HTTP/1.1" 200 85
77.35.172.105 - - [28/Apr/2009:12:49:27 +1100] "GET / HTTP/1.1" 304 -
79.137.201.45 - - [02/May/2009:12:17:26 +1100] "GET /robots.txt HTTP/1.0" 404 208
151.21.4.47 - - [17/Feb/2018:16:06:48 +0100] "GET /noindex/css/open-sans.css HTTP/1.1" 200 5081 "http://94.177.222.96/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0"
151.21.4.47 - - [17/Feb/2018:16:06:48 +0100] "GET /images/apache_pb.gif HTTP/1.1" 200 2326 "http://94.177.222.96/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0"
"""

lines = sample.split('\n')
structured_data = []
for line in lines:
    if line == '':
        continue

    parts = line.split(' ')

    remote_host = parts[0]    # IP
    time_date = parts[3]      # Log datetime
    request_method = parts[5] # GET OR POST..
    request_path = parts[6]   # Requested resource
    http_version = parts[7]
    response_status_code = parts[8]

    structured_dict = {'remote_host': remote_host,
                       'datetime': time_date,
                       'method': request_method,
                       'path': request_path,
                       'http_version': http_version,
                       'response_code': response_status_code
                       }
    structured_data.append(structured_dict)

for dict in structured_data:
    print(dict)

sample Output:

{'remote_host': '::1', 'datetime': '[03/Jan/2018:21:28:49', 'method': '"GET', 'path': '/moodle/course/view.php?id=19', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '83.198.250.175', 'datetime': '[22/Mar/2009:07:40:06', 'method': '"GET', 'path': '/style.css', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '212.31.110.34', 'datetime': '[16/May/2018:12:30:44', 'method': 'safefin.example.com', 'path': '"GET', 'http_version': '/', 'response_code': 'HTTP/1.1"'}
{'remote_host': '151.227.152.48', 'datetime': '[02/Jul/2014:14:35:55', 'method': '"GET', 'path': '/css/main.css', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '109.169.248.247', 'datetime': '[12/Dec/2015:18:25:11', 'method': '"POST', 'path': '/administrator/index.php', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '80.91.33.133', 'datetime': '[17/May/2015:08:05:24', 'method': '"GET', 'path': '/downloads/product_1', 'http_version': 'HTTP/1.1"', 'response_code': '304'}
{'remote_host': '217.168.17.5', 'datetime': '[17/May/2015:08:05:34', 'method': '"GET', 'path': '/downloads/product_1', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '192.168.0.11', 'datetime': '[27/Jun/2016:18:36:14', 'method': '"GET', 'path': '/', 'http_version': 'HTTP/1.1"', 'response_code': '302'}
{'remote_host': '51.68.152.26', 'datetime': '[09/Apr/2019:01:37:30', 'method': '"GET', 'path': '/', 'http_version': 'HTTP/1.1"', 'response_code': '302'}
{'remote_host': '71.169.154.24', 'datetime': '[01/Mar/2015:20:58:55', 'method': '"GET', 'path': '/BarHarborcemeteries/Burns-RichardsonCemeteryimages/general%20view%20(2008).jpg', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '94.90.115.82', 'datetime': '[02/Apr/2012:04:56:17', 'method': '"GET', 'path': '/manager/html', 'http_version': 'HTTP/1.1"', 'response_code': '404'}
{'remote_host': '172.20.32.1', 'datetime': '[25/Feb/2015:10:42:29', 'method': '"PUT', 'path': '/putfile?partNumber=5&uploadId=2/fFEtO5aTFYNO7tjxbbmw6QkGOmeeOFt', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '172.20.32.1', 'datetime': '[25/Feb/2015:10:42:32', 'method': '"POST', 'path': '/putfile?uploadId=2/fFEtO5aTFYNO7tjxbbmw6QkGOmeeOFt', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '172.20.32.1', 'datetime': '[25/Feb/2015:10:43:04', 'method': '"DELETE', 'path': '/putfile', 'http_version': 'HTTP/1.1"', 'response_code': '400'}
{'remote_host': '172.20.32.1', 'datetime': '[25/Feb/2015:10:43:04', 'method': '"DELETE', 'path': '/putfile', 'http_version': 'HTTP/1.1"', 'response_code': '204'}
{'remote_host': '172.20.32.1', 'datetime': '[25/Feb/2015:10:41:02', 'method': '"POST', 'path': '/putfile?uploads', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '151.227.152.48', 'datetime': '[02/Jul/2014:14:35:56', 'method': '"GET', 'path': '/img/Customers/Absolute-Steel-Framing.gif', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '159.226.202.17', 'datetime': '[31/Aug/2010:23:45:30', 'method': '"GET', 'path': '/', 'http_version': 'HTTP/1.1"', 'response_code': '403'}
{'remote_host': '65.55.3.169', 'datetime': '[01/Sep/2010:08:03:47', 'method': '"GET', 'path': '/robots.txt', 'http_version': 'HTTP/1.1"', 'response_code': '403'}
{'remote_host': '66.187.104.20', 'datetime': '[24/Apr/2009:19:15:52', 'method': '"GET', 'path': '/misc/arrow-desc.png', 'http_version': 'HTTP/1.1"', 'response_code': '404'}
{'remote_host': '77.35.168.108', 'datetime': '[28/Apr/2009:10:38:09', 'method': '"GET', 'path': '/', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '77.35.172.105', 'datetime': '[28/Apr/2009:12:49:27', 'method': '"GET', 'path': '/', 'http_version': 'HTTP/1.1"', 'response_code': '304'}
{'remote_host': '79.137.201.45', 'datetime': '[02/May/2009:12:17:26', 'method': '"GET', 'path': '/robots.txt', 'http_version': 'HTTP/1.0"', 'response_code': '404'}
{'remote_host': '151.21.4.47', 'datetime': '[17/Feb/2018:16:06:48', 'method': '"GET', 'path': '/noindex/css/open-sans.css', 'http_version': 'HTTP/1.1"', 'response_code': '200'}
{'remote_host': '151.21.4.47', 'datetime': '[17/Feb/2018:16:06:48', 'method': '"GET', 'path': '/images/apache_pb.gif', 'http_version': 'HTTP/1.1"', 'response_code': '200'}

I collected samples from various of places, Most of formats looks same the samples above.

"because of the log formats not always the same". What do the log formats look like then? It's hard to parse them if you don't know how they should look. If you don't know the format, then the best you can do is what you have above, but do error checking (i.e. after reading remote host, make sure it is valid IP, make sure datetime is valid, etc.). — Tyler Marshall
– Tyler Marshall, Commented Apr 9, 2019 at 21:21
I collected samples from various of places, Most of formats looks same the samples above — Salama
– Salama, Commented Apr 9, 2019 at 21:25
Hmm. Well as long as each source uses the same format, you can parse the logs from each source and convert them into a nice format like you have above and then just concatenate all of the data together. This problem becomes much harder if you try to do everything in one go because you have to try to figure out which format the line is in as you are trying to parse it. — Tyler Marshall
– Tyler Marshall, Commented Apr 9, 2019 at 21:33
I'm thinking that parsing with regex is better but I'm not good in regex :( — Salama
– Salama, Commented Apr 9, 2019 at 21:34
As you can see the third line in output is not correct, because the index of data not the same — Salama
– Salama, Commented Apr 9, 2019 at 21:37

Tyler Marshall · Accepted Answer · 2019-04-09 22:08:49Z

If you want to use regexes to parse the log, here are a few that could be helpful:

Capturing IP addresses is a little tougher. If you want to check it is a valid ip address, try this. Otherwise, if you want 4 groups of numbers that are at most 3 digits long separated by dots:

\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}

For the datetime, it looks like you can grab the first occurrence of characters surrounded by square brackets

\[([^\]]+)\]

For the method, path, and response, it looks like you can grab the first occurrence of characters surrounded by quotes and then the number directly after it

"([^"]+)"\s+(\d{1,3})

because multiple things are matched here, you can make use of groups to grab the individual pieces. Using this regex, you would take the first group and simply take off the "GET, POST, DELETE, etc" and what's remaining would be the path.

Use python's re library and apply each regex to a line from your input and see what you get

#!/usr/bin/env python
import re

bad_ip_regex = re.compile("\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}")
datetime_regex = re.compile("\[([^\]]+)\]")
other_regex = re.compile('"([^"]+)"\s+(\d{1,3})')
with open("input.log", "r") as f:
  for line in f:
    item = {}

    # attempt to grab IP
    ip = bad_ip_regex.search(line)
    if ip:
      item["remote_host"] = ip.group(0)
    else:
      # no ip, just skip?
      continue

    # attempt to grab datetime
    datetime = datetime_regex.search(line)
    if datetime:
      item["datetime"] = datetime.group(1)
    else:
      continue

    # attempt to grab other
    other = other_regex.search(line)
    if other:
      item["method"] = other.group(1).split()[0]
      item["path"] = other.group(1).split()[1]
      item["response"] = other.group(2)
    else:
      continue

    print(item)

Because you can't guarantee the order of these items, it doesn't make sense to try to grab all the fields at once using regular expressions. Just try one at a time on each line.

FailSafe · Accepted Answer · 2019-04-10 01:56:50Z

Hmmm... Your directions were kind of misleading but fortunately, I'd done something like this not too long ago so I just adapted some dirty code which you can use. Keep in mind that in Python dictionaries don't default to being displayed in any particular order.

But the code below should accomplish what you need and uses a single regex

>>> sample = '''
::1 - - [03/Jan/2018:21:28:49 +0100] "GET /moodle/course/view.php?id=19 HTTP/1.1" 200 78325 "http://localhost/moodle/login/index.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0"
83.198.250.175 - - [22/Mar/2009:07:40:06 +0100] "GET /style.css HTTP/1.1" 200 1692 "http://www.example.org/" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Wanadoo 6.7; Orange 8.0)" "-"
212.31.110.34 0.597 - [16/May/2018:12:30:44 +0000] safefin.example.com "GET / HTTP/1.1" 200 18193 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36"
151.227.152.48 - - [02/Jul/2014:14:35:55 +0100] "GET /css/main.css HTTP/1.1" 200 4658 "http://example.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
109.169.248.247 - - [12/Dec/2015:18:25:11 +0100] "POST /administrator/index.php HTTP/1.1" 200 4494 "http://example.net/administrator/" "Mozilla/5.0 (Windows NT 6.0; rv:34.0) Gecko/20100101 Firefox/34.0" "-"
80.91.33.133 - - [17/May/2015:08:05:24 +0000] "GET /downloads/product_1 HTTP/1.1" 304 0 "-" "Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.17)"
217.168.17.5 - - [17/May/2015:08:05:34 +0000] "GET /downloads/product_1 HTTP/1.1" 200 490 "-" "Debian APT-HTTP/1.3 (0.8.10.3)"
192.168.0.11 - - [27/Jun/2016:18:36:14 -0500] "GET / HTTP/1.1" 302 - "-" "Mozilla/5.0 (Linux; Android 5.1.1; SM-N910T Build/LMY47X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.81 Mobile Safari/537.36"
51.68.152.26 - - [09/Apr/2019:01:37:30 +0400] "GET / HTTP/1.1" 302 0 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
71.169.154.24 - - [01/Mar/2015:20:58:55 -0500] "GET /BarHarborcemeteries/Burns-RichardsonCemeteryimages/general%20view%20(2008).jpg HTTP/1.1" 200 165457 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/600.3.18 (KHTML, like Gecko) Version/7.1.3 Safari/537.85.12"
94.90.115.82 - - [02/Apr/2012:04:56:17 +0900] "GET /manager/html HTTP/1.1" 404 77 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0"
172.20.32.1 - - [25/Feb/2015:10:42:29 +0300] "PUT /putfile?partNumber=5&uploadId=2/fFEtO5aTFYNO7tjxbbmw6QkGOmeeOFt HTTP/1.1" 200 - "-" "-"
172.20.32.1 - - [25/Feb/2015:10:42:32 +0300] "POST /putfile?uploadId=2/fFEtO5aTFYNO7tjxbbmw6QkGOmeeOFt HTTP/1.1" 200 279 "-" "-"
172.20.32.1 - - [25/Feb/2015:10:43:04 +0300] "DELETE /putfile HTTP/1.1" 400 81 "-" "-"
172.20.32.1 - - [25/Feb/2015:10:43:04 +0300] "DELETE /putfile HTTP/1.1" 204 - "-" "-"
172.20.32.1 - - [25/Feb/2015:10:41:02 +0300] "POST /putfile?uploads HTTP/1.1" 200 242 "-" "-"
151.227.152.48 - - [02/Jul/2014:14:35:56 +0100] "GET /img/Customers/Absolute-Steel-Framing.gif HTTP/1.1" 200 10123 "http://example.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
159.226.202.17 - - [31/Aug/2010:23:45:30 +0100] "GET / HTTP/1.1" 403 323 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; iCafeMedia; .NET CLR 2.0.50727; CIBA)"
65.55.3.169 - - [01/Sep/2010:08:03:47 +0100] "GET /robots.txt HTTP/1.1" 403 272 "-" "msnbot/2.0b (+http://search.example.com/msnbot.htm)._"
66.187.104.20 - - [24/Apr/2009:19:15:52 +1100] "GET /misc/arrow-desc.png HTTP/1.1" 404 217
77.35.168.108 - - [28/Apr/2009:10:38:09 +1100] "GET / HTTP/1.1" 200 85
77.35.172.105 - - [28/Apr/2009:12:49:27 +1100] "GET / HTTP/1.1" 304 -
79.137.201.45 - - [02/May/2009:12:17:26 +1100] "GET /robots.txt HTTP/1.0" 404 208
151.21.4.47 - - [17/Feb/2018:16:06:48 +0100] "GET /noindex/css/open-sans.css HTTP/1.1" 200 5081 "http://94.177.222.96/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0"
151.21.4.47 - - [17/Feb/2018:16:06:48 +0100] "GET /images/apache_pb.gif HTTP/1.1" 200 2326 "http://94.177.222.96/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0"'''




>>> def text_to_dict(string):
        import re
        dict_array = []
        found_items = re.findall('(?m)^((?:[\d]{1,3}\.){3}[\d]{1,3}|[\d]*[:]*[\d]*)[\S\ ]*?\[([\S\ ]*?)\][\S\ ]*?\"([A-Z]+)[\S\ ]*?(/(?=[\s]+)|/[\s]*[\S]+)[\S\ ]*?(HTTP[\S]*?)\"[\S\ ]*?([\d]{3}(?=\s|$))', string)
        for i in range(len(found_items)):
            try:
                dict = {"remote_host":found_items[i][0], "datetime":found_items[i][1], "method":found_items[i][2], "path":found_items[i][3],"http_version":found_items[i][4], "response_code":found_items[i][5]}
                dict_array.append(dict)
            except:
                print('\n\n================Failed')
                print(found_items[i])
        return dict_array




>>> found_items = text_to_dict(sample)



>>> for elements in found_items:
        print(elements)





 #OUTPUT
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '03/Jan/2018:21:28:49 +0100', 'path': '/moodle/course/view.php?id=19', 'remote_host': '::1', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '22/Mar/2009:07:40:06 +0100', 'path': '/style.css', 'remote_host': '83.198.250.175', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '16/May/2018:12:30:44 +0000', 'path': '/', 'remote_host': '212.31.110.34', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '02/Jul/2014:14:35:55 +0100', 'path': '/css/main.css', 'remote_host': '151.227.152.48', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '12/Dec/2015:18:25:11 +0100', 'path': '/administrator/index.php', 'remote_host': '109.169.248.247', 'method': 'POST'}
 {'http_version': 'HTTP/1.1', 'response_code': '304', 'datetime': '17/May/2015:08:05:24 +0000', 'path': '/downloads/product_1', 'remote_host': '80.91.33.133', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '17/May/2015:08:05:34 +0000', 'path': '/downloads/product_1', 'remote_host': '217.168.17.5', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '302', 'datetime': '27/Jun/2016:18:36:14 -0500', 'path': '/', 'remote_host': '192.168.0.11', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '302', 'datetime': '09/Apr/2019:01:37:30 +0400', 'path': '/', 'remote_host': '51.68.152.26', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '01/Mar/2015:20:58:55 -0500', 'path': '/BarHarborcemeteries/Burns-RichardsonCemeteryimages/general%20view%20(2008).jpg', 'remote_host': '71.169.154.24', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '404', 'datetime': '02/Apr/2012:04:56:17 +0900', 'path': '/manager/html', 'remote_host': '94.90.115.82', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '25/Feb/2015:10:42:29 +0300', 'path': '/putfile?partNumber=5&uploadId=2/fFEtO5aTFYNO7tjxbbmw6QkGOmeeOFt', 'remote_host': '172.20.32.1', 'method': 'PUT'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '25/Feb/2015:10:42:32 +0300', 'path': '/putfile?uploadId=2/fFEtO5aTFYNO7tjxbbmw6QkGOmeeOFt', 'remote_host': '172.20.32.1', 'method': 'POST'}
 {'http_version': 'HTTP/1.1', 'response_code': '400', 'datetime': '25/Feb/2015:10:43:04 +0300', 'path': '/putfile', 'remote_host': '172.20.32.1', 'method': 'DELETE'}
 {'http_version': 'HTTP/1.1', 'response_code': '204', 'datetime': '25/Feb/2015:10:43:04 +0300', 'path': '/putfile', 'remote_host': '172.20.32.1', 'method': 'DELETE'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '25/Feb/2015:10:41:02 +0300', 'path': '/putfile?uploads', 'remote_host': '172.20.32.1', 'method': 'POST'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '02/Jul/2014:14:35:56 +0100', 'path': '/img/Customers/Absolute-Steel-Framing.gif', 'remote_host': '151.227.152.48', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '403', 'datetime': '31/Aug/2010:23:45:30 +0100', 'path': '/', 'remote_host': '159.226.202.17', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '403', 'datetime': '01/Sep/2010:08:03:47 +0100', 'path': '/robots.txt', 'remote_host': '65.55.3.169', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '404', 'datetime': '24/Apr/2009:19:15:52 +1100', 'path': '/misc/arrow-desc.png', 'remote_host': '66.187.104.20', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '28/Apr/2009:10:38:09 +1100', 'path': '/', 'remote_host': '77.35.168.108', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '304', 'datetime': '28/Apr/2009:12:49:27 +1100', 'path': '/', 'remote_host': '77.35.172.105', 'method': 'GET'}
 {'http_version': 'HTTP/1.0', 'response_code': '404', 'datetime': '02/May/2009:12:17:26 +1100', 'path': '/robots.txt', 'remote_host': '79.137.201.45', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '17/Feb/2018:16:06:48 +0100', 'path': '/noindex/css/open-sans.css', 'remote_host': '151.21.4.47', 'method': 'GET'}
 {'http_version': 'HTTP/1.1', 'response_code': '200', 'datetime': '17/Feb/2018:16:06:48 +0100', 'path': '/images/apache_pb.gif', 'remote_host': '151.21.4.47', 'method': 'GET'}

Collectives™ on Stack Overflow

Python parse web access logs

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related