0

i am trying to extract data from the log file using grok.my log lines looks like this.

[Server 192.178.35.40] testweb.de 63.239.73.83 - - [19/Nov/2017:23:27:26 +0100] \"GET /service/want/teaser2/Buk/ HTTP/1.1\" 200 319 \"-\" \"https://testweb.de/Suche/Buk/Bonn\" \"Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\" \"65.259.77.67\" 0

i am expecting something like this

server : 192.178.35.40
website : testweb.de
clientip : 63.239.73.83
timestamp:19/Nov/2017:23:27:26 +0100
method:GET
RESOURCE:/service/want/teaser2/Buk/ HTTP/1.1
RESPONCE:200
TIMETAKEN:319
USERAGENT:Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile 
Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
COOKIE:0

tried on https://grokdebug.herokuapp.com/ by giving pattern,

 %{ip:SERVER} 

and received the results but unable to parse the remaining data

0

1 Answer 1

1

How did you expect to extract everything in their own fields with just a single pattern to match?

You need to match every field separately in order to get your desired output. Can you try this?

%{IPV4:server}\] %{HOSTNAME:website} %{IPV4:client} - - \[%{HTTPDATE:timestamp}\] \\"%{WORD:method} (?<resource>%{NOTSPACE} HTTP/%{NUMBER})\\" %{NUMBER:response} %{NUMBER:TimeTaken} \\"-\\" \\"%{URI}\\" \\"%{GREEDYDATA:useragent}\).*%{NUMBER:cookie}

This will output,

{
  "server": [
    [
      "192.178.35.40"
    ]
  ],
  "website": [
    [
      "testweb.de"
    ]
  ],
  "client": [
    [
      "63.239.73.83"
    ]
  ],
  "timestamp": [
    [
      "19/Nov/2017:23:27:26 +0100"
    ]
  ],
  "MONTHDAY": [
    [
      "19"
    ]
  ],
  "MONTH": [
    [
      "Nov"
    ]
  ],
  "YEAR": [
    [
      "2017"
    ]
  ],
  "TIME": [
    [
      "23:27:26"
    ]
  ],
  "HOUR": [
    [
      "23"
    ]
  ],
  "MINUTE": [
    [
      "27"
    ]
  ],
  "SECOND": [
    [
      "26"
    ]
  ],
  "INT": [
    [
      "+0100"
    ]
  ],
  "method": [
    [
      "GET"
    ]
  ],
  "resource": [
    [
      "/service/want/teaser2/Buk/ HTTP/1.1"
    ]
  ],
  "NOTSPACE": [
    [
      "/service/want/teaser2/Buk/"
    ]
  ],
  "NUMBER": [
    [
      "1.1"
    ]
  ],
  "BASE10NUM": [
    [
      "1.1",
      "200",
      "319",
      "0"
    ]
  ],
  "response": [
    [
      "200"
    ]
  ],
  "TimeTaken": [
    [
      "319"
    ]
  ],
  "URI": [
    [
      "https://testweb.de/Suche/Buk/Bonn"
    ]
  ],
  "URIPROTO": [
    [
      "https"
    ]
  ],
  "USER": [
    [
      null
    ]
  ],
  "USERNAME": [
    [
      null
    ]
  ],
  "URIHOST": [
    [
      "testweb.de"
    ]
  ],
  "IPORHOST": [
    [
      "testweb.de"
    ]
  ],
  "HOSTNAME": [
    [
      "testweb.de"
    ]
  ],
  "IP": [
    [
      null
    ]
  ],
  "IPV6": [
    [
      null
    ]
  ],
  "IPV4": [
    [
      null
    ]
  ],
  "port": [
    [
      null
    ]
  ],
  "URIPATHPARAM": [
    [
      "/Suche/Buk/Bonn"
    ]
  ],
  "URIPATH": [
    [
      "/Suche/Buk/Bonn"
    ]
  ],
  "URIPARAM": [
    [
      null
    ]
  ],
  "useragent": [
    [
      "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html"
    ]
  ],
  "cookie": [
    [
      "0"
    ]
  ]
}
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.