write log entry to splunk via HTTP in python

Question

We have a python program that needs to send logs to splunk. Our splunk admins have created a service collector HTTP endpoint to publish logs to with the following:

index
token
hostname
URI

We can't find where to input the URI in the splunk python SDK client. For example:

import splunklib.client as client
import splunklib.results as results_util

HOST="splunkcollector.hostname.com"
URI="services/collector/raw"
TOKEN="ABCDEFG-8A55-4ABB-HIJK-1A7E6637LMNO"
PORT=443

# Create a Service instance and log in
service = client.connect(
    host=HOST,
    port=PORT,
    token=TOKEN)

# Retrieve the index for the data
myindex = service.indexes["cloud_custodian"]

# Submit an event over HTTP
myindex.submit("Dummy test python client log")

As you can see I never use the URI variable. The above code results in:

Traceback (most recent call last):
  File "splunk_log.py", line 15, in <module>
    myindex = service.indexes["cloud_custodian"]
  File "/usr/local/lib/python2.7/site-packages/splunklib/client.py", line 1230, in __getitem__
    raise KeyError(key)
KeyError: UrlEncoded('cloud_custodian')

What do you get when you try to list indexes with service.indexes.keys() ? — zwer
– zwer, Commented Dec 8, 2017 at 15:11
Ah, gotta love custom types... How about: print(", ".join([str(x) for x in service.indexes])) — zwer
– zwer, Commented Dec 8, 2017 at 18:54
splunklib.binding.HTTPError: HTTP 404 Not Found -- {"text":"The requested URL was not found on this server.","code":404} — Lightbeard
– Lightbeard, Commented Dec 8, 2017 at 18:59
Ok, then try to get the app endpoint (looking at their API docs): service.get(URI) and see what happens. — zwer
– zwer, Commented Dec 8, 2017 at 19:17

Lightbeard · Accepted Answer · 2017-12-11 16:00:43Z

5

Ended up performing a stock POST with requests. I'm not sure if the splunk client is even intended to support the HTTP Event Collector.

import requests

url='https://splunkcollector.hostname.com:443/services/collector/event'
authHeader = {'Authorization': 'Splunk {}'.format('ABCDEFG-8A55-4ABB-HIJK-1A7E6637LMNO')}
jsonDict = {"index":"cloud_custodian", "event": { 'message' : "Dummy test python client log" } }

r = requests.post(url, headers=authHeader, json=jsonDict, verify=False)
print r.text

answered Dec 11, 2017 at 16:00

Lightbeard

4,13110 gold badges52 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ObjectNameDisplay Over a year ago

works for me, would also recommend including time in jsonDict so your messages are received in order. And you may need, depending on your config, X-Splunk-Request-Channel (a random GUID) in your header as well

SoloPilot Over a year ago

I also add 'sourcetype' like so: jsonDict = {"index":"cloud_custodian", "sourcetype": "Thor", "event": {...} }

skoelpin · Accepted Answer · 2017-12-10 17:48:59Z

0

You should look into the HTTP Event Collector in Splunk. It's as simple as enabling it, generating a token, and making the call.

If you wanted to send data to Splunk HEC, it would look like this

<protocol>://<host>:<port>/<endpoint>

https://docs.splunk.com/Documentation/SplunkCloud/6.6.0/Data/UsetheHTTPEventCollector

answered Dec 10, 2017 at 17:48

skoelpin

2071 silver badge5 bronze badges

Collectives™ on Stack Overflow

write log entry to splunk via HTTP in python

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related