Composing a splunk query from the file content using Python

Question

I am trying to composing a Splunk query by fetching the values from the text file content. Here i dont want to use any Splunk modules/libraries.

This is my simple code -

import pandas as pd
from pandas import ExcelWriter
from pandas import ExcelFile
import sys

df = pd.read_excel("I:\\splunk_dashboards\\FID_list.xlsx", sheetname='FID_lastweek')
sys.stdout = open("I:\\splunk_dashboards\\FID.txt", "w")


v = df['FID']
#print(df['FID'])

print(v)

This is the simple code where it retrieves the particular column values and store it in a text file.

The next step is to form a splunk query with the results stored in the text file.

For example below is the result from the text file -

0                            CollectionLimitsValidation
1                               PaymentLimitsValidation
2                              AccountDetailsFacadeBean
3                              AccountDetailsFacadeBean

I do have a splunk query like below in another text file -

index=hfc_new_98764 host=QA FID=$(Value1_from_text_file) OR FID=$(value2_from _text_file) OR.... it goes on upto the final values

From the above template i need a splunk query like below -

index=hfc_new_98764 host=QA FID=CollectionLimitsValidation OR FID=PaymentLimitsValidation OR FID=.... it goes on upto the final values

I need help to iterate the values from the text file and to store in the template file file.

Harish P C · Accepted Answer · 2018-05-31 08:40:44Z

I am able to achieve the above scenario with file operations and here is my complete code -

# -*- coding: utf-8 -*-
"""
Created on Wed May 30 18:24:04 2018

@author: Harish
"""

import pandas as pd
from pandas import ExcelWriter
from pandas import ExcelFile
import sys
import fileinput
#import os


#Getting the values from Excel sheet

df = pd.read_excel("I:\\splunk_dashboards\\FID_list.xlsx", sheetname='FID_lastweek')
sys.stdout = open("I:\\splunk_dashboards\\new.txt", "w")
df.FID.unique()
v = df['FID'].to_string(index=False)
pd.options.display.max_colwidth = 200
#print(df['FID'])
#print('"{}"'.format(v))
print(v)


#os.system("script_to_create_FID.py")

#left alignment script
sys.stdout = open("I:\\splunk_dashboards\\aligned_file.txt", "w")
with open("I:\\splunk_dashboards\\new.txt") as f:
    for line in f:
        s = line.lstrip()
        m = s.strip()
        print('"{}"'.format(m))
        #print(m)

#FID and OR values 
prefix = 'FID='
suffix = '  OR'

with open('I:\\splunk_dashboards\\aligned_file.txt', 'r') as src:
    with open('I:\\splunk_dashboards\\final_FID.txt', 'w') as dest:
       for line in src:
           dest.write('%s%s%s\n' % (prefix, line.rstrip('\n'), suffix))


#Added Splunk index here      
for linenum,line in enumerate( fileinput.FileInput("I:\\splunk_dashboards\\final_FID.txt",inplace=1) ):
    if linenum==0 :
        print 'index=hfc_new_98764 host=QA" NOT(WARN=yes)'
        print line.rstrip()
    else:
        print line.rstrip()

#Add sort function at the end
a = '| stats count As NumberOfCalls, count(eval(ERCD=0)) AS "Success" ,count(eval(ERCD!=0)) AS "Failures" by FID | sort – Failures'
with open("I:\\splunk_dashboards\\final_FID.txt","a") as text:    
    text.writelines(a)

Step 1 - Create a new text file with the list of Fetched FID's from the excel Step 2 - Format the Text file Step 3 - Append 'FID' and 'OR' at front and last of the query Step 4 - Generate the Query

Collectives™ on Stack Overflow

Composing a splunk query from the file content using Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related