How to parse logs and extract lines containing specific text strings?

Question

I've got several hundred log files that I need to parse searching for text strings. What I would like to be able to do is run a Python script to open every file in the current folder, parse it and record the results in a new file with the original_name_parsed_log_file.txt. I had the script working on a single file but now I'm having some issues doing all files in the directory.

Below is what I have so far but it's not working atm. Disregard the first def... I was playing around with changing font colors.

import os
import string
from ctypes import *

title = ' Log Parser '
windll.Kernel32.GetStdHandle.restype = c_ulong
h = windll.Kernel32.GetStdHandle(c_ulong(0xfffffff5))

def display_title_bar():
    windll.Kernel32.SetConsoleTextAttribute(h, 14)
    print '\n'
    print '*' * 75 + '\n'
    windll.Kernel32.SetConsoleTextAttribute(h, 13)
    print title.center(75, ' ')
    windll.Kernel32.SetConsoleTextAttribute(h, 14)
    print '\n' + '*' * 75 + '\n' 
    windll.Kernel32.SetConsoleTextAttribute(h, 11)

def parse_files(search):
    for filename in os.listdir(os.getcwd()):
        newname=join(filename, '0_Parsed_Log_File.txt')
        with open(filename) as read:
            read.seek(0)
           # Search line for values if found append line with spaces replaced by tabs to new file.
            with open(newname, 'ab') as write:
                for line in read:
                    for val in search:
                        if val in line:
                            write.write(line.replace(' ', '\t'))
                            line = line[5:]
            read.close()
            write.close()
print'\n\n'+'Parsing Complete.'
windll.Kernel32.SetConsoleTextAttribute(h, 15)

display_title_bar()

search = raw_input('Please enter search terms separated by commas:  ').split(',')
parse_files(search)

Easiest way to get all files in a directory by matching a name wildcard string is to use the builtin glob module. — PaulMcG
– PaulMcG, Commented Aug 1, 2016 at 16:53

joel goldstick · Accepted Answer · 2016-08-01 16:15:31Z

1

This line is wrong:

    newname=join(filename, '0_Parsed_Log_File.txt')

use:

    newname= "".join([filename, '0_Parsed_Log_File.txt'])

join is a string method which requires a list of strings to be joined

answered Aug 1, 2016 at 16:15

joel goldstick

4,5136 gold badges34 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

digital_alchemy Over a year ago

Thanks, that worked. After I posted I realized I had modified that line and not corrected it. That was definitely the source of my problem.

Collectives™ on Stack Overflow

How to parse logs and extract lines containing specific text strings?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related