converting txt data (from log file) into json using python?

Question

I have a log file which has the format as shown below:

parsed: {'priority': '14', 'timestamp': '2021-04-13 13:42:07', 'hostname': 'invi-dev-gw2', 'rootname': 'root', 'pid': '27889', 'message': 'Session STARTED - Client[ID:8242, Physical: 111.119.187.47, Virtual: 10.1.0.66] <--> Service[Name:Attendance1, ID:704, Physical: 192.168.3.18, Virtual: 10.1.0.67]'}
parsed: {'priority': '15', 'timestamp': '2021-04-13 13:42:07', 'hostname': 'invi-dev-gw3', 'rootname': 'root', 'pid': '27890', 'message': 'Session STOPPED - Client[ID:8242, Physical: 111.119.187.47, Virtual: 10.1.0.66] <--> Service[Name:Attendance1, ID:704, Physical: 192.168.3.18, Virtual: 10.1.0.67]'}

Basically there are two data's in the text file. The next step is to convert the text data into a JSON using Python. So far I have the python script for the JSON conversion as shown below:

# Python program to convert text 
# file to JSON 
import json 


# the file to be converted to 
# json format 
filename = 'output.txt'

# dictionary where the lines from 
# text will be stored 
dict1 = {} 

# creating dictionary 
with open(filename) as fh: 

    for line in fh: 

        # reads each line and trims of extra the spaces 
        # and gives only the valid words 
        command, description = line.strip().split(None, 1) 

        dict1[command] = description.strip() 

# creating json file 
# the JSON file is named as test1 
out_file = open("test.json", "w") 
json.dump(dict1, out_file, indent = 4, sort_keys = False) 
out_file.close()

Now the JSON file is created but it showed only one data (The output should show 2 data) as shown below:

    "parsed:": "{'priority': '15', 
                 'timestamp': '2021-04-13 13:42:07',
                  'hostname': 'invi-dev-gw3',
                  'rootname': 'root', 
                  'pid': '27890', 
                  'message': 'Session STOPPED - Client[ID:8242, Physical: 111.119.187.47, Virtual: 10.1.0.66] <--> Service[Name:Attendance1, ID:704, Physical: 192.168.3.18, Virtual: 10.1.0.67]'
}"

I don't know why it's not printing the whole data. It should show another data in the JSON file but showed only one. Can anyone help me with this?

You've put the same parsed: key into the dictionary twice, the second one overwriting the first. Build a list instead, perhaps throwing away the parsed: part in the process and keeping the description only. Consider showing what output you expect. — tevemadar
– tevemadar, Commented Apr 13, 2021 at 17:25

JonSG · Accepted Answer · 2021-04-13 18:10:50Z

Where you have dict1[command] = description.strip() you really want something more like: dict1[command].append(description.strip()) (so dict1 really wants to be a list). There is one additional wrinkle in that you likely wanted to use json.loads() but your input data uses single quotes so let's parse it with ast

I would try something like:

import ast
import collections
import json

data_in = [
    "parsed: {'priority': '14', 'timestamp': '2021-04-13 13:42:07', 'hostname': 'invi-dev-gw2', 'rootname': 'root', 'pid': '27889', 'message': 'Session STARTED - Client[ID:8242, Physical: 111.119.187.47, Virtual: 10.1.0.66] <--> Service[Name:Attendance1, ID:704, Physical: 192.168.3.18, Virtual: 10.1.0.67]'}",
    "parsed: {'priority': '15', 'timestamp': '2021-04-13 13:42:07', 'hostname': 'invi-dev-gw3', 'rootname': 'root', 'pid': '27890', 'message': 'Session STOPPED - Client[ID:8242, Physical: 111.119.187.47, Virtual: 10.1.0.66] <--> Service[Name:Attendance1, ID:704, Physical: 192.168.3.18, Virtual: 10.1.0.67]'}"
]

data_out = collections.defaultdict(list)
for row in data_in:
    command, command_text = [value.strip() for value in row.split(":", 1)]
    data_out[command].append(ast.literal_eval(command_text))

print(json.dumps(data_out, indent=2))

Collectives™ on Stack Overflow

converting txt data (from log file) into json using python?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related