0

I'm trying to extract a specific value from log files in a directory. Now the log files contains JSON data and i want to extract the value for the id field.

JSON Data look something like this

{
    id: "123",
    name: "foo"
    description: "bar baz" 
}

Code Looks like this

def test_load_json_directly(self):
    with open('source_data/testing123.json') as log_file:
        data = json.load(log_file)
        print data

def test_load_json_from_iteration(self, dir_path, file_ext):
    path_name = os.path.join(dir_path, '*.' + file_ext)

    files = glob.glob(path_name)
    for filename in files:
    with open(filename) as log_file:
        data = json.load(log_file)
        print data

Now I try to call the function test_load_json_directly the JSON string gets loaded correctly. No problem there. This is just to check the correct behavior of the json.load function.

The issue is when I try to call the function test_load_json_from_iteration, the JSON string is not being recognized and returns an error.

ValueError: No JSON object could be decoded

What am I doing wrong here?

18
  • 1
    Is it definitely opening the same file? Are you sure? Try printing each filename in the loop. Commented May 23, 2019 at 10:24
  • 1
    @PunctuationMark : Please paste your JSON here => jsonlint.com and validate. Commented May 23, 2019 at 10:39
  • 4
    The only issue I can see is that there's a file in your directory with invalid json data. Can you remove all json files and just keep one and test it? Commented May 23, 2019 at 10:46
  • 1
    then only issue in file path. Try by printing files list before looping on single file. Check its type. It should be list. No any syntax or logical issue is there.. Commented May 23, 2019 at 10:48
  • 2
    @PunctuationMark and also, instead of using json, try using simplejson. Like this: data = simplejson.load(log_file). It will show you more detailed errors like where the json failed. Commented May 23, 2019 at 10:50

1 Answer 1

1

Your json is invalid. The property names and the values must be wrapped with quotes (except if they are numbers). You're also missing the commas.

The most probable reason for this error is an error in a json file. Since json module doesn't show detailed errors, you can use the simplejson module to see what's actually happening.

Change your code to:

import simplejson
.
.
.

data = simplejson.load(log_file)

And look at the error message. It will show you the line and the column where it fails.

Ex:

simplejson.errors.JSONDecodeError: Expecting value: line 5 column 17 (char 84)

Hope it helps :) Feel free to ask if you have any doubts.

Sign up to request clarification or add additional context in comments.

1 Comment

I've added a try catch to handle errors and iterated on more than 1k files. I found out that there is 1 file that does not contain any data. Thanks for the help!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.