Python 3 Read a json file with missing objects within lines

Question

I'm reading a json file with the structure below:

    [{"id":1,"gender":"Male","first_name":"Andrew","last_name":"Scott","email":"[email protected]","ville":"Connecticut"},
{"id":3,"first_name":"Mary","last_name":"Richards","email":"[email protected]","ville":"Minnesota"}]

So, as you can see in the second "line" the field "gender" it'is not present.I realize that because my code to read the file got wrong in this line.

my code:

import json

def jsonreader():
##Reader for json files
    ##Open files using json library
    with open('cust_data.json') as file:
        data = json.load(file)
    resultlist = list()
    for line in data:
        print(line["id"],line["gender"])

I got the error:-

C:/xxxxx/x.py
1 Male
Traceback (most recent call last):
2 Female
File "C:/xxxxx/x", line 67, in <module>
jsonreader()
File "C:/xxxxx/x", line 56, in jsonreader
print(line["id"],line["gender"])
KeyError: 'gender'

Before answer guys, you should know that I have a method to define the default value in "gender", voila my method:

def definegender(x):
    if x is None:
        x = 'unknown'
        return x
    elif (x =='Male') or (x=='Female'):#not None:
        return {
         'Male':'M',
         'Female': 'F'
        }.get(x)
    else:
        return x

So, in this case, I could not use something like a default value reading the values because I need to send some value to my method.

Some one of you guys would know how should be the best way to read this kind of files when we have missing objects. Thanks

Before answer guys, you should know that I have a method to define the default value in "gender", voila my method: — Andres Urrego Angel
– Andres Urrego Angel, Commented Jul 26, 2017 at 13:41
What you are asking for is not clear. You have a method to always give gender field a value, but your json has no gender field in some cases. Now, you want handle these cases (that should never happens due to your definegender()), but don't want to use error handling nor default values... so, what's the results do you expect? — mrnfrancesco
– mrnfrancesco, Commented Jul 26, 2017 at 13:54
@mrnfrancesco here is the thing. I need to read the file and check line by line the fields to create a class object called "person" so to ensure that always the gender is M or F or UNKNOWN I created this method. the thing is that i can't just pass the field to my method cuz sometimes it's not coming from the source file. :( — Andres Urrego Angel
– Andres Urrego Angel, Commented Jul 26, 2017 at 13:58
Then just use: print(line["id"],definegender(line.get("gender"))) In this way you get the gender from the dict if it exists, otherwise None. Giving it to your function does the rest — mrnfrancesco
– mrnfrancesco, Commented Jul 26, 2017 at 14:01

Jean-François Fabre · Accepted Answer · 2017-07-26 14:12:05Z

6

why not using a default value for your dictionary in dict.get?

print(line["id"],line.get("gender","unknown"))

And since you want to transform input further, you could nest two dict.get together, the first one with None as default value, and a new table, like this:

gender_dict = {"Male":"M", "Female":"F", None : "unknown"}

print(line["id"],gender_dict.get(line.get("gender")))

(note that you don't need your overcomplex gender conversion method anymore)

edited Jul 26, 2017 at 14:12

answered Jul 26, 2017 at 13:39

Jean-François Fabre♦

141k24 gold badges179 silver badges246 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Andres Urrego Angel Over a year ago

I have just to add another detail in my question cuz I need run a method to set a default value for gender. thanks

Jean-François Fabre Over a year ago

@AndresAngel I think my new edit will suit you (and is simpler than your original method)

Anton vBR · Accepted Answer · 2017-07-27 07:31:49Z

1

Although this already has a perfect answer, my point of view is that there can be alternatives too. So here it is:

for line in data:
    try:
        print(line["id"],line["gender"])
    except KeyError:
        print(line["id"],"Error!!! no gender!")

This is called ErrorHandling. Read the docs here: https://docs.python.org/3.6/tutorial/errors.html

update: Do you mean this? update2 corrected misstake

try:
    gender = definegender(line["gender"])
except KeyError:
    gender = definegender(None)
print(line["id"],gender)

update3: (for future purposes)

as .get() returns None by default the most simple solution would be

gender = definegender(line.get("gender"))
print(line["id"],gender)

edited Jul 27, 2017 at 7:31

answered Jul 26, 2017 at 13:43

Anton vBR

19k6 gold badges47 silver badges47 bronze badges

5 Comments

Andres Urrego Angel Over a year ago

look my edition about how to play with the method to validate the gender. thanks

Anton vBR Over a year ago

ok @AndresAngel I looked and updated my answer. Although I am not 100% sure what you are asking for.

Andres Urrego Angel Over a year ago

you right the thing is that I'm trying to create a "human" object from a class using this for loop to read the json file. I will try your approach thanks buddy

Andres Urrego Angel Over a year ago

thanks so much buddy I guess I figure it out with your approach. awesome

mrnfrancesco Over a year ago

@AntonvBR the code in your update is wrong. The default value for gender when it is not present or equals to None is "unkown", but your solution just print None instead

giran · Accepted Answer · 2017-07-26 13:45:22Z

1

Why not simplify this with an if-statement?

for line in data:
    if "gender" in line:
        print(line)

answered Jul 26, 2017 at 13:45

giran

3781 gold badge2 silver badges11 bronze badges

1 Comment

Andres Urrego Angel Over a year ago

look my edition post about how to play with the method to validate the gender. thanks

Collectives™ on Stack Overflow

Python 3 Read a json file with missing objects within lines

3 Answers 3

2 Comments

5 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

5 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related