Handling weird JSON from API call with json.loads

Question

I am having trouble efficiently storing the PhoneNumber value in the block below

    response = requests.get(api)
    data = json.loads(response.text)

So after doing the above, I have a dictionary 'data' which looks like this:

{'Key1': 'Some Value ', 'Key2': [{'PhoneNumber': '180000000000'}]}

The goal is to get just the PhoneNumber value i.e. '180000000000', and my first reaction was the below which works, but I know this will not scale well. Wondering if within the json.loads call above I can pass an argument or if anyone has other advice on how to more effectively get this value I am interested.

    data = data['Key2'][0]['PhoneNumber'])

So after the above data is now = '180000000000' and I want to store it in s3, I had old code (below) for a JSON file, and this works but is pointless. I just need to store the phone numbers as text file ( I know how to use with open for .txt but I am confused on how to use the io.Bytes and if it should be SringIO instead of BytesIO)

    inmemory = io.BytesIO() 
    with gzip.GzipFile(fileobj=inmemory, mode='wb') as fh:

        with io.TextIOWrapper(fh, encoding='utf-8',errors='replace') as wrapper:
            wrapper.write(json.dumps(data, ensure_ascii=False,indent=2))
    inmemory.seek(0)
    s3_resource.Object(s3bucket, s3path + '.json.gz').upload_fileobj(inmemory)  
    inmemory.close()

And when it is stored like this, I have when I pull the value from S3, I have to use 'with gzip.open' and decode it ect

You're making things harder than they need to be — that's a list holding a dictionary. Don't turn the data you parsed back into string: data['Key2'][0]['PhoneNumber'] is what you need. — Mark
– Mark, Commented Dec 31, 2019 at 4:04
Yes, assuming you are starting with a json string — loads converts it to a python data structure. — Mark
– Mark, Commented Dec 31, 2019 at 4:22
I may be misunderstanding, but this reads like two separate questions (since the edit): one asks how to more efficiently retrieve the phone number, and the other about BytesIO. Perhaps this should be split into two separate SO posts? — chris
– chris, Commented Dec 31, 2019 at 4:23
Hi Chris, you are correct , I apologize on second thought that should be a separate question — 0004
– 0004, Commented Dec 31, 2019 at 4:24

chris · Accepted Answer · 2019-12-31 04:14:58Z

1

Stringifying data['Key2'] is unnecessary, standard dictionary and list access syntax should work for your case. Condensed into one line:

>>> data = {'Key1': 'Some Value ', 'Key2': [{'PhoneNumber': '180000000000'}]}
>>> data['Key2'][0]['PhoneNumber']
'180000000000'

You can (and probably should) handle error cases in here, as well, perhaps by splitting this out into multiple steps or by trapping the KeyError and IndexError together. As one option, take a look at the newish suppress util in contextlib (docs). Would look something like this:

>>> with suppress(KeyError, IndexError):
...     ph = data['Key2'][0]['PhoneNumber']
...
>>> ph
'180000000000'

edited Dec 31, 2019 at 4:14

answered Dec 31, 2019 at 4:08

chris

2,0832 gold badges15 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

0004 Over a year ago

Hi Chris, thank you for your help, I will input a check because your response did make things more efficient and I updated the code in my question to reflect so now, wondering if there is still room for improvement and if json.loads was necessary because the inital block is an api response in text, then a dictionary then back as a string (ie when it is set back to 'data' after the dictionary parsing) . Also, what the best way to write that data to/from s3

0004 Over a year ago

I broke out the second question here: stackoverflow.com/questions/59539656/…

0004 Over a year ago

Why would I want to handle error cases in here? I will lookinto suppress , ty for docs link

chris Over a year ago

You'd want to check for errors in case the response body doesn't contain the data exactly as you expect. (For instance, if data['Key2'] is [].) This isn't necessary if something else is validating your input.

Collectives™ on Stack Overflow

Handling weird JSON from API call with json.loads

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related