python json nested dict and list decoding

Question

I am trying to parse the output of an API calls which is a json object with nested dict and lists, with depth not known. I saw several example of recursive functions getting a key value from nested list/dict but I want to get the specific order as well for the correctness of the structure. so for example:

    response = {'a': {'b': {'b1': 1, 'b2': 2},
                'c': [{'c1': 3,'c2': [{'x':55,'y':56},{'x':65,'y':66}]},
                      {'c1': 5,'c2': [{'x':75,'y':76},{'x':85,'y':86}]}]
                   }
                 }

fun(response,"a-c-c2-x") should return [55,65,75,85]
fun(response,"a-b-b2") should return [2]
fun(response,"a-b2-b") should return []

what I have so far that is not working :)

def get_response_values(response=None, key=None, result=None):
    print("1-r: {} - key: {} -r:{}".format(response, key, result))
    if result is None:
        result = []

    if '-' in key:
        if isinstance(response, dict):
            index = key.find('-')
            response = response[key[:index]]
            key = key[index+1:]
            print("3-r: {} - key: {} -r:{}".format(response, key, result))
            get_response_values(response, key, result)

        if isinstance(response, list):
            index = key.find('-')
            key = key[index+1:]
            for item in response:
                print("4-r: {} - key: {} -r:{}".format(item, key, result))
                get_response_values(item, key, result)

    else:
        try:
            result.append(response[key])
            print("2-r: {} - key: {} -r:{}".format(response, key, result))
            return result
        except:
            print("5-r: {} - key: {} -r:{}".format(response, key, result))
            return result


    return result

Were you ever going to return the results of your recursion? — Ignacio Vazquez-Abrams
– Ignacio Vazquez-Abrams, Commented May 22, 2015 at 3:22

James Mills · Accepted Answer · 2015-05-22 03:53:49Z

1

You could use a library like jsonselect which implelments the JSON Select query specification language.

Basic programatic Example:

from jsonselect import select

result = select(query, data)

Update: Alternatively it looks like you could also use funcy's funcy.get_in() function.

Don't re-invent the wheel; there are many good ready-to-use-solutions.

Update #2: Given your comments below here is a simpler to understand and read example that will get you 80% of the way there (without using recursion which runs the risk of blowing the stack!):

def query(data, path):
    elem = data
    try:
        for x in path.strip("/").split("/"):
            elem = elem.get(x)
    except:
        pass

    return elem

edited May 22, 2015 at 3:53

answered May 22, 2015 at 3:22

James Mills

19.1k4 gold badges53 silver badges63 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

curious07 Over a year ago

wow, yes, they both would work for my problem. I just don't want to install whole 3rd party package just for one function. I will check their code to see how they implement it. see if I can figure it out :) thank you.

James Mills Over a year ago

What's the big deal with installing and re-using a 3rd-party library?

curious07 Over a year ago

this is part of test automation, which has its own virtual env, and it is being deployed over 100 machines, updated over the air etc. whole funcy library is 400K.

James Mills Over a year ago

Ahh I see; well see my update response. You'll have to tweak/modify this to suit your exact needs but it should get you most of the way there!

James Mills Over a year ago

I know; but implementing exactly what you want to cover all edge-cases is not trivial; that's why I linked you to two good implementations that do :)

Collectives™ on Stack Overflow

python json nested dict and list decoding

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related