Python: Extracting data from nested dictionary

Question

I have downloaded some data that returns a dictionary, I've provided a sample below. I thought it might be a nested dictionary, but it doesn't really look like one. (I'm new to python) I don't understand the structure of this dictionary and how to extract data from it. I want to extract the values for given keys. So to start with, return value where key = 'id'

Here is the sample dictionary and one type of code that I've tried to use:

 my_dict = {'id': 5729283,
 'title': 'Auto & Transport(L1)',
 'colour': None,
 'is_transfer': None,
 'is_bill': None,
 'refund_behaviour': None,
 'children': [{'id': 5698724,
   'title': 'Car Insurance',
   'colour': None,
   'is_transfer': False,
   'is_bill': True,
   'refund_behaviour': 'credits_are_refunds',
   'children': [],
   'parent_id': 5729283,
   'roll_up': False,
   'created_at': '2019-07-25T12:38:46Z',
   'updated_at': '2021-01-19T08:12:28Z'},
  {'id': 5729295,
   'title': 'Toll',
   'colour': None,
   'is_transfer': False,
   'is_bill': False,
   'refund_behaviour': None,
   'children': [],
   'parent_id': 5729283,
   'roll_up': False,
   'created_at': '2019-08-05T04:30:55Z',
   'updated_at': '2021-01-08T02:33:11Z'}],
 'parent_id': None,
 'roll_up': True,
 'created_at': '2019-08-05T04:28:04Z',
 'updated_at': '2021-01-08T02:44:09Z'}


temp = 'id'
[val[temp] for key, val in my_dict.items() if temp in val]

do you just want to extract the value of key 'id' OR you want to extract some other values? I just want to know why did you try to do this [val[temp] for key, val in my_dict.items() if temp in val] — Cool Breeze
– Cool Breeze, Commented May 29, 2021 at 4:56
yes I will want to extract other values too. I used that code because I thought it was a nested dictionary and that's the code I found to search nested dictionaries. Although now I don't think it is that — jmich738
– jmich738, Commented May 29, 2021 at 5:01

Mark · Accepted Answer · 2021-05-29 05:05:45Z

3

The structure looks like a tree. You have a root node with id 5729283 then a list of children, which themselves have zero or more children (ids 5698724 and 5729295).

A common way to deal with this is a recursive function (or generator). The function yields the root value and then recursively processes the children:

def getKey(d, key):
    yield d[key]                       # the root element
    for child in d['children']:
        yield from getKey(child, key)  # the same for children

temp = 'id'
      
list(getKey(my_dict, temp))
# [5729283, 5698724, 5729295]

This will perform a depth first traversal of the tree (although that doesn't really matter if it only has one level children).

edited May 29, 2021 at 5:05

answered May 29, 2021 at 4:59

Mark

92.7k8 gold badges116 silver badges156 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Cool Breeze · Accepted Answer · 2021-05-29 04:57:02Z

1

To extract the value of temp = 'id' you can do this

print(my_dict[temp])

answered May 29, 2021 at 4:57

Cool Breeze

9843 gold badges12 silver badges28 bronze badges

2 Comments

jmich738 Over a year ago

surprisingly (to me) that actually works. But my actual case is more complex.

Cool Breeze Over a year ago

I'll update the answer to extract all values.

blackraven · Accepted Answer · 2021-05-29 12:51:32Z

Although the answer has been selected, please allow me to present below suggestion in the spirit of sharing. Below code using recursive function can extract values using specific key in the nested dictionary or 'lists of dictionaries'

You do not have to analyse the nested structure of the dictionary/list. Simply specify the key, the function will extract all values paired to the key.

my_dict = {...blah...}

def get_vals(nested, key):
    result = []
    if isinstance(nested, list) and nested != []:   #non-empty list
        for lis in nested:
            result.extend(get_vals(lis, key))
    elif isinstance(nested, dict) and nested != {}:   #non-empty dict
        for val in nested.values():
            if isinstance(val, (list, dict)):   #(list or dict) in dict
                result.extend(get_vals(val, key))
        if key in nested.keys():   #key found in dict
            result.append(nested[key])
    return result

get_vals(my_dict, 'id')

Output

[5698724, 5729295, 5729283]

Collectives™ on Stack Overflow

Python: Extracting data from nested dictionary

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related