Auto-generate json paths from given json structure using Python

Question

I am trying to generate auto json paths from given json structure but stuck in the programatic part. Can someone please help out with the idea to take it further?

Below is the code so far i have achieved.

def iterate_dict(dict_data, key, tmp_key):
    for k, v in dict_data.items():

        key = key + tmp_key + '.' + k
        key = key.replace('$$', '$')
        if type(v) is dict:
            tmp_key = key
            key = '$'
            iterate_dict(v, key, tmp_key)
        elif type(v) is list:
            str_encountered = False
            for i in v:
                if type(i) is str:
                    str_encountered = True
                    tmp_key = key
                    break
                tmp_key = key
                key = '$'
                iterate_dict(i, key, tmp_key)
            if str_encountered:
                print(key, v)
                if tmp_key is not None:
                    tmp_key = str(tmp_key)[:-str(k).__len__() - 1]
                key = '$'
        else:
            print(key, v)
            key = '$'


import json
iterate_dict_new(dict(json.loads(d_data)), '$', '')

consider the below json structure

{
    "id": "1",
    "categories": [
        {
            "name": "author",
            "book": "fiction",
            "leaders": [
                 {
                      "ref": ["wiki", "google"],
                      "athlete": {
                            "$ref": "some data"
                       },
                      "data": {
                            "$data": "some other data"
                       }
                 }
             ]
        },
        {
            "name": "dummy name"
        }
    ]
}

Expected output out of python script:

$id = 1
$categories[0].name = author
$categories[0].book = fiction
$categories[0].leaders[0].ref[0] = wiki
$categories[0].leaders[0].ref[1] = google
$categories[0].leaders[0].athlete.$ref = some data
$categories[0].leaders[0].data.$data = some other data
$categories[1].name = dummy name

Current output with above python script:

$.id 1
$$.categories.name author
$$.categories.book fiction
$$$.categories.leaders.ref ["wiki", "google"]
$$$$$.categories.leaders.athlete.$ref some data
$$$$$$.categories.leaders.athlete.data.$data some other data
$$.name dummy name

I am working on a sol, but for recursion to work properly, I think this output line: $categories[0].leaders[0].ref = ['wiki', 'google'] should be turned into 2 more lines: ...leaders[0].ref[0] = wiki and ...leaders[0].ref[1] = google — Joe Iddon
– Joe Iddon, Commented Jan 23, 2018 at 17:09
Thanks @JoeIddon, even i am trying out different other ways. Yes, it should be ...leaders[0].ref[0] = wiki and ...leaders[0].ref[1] = google i have modified my question accordingly. Other way i could think of is, to convert json to xml structure and auto generate xpaths using sax parser. — user2649233
– user2649233, Commented Jan 23, 2018 at 17:28

Joe Iddon · Accepted Answer · 2018-01-23 17:39:08Z

2

The following recursive function is similar to yours, but instead of just displaying a dictionary, it can also take a list. This means that if you passed in a dictionary where one of the values was a nested list, then the output would still be correct (printing things like dict.key[3][4] = element).

def disp_paths(it, p='$'):
    for k, v in (it.items() if type(it) is dict else enumerate(it)):
        if type(v) is dict:
            disp_paths(v, '{}.{}'.format(p, k))
        elif type(v) is list:
            for i, e in enumerate(v):
                if type(e) is dict or type(e) is list:
                    disp_paths(e, '{}.{}[{}]'.format(p, k, i))
                else:
                    print('{}.{}[{}] = {}'.format(p, k, i, e))
        else:
            f = '{}.{} = {}' if type(it) is dict else '{}[{}] = {}'
            print(f.format(p, k, v))

which, when ran with your dictionary (disp_paths(d)), gives the expected output of:

$.categories[0].leaders[0].athlete.$ref = some data
$.categories[0].leaders[0].data.$data = some other data
$.categories[0].leaders[0].ref[0] = wiki
$.categories[0].leaders[0].ref[1] = google
$.categories[0].book = fiction
$.categories[0].name = author
$.categories[1].name = dummy name
$.id = 1

Note that this is unfortunately not ordered, but that is unavoidable as dictionaries have no inherent order (they are just sets of key:value pairs)

If you need help understanding my modifications, just drop a comment!

answered Jan 23, 2018 at 17:39

Joe Iddon

20.5k7 gold badges38 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user2649233 Over a year ago

Thanks for the quick solution. Works like a charm.

user2649233 Over a year ago

A little modification to handle nested arrays and dict. '{}.{}'.format(p, k) if type(k) is str else '{}[{}]'.format(p, k) this needs to be called where ever we are calling the recursion and printing values.

Collectives™ on Stack Overflow

Auto-generate json paths from given json structure using Python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related