get list of json paths in python

Question

I'm looking to get list of all possible json paths in a json file - can recommend any one?

Eg: if input is below

{
   "_id":{
      "$oid":""
   },
   "aa":false,
   "bb":false,
   "source":"",
   "email":"",
   "createdAt":{
      "$date":""
   },
   "updatedAt":{
      "$date":""
   },
   "cc":"",
   "vv":"",
   "metadata":{
      "vv":"",
      "xx":[{}]
   }
}

o/p:

obj
obj._id
obj._id.$oid
obj.aa
obj.bb
obj.source
obj.email
obj.createdAt
obj.createdAt.$date
obj.updatedAt
obj.updatedAt.$date
obj.cc
obj.vv
obj.metadata
obj.metadata.vv
obj.metadata.xx
obj.metadata.xx[0]

I'm basically looking. a python version of this: https://www.convertjson.com/json-path-list.htm

I want to build a general solution, if any json file - it will be a single value for schema generation (ie one line in a newline delimeted json) Any suggestions ?

Mark · Accepted Answer · 2021-05-02 01:50:02Z

8

You can do this in a reasonably succinct way with a recursive generator. The string "obj" is a little awkward since it doesn't occur in the data structure. On the other hand, adding it at the end is simple:

def get_paths(d):
    if isinstance(d, dict):
        for key, value in d.items():
            yield f'.{key}'
            yield from (f'.{key}{p}' for p in get_paths(value))
        
    elif isinstance(d, list):
        for i, value in enumerate(d):
            yield f'[{i}]'
            yield from (f'[{i}]{p}' for p in get_paths(value))

paths = ['obj'+s for s in get_paths(d)]

Gives you paths as a list of strings:

['obj._id',
 'obj._id.$oid',
 'obj.aa',
 'obj.bb',
 'obj.source',
 'obj.email',
 'obj.createdAt',
 'obj.createdAt.$date',
 'obj.updatedAt',
 'obj.updatedAt.$date',
 'obj.cc',
 'obj.vv',
 'obj.metadata',
 'obj.metadata.vv',
 'obj.metadata.xx',
 'obj.metadata.xx[0]']

Of course, you can wrap that last step in a function like and accept a root object string:

def get_paths(d, root="obj"):
    def recur(d):
        if isinstance(d, dict):
            for key, value in d.items():
                yield f'.{key}'
                yield from (f'.{key}{p}' for p in get_paths(value))

        elif isinstance(d, list):
            for i, value in enumerate(d):
                yield f'[{i}]'
                yield from (f'[{i}]{p}' for p in get_paths(value))

    return (root + p for p in recur(d))

list(get_paths(d))
# same result

edited May 2, 2021 at 1:50

answered May 2, 2021 at 1:42

Mark

92.7k8 gold badges116 silver badges156 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

kenzen Over a year ago

Thanks for sharing this - I was basically looking for basic json filter - basically input will be a template json for filtering and the output data i required is filtered one - is there any easy way to do this

Mark Over a year ago

I'm not sure I understand @kenzen. Filters typically have some sort of predicate that determines what is filtered and what isn't — not sure where that fits here. Might make a good question on its own if you can explain it clearly.

Ozzie Ghani Over a year ago

@Mark, may I ask you to enhance the solution to provide the key paths of the dictionary where the value of the given key is neither dict or list, so only get the paths of the keys where the values are either string, integer or float etc?

UCYT5040 · Accepted Answer · 2021-05-02 02:04:05Z

0

You can do this with this code:

mylist = []
def getKeys(obj, parent="obj"):
  global mylist
  for i in obj.keys():
    mylist.append(parent+"."+i)
    try:
      getKeys(obj[i], parent+"."+i)
    except AttributeError: 
      pass
getKeys({
   "_id":{
      "$oid":""
   },
   "aa":False,
   "bb":False,
   "source":"",
   "email":"",
   "createdAt":{
      "$date":""
   },
   "updatedAt":{
      "$date":""
   },
   "cc":"",
   "vv":"",
   "metadata":{
      "vv":"",
      "xx":[{}]
   }
})
print(mylist)

I changed false to False. If you are using JSON and not a dictionary you may want to use the JSON library to convert it to a dictionary.

import json
myDict = json.loads("{"_id":{"$oid":""},"aa":false,"bb":false,"source":"","email":"","createdAt":{"$date":""},"updatedAt":{"$date":""},"cc":"","vv":"","metadata":{"vv":"","xx":[{}]}}")

edited May 2, 2021 at 2:04

answered May 2, 2021 at 1:57

UCYT5040

3846 silver badges19 bronze badges

1 Comment

Mark Over a year ago

This seems to miss the lists like: obj.metadata.xx[0].

Collectives™ on Stack Overflow

get list of json paths in python

2 Answers 2

3 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related