Dot notation to Json in python

Question

I receive data from the Loggly service in dot notation, but to put data back in, it must be in JSON.

Hence, I need to convert:

{'json.message.status.time':50, 'json.message.code.response':80, 'json.time':100}

Into:

{'message': {'code': {'response': 80}, 'status': {'time': 50}}, 'time': 100}

I have put together a function to do so, but I wonder if there is a more direct and simpler way to accomplish the same result.

def dot_to_json(a):

    # Create root for JSON tree structure
    resp = {}

    for k,v in a.items():
        # eliminate json. (if metric comes from another type, it will keep its root)
        k = re.sub(r'\bjson.\b','',k)
        if '.' in k:
            # Field has a dot
            r = resp
            s = ''
            k2 =  k.split('.')
            l = len(k2)
            count = 0
            t = {}
            for f in k2:
                count += 1
                if f not in resp.keys():
                    r[f]={}
                r = r[f]
                if count < l:
                    s += "['" + f + "']"
                else:
                    s = "resp%s" % s
                    t = eval(s)
                    # Assign value to the last branch
                    t[f] = v
        else:
            r2 = resp
            if k not in resp.keys():
                r2[k] = {}
            r2[k] = v
    return resp

What is that business with eval-ing "resp%s"?

BrenBarn
– BrenBarn

2014-08-19 17:59:43 +00:00
Commented Aug 19, 2014 at 17:59 — BrenBarn
– BrenBarn, Commented Aug 19, 2014 at 17:59

Martijn Pieters · Accepted Answer · 2014-08-19 18:14:35Z

8

You can turn the path into dictionary access with:

def dot_to_json(a):
    output = {}
    for key, value in a.iteritems():
        path = key.split('.')
        if path[0] == 'json':
            path = path[1:]
        target = reduce(lambda d, k: d.setdefault(k, {}), path[:-1], output)
        target[path[-1]] = value
    return output

This takes the key as a path, ignoring the first json part. With reduce() you can walk the elements of path (except for the last one) and fetch the nested dictionary with it.

Essentially you start at output and for each element in path fetch the value and use that value as the input for the next iteration. Here dict.setdefault() is used to default to a new empty dictionary each time a key doesn't yet exist. For a path ['foo', 'bar', 'baz'] this comes down to the call output.setdefault('foo', {}).setdefault('bar', {}).setdefault('baz', {}), only more compact and supporting arbitrary length paths.

The innermost dictionary is then used to set the value with the last element of the path as the key.

Demo:

>>> def dot_to_json(a):
...     output = {}
...     for key, value in a.iteritems():
...         path = key.split('.')[1:]  # ignore the json. prefix
...         target = reduce(lambda d, k: d.setdefault(k, {}), path[:-1], output)
...         target[path[-1]] = value
...     return output
... 
>>> dot_to_json({'json.message.status.time':50, 'json.message.code.response':80, 'json.time':100}))
{'message': {'status': {'time': 50}, 'code': {'response': 80}}, 'time': 100}

edited Aug 19, 2014 at 18:14

answered Aug 19, 2014 at 17:59

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

BrenBarn Over a year ago

It appears he only wants to ignore the prefix if it is "json", while leaving it if the key begins with something other than "json".

MauricioRoman Over a year ago

I am trying to understand how your answer works. From page 577 in "Learning Python" by O'Reilly, I gather that the reduce function can be expressed in equivalent terms as: def myreduce(function, sequence): tally = sequence[0] for next in sequence[1:]: tally = function(tally, next) return tally However, when applied to the solution above, I get "myreduce() takes exactly 2 arguments (3 given)". Is there an equivalent expression to the reduce function you have invoqued?

Martijn Pieters Over a year ago

@user3394978: I added a start value (the output value); if present it is used instead of sequence[0] for tally, and the loop is over for next in sequence: instead of skipping the first value.

Martijn Pieters Over a year ago

@user3394978: the documentation or reduce() gives a better equivalent.

MauricioRoman Over a year ago

I see from the comment in stackoverflow.com/questions/19589233/… that it indeed has 3 parameters, with the 3rd being the optional initializer.

Collectives™ on Stack Overflow

Dot notation to Json in python

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related