How to convert a string containing a list of dict into Python object?

Question

ugly = "[{'ride': 1, 'pickup_time': datetime(2016, 3, 17, 15, 36, 35, 976202)},
         {'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
# The actual variable contains a lot more dictionnaries...`

I'd like to convert the ugly variable as a real Python object. I tried json.loads() and ast.literal_eval() but it's for one dict only.

That's why, before that, I tried to split this string into several dictionnaries, but the split() method has only one delimiter, so it seems I could need a REGEX to do this.

What is the simplest way to do it? Thanks.

How did your list of dictionaries become a string in the first place? — jDo
– jDo, Commented Mar 17, 2016 at 18:35
Just to note, ast.literal_eval would work fine with a list of dicts; it doesn't like the datetime call. — Aaron Christiansen
– Aaron Christiansen, Commented Mar 17, 2016 at 18:36
@jDo Someone who prefers to store object as string rather than structured way in my company because we don't have time... — David Dahan
– David Dahan, Commented Mar 17, 2016 at 18:42

Serge Ballesta · Accepted Answer · 2016-03-17 18:45:39Z

10

Well first you should be consistent in you calls to datetime method. In first dict, you use datetime and in second datetime.datetime.

Whatever way you try to have Python eval the string, you cannot have at the same time datetime be a function (first dict) and a module (second). Once you fix that, you will be forced to use the evil eval function, because neither json.loads not ast.litteral_eval will accept a function. They are normally used precisely to avoid the evaluation to call any function...

But if this is what you want to do (and you are sure that ugly contains no harmful code), this works:

>>> ugly = "[{'ride': 1, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 36, 35, 976202)},{'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
>>> import datetime
>>> dlist = eval(ugly)
>>> dlist
[{'ride': 1, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 36, 35, 976202)}, {'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]

answered Mar 17, 2016 at 18:45

Serge Ballesta

150k13 gold badges137 silver badges267 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

David Dahan Over a year ago

Thanks! (and the other ones too for their answer)

kpie · Accepted Answer · 2016-03-17 18:35:08Z

6

You can use eval

a = eval(ugly)

At this point a is a list of dictionaries and I am sure you got it from there.

answered Mar 17, 2016 at 18:35

kpie

11.2k7 gold badges33 silver badges53 bronze badges

5 Comments

David Dahan Over a year ago

TypeError: 'module' object is not callable

Bhargav Rao Over a year ago

Please note that eval is dangerous. Use caution while you are using it. See nedbatchelder.com/blog/201206/eval_really_is_dangerous.html

user325117 Over a year ago

@DavidD.you've got a datetime in one record and datetime.datetime in another. You need to make them consistent.

kpie Over a year ago

Bummer, looks like an issue with not having datetime.datetime on pickup-time... like Martin so quickly pointed out.

Steven Rumbalski Over a year ago

eval(ugly.replace(' datetime(', ' datetime.datetime('))

user325117 · Accepted Answer · 2016-03-17 18:50:07Z

0

You can use eval to get the array of dictionaries (after fixing the datetime issue), and then use setattr to turn the dictionaries into real class objects:

from datetime import datetime

ugly = "[{'ride': 1, 'pickup_time': datetime(2016, 3, 17, 15, 36, 35, 976202)}, {'ride': 2, 'pickup_time': datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
array = eval(ugly)

class Ride(object):
    pass

rides = []
for record in array:
    ride = Ride()
    for k, v in record.iteritems():
        setattr(ride, k, v)
    rides.append(ride)

for ride in rides:
    print "ride: {0}: pickup_time: {1}".format(ride.ride, ride.pickup_time)

answered Mar 17, 2016 at 18:50

user325117

Comments

Padraic Cunningham · Accepted Answer · 2016-03-17 20:13:21Z

You could do it with a regex by extracting the datetimes and their keys only calling eval on those pairings:

from datetime import datetime
from ast import literal_eval
import re


def parse(ug):
    ug = ug.replace("datetime.", "")
    pairs = ("{{{}}}".format(p) for p in re.findall("('\w+':\s+datetime\(.*\))", ug))
    _parsed = literal_eval(re.sub("datetime\(.*\)","''", ug))
    for d in _parsed:
        d.update(eval(next(pairs)))
    return _parsed

The lists are going to be ordered so the correct pairing will be put back in the correct dicts:

In [4]: parse(ugly)
Out[4]: 
[{'pickup_time': datetime.datetime(2016, 3, 17, 15, 36, 35, 976202),
  'ride': 1},
 {'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202),
  'ride': 2}]

jDo · Accepted Answer · 2016-03-17 20:21:22Z

0

Depending on how you want to import datetime:

import datetime

ugly = "[{'ride': 1, 'pickup_time': datetime(2016, 3, 17, 15, 36, 35, 976202)},{'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
ugly = ugly.replace(" datetime(", " datetime.datetime(")
ugly = eval(ugly)

or

from datetime import datetime

ugly = "[{'ride': 1, 'pickup_time': datetime(2016, 3, 17, 15, 36, 35, 976202)},{'ride': 2, 'pickup_time': datetime.datetime(2016, 3, 17, 15, 41, 35, 976202)}]"
ugly = ugly.replace("datetime.", "")
ugly = eval(ugly)

Both work without prior cleaning of ugly

edited Mar 17, 2016 at 20:21

answered Mar 17, 2016 at 18:58

jDo

4,0301 gold badge13 silver badges30 bronze badges

Collectives™ on Stack Overflow

How to convert a string containing a list of dict into Python object?

5 Answers 5

1 Comment

5 Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

5 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related