How to convert json response into Python list

Question

I get the JSON response by requests.get

req = requests.get(SAMPLE_SCHEDULE_API)

and convert it into dictionary

data = json.loads(req.text)["data"]

When I tried to convert the string into Python dict,

I got ValueError: malformed node or string:

ast.literal_eval(data)

I have no idea how to do this task.

code snippets

    def schedules(cls, start_date=None, end_date=None):
        import ast
        req = requests.get(SAMPLE_SCHEDULE_API)
        data = json.loads(req.text)["data"]
        ast.literal_eval(data)
        return pd.DataFrame(json.loads(req.text)["data"])

JSON response

{
status: "ok",
version: "v1",
data: "[
{"_id":"2015-01-28","end_date":"2015-01-28","estimated_release":1422453600000,"is_projection":false,"is_statement":true,"material_link":null,"start_date":"2015-01-27"},
{"_id":"2015-03-18","end_date":"2015-03-18","estimated_release":1426687200000,"is_projection":false,"is_statement":false,"material_link":null,"start_date":"2015-03-17"},
{"_id":"2015-04-29","end_date":"2015-04-29","estimated_release":1430316000000,"is_projection":false,"is_statement":false,"material_link":null,"start_date":"2015-04-28"},
{"_id":"2015-06-17","end_date":"2015-06-17","estimated_release":1434549600000,"is_projection":false,"is_statement":false,"material_link":null,"start_date":"2015-06-16"},
{"_id":"2015-07-29","end_date":"2015-07-29","estimated_release":1438178400000,"is_projection":false,"is_statement":false,"material_link":null,"start_date":"2015-07-28"}]"
}

Detail error message

Traceback (most recent call last):
  File "fomc.py", line 25, in <module>
    schedules = FOMC.schedules()
  File "fomc.py", line 21, in schedules
    ast.literal_eval(data)
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 86, in literal_eval
    return _convert(node_or_string)
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 58, in _convert
    return list(map(_convert, node.elts))
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 63, in _convert
    in zip(node.keys, node.values))
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 62, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 85, in _convert
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Name object at 0x10a19c990>

Antti Haapala · Accepted Answer · 2015-03-16 06:41:31Z

3

You have encoded the data twice (which would strictly not be necessary). You just need to decode the data again with json.loads:

def schedules(cls, start_date=None, end_date=None):
    req = requests.get(SAMPLE_SCHEDULE_API)
    data_json = json.loads(req.text)["data"]
    data = json.loads(data_json)
    return pd.DataFrame(data)

Do note that ast.literal_eval is for Python code, whereas json.loads is for JSON that closely follows JavaScript code; the differences are for example true , false and null vs True, False and None. The former are the javascript syntax as used in JSON (and thus you would need json.loads; the latter is Python code, for which you would use ast.literal_eval.

answered Mar 16, 2015 at 6:41

Antti Haapala

135k23 gold badges298 silver badges349 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

BlackFox Over a year ago

i get "list indices must be integers, not str"

salmanwahed · Accepted Answer · 2015-03-16 14:02:59Z

1

As the response already is json format, you do not need to encode it. Approach like this,

req = requests.get(SAMPLE_SCHEDULE_API)
data_str = req.json().get('data')
json_data = json.loads(data_str)

json() method will return the json-encoded content of a response.

edited Mar 16, 2015 at 14:02

answered Mar 16, 2015 at 7:05

salmanwahed

9,6878 gold badges37 silver badges56 bronze badges

Comments

Ulrich Eckhardt · Accepted Answer · 2015-03-16 06:40:11Z

0

The field "data" is a string, not a list. The content of that string seems to be JSON, too, so you have JSON encapsulated in JSON for some reason. If you can, fix that so that you only encode as JSON once. If that doesn't work, you can retrieve that field and decode it separately.

answered Mar 16, 2015 at 6:40

Ulrich Eckhardt

17.7k5 gold badges31 silver badges63 bronze badges

Collectives™ on Stack Overflow

How to convert json response into Python list

code snippets

JSON response

Detail error message

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

code snippets

JSON response

Detail error message

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related