Numpy Array Not JSON Serializable Even after tolist()

Question

I have a nested list like this:

my_numpy_values = my_numpy_array.tolist()
>> [[1,2,3,4],
[5,6,7,8],
[9,10,11,12]]

I then set my preexisting dict values equal to the above so that I have:

my_dict
>>{'key1':[[1,2,3,4],[5,6,7,8]...], 'key2':[[1,2,3,4],[5,6,7,8]...],'key3'...}

I then simply wrapped it into a list:

json_data = [my_dict]

with open('data.json', 'w') as outfile:
    json.dump(json_data, outfile)

This is the point where I got the not json serializable error. However, from what I read tolist() is supposed to be supported. What should I be doing differently? Surely I do not have to write my own encoding serializer for something so simple?

Update

It's a little challenging to make a Minimal Complete Verifiable Example for this, please bear with me. I tried for minimal, and for whatever reason it made the error go away. Completeness may be harder to reproduce, many libraries and calculations are needed to get the values I'm working with. The best I can do is offer a screenshot of the error in action:

This is only a small part of the verbose, so amidst all the bells and whistles I missed the circular reference. I'm not sure if I have a circular reference problem in addition to non-json serializable problem, or if it's more of a cause and effect type of thing.

Either way, I don't under stand why my minimal [[1,2,3,4],[5,6,7,8]...] example as seen at the beginning of my post works and my real data doesn't. The only difference I can see are the values; the structure of the data looks identical. Am I missing something?

This is hard to answer without a minimal reproducible example. — Kevin
– Kevin, Commented May 17, 2017 at 18:57
Are you somehow appending the list to itself? The ellipses make me suspicious. If the list recursively contains itself, it would not be serializable. But it should give a message that says 'circular reference detected.' — ely
– ely, Commented May 17, 2017 at 19:01
Sounds like you might have gotten your shapes mixed up at some point, resulting in an array with other array objects inside. — user2357112
– user2357112, Commented May 17, 2017 at 19:26
When you say your dict looks like {'key1':[[1,2,3,4],[5,6,7,8]...], 'key2':[[1,2,3,4],[5,6,7,8]...],'key3'...}, are those "..."s actually present in your actual output, or did you put them into your example to indicate that the data is larger than just those values? If it's the latter, you probably don't have a circular reference problem. — Kevin
– Kevin, Commented May 17, 2017 at 19:28
@Kevin yea the "..."s are just for the example. Thanks for helping me troubleshoot it! — Arash Howaida
– Arash Howaida, Commented May 17, 2017 at 19:30

ely · Accepted Answer · 2017-05-17 19:06:00Z

1

I cannot reproduce a problem with your data nor the particular format passed to json.dump:

In [16]: test = [
    {"key1": [[1,2,3,4], [5,6,7,8], [9,10,11,12]], 
     "key2": [[1,2,3,4], [5,6,7,8], [9,10,11,12]], 
     "key3": [[1,2,3,4], [5,6,7,8], [9,10,11,12]]}
]

In [17]: with open("test.json", "w") as f:
    ...:     json.dump(test, f)
    ...:     

In [18]: !cat test.json
[{"key1": [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], "key2": [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], "key3": [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]}]

answered May 17, 2017 at 19:06

ely

77.8k36 gold badges158 silver badges234 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Arash Howaida Over a year ago

Interesting developments, I did confirm that my minimal [[1,2,3,4],[5,6,7,8]... example does work, but when using my real data, it doesn't. I have updated my post with as much information as I could.

Collectives™ on Stack Overflow

Numpy Array Not JSON Serializable Even after tolist()

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related