0

Hey I am new to python and am working on some project where I have a list of arrays like -

some_list = [array([-12, 23]), array([-13, 22])]

These array represent unique property of some stuff. So I want to save this in a file like Json with some Id assigned to each array and then extract it back as the same list of arrays.

11
  • 2
    Are these Numpy arrays, stdlib array.arrays, or something different? Commented May 30, 2018 at 18:09
  • Yes they are Numpy arrays. Commented May 30, 2018 at 18:10
  • Also, what do you mean by "some Id assigned to each array"? Getting them back in the same order as the original list makes sense, but you don't need IDs for that, and it's hard to see what you're do with them—you just want to preserve the order when you store and load them, which is exactly what happens when you save a Python list as a JSON array. Commented May 30, 2018 at 18:10
  • These array represent property of some known objects. Now I want to save them in a file and when I get an unknown object I want to compare it with my stored list of arrays and If matched return its id or else append it to the list with a unique id. Commented May 30, 2018 at 18:13
  • Related (?) Numpy array is not JSON serializable.. there are others. Commented May 30, 2018 at 18:13

1 Answer 1

1

The json module only knows how to handle the basic Python types that it maps to the basic JSON types—list, dict, str, float, bool, and NoneType.

But you can override the JSON encoder and decoder objects to add in code to handle additional types in any way you want. This is documented, with some examples, but isn't entirely trivial.


But, more simply, if you know the structure of the data you're trying to save and load, you can just transform it on the fly. For example:

def save_stuff(f, list_of_arrays):
    list_of_lists = [list(arr) for arr in list_of_arrays]
    json.dump(f, list_of_lists)

def load_stuff(f):
    list_of_lists = json.load(f)
    list_of_arrays = [np.array(lst) for lst in list_of_lists]
    return list_of_arrays

Or you can go the opposite direction: convert the list of arrays to an array of arrays and use Numpy to save that.

Either way, this is less efficient, because you have to create those temporary lists. If you have gigantic arrays, or if you're doing this zillions of times, you will probably want to do the more complicated work of overriding the encoder and decoder.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.