How do you import a CSV whose elements are Numpy arrays?

Question

Due to using a combination of not entirely compatible software packages, I need to be able to read a CSV file whose elements are stylized as Numpy arrays. An example of such a CSV would be the one below:

When using genfromtxt('Input.csv',delimiter=',',dtype=None,encoding=None), I get output of the form:

[['[4 3 2 1]' '[1 1 0 0]']
 ['[1 3 4 2]' '[0 1 1 0]']]

What I would like is to get this into the form of a three-dimensional Numpy array like below:

[[[4 3 2 1] [1 1 0 0]]
 [[1 3 4 2] [0 1 1 0]]]

I'm not too familiar with numpy, but you should be able to use an answer from convert string representation of array to numpy array in python and apply it to each element of the 2D array. — SuperStormer
– SuperStormer, Commented Jul 2, 2022 at 18:31
Try not to save arrays in that format. Parsing it isn't fun. — hpaulj
– hpaulj, Commented Jul 2, 2022 at 18:58

Aaron Meese · Accepted Answer · 2022-07-04 15:09:56Z

1

You may create an array from these strings dynamically (at run-time):

import numpy as np
import ast

# data represents the string returned by str(genfromtxt(...))
data = '[[\'[4 3 2 1]\' \'[1 1 0 0]\'] [\'[1 3 4 2]\' \'[0 1 1 0]\']]'

# remove quotes
data = data.replace('\'', '')
# insert commas
data = data.replace(' ', ', ')

# create an array 
np_data = np.array(ast.literal_eval(data))

# here's your 3D numpy array
print(np_data.shape)
print(np_data)

edited Jul 4, 2022 at 15:09

Aaron Meese

2,3737 gold badges30 silver badges48 bronze badges

answered Jul 2, 2022 at 18:08

Andrey Sozykin

265 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

SuperStormer Over a year ago

unnecessary use of exec when ast.literal_eval or json.loads would work.

SuperStormer Over a year ago

Furthermore, I'm pretty sure genfromtext returns a numpy array, not a string.

Andrey Sozykin Over a year ago

@SuperStormer thanks, I've removed unnecessary use of exec()

Collectives™ on Stack Overflow

How do you import a CSV whose elements are Numpy arrays?

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related