Convert np array of arrays to torch tensor when inner arrays are of different sizes

Question

I have several videos, which I have loaded frame by frame into a numpy array of arrays. For example if I have 8 videos, they are converted into an 8 dimensional numpy array of arrays where each inner array has a different dimension depending on the number of frames of the individual video. When I print

array.shape

my output is (8,)

Now I would like to create a dataloader for this data, and for that I would like to convert this numpy array into a torch tensor. However when I try to convert it using the torch.from_numpy or even simply the torch.tensor functions I get the error

TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.

which I assume is because my inner arrays are of different sizes. One possible solution is to artificially add a dimension to my videos to make them be of the same size and then use np.stack but that may lead to possible problems later on. Is there any better solution?

Edit: Actually adding a dimension won't work because np.stack requires all dimensions to be the same.

Edit: Sample Array would be something like:

[ [1,2,3], [1,2], [1,2,3,4] ]

This is stored as a (3,) shaped np array. The real arrays are actually 4-dimensional( Frames x Height x Width x Channels), so this is just an example.

One option is to adjust all sizes to be the same by padding with zeros. Second and in my opinion better way is just create separate tensor for each video and store them in a list. — V. Ayrat
– V. Ayrat, Commented May 23, 2020 at 10:31
@V.Ayrat Yeah for the moment I guess I am going to create a list and then use a dataloader on the list. — Rijul Ganguly
– Rijul Ganguly, Commented May 23, 2020 at 12:41

Dishin Goyani · Accepted Answer · 2020-05-23 10:36:54Z

1

You can use rnn util function pad_sequence to make them same size.

ary
array([list([1, 2, 3]), list([1, 2]), list([1, 2, 3, 4])], dtype=object)

from torch.nn.utils.rnn import pad_sequence
t = pad_sequence([torch.tensor(x) for x in ary], batch_first=True)

t
tensor([[1, 2, 3, 0],
        [1, 2, 0, 0],
        [1, 2, 3, 4]])
t.shape
torch.Size([3, 4])

edited May 23, 2020 at 10:36

answered May 23, 2020 at 10:29

Dishin Goyani

7,7533 gold badges33 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Rijul Ganguly Over a year ago

This would work fine in most cases, however I would like to avoid having to modify my images in any way before passing into the network, apart from normalization.

Collectives™ on Stack Overflow

Convert np array of arrays to torch tensor when inner arrays are of different sizes

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related