numpy ndarray with more that 32 dimensions

Question

When I try to create an numpy array with more than 32 dimensions I get an error:

import numpy as np

np.ndarray([1] * 33)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-78103e601d91> in <module>()
----> 1 np.ndarray([1] * 33)

ValueError: sequence too large; cannot be greater than 32

I found this: Using numpy.array with large number of dimensions to be related to this question but I want to do this without building my own version.

My use case:
I am working with Joint Probability Distributions and I am trying to represent each variable on an axis so that computations on it (marginalize, reduce) is a single line operation. For example for a marginalize operation I can simply do a sum over the axis of that variable. For multiplication I can simply do a simple numpy multiplication (after checking if the axes are the same).

Is there a possible workaround this?

The workaround depends on the use-case I suppose. Care to give more background information? — Eelco Hoogendoorn
– Eelco Hoogendoorn, Commented Sep 5, 2016 at 8:15
You are aware that a 32-dim array, even if it only has a minimal shape of [2]*32 and float32 dtype, would take up 16gb of memory? — Eelco Hoogendoorn
– Eelco Hoogendoorn, Commented Sep 5, 2016 at 13:58

FMAPR · Accepted Answer · 2021-01-04 17:42:17Z

4

Easy workaround

If you create a np.array(np.array(li))
Where li is a list and len(li) > 32, it'll return an ndarray as intended.

answered Jan 4, 2021 at 17:42

FMAPR

615 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

R_Z · Accepted Answer · 2023-02-14 16:22:34Z

4

Use np.array but not np.ndarray

answered Feb 14, 2023 at 16:22

R_Z

2492 silver badges4 bronze badges

Comments

hpaulj · Accepted Answer · 2016-09-05 16:17:41Z

How about maintaining the data in a 1d, and selectively reshaping to focus on a given dimension

To illustrate

In [252]: x=np.arange(2*3*4*5)

With full reshape

In [254]: x.reshape(2,3,4,5).sum(2)
Out[254]: 
array([[[ 30,  34,  38,  42,  46],
        [110, 114, 118, 122, 126],
        [190, 194, 198, 202, 206]],

       [[270, 274, 278, 282, 286],
        [350, 354, 358, 362, 366],
        [430, 434, 438, 442, 446]]])

With partial reshape - same numbers, different result shape

In [255]: x.reshape(6,4,5).sum(1)
Out[255]: 
array([[ 30,  34,  38,  42,  46],
       [110, 114, 118, 122, 126],
       [190, 194, 198, 202, 206],
       [270, 274, 278, 282, 286],
       [350, 354, 358, 362, 366],
       [430, 434, 438, 442, 446]])

I'm not going to test this on anything approaching 32+ dimensions. As was noted in the comment, if many of the dimensions are larger than 1 the overall array size will be excessively large.

Patrik · Accepted Answer · 2020-11-17 19:36:31Z

0

As of 11/17/2020, the limit to the number of dimensions for numpy arrays remains 32. To find out I ran the following code:

for dim in range (1, 100):
    arr_n_dim_list = [1]*dim
    arr_n_dim = np.ones((arr_n_dim_list))
    print(arr_n_dim.shape)

The last line of the outputs is:

ValueError: maximum supported dimension for an ndarray is 32, found 33

edited Nov 17, 2020 at 19:36

Patrik

4991 gold badge7 silver badges24 bronze badges

answered Nov 17, 2020 at 17:27

Winston Dodson

294 bronze badges

Collectives™ on Stack Overflow

numpy ndarray with more that 32 dimensions

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related