Numpy: creating an empty array results in memory error?

Question

I need to store a ton of information in a numpy array. It needs to be of the following shape:

facefeature1s = np.empty([2000,64,64,64,32])

When I run this, i get a memory error. What can I do about this?

Error is:

    MemoryError                               Traceback (most recent call last)
<ipython-input-271-2c56a37b4a7c> in <module>()
----> 1 facefeature1s = np.empty([2000,64,64,64,32])

Either buy 134 GB of RAM or make a smaller array.

Jaime
– Jaime

2014-05-16 04:24:54 +00:00
Commented May 16, 2014 at 4:24 — Jaime
– Jaime, Commented May 16, 2014 at 4:24

Saullo G. P. Castro · Accepted Answer · 2014-05-16 05:39:43Z

8

As @Jaime says in the comments, your array is too big. IF you really need such a huge array, you can use numpy.memmap() to work on the array using the hard drive:

a = np.memmap('filename.myarray', dtype=np.float64, mode='w+',
              shape=(2000, 64, 64, 64, 32))

The next time you open the array, use mode='r', or mode='r+'.

answered May 16, 2014 at 5:39

Saullo G. P. Castro

59.4k28 gold badges191 silver badges244 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

wolfsatthedoor Over a year ago

Weird, I have hundreds of Gigs left but it says the following error: [Errno 28] No space left on device

Collectives™ on Stack Overflow

Numpy: creating an empty array results in memory error?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related