MemoryError in numpy normalization

Question

I am working on a large dataset consisting images.

When I run the following code:

data=[]
def image_to_feature_vector(image, size=(128, 128)):
   return cv2.resize(image, size).flatten()

for i in range(0,len(imagePath)):
    image = cv2.imread(imagePath[i])
    features = image_to_feature_vector(image)
    data.append(features)

data = np.array(data) / 255.0

I got an error as:

np.array(data) / 255.0

MemoryError

How to fix this? Thanks in advance!!!

Let me guess, 32bit build of Python?

Dan Mašek
– Dan Mašek

2017-11-29 16:56:56 +00:00
Commented Nov 29, 2017 at 16:56 — Dan Mašek
– Dan Mašek, Commented Nov 29, 2017 at 16:56
@DanMašek No, I am using 64bit build version.

snfs
– snfs

2017-11-29 17:07:24 +00:00
Commented Nov 29, 2017 at 17:07 — snfs
– snfs, Commented Nov 29, 2017 at 17:07

Paul Panzer · Accepted Answer · 2017-11-29 17:57:25Z

2

Some easy memory saving strategies include

1 preallocate data and avoid creating a temporary list

data = np.empty((len(imagePath),) + features_shape)
for i, slc in enumerate(data):
     ...
     slc[...] = features

2 use in-place operations where possible

data /= 255.0

edited Nov 29, 2017 at 17:57

answered Nov 29, 2017 at 16:59

Paul Panzer

53.3k3 gold badges60 silver badges103 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

MemoryError in numpy normalization

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related