Issue with numpy array shape when creating custom image dataset

Question

I am trying to create a custom dataset for a Deep Learning project using jpg images. I need to read them all in one batch. Doing so using the code below, but my array shape is (100, 1, 224, 224, 3) instead of (100,224, 224, 3). Any suggestions?

path = '/content/drive/My Drive/Dataset/Training'
X=[]
for img in os.listdir(path):
    pic = cv2.imread(os.path.join(path,img))
    pic = cv2.cvtColor(pic,cv2.COLOR_BGR2RGB)
    pic = cv2.resize(pic,(224,224))
    X.append([pic])
X=np.array(X)
print(X.shape)

(100, 1, 224, 224, 3)

X.append(pic) It's the [] that are adding the size 1 dimension. — hpaulj
– hpaulj, Commented Mar 20, 2021 at 15:48
@hpaulj, did you read my answer? I wrote exactly the same thing as you commented now. — Alexander Korovin
– Alexander Korovin, Commented Mar 20, 2021 at 15:53

Alexander Korovin · Accepted Answer · 2021-03-20 21:59:43Z

2

From general point of view, use squeeze from numpy to remove unused dimension (with unit length) from tensor.

For example:

print(np.squeeze(X).shape)

gives you:

(100, 224, 224, 3)

But perhaps in your case it is enough to use X.append(pic) in the line 7 (try to check this).

Tip: try to avoid lists when using numpy. About @hpaulj comment, you can use the concatenate function of numpy instead of lists:

# initialization like X = []
X = np.zeros([0]+list(pic.shape))
...
# append
X = np.concatenate((X, pic.reshape([1]+list(pic.shape))), axis=0)

edited Mar 20, 2021 at 21:59

answered Mar 20, 2021 at 14:36

Alexander Korovin

1,4851 gold badge13 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

hpaulj Over a year ago

lists are best when collecting values iteratively.

Alexander Korovin Over a year ago

@hpaulj I only wrote "try to avoid" (this is not a restriction), as using lists can increase the calculation time.

Collectives™ on Stack Overflow

Issue with numpy array shape when creating custom image dataset

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related