6

I am trying to use Pytorch to run classification on a dataset of images of cats and dogs. In my code I am so far downloading the data and going into the folder train which has two folders in it called "cats" and "dogs." I am then trying to load this data into a dataloader and iterate through batches, but it is giving me some error I don't understand in the iteration step.

Since it is Google Colabs I have code in there for downloading data and installing libraries. Any other advice on my code so far would be appreciated as well.

!pip install torch
!pip install torchvision

from __future__ import print_function, division
import os
import torch
import pandas as pd
import numpy as np
# For showing and formatting images
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

# For importing datasets into pytorch
import torchvision.datasets as dataset

# Used for dataloaders
import torch.utils.data as data

# For pretrained resnet34 model
import torchvision.models as models

# For optimisation function
import torch.nn as nn
import torch.optim as optim


!wget http://files.fast.ai/data/dogscats.zip
!unzip dogscats.zip    

batch_size = 256

train_raw = dataset.ImageFolder(PATH+"train", transform=transforms.ToTensor())
train_loader = data.DataLoader(train_raw, batch_size=batch_size, shuffle=True)

for batch_idx, (data, target) in enumerate(train_loader):
  print("Data: ", batch_idx)

The error comes up on the last lines and is below:

RuntimeErrorTraceback (most recent call last)
<ipython-input-66-c32dd0c1b880> in <module>()
----> 1 for batch_idx, (data, target) in enumerate(train_loader):
      2   print("Data: ", batch_idx)
      3 

/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.pyc in __next__(self)
    257         if self.num_workers == 0:  # same-process loading
    258             indices = next(self.sample_iter)  # may raise StopIteration
--> 259             batch = self.collate_fn([self.dataset[i] for i in indices])
    260             if self.pin_memory:
    261                 batch = pin_memory_batch(batch)

/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.pyc in default_collate(batch)
    133     elif isinstance(batch[0], collections.Sequence):
    134         transposed = zip(*batch)
--> 135         return [default_collate(samples) for samples in transposed]
    136 
    137     raise TypeError((error_msg.format(type(batch[0]))))

/usr/local/lib/python2.7/dist-packages/torch/utils/data/dataloader.pyc in default_collate(batch)
    110             storage = batch[0].storage()._new_shared(numel)
    111             out = batch[0].new(storage)
--> 112         return torch.stack(batch, 0, out=out)
    113     elif elem_type.__module__ == 'numpy' and elem_type.__name__ != 'str_' \
    114             and elem_type.__name__ != 'string_':

/usr/local/lib/python2.7/dist-packages/torch/functional.pyc in stack(sequence, dim, out)
     62     inputs = [t.unsqueeze(dim) for t in sequence]
     63     if out is None:
---> 64         return torch.cat(inputs, dim)
     65     else:
     66         return torch.cat(inputs, dim, out=out)

RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 400 and 487 in dimension 2 at /pytorch/torch/lib/TH/generic/THTensorMath.c:2897

Thanks

Improve this question

3 Answers 3

Reset to default
3

I think the main problem was images being of different size . I may have understood ImageFolder in other way but, i think you don't need labels for images if the directory structure is as specified in pytorch and pytorch will figure out the labels for you. I would also add more things to your transform that automatically resizes every images from the folder such as:

   normalize = transforms.Normalize(
                        mean=[0.485, 0.456, 0.406],
                        std=[0.229, 0.224, 0.225]
                        )
   transform = transforms.Compose(
        [transforms.ToTensor(),transforms.Resize((224,224)),
         normalize])

Also you can use other tricks to make your DataLoader much faster such as adding batch_size and number of cpu workers such as:

    testloader = DataLoader(testset, batch_size=16,
                         shuffle=False, num_workers=4)

I think this will make you pipeline much faster.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Wow, thanks Manoj. I ran this last night my way and after a few hours it had barley gotten through all the training data and the loss had barley moved. Now I implemented your changes and after a few minutes its already at 8% and the loss has fallen drastically to less than my'n was last night after a few hours and lots more data. Two more questions if you don't mind. 1) How did you get the values for the mean and std in Normalize 2) How did you come up with the batch_size.
I found the following documentation, but it doesn't explain how to derive the values for it. pytorch.org/docs/master/torchvision/transforms.html
These values are given for pretrained pytorch model in the documentation here pytorch.org/docs/master/torchvision/models.html. For batch size you just experiment but usually 256 images in a batch is fine for a GPU.
1

I see two problems in your code first you are importing import torch.utils.data as data and again replacing that in the data loader. Please keep the imported module and your variable name in separate namespace. I think this error could be because of different sizes of data returned by dataloder(images) and labels. As you can see there is an error in concatenation because the first dimension ie. the label size and number of images in folder do not match. Hope this helps.

Improve this answer

1 Comment

Thanks for your reply Manoj Acharya, I have fixed up the first problem renaming torch.utils.data as data_util, but was a bit stuck with the second problem. I don't think I quite understand exactly how dataset.ImageFolder works. In the path I gave it there are two folders dogs and cats each of which is filled with images of the animal. So I don't understand how the labels can't match the images as the image is labelled based on what folder it is in. Does the dataset.ImageFolder pick this up? or do I need another labels file as a .csv or something?
0

I think I was wrong in my comment to Manoj Acharya, the problem was in the batch_size being put into the dataloader. I read the below source and it seems you can't batch images together with different sizes:

https://medium.com/@yvanscher/pytorch-tip-yielding-image-sizes-6a776eb4115b

So in my code after changing the data variable Manoj points out I changed the batch_size to 1 and the program stopped failing. I want to put it in batches though so I added a further transform CenterCrop() to resize all images to the same size. Below is my new code:

!pip install torch
!pip install torchvision

from __future__ import print_function, division
import os
import torch
import pandas as pd
import numpy as np
# For showing and formatting images
import matplotlib.pyplot as plt
import matplotlib.image as mpimg    
# For importing datasets into pytorch
import torchvision.datasets as dataset    
# Used for dataloaders
from torch.utils.data import DataLoader
# For pretrained resnet34 model
import torchvision.models as models    
# For optimisation function
import torch.nn as nn
import torch.optim as optim    
# For turning data into tensors
import torchvision.transforms as transforms

!wget http://files.fast.ai/data/dogscats.zip
!unzip dogscats.zip

batch_size = 256
sz = 224

train_raw = dataset.ImageFolder(PATH+"train", transform=transforms.Compose([transforms.CenterCrop(sz),transforms.ToTensor()]))
train_loader = DataLoader(train_raw,batch_size=batch_size, shuffle=True)

for batch_idx, (data, target) in enumerate(train_loader):
  print("Data: ", batch_idx)

Thanks

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.