PyTorch: Error>> expected scalar type float but found double

Question

I've just started using pytorch and I am trying a simple multi-layer perceptron . My ReLU Activation Function is the following:

def ReLU_activation_func(outputs):
    print(type(outputs))
    result = torch.where(outputs > 0, outputs, 0.)
    result = float(result)
    return result

So I am trying to maintain the value which is greater than 0 and change the value to 0 if the value is smaller than 0. And this is a part of the main code where I use the ReLU Function (where I have the error):

def forward_pass(train_loader):
    for batch_idx, (image, label) in enumerate(train_loader):
        print(image.size())
        x = image.view(-1, 28 * 28)
        print(x.size())
    
        input_node_num = 28 * 28
        hidden_node_num = 100
        output_node_num = 10
        W_ih = torch.rand(input_node_num, hidden_node_num)
        W_ho = torch.rand(hidden_node_num, output_node_num)
        final_output_n = ReLU_activation_func(torch.matmul(x, W_ih))

and when I run the code, I get the following error:

RuntimeError:
1 forward_pass(train_loader)

in forward_pass(train_loader)
-----14         W_ih = torch.rand(input_node_num, hidden_node_num)
-----15         W_ho = torch.rand(hidden_node_num, output_node_num)
---->16         final_output_n = ReLU_activation_func(torch.matmul(x, W_ih))

in ReLU_activation_func(outputs)
-----10     print(type(outputs))
---->11     result = torch.where(outputs > 0, outputs, 0.)
-----12     result = float(result)
-----13     return result

RuntimeError: expected scalar type float but found double

Any help?

Does this answer your question? How to fix RuntimeError "Expected object of scalar type Float but got scalar type Double for argument"? — Gusti Adli
– Gusti Adli, Commented May 22, 2021 at 3:46
I changed the type using "result.float()" but the same error shows up at the same line. — zzz11411
– zzz11411, Commented May 22, 2021 at 3:53

Gusti Adli · Accepted Answer · 2021-05-22 04:29:10Z

1

The issue is not on result, it's either on X, W_ih, or torch.where(outputs > 0, outputs, 0.).

If you don't set an argument for the dtype of torch.rand(), it will assign the dtype based on the pytorch's global default value.

The global variable can be changed using torch.set_default_tensor_type().

Or go the easy route:

def ReLU_activation_func(outputs):
    print((outputs).dtype)
    result = torch.where(outputs > 0, outputs, torch.zeros_like(outputs)).float()
    return result

# for the forward pass function, convert the tensor to floats before matmul
def forward_pass(train_loader):
    for batch_idx, (image, label) in enumerate(train_loader):
        ... <your code>
        X, W_ih = X.float(), W_ih.float()
        final_output_n = ReLU_activation_func(torch.matmul(x, W_ih))

answered May 22, 2021 at 4:29

Gusti Adli

1,2236 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

zzz11411 Over a year ago

I used your easy route but then I get this error: "zeros_like(): argument 'input' (position 1) must be Tensor, not numpy.ndarray" at code line "result = torch.where(outputs > 0, outputs, torch.zeros_like(outputs)).float()"

zzz11411 Over a year ago

when I print(type(outputs)), it shows <class 'torch.Tensor'>, <class 'numpy.ndarray'> as if outputs has two data types

Gusti Adli Over a year ago

that means that the output is a numpy.array. should've used ` result = numpy.where(th.numpy() > 0, th.numpy(), numpy.zeros_like(th.numpy())) and if you want to the output to be a torch tensor you can add result = torch.from_numpy(result) before returning result

zzz11411 Over a year ago

This solved my problem! Thank you so much :) Love you <3

Collectives™ on Stack Overflow

PyTorch: Error>> expected scalar type float but found double

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related