I want to use a learnable parameter that only takes values between 0 and 1. How can I do this in pytorch?
Currently I am using:
self.beta = Parameter(torch.Tensor(1))
#initialize
zeros(self.beta)
But I am getting zeros and NaN for this parameter, as I train.
You can have a "raw" parameter taking any values, and then pass it through a sigmoid function to get a values in range (0, 1) to be used by your function.
For example:
class MyZeroOneLayer(nn.Module):
def __init__(self):
self.raw_beta = nn.Parameter(data=torch.Tensor(1), requires_grad=True)
def forward(self): # no inputs
beta = torch.sigmoid(self.raw_beta) # get (0,1) value
return beta
Now you have a module with trainable parameter that is effectively in range (0,1)
self.raw_beta will always be a nn.Parameter. The local variable beta inside forward method is not a parameter.