Apply own activation function to layer in tensorflow

Question

I'm using a model where the tensorflow relu function is used for the activation of the hidden layers. So basically the model does this

h = tf.nn.relu(zw)

where zw are all the elements from the output from the previous layer times weights. According to the definition of relu of tensorflow it will return

max(zw,0)

so the max number between 0 and the value of each element of zw for each element of the tensor.

How can I apply my own relu function where I return the element zw if it is above 0 and the zw element times 0.1 if it is smaller than 0?

Avishkar Bhoopchand · Accepted Answer · 2017-08-24 15:22:54Z

1

You could do something like this:

h = tf.where(zw < 0, 0.1 * zw, zw)

answered Aug 24, 2017 at 15:22

9298 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1