Conditional execution in TensorFlow

Question

How can I choose to execute a portion of the graph based on a condition?

I have a part of my network which is to be executed only if a placeholder value is provided in feed_dict. An alternate path is taken if the value is not provided. How do I go about implementing this using tensorflow?

Here are the relevant portions of my code:

sess.run(accuracy, feed_dict={inputs: mnist.test.images, outputs: mnist.test.labels})

N = tf.shape(outputs)
    cost = 0
    if N > 0:
        y_N = tf.slice(h_c, [0, 0], N)
        cross_entropy = tf.nn.softmax_cross_entropy_with_logits(y_N, outputs, name='xentropy')
        cost = tf.reduce_mean(cross_entropy, name='xentropy_mean')

In the above code, I'm looking for something to use in the place of if N > 0:

dga · Accepted Answer · 2015-11-13 06:29:39Z

Hrm. It's possible that what you want is tf.control_flow_ops.cond() https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/control_flow_ops.py#L597

But that's not exported into the tf namespace, and I'm answering without checking how guaranteed-stable this interface is, but it's used in released models, so go for it. :)

However: Because you actually know in advance what path you want when you construct the feed_dict, you could also take a different approach of invoking a separate path through your model. The standard way to do this is to, e.g., set up code like:

def model(input, n_greater_than):
  ... cleverness ...
  if n_greater_than:
     ... other cleverness...
  return tf.reduce_mean(input)


out1 = model(input, True)
out2 = model(input, False)

And then pull the out1 or out2 nodes depending upon what you know when you're about to run your computation and set the feed_dict. Remember that by default, if the model references the same variables (create them outside the model() func), then you'll basically have two separate paths through.

You can see an example of this in the convolutional mnist example: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/models/image/mnist/convolutional.py#L165

I'm a fan of doing it this way without introducing control flow dependencies if you can.

Salvador Dali · Accepted Answer · 2015-11-13 06:25:26Z

0

Here is a simple example, that can get you started. It executes different parts of the graph based on the shape of the tensor:

import tensorflow as tf

a = tf.Variable([[3.0, 3.0], [3.0, 3.0]])
b = tf.Variable([[1.0, 1.0], [2.0, 2.0]])
l = tf.shape(a)

add_op, sub_op = tf.add(a, b), tf.sub(a, b)

sess = tf.Session()
init = tf.initialize_all_variables()
sess.run(init)
t = sess.run(l)

print sess.run(sub_op if t[0] == 3 else add_op)

sess.close()

Change 3 to 2 to see how tensor will be subtracted. As you see I initiated the nodes for add and sub and shape, then in the graph I check for the shape and go run the specific part.

answered Nov 13, 2015 at 6:25

Salvador Dali

224k151 gold badges727 silver badges766 bronze badges

Collectives™ on Stack Overflow

Conditional execution in TensorFlow

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related