Error when computing summaries in TensorFlow

Question

I am trying to use TensorFlow to produce summaries and visualize them using TensorBoard. However, I am getting an error (InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_1' with dtype float) that I do not understand.

This is the full source of my program:

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

import tensorflow as tf

x = tf.placeholder(tf.float32, [None, 784])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
y = tf.nn.softmax(tf.matmul(x, W) + b)

_ = tf.histogram_summary("weights", W)
_ = tf.histogram_summary("biases", b)
_ = tf.histogram_summary("y", y)


y_ = tf.placeholder(tf.float32, [None, 10])

with tf.name_scope("xent") as scope:
    cross_entropy = -tf.reduce_sum(y_*tf.log(y))
    _ = tf.scalar_summary("cross entropy", cross_entropy)

with tf.name_scope("train") as scope:
    train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)

init = tf.initialize_all_variables()

sess =  tf.Session()
sess.run(init)

with tf.name_scope("test") as scope:
    correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
    _ = tf.scalar_summary("accuracy", accuracy)

merged = tf.merge_all_summaries()
writer = tf.train.SummaryWriter("/tmp/mnist_nn", sess.graph_def)

for i in range(1000):
    if (i % 10) == 0:
        feed = {x: mnist.test.images, y_: mnist.test.labels}
        result = sess.run([merged, accuracy], feed_dict=feed) 
        summary_str = result[0]
        acc = result[1]
        print("Accuracy at step %s: %s" % (i, acc))
    else:
        batch_xs, batch_ys = mnist.train.next_batch(100)
        feed = {x: batch_xs, y_: batch_ys}
        sess.run(train_step, feed_dict=feed)

print(accuracy.eval({x: mnist.test.images, y_: mnist.test.labels}))

However, when I attempt to run the above code, the following error is raised:

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-23-584a7bc91816> in <module>()
     39     if (i % 10) == 0:
     40         feed = {x: mnist.test.images, y_: mnist.test.labels}
---> 41         result = sess.run([merged, accuracy], feed_dict=feed)
     42         summary_str = result[0]
     43         acc = result[1]

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in run(self, fetches, feed_dict)
    366 
    367     # Run request and get response.
--> 368     results = self._do_run(target_list, unique_fetch_targets, feed_dict_string)
    369 
    370     # User may have fetched the same tensor multiple times, but we

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in _do_run(self, target_list, fetch_list, feed_dict)
    442         # pylint: disable=protected-access
    443         raise errors._make_specific_exception(node_def, op, error_message,
--> 444                                               e.code)
    445         # pylint: enable=protected-access
    446       six.reraise(e_type, e_value, e_traceback)

InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_1' with dtype float
     [[Node: Placeholder_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Caused by op u'Placeholder_1', defined at:

It seems from the error that a placeholder in my source has not been fed an appropriate value. As far as I can tell, I am feeding values for all of the placeholders (x and y_).

If you need I'll add the full log to this issue.

I also found that when I first fetch mnist, it does work (with the following output) but still no TensorBoard visualization is produced:

Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting MNIST_data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
Tensor("MergeSummary/MergeSummary:0", shape=TensorShape([]), dtype=string)
merged
Accuracy at step 0: 0.098
Accuracy at step 10: 0.7404
Accuracy at step 20: 0.8041
Accuracy at step 30: 0.814 ...

mrry · Accepted Answer · 2016-01-31 18:40:24Z

13

From your error message, it looks like you are using IPython. One pitfall when using IPython to build a TensorFlow model is that functions like tf.merge_all_summaries() will remember every summary created in the current session, including cells that failed with an error. This is a result of TensorFlow using a default graph to collect all of the operations, summaries, etc. that are created in a process, unless you specify the graph explicitly. I suspect that your call to tf.merge_all_summaries() is returning more than the three histogram summaries that you created in your code, and the older ones will have a dependency on a previously created placeholder.

There are two main ways that you could fix that. The simplest is to explicitly merge the summaries, rather than using tf.merge_all_summaries():

weights_summary = tf.histogram_summary("weights", W)
biases_summary = tf.histogram_summary("biases", b)
y_summary = tf.histogram_summary("y", y)

merged = tf.merge_summary([weights_summary, biases_summary, y_summary])

An alternative would be to set an explicit default graph before constructing your model. This is awkward if you want to split your model across multiple IPython cells, but should also work:

# Sets a new default graph, and stores it in `g`.
with tf.Graph().as_default() as g:  

    x = tf.placeholder(tf.float32, [None, 784])
    # ...

    # Summaries are added to `g`.
    _ = tf.histogram_summary("weights", W)
    _ = tf.histogram_summary("biases", b)
    _ = tf.histogram_summary("y", y)

    # ...

    # `merged` contains only summaries from `g`.
    merged = tf.merge_all_summaries()

    # ...

answered Jan 31, 2016 at 18:40

mrry

126k27 gold badges404 silver badges401 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

seanv507 Over a year ago

So is it 'best practise' to not to use interactive sessions - neither of the solutions seem ideal. One would have hoped there would be a way of 'clearing' the default graph

mrry Over a year ago

You can clear the default graph by calling tf.reset_default_graph(), but any previously created tf.InteractiveSession objects won't see any newly created nodes. I'd recommend steering clear of any functions that implicitly depend on "all" previously created objects (tf.merge_all_summaries(), tf.initialize_all_variables()) if you have complex IPython sessions that span multiple cells.

BlackBear · Accepted Answer · 2018-03-22 18:38:19Z

6

for keras users

you might encounter this error when using the TensorBoard callback and fitting a new model from scratch. In this case, the solution is to call

from keras import backend as K
K.clear_session()

before creating the new model. verified with keras 2.1.5 and tensorflow 1.6.0

answered Mar 22, 2018 at 18:38

BlackBear

23.1k10 gold badges53 silver badges90 bronze badges

1 Comment

Ezekiel Kruglick Over a year ago

Wish I could upvote this more, this should be highly featured in the keras manual. Bit me too.

Collectives™ on Stack Overflow

Error when computing summaries in TensorFlow

2 Answers 2

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related