1

I have some questions regarding this Code - neural networks in TensorFlow. 

#!/usr/bin/env python

import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data


def init_weights(shape):
    return tf.Variable(tf.random_normal(shape, stddev=0.01))


def model(X, w_h, w_o):
    h = tf.nn.sigmoid(tf.matmul(X, w_h)) # this is a basic mlp, think 2 stacked logistic regressions
    return tf.matmul(h, w_o) # note that we dont take the softmax at the end because our cost fn does that for us


mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
trX, trY, teX, teY = mnist.train.images, mnist.train.labels, mnist.test.images, mnist.test.labels

X = tf.placeholder("float", [None, 784])
Y = tf.placeholder("float", [None, 10])

w_h = init_weights([784, 625]) # create symbolic variables
w_o = init_weights([625, 10])

py_x = model(X, w_h, w_o)

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=py_x, labels=Y)) # compute costs
train_op = tf.train.GradientDescentOptimizer(0.05).minimize(cost) # construct an optimizer
predict_op = tf.argmax(py_x, 1)

# Launch the graph in a session
with tf.Session() as sess:
    # you need to initialize all variables
    tf.global_variables_initializer().run()

    for i in range(100):
        for start, end in zip(range(0, len(trX), 128), range(128, len(trX)+1, 128)):
            sess.run(train_op, feed_dict={X: trX[start:end], Y: trY[start:end]})
        print(i, np.mean(np.argmax(teY, axis=1) ==
                         sess.run(predict_op, feed_dict={X: teX})))

  • After a single run of the loop at line 37, how do i call model() with X[0] and the newly learnt w_h and w_o , so that I can see the function returns

  • Similarly, how to print the values of h inside the model() function ?

Thanks in advance. I am new to tensorFlow :)

Improve this question
2
  • After each loop at line 37, at line 40 model is being called. X[0] does not make any sense as X is merely a placeholder. teX in line 41 fulfills the purpose of X Commented Feb 26, 2017 at 17:49
  • Just to clarify this: 'model is being used' is probably more exact since the function model() is called only once. Commented Feb 27, 2017 at 21:57

2 Answers 2

Reset to default
3

The feed_dict translates the placeholders into real value(s). So provide a single entry for feed_dicts and evaluate py_x.

The following should work:

For the result (px_y):

print(sess.run(py_x, feed_dict={X: [yoursample]}))

For h it's (almost) the same. But as in the linked code h is a private member of model() you'll need a reference to h in order to evaluate it. The easiest way most probably is to replace lines:

(14) return tf.matmul(h, w_o)
with
(14) return (tf.matmul(h, w_o), h)

(26) py_x = model(X, w_h, w_o)
with
(26) py_x, h = model(X, w_h, w_o)

and use:

print(sess.run(h, feed_dict={X: [yoursample]}))

or alternatively (evaluate multiple variables):

py_val, h_val = sess.run([py_x, h], feed_dict={X: [yoursample]})
print(py_val, h)

Explanation: By the way we told Tensorflow how our Network is constructed we did not need an explicit reference to the (inner/hidden) variable h. But in order to evaluate it we do need the reference to define what exactly to evaluate.

There are other ways to get the variables out of the guts of Tensorflow, but as we explicitly create this variable a few lines above I'd avoid dropping something into a black box and ask the very same black box later on to give it back.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

thanks. what about the 2nd bullet point ? Should another function be written that just returns h ? Or there a better way to do this ?
As this question might be interesting for others as well I covered that as well and added some comments.
thank you, got what i was looking :) Can you suggest some good tutorials for TensorFlow, its a bit tough to understand their documentation. Thanks again.
Indeed it is.. Personally I started trying to understand their tutorials but failed to do so and I so used their final code per tutorial to ping-pong try and understand what they do and their explanations. I like the MNIST examples as images (of written digits) are easier to visualize, reproduce and understand.
0

Regarding the second question:

  • Similarly, how to print the values of h inside the model() function ?

With the function tf.Print

def model(X, w_h, w_o):
    print_h = tf.nn.sigmoid(tf.matmul(X, w_h)) # this is a basic mlp, think 2 stacked logistic regressions
    h = tf.Print(print_h,[print_h]) # add a node to the execution graph that prints h when ever h is needed and executed in the graph
    return tf.matmul(h, w_o) # note that we dont take the softmax at the end because our cost fn does that for us

Good explanation can be found here: https://towardsdatascience.com/using-tf-print-in-tensorflow-aa26e1cff11e

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.