Updating variable values in tensorflow

Question

I've have a basic question about updating the values of tensors via the tensorflow python api.

Consider the code snippet:

x = tf.placeholder(shape=(None,10), ... )
y = tf.placeholder(shape=(None,), ... )
W = tf.Variable( randn(10,10), dtype=tf.float32 )
yhat = tf.matmul(x, W)

Now let's assume I want to implement some sort of algorithm that iteratively updates the value of W (e.g. some optimization algo). This will involve steps like:

for i in range(max_its):
     resid = y_hat - y
     W = f(W , resid) # some update

the problem here is that W on the LHS is a new tensor, not the W that is used in yhat = tf.matmul(x, W)! That is, a new variable is created and the value of W used in my "model" doesn't update.

Now one way around this would be

 for i in range(max_its):
     resid = y_hat - y
     W = f(W , resid) # some update 
     yhat = tf.matmul( x, W)

which results in the creation of a new "model" for each iteration of my loop !

Is there a better way to implement this (in python) without creating a whole bunch of new models for each iteration of the loop - but instead updating the original tensor W "in-place" so to speak?

aarbelle · Accepted Answer · 2016-12-01 20:49:41Z

8

Variables have an assign method. Try:W.assign(f(W,resid))

answered Dec 1, 2016 at 20:49

aarbelle

1,0338 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Lance · Accepted Answer · 2017-09-24 09:07:23Z

2

@aarbelle's terse answer is correct, I'll expand it a bit in case someone needs more info. The last 2 lines below is used for updating W.

x = tf.placeholder(shape=(None,10), ... )
y = tf.placeholder(shape=(None,), ... )
W = tf.Variable(randn(10,10), dtype=tf.float32 )
yhat = tf.matmul(x, W)

...

for i in range(max_its):
    resid = y_hat - y
    update = W.assign(f(W , resid)) # do not forget to initialize tf variables. 
    # "update" above is just a tf op, you need to run the op to update W.
    sess.run(update)

answered Sep 24, 2017 at 9:07

Lance

5514 silver badges5 bronze badges

Comments

guorui · Accepted Answer · 2018-10-23 12:09:42Z

0

Precisely, the answer should be sess.run(W.assign(f(W,resid))). Then use sess.run(W) to show the change.

answered Oct 23, 2018 at 12:09

guorui

9012 gold badges11 silver badges21 bronze badges

1 Comment

Hadij Over a year ago

You forgot to define sess.

Collectives™ on Stack Overflow

Updating variable values in tensorflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related