Tensorflow and Numpy mismatch data format

Question

Whit this code:

import tensorflow as tf  

w = tf.Variable(tf.random_normal( [ 3 , 3 , 1 , 1  ], stddev = 0.01 )) 
if __name__ == '__main__': 
    initVar = tf.global_variables_initializer()  
    with tf.Session() as sess:  
        sess.run(initVar) 
        print w.eval()

Because of the data format w = tf.Variable(tf.random_normal( [kernel_height, kernel_width, input_channel, output_chhannel], stddev = 0.01 )), I would expect to see a matrix like this:

[[[[ -0.004  0.003  0.006]
   [ -0.005 -0.008  0.001]
   [  0.006  0.007  0.002]]]]

but it prints this:

[[[[ 0.001]] 
  [[-0.031]] 
  [[-0.005]]]

 [[[ 0.006]] 
  [[ 0.011]] 
  [[ 0.006]]]

 [[[ 0.008]] 
  [[-0.001]] 
  [[ 0.014]]]]

What I want is to multiply my weight tensor values one-by-one with a constant tensor of 0 and 1 to have masked weights like:

w = [[[[ -0.004  0.003  0.006]
       [ -0.005 -0.008  0.001]
       [  0.006  0.007  0.002]]]]

mask = [[[[ 1  1  1]
          [ 1  1  0]
          [ 0  0  0]]]]

w * mask =  [[[[ -0.004  0.003  0.006]
               [ -0.005 -0.008  0.   ]
               [  0.     0.     0.   ]]]]

The code I used it this:

    mask = np.ones((3, 3, 1, 1), dtype=np.float32)
    mask[1, 2, :, :] = 0.
    mask[2, :, :, :] = 0. 

    weight = tf.get_variable("weight", [3, 3, 1, 1], tf.float32, tf.contrib.layers.xavier_initializer()) 

    weight *= tf.constant(mask, dtype=tf.float32)

But seems it doesn't work properly. I appreciate your help.

danche · Accepted Answer · 2017-06-30 14:47:18Z

2

You need

w = tf.Variable(tf.random_normal([1, 1, 3, 3], stddev=0.01))

and finally, you can use

import tensorflow as tf 
import numpy as np

mask = np.ones((1, 1, 3, 3), dtype=np.float32)
mask[:, :, 1, 2] = 0.
mask[:, :, 2, :] = 0. 

print(mask)

weight = tf.get_variable("weight", [3, 3, 1, 1], tf.float32, tf.contrib.layers.xavier_initializer())
weight *= tf.transpose( tf.constant(mask, dtype=tf.float32) )

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(tf.transpose(weight).eval())

you will get

[[[[ 1.  1.  1.]
   [ 1.  1.  0.]
   [ 0.  0.  0.]]]]

[[[[ 0.88993669  0.80872607  0.57259583]
   [ 0.5067296  -0.20804334 -0.        ]
   [ 0.          0.          0.        ]]]]

edited Jun 30, 2017 at 14:47

answered Jun 29, 2017 at 12:47

danche

1,81517 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Ali Over a year ago

thank you for your detailed answer, but as I mention I use this data format [kernel_height, kernel_width, input_channel, output_channel] in my model. So your proposed approach doesn't match in my task. Since you used [ input_channel, output_channel, kernel_height, kernel_width]

danche Over a year ago

@AliAbbasi tf has fixed data format as other dl platform. If you are processing image, W should be [BHWC] format.

Ali Over a year ago

So I wonder the Tensorflow use BHWC but with Numpy, it in reverse order BCHW, so you say that it doesn't problem, and TF can handle it? am I right?

danche Over a year ago

@AliAbbasi sorry I couldn't get your point. I think you just need to format the mask with appropriate shape.

Ali Over a year ago

I mean in Tensorflow the weights are in [kernel_height, kernel_width, input_channel, output_channel] format, and when I want to create a 4 dimension Numpy array as Mask, Numpy create it as [ input_channel, output_channel, kernel_height, kernel_width] format, my question is that how to handle this mismatching format?

|

Collectives™ on Stack Overflow

Tensorflow and Numpy mismatch data format

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related