TypeError: Input 'b' of 'MatMul' Op has type float32 that does not match type int32 of argument 'a'

Question

I'm trying to follow word2vec example, but I'm getting this error:

TypeError: Input 'b' of 'MatMul' Op has type float32 that does not match type int32 of argument 'a'.

At this line

similarity = tf.matmul( tf.cast(valid_embeddings,tf.int32), tf.cast(normalized_embeddings,tf.int32), transpose_b=True)

This is the entire code:

graph = tf.Graph()

with graph.as_default():
  # Input data.
  train_inputs = tf.placeholder(tf.int32, shape=[batch_size])
  train_labels = tf.placeholder(tf.int32, shape=[batch_size, 1])
  valid_dataset = tf.constant(valid_examples, dtype=tf.int32)
  # Ops and variables pinned to the CPU because of missing GPU implementation
  with tf.device('/cpu:0'):
    # Look up embeddings for inputs.
    embeddings = tf.Variable(
        tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))
    embed = tf.nn.embedding_lookup(embeddings, train_inputs)
    # Construct the variables for the NCE loss
    nce_weights = tf.Variable(
        tf.truncated_normal([vocabulary_size, embedding_size],
                            stddev=1.0 / math.sqrt(embedding_size)))
    nce_biases = tf.Variable(tf.zeros([vocabulary_size]))
  # Compute the average NCE loss for the batch.
  # tf.nce_loss automatically draws a new sample of the negative labels each
  # time we evaluate the loss.
  loss = tf.reduce_mean(
      tf.nn.nce_loss(nce_weights, nce_biases, embed, train_labels,
                     num_sampled, vocabulary_size))
  # Construct the SGD optimizer using a learning rate of 1.0.
  optimizer = tf.train.GradientDescentOptimizer(1.0).minimize(loss)
  # Compute the cosine similarity between minibatch examples and all embeddings.
  norm = tf.sqrt(tf.reduce_sum(tf.square(embeddings), 1, keep_dims=True))
  normalized_embeddings = embeddings / norm
  valid_embeddings = tf.nn.embedding_lookup(
      normalized_embeddings, valid_dataset)
  similarity = tf.matmul(
      tf.cast(valid_embeddings,tf.int32), tf.cast(normalized_embeddings,tf.int32), transpose_b=True)
  # Add variable initializer.
  init = tf.initialize_all_variables()

How can I fix this?

Charles Liu · Accepted Answer · 2019-03-11 02:00:27Z

I've met the same problem using Tensorflow r1.4 with Python 3.4.

Indeed, I think you need to change the code

tf.nn.nce_loss(nce_weights, nce_biases, embed, train_labels,
                 num_sampled, vocabulary_size))

into

tf.nn.nce_loss(nce_weights, nce_biases, train_labels, embed,
                 num_sampled, vocabulary_size))

or

loss = tf.reduce_mean(tf.nn.nce_loss(
        weights = softmax_weights,
        biases = softmax_biases, 
        inputs = embed, 
        labels = train_labels, 
        num_sampled = num_sampled, 
        num_classes = vocabulary_size))

Meanwhile, you need to change the code back to

similarity = tf.matmul(valid_embeddings, tf.transpose(normalized_embeddings))

It wrong to use tf.cast(..., tf.int32) and actually, there is no need to use tf.cast(..., tf.float32) because it's already been tf.float32.

p.s.

The solution is also useful when you meet the problem while using tf.nn.sampled_softmax_loss() because the usage of sampled_softmax_loss() is quite similar to nce_loss().

Alexandre Passos · Accepted Answer · 2017-08-28 16:29:45Z

1

Why are you doing the matrix multiplication in integer space? You probably want both of those tf.cast to be to tf.float32.

answered Aug 28, 2017 at 16:29

Alexandre Passos

5,2061 gold badge17 silver badges19 bronze badges

Collectives™ on Stack Overflow

TypeError: Input 'b' of 'MatMul' Op has type float32 that does not match type int32 of argument 'a'

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related