TensorFlow Parallel CPU with Numpy

Question

I am trying to find a way to speed my code up.

In short, I have a trained model which I use something like to obtain predicts, sort them and output a rank.

def predict(feed_dict, truth):
    # Feed dict contains about 10K candidates to obtain scores
    pred = self.sess.run([self.mdl.predict_op], feed_dict)
    pred = np.array(pred)
    # With the scores, I sort them by likelihood
    sort = np.argsort(pred)[::-1]
    # I find the rank of the ground truth 
    rank = np.where(sort==truth)[0][0] + 1
    return rank

However, this process is extremely slow. I have about 10K test samples. I believe session doesn't work well with standard multiprocessing libraries in python while the multi-cpu/gpu support is only available for tensorflow ops.

Is there any elegant way to speed this up via multiprocessing? Or do I have to implement it as part of the computational graph in TF.

Thanks a lot!

BTW, tf.nn.top_k(pred)[1] is the same as your np.argsort line. If you turn everything into TF graph you won't need multiprocessing -- parallel session.run calls can be started from different Python threads in the same process. — Yaroslav Bulatov
– Yaroslav Bulatov, Commented Dec 23, 2016 at 16:55
The slow comes from the fact that I have to call this 10K+ times each time either on the valid or test set. — op10no4
– op10no4, Commented Dec 23, 2016 at 17:20
Thanks! Btw, do you know what is the equivalent for np.where in TF? Thanks so much — op10no4
– op10no4, Commented Dec 23, 2016 at 17:21
it's tf.where in version 0.12 (tf.select in earlier versions) — Yaroslav Bulatov
– Yaroslav Bulatov, Commented Dec 23, 2016 at 17:30

Yaroslav Bulatov · Accepted Answer · 2016-12-23 17:45:23Z

3

You can translate whole thing into TensorFlow graph:

pred_op = tf.constant([1,2,0])
truth = [0, 1, 2]
sess = tf.Session()
sort = tf.nn.top_k(pred_op)[1] # same as np.argsort(x)[::-1]
rank = tf.where(tf.equal(sort,truth))[0][0] + 1
print(sess.run(rank))  # => 2

answered Dec 23, 2016 at 17:45

Yaroslav Bulatov

58.1k23 gold badges147 silver badges210 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

TensorFlow Parallel CPU with Numpy

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related