Input parameters of tf.contrib.learn.read_batch_features

Question

I am working through these tensorflow codes which implement a LSTM in tensorflow. While going through the codes, I came across this function (in input_fn code - line 38) tf.contrib.learn.read_batch_features. I looked up the documentation of tf.contrib.learn.read_batch_features here. This is what I got -

file_pattern: List of files or pattern of file paths containing Example records. 
batch_size: An int or scalar Tensor specifying the batch size to use.
features: A dict mapping feature keys to FixedLenFeature or VarLenFeature values.
randomize_input: Whether the input should be randomized.
num_epochs: Integer specifying the number of times to read through the dataset. If None, cycles through the dataset forever. NOTE - If specified, creates a variable that must be initialized, so call tf.local_variables_initializer() as shown in the tests.
queue_capacity: Capacity for input queue.
reader_num_threads: The number of threads to read examples.
name: Name of resulting op.

There are few input parameters that I am not able to understand and was hoping someone could help me with it.

The randomize_input parameter. Does it mean it will shuffle the entire dataset?
For num_epochs, if I specify None does it mean that my input_fn will keep feeding to the model_fn. In that case the training wouldn't stop. This doesn't make sense to me. I guess I'm going wrong somewhere here.
queue_capacity I am not sure what this means

Would appreciate any help around these questions. Thanks in advance!

Alexandre Passos · Accepted Answer · 2017-03-08 23:02:54Z

1

randomize_input means both that the filenames are randomized and that a shuffling queue is used to shuffle examples once in memory.

For num_epochs, training can stop due to other reasons (often training stops at a given global_step regardless of the input function).

For queue_capacity, TensorFlow uses queues internally to buffer I/O for maximum throughput. This parameter allows you to control the size of that queue.

answered Mar 8, 2017 at 23:02

Alexandre Passos

5,2061 gold badge17 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Input parameters of tf.contrib.learn.read_batch_features

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related