tf.contrib.learn.read_keyed_batch_examples_shared_queue(
file_pattern,
batch_size,
reader,
randomize_input=True,
num_epochs=None,
queue_capacity=10000,
num_threads=1,
read_batch_size=1,
parse_fn=None,
name=None,
seed=None
)
Defined in tensorflow/contrib/learn/python/learn/learn_io/graph_io.py
.
Adds operations to read, queue, batch Example
protos. (deprecated)
Given file pattern (or list of files), will setup a shared queue for file
names, setup a worker queue that pulls from the shared queue, read Example
protos using provided reader
, use batch queue to create batches of examples
of size batch_size
. This provides at most once visit guarantees. Note that
this only works if the parameter servers are not pre-empted or restarted or
the session is not restored from a checkpoint since the state of a queue
is not checkpointed and we will end up restarting from the entire list of
files.
All queue runners are added to the queue runners collection, and may be
started via start_queue_runners
.
All ops are added to the default graph.
Use parse_fn
if you need to do parsing / processing on single examples.
Args:
file_pattern
: List of files or patterns of file paths containingExample
records. Seetf.gfile.Glob
for pattern rules.batch_size
: An int or scalarTensor
specifying the batch size to use.reader
: A function or class that returns an object withread
method, (filename tensor) -> (example tensor).randomize_input
: Whether the input should be randomized.num_epochs
: Integer specifying the number of times to read through the dataset. IfNone
, cycles through the dataset forever. NOTE - If specified, creates a variable that must be initialized, so calltf.local_variables_initializer()
and run the op in a session.queue_capacity
: Capacity for input queue.num_threads
: The number of threads enqueuing examples.read_batch_size
: An int or scalarTensor
specifying the number of records to read at once.parse_fn
: Parsing function, takesExample
Tensor returns parsed representation. IfNone
, no parsing is done.name
: Name of resulting op.seed
: An integer (optional). Seed used if randomize_input == True.
Returns:
Returns tuple of:
- Tensor
of string keys.
- String Tensor
of batched Example
proto.
Raises:
ValueError
: for invalid inputs.