tf.contrib.data.bucket_by_sequence_length(
element_length_func,
bucket_boundaries,
bucket_batch_sizes,
padded_shapes=None,
padding_values=None,
pad_to_bucket_boundary=False,
no_padding=False
)
Defined in tensorflow/contrib/data/python/ops/grouping.py
.
A transformation that buckets elements in a Dataset
by length. (deprecated)
Elements of the Dataset
are grouped together by length and then are padded
and batched.
This is useful for sequence tasks in which the elements have variable length. Grouping together elements that have similar lengths reduces the total fraction of padding in a batch which increases training step efficiency.
Args:
element_length_func
: function from element inDataset
totf.int32
, determines the length of the element, which will determine the bucket it goes into.bucket_boundaries
:list<int>
, upper length boundaries of the buckets.bucket_batch_sizes
:list<int>
, batch size per bucket. Length should belen(bucket_boundaries) + 1
.padded_shapes
: Nested structure oftf.TensorShape
to pass totf.data.Dataset.padded_batch
. If not provided, will usedataset.output_shapes
, which will result in variable length dimensions being padded out to the maximum length in each batch.padding_values
: Values to pad with, passed totf.data.Dataset.padded_batch
. Defaults to padding with 0.pad_to_bucket_boundary
: bool, ifFalse
, will pad dimensions with unknown size to maximum length in batch. IfTrue
, will pad dimensions with unknown size to bucket boundary minus 1 (i.e., the maximum length in each bucket), and caller must ensure that the sourceDataset
does not contain any elements with length longer thanmax(bucket_boundaries)
.no_padding
:bool
, indicates whether to pad the batch features (features need to be either of typetf.SparseTensor
or of same shape).
Returns:
A Dataset
transformation function, which can be passed to
tf.data.Dataset.apply
.
Raises:
ValueError
: iflen(bucket_batch_sizes) != len(bucket_boundaries) + 1
.