Class ReductionToOneDeviceCrossDeviceOps
Inherits From: CrossDeviceOps
Defined in tensorflow/python/distribute/cross_device_ops.py
.
Always do reduction to one device first and then do broadcasting.
Batch reduction is done by reduction on each element one by one.
__init__
__init__(
reduce_to_device=None,
accumulation_fn=tf.math.add_n
)
Constructor.
Args:
reduce_to_device
: the intermediate device to reduce to. If None, reduce to the first device indestinations
of the reduce() method.accumulation_fn
: a function that does accumulation.
Methods
tf.contrib.distribute.ReductionToOneDeviceCrossDeviceOps.batch_reduce
batch_reduce(
reduce_op,
value_destination_pairs
)
Reduce PerReplica objects in a batch.
Reduce each first element in value_destination_pairs
to each second
element which indicates the destinations.
Args:
reduce_op
: Indicates how per_replica_value will be reduced. Accepted values aretf.distribute.ReduceOp.SUM
,tf.distribute.ReduceOp.MEAN
.value_destination_pairs
: a list or a tuple of tuples of PerReplica objects (or tensors with device set if there is one device) and destinations.
Returns:
a list of Mirrored objects.
Raises:
ValueError
: ifvalue_destination_pairs
is not a list or a tuple of tuples of PerReplica objects and destinations
tf.contrib.distribute.ReductionToOneDeviceCrossDeviceOps.broadcast
broadcast(
tensor,
destinations
)
Broadcast the tensor
to destinations.
Args:
tensor
: the tensor to broadcast.destinations
: the broadcast destinations.
Returns:
a Mirrored object.
tf.contrib.distribute.ReductionToOneDeviceCrossDeviceOps.reduce
reduce(
reduce_op,
per_replica_value,
destinations
)
Reduce per_replica_value
to destinations
.
It runs the reduction operation defined by reduce_op
and put the
result on destinations
.
Args:
reduce_op
: Indicates how per_replica_value will be reduced. Accepted values aretf.distribute.ReduceOp.SUM
,tf.distribute.ReduceOp.MEAN
.per_replica_value
: a PerReplica object or a tensor with device set.destinations
: the reduction destinations.
Returns:
a Mirrored object.
Raises:
ValueError
: if per_replica_value is not a PerReplica object.