View source on GitHub
|
View source on GitHub
|
Types of loss reduction.
Contains the following values:
AUTO: Indicates that the reduction option will be determined by the usage
context. For almost all cases this defaults to SUM_OVER_BATCH_SIZE. When
used with tf.distribute.Strategy, outside of built-in training loops such
as tf.keras compile and fit, we expect reduction value to be
SUM or NONE. Using AUTO in that case will raise an error.NONE: Weighted losses with one dimension reduced (axis=-1, or axis
specified by loss function). When this reduction type used with built-in
Keras training loops like fit/evaluate, the unreduced vector loss is
passed to the optimizer but the reported loss will be a scalar value.SUM: Scalar sum of weighted losses.SUM_OVER_BATCH_SIZE: Scalar SUM divided by number of elements in losses.
This reduction type is not supported when used with
tf.distribute.Strategy outside of built-in training loops like tf.keras
compile/fit.
You can implement 'SUM_OVER_BATCH_SIZE' using global batch size like:
with strategy.scope():
loss_obj = tf.keras.losses.CategoricalCrossentropy(
reduction=tf.keras.losses.Reduction.NONE)
....
loss = tf.reduce_sum(loss_object(labels, predictions)) *
(1. / global_batch_size)
Please see the custom training guide # pylint: disable=line-too-long for more details on this.
all@classmethod
all()
validate@classmethod
validate(
key
)