tf.keras.losses 中的“减少”参数

Question

根据文档，该Reduction参数有 3 个值 - SUM_OVER_BATCH_SIZE、SUM和NONE。

y_true = [[0., 2.], [0., 0.]]
y_pred = [[3., 1.], [2., 5.]]

mae = tf.keras.losses.MeanAbsoluteError(reduction=tf.keras.losses.Reduction.SUM)
mae(y_true, y_pred).numpy()
> 5.5

mae = tf.keras.losses.MeanAbsoluteError()
mae(y_true, y_pred).numpy()
> 2.75

经过各种试验后我可以推断出的计算结果是：-

• 什么时候REDUCTION = SUM，

  Loss = Sum over all samples {(Sum of differences between y_pred and y_target vector of each sample / No of element in y_target of the sample )} = { (abs(3-0) + abs(1-2))/2 } + { (abs(2-0) + abs(5-0))/2 } = {4/2} + {7/2} = 5.5。

• 什么时候REDUCTION = SUM_OVER_BATCH_SIZE，

  Loss = [Sum over all samples {(Sum of differences between y_pred and y_target vector of each sample / No of element in y_target of the sample )}] / Batch_size or No of Samples = [ { (abs(3-0)} + abs(1-2))/2 } + { (abs(2-0) + abs(5-0))/2 } ]/2 = [ {4/2} + {7/2} ]/2 = [5.5]/2 = 2.75。

结果，SUM_OVER_BATCH_SIZE无非是SUM/batch_size。那么，为什么要调用它呢SUM_OVER_BATCH_SIZE？实际上是SUM将整个批次的损失相加，同时SUM_OVER_BATCH_SIZE计算该批次的平均损失。

SUM_OVER_BATCH_SIZE我关于和的运作的假设是否SUM正确？

Answer 1

据我了解，您的假设是正确的。

如果您检查 github [keras/losses_utils.py][1] 第 260-269 行，您将看到它确实按预期执行。 SUM将总结批量维度中的损失，并SUM_OVER_BATCH_SIZE除以总SUM损失数（批量大小）。

def reduce_weighted_loss(weighted_losses,
                     reduction=ReductionV2.SUM_OVER_BATCH_SIZE):
  if reduction == ReductionV2.NONE:
     loss = weighted_losses
  else:
     loss = tf.reduce_sum(weighted_losses)
     if reduction == ReductionV2.SUM_OVER_BATCH_SIZE:
        loss = _safe_mean(loss, _num_elements(weighted_losses))
  return loss

您只需添加一对损失为零的输出即可对前面的示例进行简单检查。

y_true = [[0., 2.], [0., 0.],[1.,1.]]
y_pred = [[3., 1.], [2., 5.],[1.,1.]]

mae = tf.keras.losses.MeanAbsoluteError(reduction=tf.keras.losses.Reduction.SUM)
mae(y_true, y_pred).numpy()
> 5.5

mae = tf.keras.losses.MeanAbsoluteError()
mae(y_true, y_pred).numpy()
> 1.8333

所以，你的假设是正确的。[1]：https://github.com/keras-team/keras/blob/v2.7.0/keras/utils/losses_utils.py#L25-L84