小编seb*_*iic的帖子

在我的网络中，我想在前向传播中计算网络的前向传播和后向传播。为此，我必须手动定义前向传递层的所有后向传递方法。
对于激活函数来说，这很简单。对于线性层和转换层来说，它也运行良好。但我真的很挣扎于 BatchNorm。由于 BatchNorm 论文仅讨论一维情况：到目前为止，我的实现如下所示：

def backward_batchnorm2d(input, output, grad_output, layer):
    gamma = layer.weight
    beta = layer.bias
    avg = layer.running_mean
    var = layer.running_var
    eps = layer.eps
    B = input.shape[0]

    # avg, var, gamma and beta are of shape [channel_size]
    # while input, output, grad_output are of shape [batch_size, channel_size, w, h]
    # for my calculations I have to reshape avg, var, gamma and beta to [batch_size, channel_size, w, h] by repeating the channel values over the whole image and batches

    dL_dxi_hat …