在y_true和y_pred中使用不同大小的keras实现自定义损失函数

Question

我是Keras的新手.我需要一些帮助,使用TensorFlow后端在keras中编写自定义损失函数,以获得以下损失方程.

传递给损失函数的参数是:

1. y_true会是好形的(batch_size, N, 2).在这里,我们(x, y)在批处理中的每个样本中传递N个坐标.
2. y_pred会是好形的(batch_size, 256, 256, N).在这里,我们256 x 256在批次中的每个样本中传递N个预测的像素热图.

i ∈ [0, 255]

j ∈ [0, 255]

Mn(i, j)表示(i, j)第n ^个预测热图的像素位置的值.

Mn∼(i, j) = Guassian2D((i, j), y_truen, std) 哪里

std = standard deviation,两个尺寸(5 px)的标准偏差相同.

y_true _n是第n ^个(x,y)坐标.这就是意思.

有关详细信息,请检查本文" 人体姿势估计"中描述的l ₂损失 .

注意:我在y_true和y_pred的形状中提到了batch_size.我假设Keras称整个批次的损失函数,而不是批次中的单个样本.如果我错了,请纠正我.

def l2_loss(y_true, y_pred):
     loss = 0
     n = y_true.shape[0]
     for j in range(n):
        for i in range(num_joints):
            yv, xv = tf.meshgrid(tf.arange(0, im_height), tf.arange(0, im_width))
            z = np.array([xv, yv]).transpose(1, 2, 0)
            ground = np.exp(-0.5*(((z - y_true[j, i, :])**2).sum(axis=2))/(sigma**2))
            loss = loss + np.sum((ground - y_pred[j,:, :, i])**2)
     return loss/num_joints

这是我到目前为止编写的代码.我知道这不会运行,因为我们不能在keras损失函数中使用直接numpy ndarray.另外,我需要消除循环!

Answer 1

您几乎可以将numpy函数转换为Keras后端函数.唯一需要注意的是设置正确的广播形状.

def l2_loss_keras(y_true, y_pred):
    # set up meshgrid: (height, width, 2)
    meshgrid = K.tf.meshgrid(K.arange(im_height), K.arange(im_width))
    meshgrid = K.cast(K.transpose(K.stack(meshgrid)), K.floatx())

    # set up broadcast shape: (batch_size, height, width, num_joints, 2)
    meshgrid_broadcast = K.expand_dims(K.expand_dims(meshgrid, 0), -2)
    y_true_broadcast = K.expand_dims(K.expand_dims(y_true, 1), 2)
    diff = meshgrid_broadcast - y_true_broadcast

    # compute loss: first sum over (height, width), then take average over num_joints
    ground = K.exp(-0.5 * K.sum(K.square(diff), axis=-1) / sigma ** 2)
    loss = K.sum(K.square(ground - y_pred), axis=[1, 2])
    return K.mean(loss, axis=-1)

要验证它:

def l2_loss_numpy(y_true, y_pred):
     loss = 0
     n = y_true.shape[0]
     for j in range(n):
        for i in range(num_joints):
            yv, xv = np.meshgrid(np.arange(0, im_height), np.arange(0, im_width))
            z = np.stack([xv, yv]).transpose(1, 2, 0)
            ground = np.exp(-0.5*(((z - y_true[j, i, :])**2).sum(axis=2))/(sigma**2))
            loss = loss + np.sum((ground - y_pred[j,:, :, i])**2)
     return loss/num_joints

batch_size = 32
num_joints = 10
sigma = 5
im_width = 256
im_height = 256

y_true = 255 * np.random.rand(batch_size, num_joints, 2)
y_pred = 255 * np.random.rand(batch_size, im_height, im_width, num_joints)

print(l2_loss_numpy(y_true, y_pred))
45448272129.0

print(K.eval(l2_loss_keras(K.variable(y_true), K.variable(y_pred))).sum())
4.5448e+10

该数字在默认的dtypefloat32 下被截断.如果你运行它dtype设置为float64:

y_true = 255 * np.random.rand(batch_size, num_joints, 2)
y_pred = 255 * np.random.rand(batch_size, im_height, im_width, num_joints)

print(l2_loss_numpy(y_true, y_pred))
45460126940.6

print(K.eval(l2_loss_keras(K.variable(y_true), K.variable(y_pred))).sum())
45460126940.6

编辑:

似乎Keras需要y_true并且y_pred具有相同数量的维度.例如,在以下测试模型上:

X = np.random.rand(batch_size, 256, 256, 3)
model = Sequential([Dense(10, input_shape=(256, 256, 3))])
model.compile(loss=l2_loss_keras, optimizer='adam')
model.fit(X, y_true, batch_size=8)

ValueError: Cannot feed value of shape (8, 10, 2) for Tensor 'dense_2_target:0', which has shape '(?, ?, ?, ?)'

要处理此问题,您可以expand_dims在y_true输入模型之前添加虚拟维度:

def l2_loss_keras(y_true, y_pred):
    ...

    y_true_broadcast = K.expand_dims(y_true, 1)  # change this line

    ...

model.fit(X, np.expand_dims(y_true, axis=1), batch_size=8)