如何处理 KerasTensor 和 Tensor？

Question

我正在尝试创建变分自动编码器，这意味着我需要自定义损失函数。问题是，在损失函数内部，我有 2 个不同的损失 - mse 和散度。mse是Tensor，divergence是KerasTensor（由于色散和mu，我从编码器中退出）。我收到这样的错误：

类型错误：无法将符号 Keras 输入/输出转换为 numpy 数组。此错误可能表明您正在尝试将符号值传递给 NumPy 调用，但该调用不受支持。或者，您可能尝试将 Keras 符号输入/输出传递到未注册调度的 TF API，从而阻止 Keras 自动将 API 调用转换为功能模型中的 lambda 层。

这是我的架构：

import tensorflow.keras as keras
from keras.layers import Input, Dense, Flatten, Reshape
from keras.layers import Conv2D, MaxPooling2D, UpSampling2D, Conv2DTranspose
from keras.models import Model
import tensorflow as tf
import keras.backend as K


encoded_dim = 2

class Sampling(keras.layers.Layer):
    """Uses (z_mean, z_log_var) to sample z, the vector encoding a digit."""

    def call(self, inputs):
        z_mean, z_log_var = inputs
        batch = tf.shape(z_mean)[0]
        dim = tf.shape(z_mean)[1]
        epsilon = K.random_normal(shape=(batch, dim))
        return z_mean + tf.exp(0.5 * z_log_var) * epsilon


img = Input((28,28,1), name='img')

x = Conv2D(32, (3,3), padding='same', activation='relu')(img)
x = MaxPooling2D()(x)
x = Conv2D(64, (3,3), padding='same', activation='relu')(x)
x = MaxPooling2D()(x)
x = Flatten()(x)
x = Dense(16, activation='relu')(x)
mu = Dense(encoded_dim, name='mu')(x)
sigma = Dense(encoded_dim, name='sigma')(x)
z = Sampling()([mu,sigma])
# print(mu)
xx = Input((encoded_dim,))

x = Dense(7*7*64, activation='relu')(xx)
x = Reshape((7,7,64))(x)

x = Conv2DTranspose(64, 3, activation="relu", strides=2, padding="same")(x)
x = Conv2DTranspose(32, 3, activation="relu", strides=2, padding="same")(x)

out = Conv2DTranspose(1, 3, activation="sigmoid", padding="same")(x)

encoder = Model(img,z, name='encoder')
decoder = Model(xx,out,name='decoder')

autoencoder = Model(img,decoder(encoder(img)),name='autoencoder')

以及损失函数：

def vae_loss(x, y):
    loss = tf.reduce_mean(K.square(x-y))
    kl_loss = -0.5 * tf.reduce_mean(1 + sigma - tf.square(mu) - tf.exp(sigma))
    print(type(loss))
    print(type(kl_loss))
    return loss + kl_loss

autoencoder.compile(optimizer='adam',
                    loss = vae_loss)

autoencoder.fit(train,train,
                epochs=1,
                batch_size=60,
                shuffle=True,
                verbose = 2)

损失和lk_loss的类型：

类“tensorflow.python.framework.ops.Tensor”

类“tensorflow.python.keras.engine.keras_tensor.KerasTensor”

Answer 1

你需要将mu和传递sigma给你的损失函数。vae_loss现在接受 4 个输入：

def vae_loss(x, y, mu, sigma):
    loss = tf.reduce_mean(K.square(x-y))
    kl_loss = -0.5 * tf.reduce_mean(1 + sigma - tf.square(mu) - tf.exp(sigma))
    return loss + kl_loss

您可以在模型中使用它，只需使用autoencoder.add_loss.

同样重要的是，encoder不仅返回z，还mu返回 和sigma。

z, mu, sigma = encoder(img)
out = decoder(z)
autoencoder = Model(img, out, name='autoencoder')
autoencoder.add_loss(vae_loss(img, out, mu, sigma))  # <======= add_loss
autoencoder.compile(loss=None, optimizer='adam')

这里是正在运行的笔记本https://colab.research.google.com/drive/1r5lMZ2Dc_Lq4KJDqiirXla1qfDVmdwxW?usp=sharing