"ValueError:在尝试使用get_variable时尝试共享变量$ var,但指定了dtype float32并找到了dtype float64_ref"

Question

我正在尝试构建一个自定义变量自动编码器网络,我在初始化解码器权重时使用来自编码器层的权重转置,我找不到原生的东西,tf.contrib.layers.fully_connected所以我用了tf.assign代替,这里是我的代码层:

def inference_network(inputs, hidden_units, n_outputs):
    """Layer definition for the encoder layer."""
    net = inputs
    with tf.variable_scope('inference_network', reuse=tf.AUTO_REUSE):
        for layer_idx, hidden_dim in enumerate(hidden_units):
            net = layers.fully_connected(
                net,
                num_outputs=hidden_dim,
                weights_regularizer=layers.l2_regularizer(training_params.weight_decay),
                scope='inf_layer_{}'.format(layer_idx))
            add_layer_summary(net)
        z_mean = layers.fully_connected(net, num_outputs=n_outputs, activation_fn=None)
        z_log_sigma = layers.fully_connected(
            net, num_outputs=n_outputs, activation_fn=None)

    return z_mean, z_log_sigma


def generation_network(inputs, decoder_units, n_x):
    """Define the decoder network."""
    net = inputs  # inputs here is the latent representation.
    with tf.variable_scope("generation_network", reuse=tf.AUTO_REUSE):
        assert(len(decoder_units) >= 2)
        # First layer does not have a regularizer
        net = layers.fully_connected(
            net,
            decoder_units[0],
            scope="gen_layer_0",
        )
        for idx, decoder_unit in enumerate([decoder_units[1], n_x], 1):
            net = layers.fully_connected(
                net,
                decoder_unit,
                scope="gen_layer_{}".format(idx),
                weights_regularizer=layers.l2_regularizer(training_params.weight_decay)
            )
    # Assign the transpose of weights to the respective layers
    tf.assign(tf.get_variable("generation_network/gen_layer_1/weights"),
              tf.transpose(tf.get_variable("inference_network/inf_layer_1/weights")))
    tf.assign(tf.get_variable("generation_network/gen_layer_1/bias"),
              tf.get_variable("generation_network/inf_layer_0/bias"))
    tf.assign(tf.get_variable("generation_network/gen_layer_2/weights"),
              tf.transpose(tf.get_variable("inference_network/inf_layer_0/weights")))
    return net # x_recon

它是用这个包裹的tf.slim arg_scope:

def _autoencoder_arg_scope(activation_fn):
    """Create an argument scope for the network based on its parameters."""

    with slim.arg_scope([layers.fully_connected],
                        weights_initializer=layers.xavier_initializer(),
                        biases_initializer=tf.initializers.constant(0.0),
                        activation_fn=activation_fn) as arg_sc:
        return arg_sc

但是我收到了错误:ValueError: Trying to share variable VarAutoEnc/generation_network/gen_layer_1/weights, but specified dtype float32 and found dtype float64_ref. 我已将此缩小到get_variable调用范围,但我不知道它为什么会失败.

如果有一种方法可以在tf.contrib.layers.fully_connected没有tf.assign操作的情况下从另一个完全连接的层初始化a ,那么该解决方案对我来说没问题.

Answer 1

我无法重现你的错误。这是一个简约的可运行示例，其功能与您的代码相同：

import tensorflow as tf

with tf.contrib.slim.arg_scope([tf.contrib.layers.fully_connected],
                               weights_initializer=tf.contrib.layers.xavier_initializer(),
                               biases_initializer=tf.initializers.constant(0.0)):

  i = tf.placeholder(tf.float32, [1, 30])

  with tf.variable_scope("inference_network", reuse=tf.AUTO_REUSE):
    tf.contrib.layers.fully_connected(i, 30, scope="gen_layer_0")

  with tf.variable_scope("generation_network", reuse=tf.AUTO_REUSE):
    tf.contrib.layers.fully_connected(i, 30, scope="gen_layer_0",
      weights_regularizer=tf.contrib.layers.l2_regularizer(0.01))

  with tf.variable_scope("", reuse=tf.AUTO_REUSE):
    tf.assign(tf.get_variable("generation_network/gen_layer_0/weights"),
              tf.transpose(tf.get_variable("inference_network/gen_layer_0/weights")))

代码运行时没有出现 ValueError。如果运行此程序时出现 ValueError，那么它可能是一个已在更高版本的 TensorFlow 版本中修复的错误（我在 1.9 上进行了测试）。否则，错误是您的代码的一部分，您不会在问题中显示。

顺便说一句，分配将返回一个操作，一旦返回的操作在会话中运行，该操作将执行分配。因此，您需要返回函数中所有分配调用的输出generation_network。您可以使用 . 将所有分配操作捆绑到一个操作中tf.group。