Tensorflow:小批量中每个样品的不同过滤器的卷积

Question

我想要一个带过滤器的2d卷积,这取决于张量流中的小批量样本.任何想法如何做到这一点,特别是如果每​​个小批量的样品数量未知？

具体而言,我有输入数据inp的形式的MB x H x W x Channels,我有过滤器F的形式的MB x fh x fw x Channels x OutChannels.

假设是

inp = tf.placeholder('float', [None, H, W, channels_img], name='img_input').

我想这样做tf.nn.conv2d(inp, F, strides = [1,1,1,1]),但这是不允许的,因为F不能有一个小批量维度.知道如何解决这个问题吗？

Answer 1

我认为建议的技巧实际上是不对的。tf.conv3d()层发生的事情是输入在深度（=实际批次）维度上进行卷积，然后沿着生成的特征图求和。随着padding='SAME'所产生的产出数量则恰好是同批大小，这样一个被愚弄！

编辑：我认为对不同小批量元素使用不同过滤器进行卷积的可能方法涉及“破解”深度卷积。假设批量大小MB已知：

inp = tf.placeholder(tf.float32, [MB, H, W, channels_img])

# F has shape (MB, fh, fw, channels, out_channels)
# REM: with the notation in the question, we need: channels_img==channels

F = tf.transpose(F, [1, 2, 0, 3, 4])
F = tf.reshape(F, [fh, fw, channels*MB, out_channels)

inp_r = tf.transpose(inp, [1, 2, 0, 3]) # shape (H, W, MB, channels_img)
inp_r = tf.reshape(inp, [1, H, W, MB*channels_img])

out = tf.nn.depthwise_conv2d(
          inp_r,
          filter=F,
          strides=[1, 1, 1, 1],
          padding='VALID') # here no requirement about padding being 'VALID', use whatever you want. 
# Now out shape is (1, H, W, MB*channels*out_channels)

out = tf.reshape(out, [H, W, MB, channels, out_channels) # careful about the order of depthwise conv out_channels!
out = tf.transpose(out, [2, 0, 1, 3, 4])
out = tf.reduce_sum(out, axis=3)

# out shape is now (MB, H, W, out_channels)

如果MB未知，应该可以使用tf.shape()（我认为）动态确定它

Answer 2

您可以tf.map_fn按如下方式使用：

inp = tf.placeholder(tf.float32, [None, h, w, c_in]) 
def single_conv(tupl):
    x, kernel = tupl
    return tf.nn.conv2d(x, kernel, strides=(1, 1, 1, 1), padding='VALID')
# Assume kernels shape is [tf.shape(inp)[0], fh, fw, c_in, c_out]
batch_wise_conv = tf.squeeze(tf.map_fn(
    single_conv, (tf.expand_dims(inp, 1), kernels), dtype=tf.float32),
    axis=1
)

dtype指定非常重要map_fn。基本上，该解决方案定义了batch_dim_size2D 卷积运算。