在Tensor Flow中实现Siamese网络

Question

我想实现一个Siamese卷积神经网络,其中两个图像在卷积层中共享权重,然后在通过完全连接的层之前连接.我试过一个实现,但它似乎是一个"黑客"解决方案.特别是,我已经将张量上的操作定义为简单的Python函数,我不确定这是否允许.

这是我尝试过的代码:

images = tf.placeholder(tf.float32, shape=[None, 64 * 64])
# Convolutional layers
# ...
# ...
# Results in pool3_flat, which is the flattened output of the third convolutional layer
pool3_flat = tf.reshape(pool3, [-1, 8 * 8 * 128])

# Now, merge the image pairs, where each pair is composed of adjacent images in the batch, with a stride of 2
def merge_pairs():
  # Create a tensor to store the merged image pairs
  # The batch size is 128, therefore there will be 64 pairs (64 in the first dimension of this tensor)
  merged_pairs = tf.Variable(tf.zeros([64, 8 * 8 * 128]))
  # Split the images into 64 pairs
  pairs = tf.split(0, 64, pool3_flat)
  # For each pair, concatenate the two images across dimension 1, and set this tensor in the appropriate row of merged_pairs
  for pair_num, pair in enumerate(pairs):
      merged_pair = tf.concat(1, pair)
      merged_pairs[pair_num] = merged_pair
  return merged_pairs


# Proceed with operations on the merged_pair tensor, as if the batch size is 64
fc4 = tf.matmul(merge_pairs(), weights4)
# ...
# ...

虽然这个编译并且似乎运行良好,但结果并不像预期的那样.所以,我想知道在TensorFlow中使用内置操作是否有更好的方法来实现Siamese网络？

Answer 1

您可以使用tf.pack和tf.unpack,有点像:

pairs = tf.pack(tf.split(0, 64, pool3_flat))
left, right = tf.unpack(tf.transpose(pairs, perm=[1,0,2]))
merged_pairs = tf.concat(1, [left, right])

更简洁的方法是将您的对与开头分开,这样您就可以定义两个网络并在每个网络中使用相同的可训练变量.

你会有类似的东西(跳过卷积层):

image_left = tf.placeholder(tf.float32, shape=[None, 64, 64, 1])
image_right = tf.placeholder(tf.float32, shape=[None, 64, 64, 1])

pool_left = tf.nn.max_pool(image_left, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
pool_right = tf.nn.max_pool(image_left, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

pool_flat_left = tf.reshape(pool_left, [-1, 32*32])
pool_flat_right = tf.reshape(pool_right, [-1, 32*32])

然后简单地在维度1中左右对齐.

concat_layer = tf.concat(1, [pool_flat_left, pool_flat_right])

这样,您也可以稍后更改批量大小.确保在每个尺寸(左和右)上使用相同的重量和偏差.