张量流:一类分类

Question

我正在尝试修改Deep MNIST for Experts这个教程来检测一个类,让我们说检测一个图像是否包含一个小猫.

这是我的代码的预测部分:

y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
sess.run(tf.initialize_all_variables())
for i in range(20000):
  batch = mnist.train.next_batch(50)
  if i%100 == 0:
    train_accuracy = accuracy.eval(feed_dict={
        x:batch[0], y_: batch[1], keep_prob: 1.0})
    print("step %d, training accuracy %g"%(i, train_accuracy))
  train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})

print("test accuracy %g"%accuracy.eval(feed_dict={
    x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))

问题是,对于一个类,softmax总是以1的置信度返回该类,即使对于空白图像也是如此.我尝试修改softmax和交叉熵,但我无法解决它.

我需要知道这个问题的推荐方法.我希望预测是图像成为小猫的概率.

我知道这可以通过使用随机图像训练的第二个标签来解决,但我需要知道是否有更好的解决方案.

非常感谢你.

Answer 1

不要将softmax和多类logloss用于单个类成员预测.相反,更常见的设置是使用二进制交叉熵的sigmoid激活.除非您正在优化正确预测*的成本/收益,否则只需设置> 0.5的阈值即可归类为"正"类.

在TensorFlow中,这只会在几个地方更改您的代码.

我认为以下调整适用于您的代码的开头:

y_conv = tf.nn.sigmoid(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)

# I've split the individual loss out because the line length was too long
# The +1e-12 is for numerical stability in case of perfect 0.0 or 1.0 predictions
# Note how this loss metric penalises incorrect predictions in both directions,
# unlike the multiclass logloss which only assessed confidence in
# correct class.
loss = -(y_ * tf.log(y_conv + 1e-12) + (1 - y_) * tf.log( 1 - y_conv + 1e-12))
cross_entropy = tf.reduce_mean(tf.reduce_sum(loss, reduction_indices=[1]))

predict_is_kitty = tf.greater(y_conv,0.5)
correct_prediction = tf.equal( tf.to_float(predict_is_kitty), y_ )

*如果您正在处理您关心预测信心的问题,并且需要评估设置阈值的位置,则通常的度量而非准确度是ROC曲线下的面积,通常称为AUROC或仅AUC.