Keras 2.3.0 度量准确度、精确度和召回率的相同值

Question

我正在尝试获取准确度、准确度和召回率的 keras 指标，但它们三个都显示相同的值，这实际上是准确度。

我正在使用 TensorFlow 文档示例中提供的指标列表：

metrics = [keras.metrics.TruePositives(name='tp'),
           keras.metrics.FalsePositives(name='fp'),
           keras.metrics.TrueNegatives(name='tn'),
           keras.metrics.FalseNegatives(name='fn'),
           keras.metrics.BinaryAccuracy(name='accuracy'),
           keras.metrics.Precision(name='precision'),
           keras.metrics.Recall(name='recall'),
           keras.metrics.AUC(name='auc')]

模型是用于图像分类的非常基本的 CNN：

model = Sequential()

model.add(Convolution2D(32, 
                      (7, 7), 
                      padding ="same", 
                      input_shape=(255, 255, 3), 
                      activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Convolution2D(64, 
                      (3, 3), 
                      padding ="same"))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(256, 
              activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(n_classes, 
              activation='softmax'))

使用上面显示的指标列表进行编译：

model.compile(loss=loss,
            optimizer=optimizer,
            metrics=metrics)

这是我在训练时一直看到的问题的一个例子：

Epoch 1/15
160/160 [==============================] - 6s 37ms/step - loss: 0.6402 - tp: 215.0000 - fp: 105.0000 - tn: 215.0000 - fn: 105.0000 - accuracy: 0.6719 - precision: 0.6719 - recall: 0.6719 - auc: 0.7315 - val_loss: 0.6891 - val_tp: 38.0000 - val_fp: 42.0000 - val_tn: 38.0000 - val_fn: 42.0000 - val_accuracy: 0.4750 - val_precision: 0.4750 - val_recall: 0.4750 - val_auc: 0.7102
Epoch 2/15
160/160 [==============================] - 5s 30ms/step - loss: 0.6929 - tp: 197.0000 - fp: 123.0000 - tn: 197.0000 - fn: 123.0000 - accuracy: 0.6156 - precision: 0.6156 - recall: 0.6156 - auc: 0.6941 - val_loss: 0.6906 - val_tp: 38.0000 - val_fp: 42.0000 - val_tn: 38.0000 - val_fn: 42.0000 - val_accuracy: 0.4750 - val_precision: 0.4750 - val_recall: 0.4750 - val_auc: 0.6759

每折的指标，每次都具有相同的准确度、精确度和召回率值：

['loss', 'tp', 'fp', 'tn', 'fn', 'accuracy', 'precision', 'recall', 'auc']
[[ 0.351 70.    10.    70.    10.     0.875  0.875  0.875  0.945]
 [ 0.091 78.     2.    78.     2.     0.975  0.975  0.975  0.995]
 [ 0.253 72.     8.    72.     8.     0.9    0.9    0.9    0.974]
 [ 0.04  78.     2.    78.     2.     0.975  0.975  0.975  0.999]
 [ 0.021 80.     0.    80.     0.     1.     1.     1.     1.   ]]

sklearn.metrics.classification_report 显示正确的精度和召回率

================ Fold 1 =====================
Accuracy: 0.8875
              precision    recall  f1-score   support

      normal       0.84      0.95      0.89        38
          pm       0.95      0.83      0.89        42

    accuracy                           0.89        80
   macro avg       0.89      0.89      0.89        80
weighted avg       0.89      0.89      0.89        80

================ Fold 2 =====================
Accuracy: 0.9375
              precision    recall  f1-score   support

      normal       1.00      0.87      0.93        38
          pm       0.89      1.00      0.94        42

    accuracy                           0.94        80
   macro avg       0.95      0.93      0.94        80
weighted avg       0.94      0.94      0.94        80

================ Fold 3 =====================
Accuracy: 0.925
              precision    recall  f1-score   support

      normal       0.88      0.97      0.92        37
          pm       0.97      0.88      0.93        43

    accuracy                           0.93        80
   macro avg       0.93      0.93      0.92        80
weighted avg       0.93      0.93      0.93        80

================ Fold 4 =====================
Accuracy: 0.925
              precision    recall  f1-score   support

      normal       0.97      0.86      0.91        37
          pm       0.89      0.98      0.93        43

    accuracy                           0.93        80
   macro avg       0.93      0.92      0.92        80
weighted avg       0.93      0.93      0.92        80

================ Fold 5 =====================
Accuracy: 1.0
              precision    recall  f1-score   support

      normal       1.00      1.00      1.00        37
          pm       1.00      1.00      1.00        43

    accuracy                           1.00        80
   macro avg       1.00      1.00      1.00        80
weighted avg       1.00      1.00      1.00        80

Answer 1

精确度和召回率已经存在一些问题。

看这个问题：https://github.com/keras-team/keras/issues/5400

你可以尝试tensorflow.keras一下。这个问题应该消失。

或者，您可以使用自定义实现并在编译函数中传递它们。

from keras import backend as K

def check_units(y_true, y_pred):
    if y_pred.shape[1] != 1:
      y_pred = y_pred[:,1:2]
      y_true = y_true[:,1:2]
    return y_true, y_pred

def precision(y_true, y_pred):
    y_true, y_pred = check_units(y_true, y_pred)
    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
    predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
    precision = true_positives / (predicted_positives + K.epsilon())
    return precision

def recall(y_true, y_pred):
    y_true, y_pred = check_units(y_true, y_pred)
    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
    recall = true_positives / (possible_positives + K.epsilon())
    return recall

metrics = [keras.metrics.TruePositives(name='tp'),
           keras.metrics.FalsePositives(name='fp'),
           keras.metrics.TrueNegatives(name='tn'),
           keras.metrics.FalseNegatives(name='fn'),
           keras.metrics.BinaryAccuracy(name='accuracy'),
           precision,
           recall,
           keras.metrics.AUC(name='auc')]