Keras .fit 提供比手动 Tensorflow 更好的性能

Question

我是 Tensorflow 和 Keras 的新手。首先，我遵循了https://www.tensorflow.org/tutorials/quickstart/advanced教​​程。我现在正在对其进行调整，以在 CIFAR10 而不是 MNIST 数据集上进行训练。我重新创建了这个模型https://keras.io/examples/cifar10_cnn/并且我尝试在我自己的代码库中运行它。

从逻辑上讲，如果模型、批量大小和优化器都相同，那么两者的性能应该相同，但事实并非如此。我想可能是我在准备数据时犯了错误。因此，我将 model.fit 函数从 keras 代码复制到我的脚本中，它的性能仍然更好。使用 .fit 在 25 个 epoch 内获得了大约 75% 的准确率，而使用手动方法则需要大约 60 个 epoch。通过 .fit，我还获得了稍微更好的最大准确度。

我想知道的是： .fit 是否在幕后做一些优化训练的事情？我需要在代码中添加什么才能获得相同的性能？我做错了什么吗？

谢谢你的时间。

主要代码：

import tensorflow as tf
from tensorflow import keras
import msvcrt
from Plotter import Plotter


#########################Configuration Settings#############################

BatchSize = 32
ModelName = "CifarModel"

############################################################################


(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()

print("x_train",x_train.shape)
print("y_train",y_train.shape)
print("x_test",x_test.shape)
print("y_test",y_test.shape)

x_train, x_test = x_train / 255.0, x_test / 255.0

# Convert class vectors to binary class matrices.
y_train = keras.utils.to_categorical(y_train, 10)
y_test = keras.utils.to_categorical(y_test, 10)



train_ds = tf.data.Dataset.from_tensor_slices(
    (x_train, y_train)).batch(BatchSize)

test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(BatchSize)


loss_object = tf.keras.losses.CategoricalCrossentropy(from_logits=True)
optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.0001,decay=1e-6)

# Create an instance of the model
model = ModelManager.loadModel(ModelName,10)


train_loss = tf.keras.metrics.Mean(name='train_loss')
train_accuracy = tf.keras.metrics.CategoricalAccuracy(name='train_accuracy')

test_loss = tf.keras.metrics.Mean(name='test_loss')
test_accuracy = tf.keras.metrics.CategoricalAccuracy(name='test_accuracy')



########### Using this function I achieve better results ##################

model.compile(loss='categorical_crossentropy',
              optimizer=optimizer,
              metrics=['accuracy'])
model.fit(x_train, y_train,
              batch_size=BatchSize,
              epochs=100,
              validation_data=(x_test, y_test),
              shuffle=True,
              verbose=2)

############################################################################

########### Using the below code I achieve worse results ##################

@tf.function
def train_step(images, labels):
  with tf.GradientTape() as tape:
    predictions = model(images, training=True)
    loss = loss_object(labels, predictions)
  gradients = tape.gradient(loss, model.trainable_variables)
  optimizer.apply_gradients(zip(gradients, model.trainable_variables))

  train_loss(loss)
  train_accuracy(labels, predictions)

@tf.function
def test_step(images, labels):
  predictions = model(images, training=False)
  t_loss = loss_object(labels, predictions)

  test_loss(t_loss)
  test_accuracy(labels, predictions)

epoch = 0
InterruptLoop = False
while InterruptLoop == False:
  #Shuffle training data
  train_ds.shuffle(1000)
  epoch = epoch + 1
  # Reset the metrics at the start of the next epoch
  train_loss.reset_states()
  train_accuracy.reset_states()
  test_loss.reset_states()
  test_accuracy.reset_states()

  for images, labels in train_ds:
    train_step(images, labels)

  for test_images, test_labels in test_ds:
    test_step(test_images, test_labels)

  test_accuracy = test_accuracy.result() * 100
  train_accuracy = train_accuracy.result() * 100

  #Print update to console
  template = 'Epoch {}, Loss: {}, Accuracy: {}, Test Loss: {}, Test Accuracy: {}'
  print(template.format(epoch,
                        train_loss.result(),
                        train_accuracy ,
                        test_loss.result(),
                        test_accuracy))

  # Check if keyboard pressed
  while msvcrt.kbhit():
    char = str(msvcrt.getch())
    if char == "b'q'":
      InterruptLoop = True
      print("Stopping loop")

该模型：

from tensorflow.keras.layers import Dense, Flatten, Conv2D, Dropout, MaxPool2D
from tensorflow.keras import Model

class ModelData(Model):
  def __init__(self,NumberOfOutputs):
    super(ModelData, self).__init__()
    self.conv1 = Conv2D(32, 3, activation='relu', padding='same', input_shape=(32,32,3))
    self.conv2 = Conv2D(32, 3, activation='relu')
    self.maxpooling1 = MaxPool2D(pool_size=(2,2))
    self.dropout1 = Dropout(0.25)
    ############################
    self.conv3 = Conv2D(64,3,activation='relu',padding='same')
    self.conv4 = Conv2D(64,3,activation='relu')
    self.maxpooling2 = MaxPool2D(pool_size=(2,2))
    self.dropout2 = Dropout(0.25)
    ############################
    self.flatten = Flatten()
    self.d1 = Dense(512, activation='relu')
    self.dropout3 = Dropout(0.5)
    self.d2 = Dense(NumberOfOutputs,activation='softmax')

  def call(self, x):
    x = self.conv1(x)
    x = self.conv2(x)
    x = self.maxpooling1(x)
    x = self.dropout1(x)
    x = self.conv3(x)
    x = self.conv4(x)
    x = self.maxpooling2(x)
    x = self.dropout2(x)
    x = self.flatten(x)
    x = self.d1(x)
    x = self.dropout3(x)
    x = self.d2(x)
    return x

Answer 1

我知道这个问题已经有了答案，但我遇到了同样的问题，并且解决方案似乎有所不同，文档中没有指定。

我将在 GitHub 上找到的答案（以及相关链接）复制并粘贴到此处，这解决了我的案例中的问题：

该问题是由自定义循环中的损失函数中的广播引起的。确保预测和标签的维度相等。目前（对于 MAE）它们是 [128,1] 和 [128]。只需使用 tf.squeeze 或 tf.expand_dims。

链接： https: //github.com/tensorflow/tensorflow/issues/28394

基本翻译：计算损失时，始终确定张量的形状。