为什么 ImageDataGenerator 永远迭代？

Question

我刚刚开始使用 Keras 并且正在做一些图像预处理，我观察到从接收到的生成器ImageDataGenerator在for-loop.

image_gen = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1/255, rotation_range=45)

train_data_gen = image_gen.flow_from_directory(train_dir,
                                               shuffle=True,
                                              target_size=(IMG_SHAPE, IMG_SHAPE),
                                              batch_size=batch_size
                                              )
print('Total number of batches - {}'.format(len(train_data_gen)))
for n, i in enumerate(train_data_gen):
    if n >= 30:
        # I have to add explicit break statement to get out of loop when done with iterating over all the items present in generator.
        break
    batch_data = i[0]
    print(n, batch_data[0].shape)
# TRY to access element out of bound to see if there really exists more than 30 elements.
print(''.format(train_data_gen[32]))

输出

Found 2935 images belonging to 5 classes.
Total number of batches - 30
0 (150, 150, 3)
1 (150, 150, 3)
2 (150, 150, 3)
.
.
.
29 (150, 150, 3)
---------------------------------------------------------------------------
ValueError: Traceback (most recent call last)
<ipython-input-20-aed377bb98f7> in <module>
     13     batch_data = i[0]
     14     print(n, batch_data[0].shape)
---> 15 print(''.format(train_data_gen[32]))

~/.virtualenvs/pan_demo/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in __getitem__(self, idx)
     55                              'but the Sequence '
     56                              'has length {length}'.format(idx=idx,
---> 57                                                           length=len(self)))
     58         if self.seed is not None:
     59             np.random.seed(self.seed + self.total_batches_seen)

ValueError: Asked to retrieve element 32, but the Sequence has length 30

题

1. 这ImageDataGenerator是工作的方式吗？如果是这样，我可以if n >=30以某种方式避免检查零件吗？
2. 在准备导致这种行为的发电机时，我是否遗漏了什么？

Keras版本：tf.keras.__version__ ---> 2.2.4-tf Tensorflow版本：tf.VERSION--->1.13.1

Answer 1

实际上，train_data_gen会无限地逐批生成数据。

当我们调用时model.fit_generator()，我们将train_data_gen生成器指定为生成器，并设置steps_per_epoch（应该是len(train_data)/batch_size）。然后模型将知道单个 epoch 何时结束。