当数据位于每个类的不同目录中时，如何使用 ImageDataGenerator 将数据分为 3 部分（训练、验证、测试）

Question

ImageDataGenerator如何使用Keras将数据分成 3 个部分？ImageDataGenerator只给出validation_split参数，所以如果我使用它，我不会将我的测试集用于以后的目的。

我的数据格式为

>input_data_dir
 >class_1_dir
  > image_1.png
  > image_2.png
 > class_2_dir
 > class_3_dir

Answer 1

正如您正确地提到的，使用 Keras 在一行代码中不可能将数据分成 3 部分ImageDataGenerator。

\n\n

解决方法是将对应的图像存储Test Data在单独的文件夹中并应用ImageDataGenerator，如下所示：

\n\n

# Path to Training Directory\ntrain_dir = 'Dogs_Vs_Cats_Small/train'\n\n# Path to Test Directory\ntest_dir = 'Dogs_Vs_Cats_Small/test'\n\nTrain_Gen = ImageDataGenerator(1./255)\nTest_Gen = ImageDataGenerator(1./255)\n\n\nTrain_Generator = Train_Gen.flow_from_directory(train_dir, target_size = (150,150), batch_size = 20, class_mode = 'binary')\n\nTest_Generator = Test_Gen.flow_from_directory(test_dir, target_size = (150, 150), class_mode = 'binary', batch_size = 20)\n

\n\n

从原始目录中提取一些图像并将它们放入两个单独的文件夹中的示例代码train和test可能会对您有所帮助，如下所示：

\n\n

import os, shutil\n\n# Path to the directory where the original dataset was uncompressed\noriginal_dataset_dir = 'Dogs_Vs_Cats'\n\n# Directory where you\xe2\x80\x99ll store your smaller dataset\nbase_dir = 'Dogs_Vs_Cats_Small2'\n\nos.mkdir(base_dir)\n\n# Directory for the training splits\ntrain_dir = os.path.join(base_dir, 'train')\nos.mkdir(train_dir)\n\n# Directory for the test splits\ntest_dir = os.path.join(base_dir, 'test')\nos.mkdir(test_dir)\n\n# Directory with training cat pictures\ntrain_cats_dir = os.path.join(train_dir, 'cats')\nos.mkdir(train_cats_dir)\n\n# Directory with training dog pictures\ntrain_dogs_dir = os.path.join(train_dir, 'dogs')\nos.mkdir(train_dogs_dir)\n\n# Directory with Test Cat Pictures\ntest_cats_dir = os.path.join(test_dir, 'cats')\nos.mkdir(test_cats_dir)\n\n# Directory with Test Dog Pictures\ntest_dogs_dir = os.path.join(test_dir, 'dogs')\nos.mkdir(test_dogs_dir)\n\n# Copies the first 1,000 cat images to train_cats_dir. \nfnames = ['cat.{}.jpg'.format(i) for i in range(1000)]\nfor fname in fnames:\n    src = os.path.join(original_dataset_dir, 'train', fname)\n    dst = os.path.join(train_cats_dir, fname)\n    shutil.copyfile(src, dst)\n\n# Copies the next 500 cat images to test_cats_dir\nfnames = ['cat.{}.jpg'.format(i) for i in range(1500, 2000)]\nfor fname in fnames:\n    src = os.path.join(original_dataset_dir, 'train', fname)\n    dst = os.path.join(test_cats_dir, fname)\n    shutil.copyfile(src, dst)\n\n# Copies the first 1,000 dog images to train_dogs_dir\nfnames = ['dog.{}.jpg'.format(i) for i in range(1000)]\nfor fname in fnames:\n    src = os.path.join(original_dataset_dir, 'train', fname)\n    dst = os.path.join(train_dogs_dir, fname)\n    shutil.copyfile(src, dst)\n\n# Copies the next 500 dog images to test_dogs_dir\nfnames = ['dog.{}.jpg'.format(i) for i in range(1500, 2000)]\nfor fname in fnames:\n    src = os.path.join(original_dataset_dir, 'train', fname)\n    dst = os.path.join(test_dogs_dir, fname)\n    shutil.copyfile(src, dst)\n\n# Sanity Check to ensure that Training, Validation and Test Folders have the expected number of images\n\nprint('Number of Cat Images in Training Directory is {}'.format(len(os.listdir(train_cats_dir))))\nprint('Number of Dog Images in Training Directory is {}'.format(len(os.listdir(train_dogs_dir))))\nprint('Number of Cat Images in Testing Directory is {}'.format(len(os.listdir(test_cats_dir))))\nprint('Number of Dog Images in Testing Directory is {}'.format(len(os.listdir(test_dogs_dir))))\n

\n\n

希望这可以帮助。

\n