TensorFlow-Slim:内存数据集的数据提供程序

Question

我刚开始使用Slim并且非常喜欢它.我使用了mnist测试代码slim.dataset_data_provider,但发现它比本机tensorflow慢得多,我在内存中加载了所有数据.

我想这是因为来自硬盘的瘦数据提供者流数据？我想知道是否有一个示例使用数据提供程序进行内存数据访问.

我已经开始num_readers = 10了DatasetDataProvider,并且已经开始num_threads = 10了tf.train.batch.

谢谢.这是一个很好的工具.

还附上代码供参考:

import tensorflow as tf
import time
import sys
from tensorflow.contrib import slim

# need tensorflow_models project
sys.path.append('/home/user/projects/tf_models/slim')
from datasets import mnist

g = tf.Graph()
with g.as_default():
    tf.logging.set_verbosity(tf.logging.DEBUG)
    train_set = mnist.get_split('train', data_dir)
    provider = slim.dataset_data_provider.DatasetDataProvider(train_set, num_readers = 10, shuffle = True)
    [image, label] = provider.get(['image', 'label'])
    images, _ = tf.train.batch([image, label], batch_size = batch_size, num_threads=10, capacity = 2*batch_size)
    images = tf.cast(images, tf.float32) / 255

    recon, model = inference_ae(images, 0.5)

    sh = images.get_shape().as_list()
    loss = tf.contrib.losses.log_loss(recon, tf.reshape(images, [sh[0], -1]))

    tf.scalar_summary('loss', loss)
    optimizer = tf.train.AdadeltaOptimizer(learning_rate)
    train_op = slim.learning.create_train_op(loss, optimizer)

    final_loss_value = slim.learning.train(train_op, log_dir, number_of_steps= n_steps,
                                           log_every_n_steps = 10,
                                           save_summaries_secs=300, save_interval_secs=600)
    print("Final loss value: {}".format(final_loss_value))