我正在 MNIST 上训练自动编码器，并注意到在 128 之后增加批量大小，开始在固定数据集大小上花费更多的计算时间。
我正在使用tensorflow-gpu并拥有GeForce GTX 1070。
我尝试在 5000 个样本（784 个暗淡）的固定训练集上运行一些测试，并运行 10 个 epoch。批次是batch-size来自 5000 个训练样本的连续块，因此迭代次数实际上取决于批次大小。
我跟踪了该数据的性能（丢失）、执行时间和 python 进程的 GPU 内存使用情况（来自 nvidia-smi 输出）：
5000 datapoints 10 epochs
batch size
512: loss: 53.7472; execution took 00:00:13,787; 4281MiB
256: loss: 48.1941; execution took 00:00:04,973; 695MiB
128: loss: 42.7486; execution took 00:00:03,350; 439MiB
64: loss: 40.0781; execution took 00:00:04,191; 439MiB
32: loss: 37.7348; execution took 00:00:06,487; 441MiB
16: loss: 36.6291; execution took 00:00:12,102; 441MiB
8: loss: nan; execution took 00:00:23,115; …