在tensorflow中多次运行train op

Question

我有一些相当大的批量大小,我想采取多个渐变步骤.虽然我可以使用python for循环轻松完成此操作,但我想可能有一个更有效的方法,不涉及在每次迭代时将数据传输到gpu.我已经尝试将列车操作多次放入获取列表中,但我不确定它实际上是多次运行(运行时完全相同).

Answer 1

如果你有可变大小的批处理,那么变量不适合保存它,你可以run使用持久的张量在调用之间保留这些数据.这是一个玩具的例子

t = tf.int32
params = tf.Variable(tf.ones_initializer((), dtype=dt))
data_batches = [[1], [2, 3], [4, 5, 6]]

# op that uploads data to TF and saves it as a persistent Tensor
data_saver_placeholder = tf.placeholder(dt)
tensor_handle_op = tf.get_session_handle(data_saver_placeholder)

data_placeholder, data = tf.get_session_tensor(dt)
train_op = tf.assign_add(params, tf.reduce_prod(data)) 
init_op = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init_op)

for batch in data_batches:
    # upload tensor to TF runtime and save its handle
    tensor_handle = sess.run(tensor_handle_op, feed_dict={data_saver_placeholder: batch})
    # run train op several times reusing same data
    for i in range(3):
        sess.run(train_op, feed_dict={data_placeholder: tensor_handle.handle})


assert sess.run(params) == 382