如何使用tensorflow的Dataset API Iterator作为(周期性)神经网络的输入？

Question

当使用tensorflow的Dataset API Iterator时,我的目标是定义一个在迭代器的get_next()张量上操作的RNN 作为其输入(参见(1)代码).

但是,简单地将dynamic_rnnwith 定义get_next()为输入会导致错误:ValueError: Initializer for variable rnn/basic_lstm_cell/kernel/ is from inside a control-flow construct, such as a loop or conditional. When creating a variable inside a loop or conditional, use a lambda as the initializer.

现在我知道了一个解决方法是简单地创建了一个占位符next_batch,然后eval()张量(因为你无法通过自身张量),并用它传递feed_dict(见X并(2)在代码中).但是,如果我理解正确,这不是一个有效的解决方案,因为我们首先评估然后重新初始化张量.

有没有办法:

1. dynamic_rnn直接在迭代器的输出顶部定义;

要么:

2. 不知怎的,直接将现有的get_next()张量传递给占位符,这是输入dynamic_rnn？

完整的工作实例; 该(1)版本是我想要的工作,但它没有,虽然(2)是可行的解决方法.

import tensorflow as tf

from tensorflow.contrib.rnn import BasicLSTMCell
from tensorflow.python.data import Iterator

data = [ [[1], [2], [3]], [[4], [5], [6]], [[1], [2], [3]] ]
dataset = tf.data.Dataset.from_tensor_slices(data)
dataset = dataset.batch(2)
iterator = Iterator.from_structure(dataset.output_types,
                                   dataset.output_shapes)
next_batch = iterator.get_next()
iterator_init = iterator.make_initializer(dataset)

# (2):
X = tf.placeholder(tf.float32, shape=(None, 3, 1))

cell = BasicLSTMCell(num_units=8)

# (1):
# outputs, states = lstm_outputs, lstm_states = tf.nn.dynamic_rnn(cell, next_batch, dtype=tf.float32)

# (2):
outputs, states = lstm_outputs, lstm_states = tf.nn.dynamic_rnn(cell, X, dtype=tf.float32)

init = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)
    sess.run(iterator_init)

    # (1):
    # o, s = sess.run([outputs, states])
    # o, s = sess.run([outputs, states])

    # (2):
    o, s = sess.run([outputs, states], feed_dict={X: next_batch.eval()})
    o, s = sess.run([outputs, states], feed_dict={X: next_batch.eval()})

(使用tensorflow 1.4.0,Python 3.6.)

非常感谢你 :)

Answer 1

原来这个神秘错误很可能是tensorflow中的一个错误,请参阅https://github.com/tensorflow/tensorflow/issues/14729.更具体地说,错误实际上来自于提供错误的数据类型(在我上面的例子中,data数组包含int32值但它应该包含浮点数).

tensorflow应该返回:而不是得到ValueError: Initializer for variable rnn/basic_lstm_cell/kernel/ is from inside a control-flow construct错误
:(
TypeError: Tensors in list passed to 'values' of 'ConcatV2' Op have types [int32, float32] that don't all match.见1).

要解决此问题,只需更改
data = [ [[1], [2], [3]], [[4], [5], [6]], [[1], [2], [3]] ]
为
data = np.array([[ [1], [2], [3]], [[4], [5], [6]], [[1], [2], [3]] ], dtype=np.float32)

然后以下代码应正常工作:

import tensorflow as tf
import numpy as np

from tensorflow.contrib.rnn import BasicLSTMCell
from tensorflow.python.data import Iterator

data = np.array([[ [1], [2], [3]], [[4], [5], [6]], [[1], [2], [3]] ], dtype=np.float32)
dataset = tf.data.Dataset.from_tensor_slices(data)
dataset = dataset.batch(2)
iterator = Iterator.from_structure(dataset.output_types,
                                   dataset.output_shapes)
next_batch = iterator.get_next()
iterator_init = iterator.make_initializer(dataset)

# (2):
# X = tf.placeholder(tf.float32, shape=(None, 3, 1))

cell = BasicLSTMCell(num_units=8)

# (1):
outputs, states = lstm_outputs, lstm_states = tf.nn.dynamic_rnn(cell, next_batch, dtype=tf.float32)

# (2):
# outputs, states = lstm_outputs, lstm_states = tf.nn.dynamic_rnn(cell, X, dtype=tf.float32)

init = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)
    sess.run(iterator_init)

    # (1):
    o, s = sess.run([outputs, states])
    o, s = sess.run([outputs, states])

    # (2):
    # o, s = sess.run([outputs, states], feed_dict={X: next_batch.eval()})
    # o, s = sess.run([outputs, states], feed_dict={X: next_batch.eval()})