如何从张量流中的RNN模型中提取单元状态和隐藏状态？

Question

我是TensorFlow的新手,很难理解RNN模块.我试图从LSTM中提取隐藏/单元状态.对于我的代码,我使用的是https://github.com/aymericdamien/TensorFlow-Examples中的实现.

# tf Graph input
x = tf.placeholder("float", [None, n_steps, n_input])
y = tf.placeholder("float", [None, n_classes])

# Define weights
weights = {'out': tf.Variable(tf.random_normal([n_hidden, n_classes]))}
biases = {'out': tf.Variable(tf.random_normal([n_classes]))}

def RNN(x, weights, biases):
    # Prepare data shape to match `rnn` function requirements
    # Current data input shape: (batch_size, n_steps, n_input)
    # Required shape: 'n_steps' tensors list of shape (batch_size, n_input)

    # Permuting batch_size and n_steps
    x = tf.transpose(x, [1, 0, 2])
    # Reshaping to (n_steps*batch_size, n_input)
    x = tf.reshape(x, [-1, n_input])
    # Split to get a list of 'n_steps' tensors of shape (batch_size, n_input)
    x = tf.split(0, n_steps, x)

    # Define a lstm cell with tensorflow
    #with tf.variable_scope('RNN'):
    lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0, state_is_tuple=True)

    # Get lstm cell output
        outputs, states = rnn.rnn(lstm_cell, x, dtype=tf.float32)

    # Linear activation, using rnn inner loop last output
    return tf.matmul(outputs[-1], weights['out']) + biases['out'], states

pred, states = RNN(x, weights, biases)

# Define loss and optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

# Evaluate model
correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
# Initializing the variables
init = tf.initialize_all_variables()

现在我想为预测中的每个时间步骤提取单元格/隐藏状态.状态存储在形式为(c,h)的LSTMStateTuple中,我可以通过评估找到它print states.但是,尝试调用print states.c.eval()(根据文档应该给出张量中的值states.c),会产生一个错误,指出我的变量没有被初始化,即使我在预测某些东西后正在调用它.这个代码在这里:

# Launch the graph
with tf.Session() as sess:
    sess.run(init)
    step = 1
    # Keep training until reach max iterations
    for v in tf.get_collection(tf.GraphKeys.VARIABLES, scope='RNN'):
        print v.name
    while step * batch_size < training_iters:
        batch_x, batch_y = mnist.train.next_batch(batch_size)
        # Reshape data to get 28 seq of 28 elements
        batch_x = batch_x.reshape((batch_size, n_steps, n_input))
        # Run optimization op (backprop)
        sess.run(optimizer, feed_dict={x: batch_x, y: batch_y})

        print states.c.eval()
        # Calculate batch accuracy
        acc = sess.run(accuracy, feed_dict={x: batch_x, y: batch_y})

        step += 1
    print "Optimization Finished!"

并且错误消息是

InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float
     [[Node: Placeholder = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

状态也是不可见的tf.all_variables(),只有经过训练的矩阵/偏置张量(如下所述:Tensorflow:在LSTM中显示或保存忘记门值).我不想从头开始构建整个LSTM,因为我在states变量中有状态,我只需要调用它.

Answer 1

您可以简单地以收集states精度的方式收集值.

我想,pred, states, acc = sess.run(pred, states, accuracy, feed_dict={x: batch_x, y: batch_y})应该完美无缺.

Answer 2

关于你的假设的一个评论:"状态"确实只具有上次时间步的"隐藏状态"和"存储单元"的值.

"输出"包含你想要的每个时间步的"隐藏状态"(输出的大小是[batch_size,seq_len,hidden_​​size].所以我假设你想要"输出"变量,而不是"状态".参见文档.