如何从tensorflow RNN中选择最后一个有效输出值

Question

我在不同长度的批次序列上训练LSTM细胞.它tf.nn.rnn有一个非常方便的参数sequence_length,但在调用它之后,我不知道如何选择与批处理中每个项目的最后一个步骤相对应的输出行.

我的代码基本如下:

lstm_cell = tf.nn.rnn_cell.LSTMCell(num_lstm_units, input_size)
lstm_outputs, state = tf.nn.rnn(lstm_cell, input_list, dtype=tf.float32, sequence_length=sequence_lengths)

lstm_outputs是每个时间步都有LSTM输出的列表.但是,我的批处理中的每个项目都有不同的长度,因此我想创建一个包含最后一个LSTM输出的张量,该输出对我的批次中的每个项目都有效.

如果我可以使用numpy索引,我会做这样的事情:

all_outputs = tf.pack(lstm_outputs)
last_outputs = all_outputs[sequence_lengths, tf.range(batch_size), :]

但事实证明,开始tensorflow不支持它(我知道功能请求).

那么,我怎么能得到这些价值呢？

Answer 1

danijar在我在问题中链接的功能请求页面上发布了一个更可接受的解决方法.它不需要评估张量,这是一个很大的优点.

我让它与tensorflow 0.8一起工作.这是代码:

def extract_last_relevant(outputs, length):
    """
    Args:
        outputs: [Tensor(batch_size, output_neurons)]: A list containing the output
            activations of each in the batch for each time step as returned by
            tensorflow.models.rnn.rnn.
        length: Tensor(batch_size): The used sequence length of each example in the
            batch with all later time steps being zeros. Should be of type tf.int32.

    Returns:
        Tensor(batch_size, output_neurons): The last relevant output activation for
            each example in the batch.
    """
    output = tf.transpose(tf.pack(outputs), perm=[1, 0, 2])
    # Query shape.
    batch_size = tf.shape(output)[0]
    max_length = int(output.get_shape()[1])
    num_neurons = int(output.get_shape()[2])
    # Index into flattened array as a workaround.
    index = tf.range(0, batch_size) * max_length + (length - 1)
    flat = tf.reshape(output, [-1, num_neurons])
    relevant = tf.gather(flat, index)
    return relevant