具有辅助输入的Keras中的LSTM模型

Question

我有一个包含2列的数据集 - 每列包含一组文档.我必须将Col A中的文档与Col B中提供的文档相匹配.这是一个受监督的分类问题.所以我的训练数据包含一个标签栏,表明文件是否匹配.

为了解决这个问题,我创建了一组功能,例如f1-f25(通过比较2个文档),然后在这些功能上训练了二元分类器.这种方法运行得相当好,但现在我想评估这个问题的深度学习模型(特别是LSTM模型).

我keras在Python中使用该库.在浏览了keras文档和在线提供的其他教程后,我成功完成了以下操作:

from keras.layers import Input, Embedding, LSTM, Dense
from keras.models import Model

# Each document contains a series of 200 words 
# The necessary text pre-processing steps have been completed to transform  
  each doc to a fixed length seq
main_input1 = Input(shape=(200,), dtype='int32', name='main_input1')
main_input2 = Input(shape=(200,), dtype='int32', name='main_input2')

# Next I add a word embedding layer (embed_matrix is separately created    
for each word in my vocabulary by reading from a pre-trained embedding model)
x = Embedding(output_dim=300, input_dim=20000, 
input_length=200, weights = [embed_matrix])(main_input1)
y = Embedding(output_dim=300, input_dim=20000, 
input_length=200, weights = [embed_matrix])(main_input2)

# Next separately pass each layer thru a lstm layer to transform seq of   
vectors into a single sequence
lstm_out_x1 = LSTM(32)(x)
lstm_out_x2 = LSTM(32)(y)

# concatenate the 2 layers and stack a dense layer on top
x = keras.layers.concatenate([lstm_out_x1, lstm_out_x2])
x = Dense(64, activation='relu')(x)
# generate intermediate output
auxiliary_output = Dense(1, activation='sigmoid', name='aux_output')(x)

# add auxiliary input - auxiliary inputs contains 25 features for each document pair
auxiliary_input = Input(shape=(25,), name='aux_input')

# merge aux output with aux input and stack dense layer on top
main_input = keras.layers.concatenate([auxiliary_output, auxiliary_input])
x = Dense(64, activation='relu')(main_input)
x = Dense(64, activation='relu')(x)

# finally add the main output layer
main_output = Dense(1, activation='sigmoid', name='main_output')(x)

model = Model(inputs=[main_input1, main_input2, auxiliary_input], outputs= main_output)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

model.fit([x1, x2,aux_input], y,
      epochs=3, batch_size=32)

但是,当我在训练数据上得分时,我会得到相同的概率.所有案件的得分.问题似乎与辅助输入的输入方式有关(因为当我删除辅助输入时它会产生有意义的输出).我还尝试在网络中的不同位置插入辅助输入.但不知怎的,我无法让这个工作.

有什么指针吗？

Answer 1

嗯，这个开放了几个月，人们正在投票。我最近使用这个数据集
做了一些非常类似的事情，该数据集可用于预测信用卡违约，它包含客户的分类数据（性别、教育水平、婚姻状况等）以及时间序列的付款历史记录。所以我必须将时间序列与非序列数据合并。我的解决方案与您的解决方案非常相似，将 LSTM 与密集相结合，我尝试采用该方法来解决您的问题。对我有用的是辅助输入上的密集层。

此外，在您的情况下，共享层是有意义的，因此使用相同的权重来“读取”两个文档。我对您的数据进行测试的建议：

from keras.layers import Input, Embedding, LSTM, Dense
from keras.models import Model

# Each document contains a series of 200 words 
# The necessary text pre-processing steps have been completed to transform  
  each doc to a fixed length seq
main_input1 = Input(shape=(200,), dtype='int32', name='main_input1')
main_input2 = Input(shape=(200,), dtype='int32', name='main_input2')

# Next I add a word embedding layer (embed_matrix is separately created    
for each word in my vocabulary by reading from a pre-trained embedding model)
x1 = Embedding(output_dim=300, input_dim=20000, 
input_length=200, weights = [embed_matrix])(main_input1)
x2 = Embedding(output_dim=300, input_dim=20000, 
input_length=200, weights = [embed_matrix])(main_input2)

# Next separately pass each layer thru a lstm layer to transform seq of   
vectors into a single sequence
# Comment Manngo: Here I changed to shared layer
# Also renamed y as input as it was confusing
# Now x and y are x1 and x2
lstm_reader = LSTM(32)
lstm_out_x1 = lstm_reader(x1)
lstm_out_x2 = lstm_reader(x2)

# concatenate the 2 layers and stack a dense layer on top
x = keras.layers.concatenate([lstm_out_x1, lstm_out_x2])
x = Dense(64, activation='relu')(x)
x = Dense(32, activation='relu')(x)
# generate intermediate output
# Comment Manngo: This is created as a dead-end
# It will not be used as an input of any layers below
auxiliary_output = Dense(1, activation='sigmoid', name='aux_output')(x)

# add auxiliary input - auxiliary inputs contains 25 features for each document pair
# Comment Manngo: Dense branch on the comparison features
auxiliary_input = Input(shape=(25,), name='aux_input')
auxiliary_input = Dense(64, activation='relu')(auxiliary_input)
auxiliary_input = Dense(32, activation='relu')(auxiliary_input)

# OLD: merge aux output with aux input and stack dense layer on top
# Comment Manngo: actually this is merging the aux output preparation dense with the aux input processing dense
main_input = keras.layers.concatenate([x, auxiliary_input])
main = Dense(64, activation='relu')(main_input)
main = Dense(64, activation='relu')(main)

# finally add the main output layer
main_output = Dense(1, activation='sigmoid', name='main_output')(main)

# Compile
# Comment Manngo: also define weighting of outputs, main as 1, auxiliary as 0.5
model.compile(optimizer=adam,
              loss={'main_output': 'w_binary_crossentropy', 'aux_output': 'binary_crossentropy'},
              loss_weights={'main_output': 1.,'auxiliary_output': 0.5},
              metrics=['accuracy'])

# Train model on main_output and on auxiliary_output as a support
# Comment Manngo: Unknown information marked with placeholders ____
# We have 3 inputs: x1 and x2: the 2 strings
# aux_in: the 25 features
# We have 2 outputs: main and auxiliary; both have the same targets -> (binary)y


model.fit({'main_input1': __x1__, 'main_input2': __x2__, 'auxiliary_input' : __aux_in__}, {'main_output': __y__, 'auxiliary_output': __y__}, 
              epochs=1000, 
              batch_size=__, 
              validation_split=0.1, 
              callbacks=[____])

我不知道这有多大帮助，因为我没有你的数据，所以我无法尝试。尽管如此，这是我最好的机会。
由于显而易见的原因，我没有运行上面的代码。