Fitting a Keras model yields error "constant folding failed: Invalid argument: Unsupported type: 21"

Question

I am following the example from https://www.tensorflow.org/alpha/tutorials/load_data/text to load my own dataset and run binary classification on the sentences there (TensorFlow 2.0). The only change I've made to the example is the dataset being used (which I took from https://github.com/UKPLab/emnlp2017-claim-identification/tree/master/src/main/python), and since the labels can be only 0 or 1 I changed the loss function to binary_crossentropy and the optimizer to RMSprop. When fitting the Keras model which is identical to the model proposed in the tutorial, I'm constantly receiving the following error:

2019-04-29 13:51:15.609297: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:495] constant folding failed: Invalid argument: Unsupported type: 21
2019-04-29 13:51:15.882000: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:495] constant folding failed: Invalid argument: Unsupported type: 21

The fitting process is still advancing between those prints as evident from:

662/4508 [===>..........................] - ETA: 9:35 - loss: 11.0703 - accuracy: 0.2780

but instead of minimizing the loss, it seems to actually be maximizing it, with the accuracy going down after each iteration. (In fact, if the accuracy metric is correct, it would be a pretty good classifier if I just take not [prediction]). Is there anyone here who can explain to me what is the meaning of this error, and whether it's related to the strange behavior of the model (and hopefully how to fix it)? I've been trying to look for similar errors but couldn't find any. Thanks!

Answer 1

链接https://www.tensorflow.org/beta/tutorials/load_data/text中提到的教程与您的数据集之间存在一个关键区别。

在教程中，标签为 0、1 和 2，即 中的所有句子 都cowper.txt标记为0， 中的所有句子derby.txt都标记为1， 中的所有句子butler.txt都标记为2。但在您的数据集中，标签位于文本文件每个句子的末尾。

我已经执行了您的数据集的代码，如下所示：

FILE_NAMES = ['001.dev', '001.test', '001.train', '002.dev', '002.test', '002.train']

parent_dir = "Issue_55902068/OC"

parent_dir

为了处理上述差异，labeler应修改该函数，如下所示：

def labeler(example, index):
  Label = tf.strings.split(example, sep="")[-1] #It will give 0 or 1 in Str format
  Label = tf.strings.to_number(Label)
  return example, tf.cast(Label, tf.int64)

之后，我将损失函数binary_crossentropy和优化器更改RMSprop为如下所示：

model.compile(optimizer='RMSprop', loss='binary_crossentropy', metrics=['accuracy'])

它正在按预期工作。输出的屏幕截图如下所示。