我正在尝试解决文本分类问题.我有一些有限数量的标签可以捕获我的文本数据类别.如果传入的文本数据不适合任何标签,则标记为"其他".在下面的示例中,我构建了一个文本分类器,将文本数据分类为"早餐"或"意大利语".在测试场景中,我包含了几个不适合我用于训练的标签的文本数据.这是我面临的挑战.理想情况下,我希望模型能说 - "其他"用于"我喜欢徒步旅行"和"每个人都应该理解数学".我怎样才能做到这一点？

import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.feature_extraction.text import TfidfTransformer

X_train = np.array(["coffee is my favorite drink",
                    "i like to have tea in the morning",
                    "i like to eat italian food for dinner",
                    "i had pasta at this restaurant and it was amazing",
                    "pizza at this restaurant is the best in nyc",
                    "people like italian food these days",
                    "i like to have bagels for breakfast",
                    "olive oil …

我有一个pandas数据框,其列表为值.我想将此数据帧转换为预期结果中的格式.数据帧太大(100万行)

import pandas as pd

import numpy as np

df = pd.DataFrame(
      [[['A', 'Second'], [], 'N/A', [6]],
      [[2, 3], [3, 4, 6], [3, 4, 5, 7], [2, 6, 3, 4]]],
      columns=list('ABCD')
      )

df.replace('N/A',np.NaN, inplace=True)

df

    A           B        C          D
0   [A,Second]  []       NaN        [6]
1   [2,3]       [3,4,6] [3,4,5,7]   [2,6,3,4]

预期结果

0 A A
0 A Second
0 D 6
1 A 2
1 A 3
1 B 3
1 B 4
1 B 6
1 C 3
1 C 4 …