python pandas用文字中的复数"s"来计算字数

Question

我有以下python pandas数据帧:

Question_ID | Customer_ID | Answer
    1           234         The team worked very hard ...
    2           234         All the teams have been working together ...

我将使用我的代码来计算答案列中的单词.但事先,我想从"球队"这个词中取出"s",所以在上面的例子中我统计球队:2而不是球队:1和球队:1.

我怎么能为所有的话呢？

Answer 1

您需要使用自然语言工具包提供的标记化器(用于将句子分解为单词)和lemmmatizer(用于标准化单词形式)nltk:

import nltk
wnl = nltk.WordNetLemmatizer()
[wnl.lemmatize(word) for word in nltk.wordpunct_tokenize(sentence)]
# ['All', 'the', 'team', 'have', 'been', 'working', 'together']