如何使用spacy lemmatizer来获得基本形式的单词

Question

我是新手,知道spacy并且我想使用他的lemmatizer功能,但我不知道如何使用它,就像我进入单词串,它将返回具有基本形式单词的字符串.比如'words'=> word,'did'=>'do',谢谢.

Answer 1

以前的答案是复杂的,无法编辑,所以这是一个更传统的答案.

# make sure your downloaded the english model with "python -m spacy download en"

import spacy
nlp = spacy.load('en')

doc = nlp(u"Apples and oranges are similar. Boots and hippos aren't.")

for token in doc:
    print(token, token.lemma, token.lemma_)

输出:

Apples 6617 apples
and 512 and
oranges 7024 orange
are 536 be
similar 1447 similar
. 453 .
Boots 4622 boot
and 512 and
hippos 98365 hippo
are 536 be
n't 538 not
. 453 .

来自官方照明之旅

Answer 2

代码:

import os
from spacy.en import English, LOCAL_DATA_DIR

data_dir = os.environ.get('SPACY_DATA', LOCAL_DATA_DIR)

nlp = English(data_dir=data_dir)

doc3 = nlp(u"this is spacy lemmatize testing. programming books are more better than others")

for token in doc3:
    print token, token.lemma, token.lemma_

输出:

this 496 this
is 488 be
spacy 173779 spacy
lemmatize 1510965 lemmatize
testing 2900 testing
. 419 .
programming 3408 programming
books 1011 book
are 488 be
more 529 more
better 615 better
than 555 than
others 871 others

示例参考:这里

Answer 3

如果你只想使用Lemmatizer.你可以通过以下方式做到这一点.

from spacy.lemmatizer import Lemmatizer
from spacy.lang.en import LEMMA_INDEX, LEMMA_EXC, LEMMA_RULES

lemmatizer = Lemmatizer(LEMMA_INDEX, LEMMA_EXC, LEMMA_RULES)
lemmas = lemmatizer(u'ducks', u'NOUN')
print(lemmas)

产量

['duck']

Answer 4

我使用的是 Spacy 2.x 版

import spacy
nlp = spacy.load('en_core_web_sm', disable=['parser', 'ner'])
doc = nlp('did displaying words')
print (" ".join([token.lemma_ for token in doc]))

和输出：

do display word

希望能帮助到你 ：）