我正在尝试使用PunktSentenceTokenizerfrom nltk将文本拆分成句子。文本包含以项目符号开头的列表,但它们不会被识别为新句子。我试图添加一些参数,但没有用。还有其他方法吗?
下面是一些示例代码:
from nltk.tokenize.punkt import PunktSentenceTokenizer, PunktParameters
params = PunktParameters()
params.sent_starters = set(['•'])
tokenizer = PunktSentenceTokenizer(params)
tokenizer.tokenize('• I am a sentence • I am another sentence')
['• I am a sentence • I am another sentence']
Run Code Online (Sandbox Code Playgroud)