用nltk分割句子,同时保留引号

Question

我正在使用nltk将文本拆分为句子单元.但是,我需要将包含引号的句子作为单个单元提取.现在每个句子,即使它在一个引用中,也被提取为一个单独的部分.

这是我尝试作为单个单元提取的一些示例:

"This is a sentence. This is also a sentence," said the cat.

现在我有这个代码:

import nltk.data
tokenizer = nltk.data.load('tokenizers/punkt/english.pickle')

text = 'This is a sentence. This is also a sentence," said the cat.'

print '\n-----\n'.join(tokenizer.tokenize(text, realign_boundaries=True))

这很好用,但即使引号本身包含多个句子,我也想维持带引号的句子.

上面的代码产生:

This is a sentence.
-----
This is also a sentence," said the cat.

我试图将整个文本提取为一个单元:

"This is a sentence. This is also a sentence," said the cat.

有没有一种简单的方法可以使用nltk执行此操作,还是应该使用正则表达式？我对nltk入门是多么容易感到印象深刻,但现在被卡住了.

Answer 1

只需将打印语句更改为：

print ' '.join(tokenizer.tokenize(text, realign_boundaries=True))

这将用空格而不是 来连接句子\n-----\n。