NLTK v3.2:无法使用nltk.pos_tag()

Question

嗨文本挖掘冠军,

我在Windows 10上使用带有NLTK v3.2的Anaconda.(客户端的环境)

当我尝试POS标记时,我不断收到URLLIB2错误:

URLError: <urlopen error unknown url type: c>

看来urllib2无法识别Windows路径？我该如何解决这个问题？

该命令很简单:

nltk.pos_tag(nltk.word_tokenize("Hello World"))

编辑:有一个重复的问题,但我认为manan和alvas在这里得到的答案是一个更好的解决方案.

Answer 1

EDITED

此问题已从NLTK v3.2.1解决.升级您的NLTK版本可以解决问题,例如pip install -U nltk.

我遇到了同样的问题,遇到的错误如下;

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\tag\__init__.py", line 110, in pos_tag
tagger = PerceptronTagger()
  File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\tag\perceptron.py", line 141, in __init__
self.load(AP_MODEL_LOC)
  File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\tag\perceptron.py", line 209, in load
self.model.weights, self.tagdict, self.classes = load(loc)
  File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\data.py", line 801, in load
opened_resource = _open(resource_url)
  File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\data.py", line 924, in _open
return urlopen(resource_url)
  File "C:\Python27\lib\urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
  File "C:\Python27\lib\urllib2.py", line 391, in open
response = self._open(req, data)
  File "C:\Python27\lib\urllib2.py", line 414, in _open
'unknown_open', req)
  File "C:\Python27\lib\urllib2.py", line 369, in _call_chain
result = func(*args)
  File "C:\Python27\lib\urllib2.py", line 1206, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib2.URLError: <urlopen error unknown url type: c>

您提到的URLError是由于Windows的NLTK库中的perceptron.py文件中的错误引起的.在我的机器中,文件位于此位置

C:\Python27\Lib\site-packages\nltk-3.2-py2.7.egg\nltk\tag\perceptron.py

(无论你在哪里拥有Python27文件夹,基本上都可以查看你的同等位置)

该错误基本上是在代码中找到机器中averaged_perceptron_tagger的相应位置.可以看一下data.py文件中提到的关于此的801和924行.

我认为NLTK开发人员社区最近修复了代码中的这个错误.看看几天前对它们的代码做出的这个提交.

https://github.com/nltk/nltk/commit/d3de14e58215beebdccc7b76c044109f6197d1d9#diff-26b258372e0d13c2543de8dbb1841252

进行更改的片段如下;

self.tagdict = {}
self.classes = set()
    if load:
        AP_MODEL_LOC = 'file:'+str(find('taggers/averaged_perceptron_tagger/'+PICKLE))
          self.load(AP_MODEL_LOC)
        # Initially it was:AP_MODEL_LOC = str(find('taggers/averaged_perceptron_tagger/'+PICKLE)) 

def tag(self, tokens):

将文件更新为最新的提交对我有用,并且能够使用nltk.pos_tag命令.我相信这也可以解决你的问题(假设你已经设置了其他一切).

Answer 2

EDITED

此问题已从NLTK v3.2.1解决.请升级您的NLTK!

首先阅读@MananVyas回答原因:

/sf/answers/2513174611/

以下是使用NLTK 3.2,如果不降级到NLTK v3.1,你可以使用这个"黑客":

>>> from nltk.tag import PerceptronTagger
>>> from nltk.data import find
>>> PICKLE = "averaged_perceptron_tagger.pickle"
>>> AP_MODEL_LOC = 'file:'+str(find('taggers/averaged_perceptron_tagger/'+PICKLE))
>>> tagger = PerceptronTagger(load=False)
>>> tagger.load(AP_MODEL_LOC)
>>> pos_tag = tagger.tag
>>> pos_tag('The quick brown fox jumps over the lazy dog'.split())
[('The', 'DT'), ('quick', 'JJ'), ('brown', 'NN'), ('fox', 'NN'), ('jumps', 'VBZ'), ('over', 'IN'), ('the', 'DT'), ('lazy', 'JJ'), ('dog', 'NN')]