Python(nltk) - UnicodeDecodeError:'ascii'编解码器无法解码字节

Question

我是NLTK的新手.我收到此错误,我已经搜索了编码/解码,特别是UnicodeDecodeError,但这个错误似乎特定于NLTK源代码.

这是错误:

Traceback (most recent call last):
  File "A:\Python\Projects\Test\main.py", line 2, in <module>
    print(pos_tag(word_tokenize("John's big idea isn't all that bad.")))
  File "A:\Python\Python\lib\site-packages\nltk\tag\__init__.py", line 100, in pos_tag
    tagger = load(_POS_TAGGER)
  File "A:\Python\Python\lib\site-packages\nltk\data.py", line 779, in load
    resource_val = pickle.load(opened_resource)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcb in position 0: ordinal not in range(128)

如何解决此错误？

这是导致错误的原因:

from nltk import pos_tag, word_tokenize
print(pos_tag(word_tokenize("John's big idea isn't all that bad.")))

Answer 1

试试这个...用Python 2.7.x的NLTK 3.0.1

import io
f = io.open(txtFile, 'rU', encoding='utf-8')

Answer 2

重复：NLTK 3 POS_TAG 抛出 UnicodeDecodeError

长话短说：NLTK 与 Python 3 不兼容。您必须使用 NLTK 3，这在这一点上听起来有点实验性。