我正在尝试使用来自sckit-learn的Tf-idf Vectorizer,使用来自NLTK的西班牙语停用词:
from nltk.corpus import stopwords
vectorizer = TfidfVectorizer(stop_words=stopwords.words("spanish"))
问题是我得到以下警告:
/home/---/.virtualenvs/thesis/local/lib/python2.7/site-packages/sklearn/feature_extraction/text.py:122: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
tokens = [w for w in tokens if w not in stop_words]
有没有简单的方法来解决这个问题？