如何从nltk分类器获取精度和召回率？

Question

import nltk
from nltk.corpus import movie_reviews
from nltk.tokenize import word_tokenize

documents = [(list(movie_reviews.words(fileid)), category)
             for category in movie_reviews.categories()
             for fileid in movie_reviews.fileids(category)]


all_words = []

for w in movie_reviews.words():
    all_words.append(w.lower())

all_words = nltk.FreqDist(all_words)

word_features = list(all_words.keys())[:3000]

def find_features(document):
    words = set(document)
    features = {}
    for w in word_features:
        features[w] = (w in words)

    return features

featuresets = [(find_features(rev), category) for (rev, category) in documents]

training_set = featuresets[500:1500]
testing_set = featuresets[:1500]

classifier = nltk.DecisionTreeClassifier.train(training_set)

print "Classifier accuracy percent:",(nltk.classify.accuracy(classifier, testing_set))*100 , "%"

string = raw_input("Enter the string: ")
print (classifier.classify(find_features(word_tokenize(string))))

该代码将显示分类器的准确性，然后从用户那里获取输入。并且它返回用户输入的字符串的极性。

但是，这是我的问题：由于我可以使用来获得精度nltk.accuracy()，是否有可能获得其精度和召回率？

Answer 1

如果您使用的是nltk软件包，那么您似乎可以使用recall和中的precision函数nltk.metrics.scores（请参阅docs）。

调用后功能应该可用

from nltk.metrics.scores import (precision, recall)

然后，您需要使用reference（已知标签）和test（测试集上分类器的输出）集合来调用它们。

像下面的代码的东西应该产生这些集作为refsets和testsets

refsets = collections.defaultdict(set)
testsets = collections.defaultdict(set)

for i, (feats, label) in enumerate(testing_set):
    refsets[label].add(i)
    observed = classifier.classify(feats)
    testsets[observed].add(i)

然后，您可以看到准确度并回想正面预测，例如

print 'Precision:', nltk.metrics.precision(refsets['pos'], testsets['pos'])
print 'Recall:', nltk.metrics.recall(refsets['pos'], testsets['pos'])