评估清单:AvgP @ K和R @ K是否相同？

Question

我的目标是了解平均值Precision at K,和Recall at K.我有两个列表,一个是预测的,另一个是实际的(基本事实)

我们可以将这两个列表称为预测和实际.现在我想做precision@k和recall@k.

使用python我在K处实现了Avg精度,如下所示:

def apk(actual, predicted, k=10):
    """
    Computes the average precision at k.

    This function computes the average precision at k between two lists of items.

    Parameters
    ----------
    actual: list
            A list of elements that are to be predicted (order doesn't matter)
    predicted : list
            A list of predicted elements (order does matter)
    k: int, optional

    Returns
    -------
    score : double
            The average precision at k over the input lists

    """
    if len(predicted) > k:
        predicted = predicted[:k]

    score = 0.0
    num_hits = 0.0

    for i,p in enumerate(predicted):
        if p in actual and p not in predicted[:i]:
            num_hits += 1.0
            score += num_hits / (i + 1.0)

    if not actual:
        return 1.0
    if min(len(actual), k) == 0:
        return 0.0
    else:
        return score / min(len(actual), k)

假设我们的预测有以下顺序的5个字符串: predicted = ['b','c','a','e','d'] andactual = ['a','b','e'] since we are doing @k would the precision@k is same as召回@ k ? If not how would I do召回@ k`

如果我想做f-measure (f-score)上面列举的最佳路线？

Answer 1

我想,你已经检查了维基.基于它的公式,第3个和最大的公式(在"这个有限和相当于:"之后),让我们看看每个迭代的示例:

1. i = 1 p = 1
2. i = 2 rel = 0
3. i = 3 p = 2/3
4. i = 4 p = 3/4
5. i = 5 rel = 0

所以,avp @ 4 = avp @ 5 =(1 + 0.66 + 0.75)/ 3 = 0.805; avp @ 3 =(1 + 0.66)/ 3等等.

召回@ 5 = Recall @ 4 = 3/3 = 1; 召回@ 3 = 2/3; 召回@ 2 =召回@ 1 = 1/3

下面是精度@ k和召回@ k的代码.我保留了你的符号,而actual用于观察/返回值和expected基本事实似乎更常见(参见例如JUnit默认值).

def precision(actual, predicted, k):
    act_set = set(actual)
    pred_set = set(predicted[:k])
    result = len(act_set & pred_set) / float(k)
    return result

def recall(actual, predicted, k):
    act_set = set(actual)
    pred_set = set(predicted[:k])
    result = len(act_set & pred_set) / float(len(act_set))
    return result