如何在Python中构建升力图(又名增益图表)？

Question

我刚刚使用scikit-learn创建了一个模型,该模型估计了客户对某些优惠的响应可能性.现在我正在尝试评估我的模型.为此,我想绘制升力图.我理解lift的概念,但我很难理解如何在python中实际实现它.

Answer 1

您可以使用scikit-plot包来完成繁重的工作.

skplt.metrics.plot_cumulative_gain(y_test, predicted_probas)

例

# The usual train-test split mumbo-jumbo
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB

X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, 
test_size=0.33)
nb = GaussianNB()
nb.fit(X_train, y_train)
predicted_probas = nb.predict_proba(X_test)

# The magic happens here
import matplotlib.pyplot as plt
import scikitplot as skplt
skplt.metrics.plot_cumulative_gain(y_test, predicted_probas)
plt.show()

这应该会产生如下情节:

Answer 2

提升/累积增益图表不是评估模型的好方法(因为它不能用于模型之间的比较),而是一种评估资源有限的结果的方法.要么是因为每个结果都有成本(在营销方案中),要么你想忽略一定数量的有保证的选民,而只是采取行动.如果您的模型非常好,并且所有结果的分类准确度都很高,那么您可以放心地订购结果.

import sklearn.metrics
import pandas as pd

def calc_cumulative_gains(df: pd.DataFrame, actual_col: str, predicted_col:str, probability_col:str):

该方法如下所示,首先将数据排序到bin中,然后自信地排序.该方法返回用于可视化的数据帧.

df.sort_values(by=probability_col, ascending=False, inplace=True)

subset = df[df[predicted_col] == True]

rows = []
for group in np.array_split(subset, 10):
    score = sklearn.metrics.accuracy_score(group[actual_col].tolist(),
                                               group[predicted_col].tolist(),
                                               normalize=False)

    rows.append({'NumCases': len(group), 'NumCorrectPredictions': score})

lift = pd.DataFrame(rows)

#Cumulative Gains Calculation
lift['RunningCorrect'] = lift['NumCorrectPredictions'].cumsum()
lift['PercentCorrect'] = lift.apply(
    lambda x: (100 / lift['NumCorrectPredictions'].sum()) * x['RunningCorrect'], axis=1)
lift['CumulativeCorrectBestCase'] = lift['NumCases'].cumsum()
lift['PercentCorrectBestCase'] = lift['CumulativeCorrectBestCase'].apply(
    lambda x: 100 if (100 / lift['NumCorrectPredictions'].sum()) * x > 100 else (100 / lift[
        'NumCorrectPredictions'].sum()) * x)
lift['AvgCase'] = lift['NumCorrectPredictions'].sum() / len(lift)
lift['CumulativeAvgCase'] = lift['AvgCase'].cumsum()
lift['PercentAvgCase'] = lift['CumulativeAvgCase'].apply(
    lambda x: (100 / lift['NumCorrectPredictions'].sum()) * x)

#Lift Chart
lift['NormalisedPercentAvg'] = 1
lift['NormalisedPercentWithModel'] = lift['PercentCorrect'] / lift['PercentAvgCase']

return lift

要绘制累积增益图表,可以使用下面的代码.

import matplotlib.pyplot as plt
def plot_cumulative_gains(lift: pd.DataFrame):
    fig, ax = plt.subplots()
    fig.canvas.draw()

    handles = []
    handles.append(ax.plot(lift['PercentCorrect'], 'r-', label='Percent Correct Predictions'))
    handles.append(ax.plot(lift['PercentCorrectBestCase'], 'g-', label='Best Case (for current model)'))
    handles.append(ax.plot(lift['PercentAvgCase'], 'b-', label='Average Case (for current model)'))
    ax.set_xlabel('Total Population (%)')
    ax.set_ylabel('Number of Respondents (%)')

    ax.set_xlim([0, 9])
    ax.set_ylim([10, 100])

    labels = [int((label+1)*10) for label in [float(item.get_text()) for item in ax.get_xticklabels()]]

    ax.set_xticklabels(labels)

    fig.legend(handles, labels=[h[0].get_label() for h in handles])
    fig.show()

并可视化电梯:

def plot_lift_chart(lift: pd.DataFrame):
    plt.figure()
    plt.plot(lift['NormalisedPercentAvg'], 'r-', label='Normalised \'response rate\' with no model')
    plt.plot(lift['NormalisedPercentWithModel'], 'g-', label='Normalised \'response rate\' with using model')
    plt.legend()
    plt.show()

结果如下:

我发现这些网站可供参考:

• https://docs.microsoft.com/en-us/sql/analysis-services/data-mining/lift-chart-analysis-services-data-mining
• https://paultebraak.wordpress.com/2013/10/31/understanding-the-lift-chart/
• http://www2.cs.uregina.ca/~dbd/cs831/notes/lift_chart/lift_chart.html

编辑:

我发现MS链接在其描述中有些误导,但Paul Te Braak链接非常有用.回答评论;

@Tanguy对于上面的累积增益图表,所有计算都是基于该特定模型的准确性.正如Paul Te Braak链接说明的那样,我的模型的预测精度如何达到100%(图表中的红线)？最好的情况(绿线)是我们能够多快地达到红线在整个人口过程中达到的相同精度(例如我们的最佳累积增益情景).蓝色是我们只是随机选择人口中每个样本的分类.因此,累积收益和提升图表纯粹是为了理解该模型(以及该模型)将如何在我不打算与整个人口互动的情况下给予我更大的影响.

我使用累积收益图表的一个场景是针对欺诈案例,我想知道有多少应用程序我们可以基本忽略或优先排序(因为我知道模型可以预测它们以及它可以)前10%.在这种情况下,对于"平均模型",我改为从真实的无序数据集中选择分类(以显示如何处理现有应用程序,以及如何使用模型 - 我们可以优先考虑应用程序类型).

因此,对于比较模型,只需坚持使用ROC/AUC,一旦您对所选模型感到满意,请使用累积增益/提升图表来查看它如何响应数据.

Answer 3

您可以使用kds包来实现同样的目的。

对于累积收益图：

# pip install kds
import kds
kds.metrics.plot_cumulative_gain(y_test, y_prob)

例子

# REPRODUCABLE EXAMPLE
# Load Dataset and train-test split
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn import tree

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, 
test_size=0.33,random_state=3)
clf = tree.DecisionTreeClassifier(max_depth=1,random_state=3)
clf = clf.fit(X_train, y_train)
y_prob = clf.predict_proba(X_test)


# CUMMULATIVE GAIN PLOT
import kds
kds.metrics.plot_cumulative_gain(y_test, y_prob[:,1])

向导曲线将为模型提供最佳的曲线。

免责声明：我是这个包的作者