如何在xgboost中获取每个树的预测？

Question

使用xgboost.Booster.predict只能获得所有树的预测结果或每棵树的预测叶.但是我怎么能得到每棵树的预测值呢？

Answer 1

截至最近，xgboost引入了切片 API，Raul 的答案虽然有效，但过于复杂。

要获得单独的预测，您所需要做的就是迭代booster对象。

individual_preds = []
for tree_ in model.get_booster():
    individual_preds.append(
        tree_.predict(xgb.DMatrix(X))
    )

但请注意，这些个人预测并不是个人贡献。例如，将它们相加并不能得到最终的预测。为此，我们需要将它们转换回对数赔率，然后求和：

from scipy.special import expit as sigmoid, logit as inverse_sigmoid
individual_preds = np.vstack(individual_preds)
indivudual_logits = inverse_sigmoid(individual_preds)
final_logits = indivudual_logits.sum(axis=0)
final_preds = sigmoid(final_logits)

完全可复制的例子，复制劳尔的努力

import numpy as np
import xgboost as xgb
from sklearn import datasets
from scipy.special import expit as sigmoid, logit as inverse_sigmoid

# Load data
iris = datasets.load_iris()
X, y = iris.data, (iris.target == 1).astype(int)

# Fit a model
model = xgb.XGBClassifier(
    n_estimators=10,
    max_depth=10,
    use_label_encoder=False,
    objective='binary:logistic'
)
model.fit(X, y)
booster_ = model.get_booster()

# Extract indivudual predictions
individual_preds = []
for tree_ in booster_:
    individual_preds.append(
        tree_.predict(xgb.DMatrix(X))
    )
individual_preds = np.vstack(individual_preds)

# Aggregated individual predictions to final predictions
indivudual_logits = inverse_sigmoid(individual_preds)
final_logits = indivudual_logits.sum(axis=0)
final_preds = sigmoid(final_logits)

# Verify correctness
xgb_preds = booster_.predict(xgb.DMatrix(X))
np.testing.assert_almost_equal(final_preds, xgb_preds)