在scikit中结合随机森林模型学习

Question

我有两个RandomForestClassifier模型,我想将它们组合成一个元模型.他们都使用类似但不同的数据进行培训.我怎样才能做到这一点？

rf1 #this is my first fitted RandomForestClassifier object, with 250 trees
rf2 #this is my second fitted RandomForestClassifier object, also with 250 trees

我想创建big_rf所有树木组合成一个500树模型

Answer 1

我相信这可以通过修改RandomForestClassifier对象的estimators_和n_estimators属性来实现.林中的每个树都存储为DecisionTreeClassifier对象,这些树的列表存储在estimators_属性中.为了确保没有不连续性,更改估算器的数量也是有意义的n_estimators.

这种方法的优点是你可以在多台机器上并行构建一堆小型森林并将它们组合在一起.

以下是使用虹膜数据集的示例:

from sklearn.ensemble import RandomForestClassifier
from sklearn.cross_validation import train_test_split
from sklearn.datasets import load_iris

def generate_rf(X_train, y_train, X_test, y_test):
    rf = RandomForestClassifier(n_estimators=5, min_samples_leaf=3)
    rf.fit(X_train, y_train)
    print "rf score ", rf.score(X_test, y_test)
    return rf

def combine_rfs(rf_a, rf_b):
    rf_a.estimators_ += rf_b.estimators_
    rf_a.n_estimators = len(rf_a.estimators_)
    return rf_a

iris = load_iris()
X, y = iris.data[:, [0,1,2]], iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.33)
# in the line below, we create 10 random forest classifier models
rfs = [generate_rf(X_train, y_train, X_test, y_test) for i in xrange(10)]
# in this step below, we combine the list of random forest models into one giant model
rf_combined = reduce(combine_rfs, rfs)
# the combined model scores better than *most* of the component models
print "rf combined score", rf_combined.score(X_test, y_test)

Answer 2

除了@mgoldwasser解决方案之外,另一种方法是warm_start在训练森林时使用.在Scikit-Learn 0.16-dev中,您现在可以执行以下操作:

# First build 100 trees on X1, y1
clf = RandomForestClassifier(n_estimators=100, warm_start=True)
clf.fit(X1, y1)

# Build 100 additional trees on X2, y2
clf.set_params(n_estimators=200)
clf.fit(X2, y2)