ValueError:模型的要素数必须与输入匹配

Question

在尝试使用我在scikit中构建的模型进行预测时,我收到此错误.我知道有很多关于这个的问题,但我的看起来与它们不同,因为我在输入和模型功能之间疯狂.这是我训练模型的代码(仅供参考.csv文件有45列,其中一列是已知值):

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn import ensemble
from sklearn.metrics import mean_absolute_error
from sklearn.externals import joblib


df = pd.read_csv("Cinderella.csv")


features_df = pd.get_dummies(df, columns=['Overall_Sentiment', 'Word_1','Word_2','Word_3','Word_4','Word_5','Word_6','Word_7','Word_8','Word_9','Word_10','Word_11','Word_1','Word_12','Word_13','Word_14','Word_15','Word_16','Word_17','Word_18','Word_19','Word_20','Word_21','Word_22','Word_23','Word_24','Word_25','Word_26','Word_27','Word_28','Word_29','Word_30','Word_31','Word_32','Word_33','Word_34','Word_35','Word_36','Word_37','Word_38','Word_39','Word_40','Word_41', 'Word_42', 'Word_43'], dummy_na=True)

del features_df['Slope']

X = features_df.as_matrix()
y = df['Slope'].as_matrix()

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

model = ensemble.GradientBoostingRegressor(
    n_estimators=500,
    learning_rate=0.01,
    max_depth=5,
    min_samples_leaf=3,
    max_features=0.1,
    loss='lad'
)

model.fit(X_train, y_train)

joblib.dump(model, 'slope_from_sentiment_model.pkl')

mse = mean_absolute_error(y_train, model.predict(X_train))

print("Training Set Mean Absolute Error: %.4f" % mse)

mse = mean_absolute_error(y_test, model.predict(X_test))
print("Test Set Mean Absolute Error: %.4f" % mse)

这是我使用不同的.csv文件进行实际预测的代码(这有44列,因为它没有任何值):

from sklearn.externals import joblib
import pandas


model = joblib.load('slope_from_sentiment_model.pkl')

df = pandas.read_csv("Slaughterhouse_copy.csv")


features_df = pandas.get_dummies(df, columns=['Overall_Sentiment','Word_1', 'Word_2', 'Word_3', 'Word_4', 'Word_5', 'Word_6', 'Word_7', 'Word_8', 'Word_9', 'Word_10', 'Word_11', 'Word_12', 'Word_13', 'Word_14', 'Word_15', 'Word_16', 'Word_17','Word_18','Word_19','Word_20','Word_21','Word_22','Word_23','Word_24','Word_25','Word_26','Word_27','Word_28','Word_29','Word_30','Word_31','Word_32','Word_33','Word_34','Word_35','Word_36','Word_37','Word_38','Word_39','Word_40','Word_41','Word_42','Word_43'], dummy_na=True)

predicted_slopes = model.predict(features_df)

当我运行预测文件时,我得到:

ValueError: Number of features of the model must match the input. Model n_features is 146 and input n_features is 226.

如果有人能帮助我,我将不胜感激!提前致谢!

Answer 1

您收到错误的原因是由于您要生成虚拟值的要素中的不同值get_dummies.

假设Word_1训练集中的列具有以下不同的单词:the, dog, jumps, roof, off.这是5个不同的词,所以大熊猫将产生5个功能Word_1.现在,如果您的评分数据集在Word_1列中具有不同数量的不同单词,那么您将获得不同数量的功能.

怎么修:

您需要使用concat,apply get_dummies,然后拆分数据集来连接训练和评分数据集.这将确保您已捕获列中的所有不同值.鉴于您使用的是两个不同的csv,您可能希望生成一个指定训练与评分数据集的列.

示例解决方案

train_df = pd.read_csv("Cinderella.csv")
train_df['label'] = 'train'

score_df = pandas.read_csv("Slaughterhouse_copy.csv")
score_df['label'] = 'score'

# Concat
concat_df = pd.concat([train_df , score_df])

# Create your dummies
features_df = pd.get_dummies(concat_df, columns=['Overall_Sentiment', 'Word_1','Word_2','Word_3','Word_4','Word_5','Word_6','Word_7','Word_8','Word_9','Word_10','Word_11','Word_1','Word_12','Word_13','Word_14','Word_15','Word_16','Word_17','Word_18','Word_19','Word_20','Word_21','Word_22','Word_23','Word_24','Word_25','Word_26','Word_27','Word_28','Word_29','Word_30','Word_31','Word_32','Word_33','Word_34','Word_35','Word_36','Word_37','Word_38','Word_39','Word_40','Word_41', 'Word_42', 'Word_43'], dummy_na=True)

# Split your data
train_df = features_df[features_df['label'] == 'train']
score_df = features_df[features_df['label'] == 'score']

# Drop your labels
train_df = train_df.drop('label', axis=1)
score_df = score_df.drop('label', axis=1)

# Now delete your 'slope' feature, create your features matrix, and create your model as you have already shown in your example
...