如何使用MinMaxScaler sklearn规范化训练和测试数据

Question

所以,我怀疑并一直在寻找答案.所以问题是我何时使用,

from sklearn import preprocessing
min_max_scaler = preprocessing.MinMaxScaler()

df = pd.DataFrame({'A':[1,2,3,7,9,15,16,1,5,6,2,4,8,9],'B':[15,12,10,11,8,14,17,20,4,12,4,5,17,19],'C':['Y','Y','Y','Y','N','N','N','Y','N','Y','N','N','Y','Y']})

df[['A','B']] = min_max_scaler.fit_transform(df[['A','B']])
df['C'] = df['C'].apply(lambda x: 0 if x.strip()=='N' else 1)

这之后,我将训练和测试模型(A,B作为特征,C如标签),并得到一些准确度得分.现在我的疑问是,当我必须预测新数据集的标签时会发生什么.说,

df = pd.DataFrame({'A':[25,67,24,76,23],'B':[2,54,22,75,19]})

因为当我对列进行标准化时A,B将根据新数据更改值的值,而不是根据模型将要训练的数据.那么,现在我的数据准备步骤之后的数据如下所示.

data[['A','B']] = min_max_scaler.fit_transform(data[['A','B']])

价值A和B将相对于Max和的Min价值而变化df[['A','B']].的准备数据df[['A','B']]是相对于Min Max的df[['A','B']].

关于不同的数字,数据准备如何有效？我不明白这里的预测是否正确.

Answer 1

您应该MinMaxScaler使用training数据,然后testing在预测之前对数据应用缩放器.

综上所述:

• 步骤1:适合scaler于TRAINING data
• 第2步:使用scalertotransform the training data
• 第3步:使用transformed training datatofit the predictive model
• 第4步:使用scalertotransform the TEST data
• 第五步:predict使用trained model和transformed TEST data

使用您的数据的示例:

from sklearn import preprocessing
min_max_scaler = preprocessing.MinMaxScaler()
#training data
df = pd.DataFrame({'A':[1,2,3,7,9,15,16,1,5,6,2,4,8,9],'B':[15,12,10,11,8,14,17,20,4,12,4,5,17,19],'C':['Y','Y','Y','Y','N','N','N','Y','N','Y','N','N','Y','Y']})
#fit and transform the training data and use them for the model training
df[['A','B']] = min_max_scaler.fit_transform(df[['A','B']])
df['C'] = df['C'].apply(lambda x: 0 if x.strip()=='N' else 1)

#fit the model
model.fit(df['A','B'])

#after the model training on the transformed training data define the testing data df_test
df_test = pd.DataFrame({'A':[25,67,24,76,23],'B':[2,54,22,75,19]})

#before the prediction of the test data, ONLY APPLY the scaler on them
df_test[['A','B']] = min_max_scaler.transform(df_test[['A','B']])

#test the model
y_predicted_from_model = model.predict(df_test['A','B'])

使用虹膜数据的示例:

import matplotlib.pyplot as plt
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from sklearn.svm import SVC

data = datasets.load_iris()
X = data.data
y = data.target

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

scaler = MinMaxScaler()
X_train_scaled = scaler.fit_transform(X_train)

model = SVC()
model.fit(X_train_scaled, y_train)

X_test_scaled = scaler.transform(X_test)
y_pred = model.predict(X_test_scaled)

希望这可以帮助.