使用 python 将列文本数据转换为特征以用于机器学习

Question

左侧 CSV 文件有五列，.application列有几种应用程序类型，以;. 根据app,device和district类型，我想预测target. 但我想首先将文件转换为右侧数据框以应用机器学习。

我怎样才能使用 python 做到这一点？

Answer 1

您需要对application列应用多热编码，对其他列应用一种热编码。

这是我的解决方案！

>>> import pandas as pd
>>> import numpy as np

>>> df = pd.DataFrame({'number': np.random.randint(0,10,size=5),
                  'device': np.random.choice(['a','b'],size=5),
                  'application': ['app2;app3','app1','app2;app4', 'app1;app2', 'app1'],
                  'district': np.random.choice(['aa', 'bb', 'cc'],size=5)})

>>> df

    application device  district    number
0   app2;app3   b         aa    3
1   app1        a         cc    7
2   app2;app4   a         aa    3
3   app1;app2   b         bb    9
4   app1        a         cc    4

from sklearn.preprocessing import OneHotEncoder, MultiLabelBinarizer

mlb = MultiLabelBinarizer()
# Assuming appl names are separated by ;
mhv = mlb.fit_transform(df['application'].apply(lambda x: set(x.split(';'))))
df_out = pd.DataFrame(mhv,columns=mlb.classes_)

enc = OneHotEncoder(sparse=False)
ohe_vars = ['device','district'] # specify the list of columns here
ohv = enc.fit_transform(df.loc[:,ohe_vars])
ohe_col_names = ['%s_%s'%(var,cat) for var,cats in zip(ohe_vars, enc.categories_) for cat in cats]

df_out.assign(**dict(zip(ohe_col_names,ohv.T)))

df_out