我正在尝试使用Python将一个函数并行应用于5个交叉验证集,并multiprocessing针对不同的参数值重复执行此操作,如下所示:
import pandas as pd
import numpy as np
import multiprocessing as mp
from sklearn.model_selection import StratifiedKFold
#simulated datasets
X = pd.DataFrame(np.random.randint(2, size=(3348,868), dtype='int8'))
y = pd.Series(np.random.randint(2, size=3348, dtype='int64'))
#dummy function to apply
def _work(args):
del(args)
for C in np.arange(0.0,2.0e-3,1.0e-6):
splitter = StratifiedKFold(n_splits=5)
with mp.Pool(processes=5) as pool:
pool_results = \
pool.map(
func=_work,
iterable=((C,X.iloc[train_index],X.iloc[test_index]) for train_index, test_index in splitter.split(X, y))
)
Run Code Online (Sandbox Code Playgroud)
但是在执行过程中,出现以下错误:
Traceback (most recent call last):
File "mre.py", line 19, in <module>
with mp.Pool(processes=5) as pool:
File "/usr/lib/python3.5/multiprocessing/context.py", …Run Code Online (Sandbox Code Playgroud)