相关疑难解决方法(0)

并行python迭代

我想基于a中的值创建一个类的实例pandas.DataFrame.我已经失败了.

import itertools
import multiprocessing as mp
import pandas as pd

class Toy:
    id_iter = itertools.count(1)

    def __init__(self, row):
        self.id = self.id_iter.next()
        self.type = row['type']

if __name__ == "__main__":

    table = pd.DataFrame({
        'type': ['a', 'b', 'c'],
        'number': [5000, 4000, 30000]
        })

    for index, row in table.iterrows():
        [Toy(row) for _ in range(row['number'])]
Run Code Online (Sandbox Code Playgroud)

多处理尝试

我已经能够通过添加以下内容来并行化这种(某种程度):

pool = mp.Pool(processes=mp.cpu_count())
m = mp.Manager()
q = m.Queue()

for index, row in table.iterrows():
    pool.apply_async([Toy(row) for _ in range(row['number'])])
Run Code Online (Sandbox Code Playgroud)

如果数字row['number']显着长于长度,这似乎会更快table.但在我的实际情况中,table …

python pandas python-multiprocessing

9
推荐指数
1
解决办法
1858
查看次数

标签 统计

pandas ×1

python ×1

python-multiprocessing ×1