Python multiprocessing.Queue在put和get上死锁

Question

这段代码我遇到了死锁问题:

def _entropy_split_parallel(data_train, answers_train, weights):
    CPUS = 1 #multiprocessing.cpu_count()
    NUMBER_TASKS = len(data_train[0])
    processes = []

    multi_list = zip(data_train, answers_train, weights)

    task_queue = multiprocessing.Queue()
    done_queue = multiprocessing.Queue()

    for feature_index in xrange(NUMBER_TASKS):
        task_queue.put(feature_index)

    for i in xrange(CPUS):
        process = multiprocessing.Process(target=_worker, 
                args=(multi_list, task_queue, done_queue))
        processes.append(process)
        process.start()

    min_entropy = None
    best_feature = None
    best_split = None
    for i in xrange(NUMBER_TASKS):
        entropy, feature, split = done_queue.get()
        if (entropy < min_entropy or min_entropy == None) and entropy != None:
            best_feature = feature
            best_split = split

    for i in xrange(CPUS):
        task_queue.put('STOP')

    for process in processes:
        process.join()

    return best_feature, best_split


def _worker(multi_list, task_queue, done_queue):
    feature_index = task_queue.get()
    while feature_index != 'STOP':
        result = _entropy_split3(multi_list, feature_index)
        done_queue.put(result)
        feature_index = task_queue.get()

当我运行我的程序时,它可以正常运行几次_entropy_split_parallel,但最终会出现死锁.父进程正在阻塞done_queue.get(),并且工作进程正在阻塞done_queue.put().由于发生这种情况时队列始终为空,因此get预计会阻塞.我不明白的是为什么工人阻塞put,因为队列显然不是满的(它是空的!).我已经尝试了block和timeout关键字参数,但得到了相同的结果.

我正在使用多处理backport,因为我坚持使用Python 2.5.

---

编辑:看起来我也遇到了多处理模块提供的一个示例的死锁问题.这是底部的第三个例子.如果我多次调用测试方法,似乎只会发生死锁.例如,将脚本的底部更改为:

if __name__ == '__main__':
    freeze_support()
    for x in xrange(1000):
        test()

---

编辑:我知道这是一个老问题,但测试表明,在使用Python 2.7的Windows上,这不再是一个问题.我会尝试Linux并报告回来.

Answer 1

这个问题在较新版本的 Python 中消失了，所以我假设这是向后移植的问题。无论如何，这已经不再是问题了。