相关疑难解决方法(0)

multiprocessing.pool.MaybeEncodingError: 'TypeError("cannot serialize '_io.BufferedReader' object",)'

为什么下面的代码只适用于multiprocessing.dummy,而不适用于 simple multiprocessing

import urllib.request
#from multiprocessing.dummy import Pool #this works
from multiprocessing import Pool

urls = ['http://www.python.org', 'http://www.yahoo.com','http://www.scala.org', 'http://www.google.com']

if __name__ == '__main__':
    with Pool(5) as p:
        results = p.map(urllib.request.urlopen, urls)
Run Code Online (Sandbox Code Playgroud)

错误 :

Traceback (most recent call last):
  File "urlthreads.py", line 31, in <module>
    results = p.map(urllib.request.urlopen, urls)
  File "C:\Users\patri\Anaconda3\lib\multiprocessing\pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "C:\Users\patri\Anaconda3\lib\multiprocessing\pool.py", line 657, in get
    raise self._value
multiprocessing.pool.MaybeEncodingError: Error sending result: '[<http.client.HTTPResponse object at 0x0000016AEF204198>]'. Reason: …
Run Code Online (Sandbox Code Playgroud)

python urllib multiprocessing python-multithreading python-multiprocessing

4
推荐指数
1
解决办法
2526
查看次数