使用asyncio进行并行任务

Question

我想知道如何使用asyncio来处理类似于nodeJS的任务.我想在不打开线程的情况下同时运行任务.例:

import asyncio

@asyncio.coroutine
def my_coroutine(task_name, seconds_to_sleep=3):
    print('{0} sleeping for: {1} seconds'.format(task_name, seconds_to_sleep))
    yield from asyncio.sleep(seconds_to_sleep)
    print('{0} is finished'.format(task_name))


loop = asyncio.get_event_loop()
tasks = [
    my_coroutine('task1', 4),
    my_coroutine('task2', 3),
    my_coroutine('task3', 2)]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

将输出:

task1 sleeping for: 4 seconds
task2 sleeping for: 3 seconds
task3 sleeping for: 2 seconds
task3 is finished
task2 is finished
task1 is finished

但是当我尝试用不同的任务完成它时,它将无法正常工作.

import asyncio
import timeit

@asyncio.coroutine
def my_coroutine(task_name):
    print('order placed for ' + task_name)
    print(timeit.timeit('1 + 3 ', number=50000000))
    print(task_name + ' done')


loop = asyncio.get_event_loop()
tasks = [
    my_coroutine('task1'),
    my_coroutine('task2'),
    my_coroutine('task3')]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

输出

order placed for task2
0.6677237730912453
task2 done
order placed for task1
0.6627442526498016
task1 done
order placed for task3
0.665618849882418
task3 done

Answer 1

asyncio 文档如下所述，因此asyncio 任务可以并发运行，但不能并行运行。

asyncio 是一个使用 async/await 语法编写并发代码的库。

并且，自Python 3.7.14起不推荐使用@asyncio.coroutine，并自Python 3.11.0起删除 @asyncio.coroutine ，因此您应该使用如下所示的方法：async

# @asyncio.coroutine
async def test():
    print("Test")

例如，使用下面的代码：

import asyncio

async def test1():
    for _ in range(0, 3):
        print("Test1")
        
async def test2():
    for _ in range(0, 3):
        print("Test2")
        
async def test3():
    for _ in range(0, 3):
        print("Test3")

async def call_tests():
    await asyncio.gather(test1(), test2(), test3())

asyncio.run(call_tests())

test1()，test2()并按test3()如下所示串行运行：

Test1 # 0 second
Test1 # 0 second
Test1 # 0 second
Test2 # 0 second
Test2 # 0 second
Test2 # 0 second
Test3 # 0 second
Test3 # 0 second
Test3 # 0 second

并且，如果await asyncio.sleep(1)在其中使用，如下所示：

import asyncio

async def test1():
    for _ in range(0, 3):
        print("Test1")
        await asyncio.sleep(1) # Here
        
async def test2():
    for _ in range(0, 3):
        print("Test2")
        await asyncio.sleep(1) # Here

async def test3():
    for _ in range(0, 3):
        print("Test3")
        await asyncio.sleep(1) # Here

async def call_tests():
    await asyncio.gather(test1(), test2(), test3())

asyncio.run(call_tests())

它们交替运行，每次睡眠 1 秒，如下所示：

Test1 # 1 second
Test2 # 1 second
Test3 # 1 second
Test1 # 2 seconds
Test2 # 2 seconds
Test3 # 2 seconds
Test1 # 3 seconds
Test2 # 3 seconds
Test3 # 3 seconds

并且，如果await asyncio.sleep(0)在其中使用，如下所示：

import asyncio

async def test1():
    for _ in range(0, 3):
        print("Test1")
        await asyncio.sleep(0) # Here
        
async def test2():
    for _ in range(0, 3):
        print("Test2")
        await asyncio.sleep(0) # Here

async def test3():
    for _ in range(0, 3):
        print("Test3")
        await asyncio.sleep(0) # Here

async def call_tests():
    await asyncio.gather(test1(), test2(), test3())

asyncio.run(call_tests())

它们交替运行而不休眠，如下所示：

Test1 # 0 second
Test2 # 0 second
Test3 # 0 second
Test1 # 0 second
Test2 # 0 second
Test3 # 0 second
Test1 # 0 second
Test2 # 0 second
Test3 # 0 second

Answer 2

asyncio不会并行运行.它运行一个任务,直到等待,然后继续下一个任务.sleep第一个例子中的s是使任务相互控制的原因.您的第二个示例没有等待任何内容,因此每个任务都会运行直到完成,然后事件循环才能控制另一个任务.

如果你将一些等待的东西(例如asyncio.sleep)添加到你的协程中,每一个都会产生控制权并让其他人有机会跑.

@asyncio.coroutine
def my_coroutine(task_name):
    print('order placed for ' + task_name)
    yield from asyncio.sleep(0)  # Another coroutine will resume here.
    print(timeit.timeit('1 + 3 ', number=50000000))
    yield from asyncio.sleep(0)  # Another coroutine will resume here.
    print(task_name + ' done')