"火与忘记"python async/await

Mik*_*e N 86 python python-asyncio python-3.5

有时需要发生一些非关键的异步操作,但我不想等待它完成.在Tornado的协程实现中,您可以通过简单地省略yield关键字来"触发并忘记"异步功能.

我一直试图弄清楚如何使用Python 3.5中发布的新async/ await语法来"解雇" .例如,简化的代码段:

async def async_foo():
    print("Do some stuff asynchronously here...")

def bar():
    async_foo()  # fire and forget "async_foo()"

bar()
Run Code Online (Sandbox Code Playgroud)

但是会发生什么,bar()从不执行,而是我们得到运行时警告:

RuntimeWarning: coroutine 'async_foo' was never awaited
  async_foo()  # fire and forget "async_foo()"
Run Code Online (Sandbox Code Playgroud)

Mik*_*mov 138

UPD:

如果你正在使用Python> = 3.7那就替换asyncio.ensure_futureasyncio.create_task任何地方它是更新,更好的方法来产生任务.


asyncio.Task"发射并忘记"

根据python文档,asyncio.Task可以启动一些协程来执行"在后台".asyncio.ensure_future 函数创建的任务不会阻止执行(因此函数将立即返回!).这看起来像你要求的"发射并忘记"的方式.

import asyncio


async def async_foo():
    print("async_foo started")
    await asyncio.sleep(1)
    print("async_foo done")


async def main():
    asyncio.ensure_future(async_foo())  # fire and forget async_foo()

    # btw, you can also create tasks inside non-async funcs

    print('Do some actions 1')
    await asyncio.sleep(1)
    print('Do some actions 2')
    await asyncio.sleep(1)
    print('Do some actions 3')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
Run Code Online (Sandbox Code Playgroud)

输出:

Do some actions 1
async_foo started
Do some actions 2
async_foo done
Do some actions 3
Run Code Online (Sandbox Code Playgroud)

如果在事件循环完成后执行任务怎么办?

请注意,asyncio期望任务将在事件循环完成时完成.所以如果你改成main():

async def main():
    asyncio.ensure_future(async_foo())  # fire and forget

    print('Do some actions 1')
    await asyncio.sleep(0.1)
    print('Do some actions 2')
Run Code Online (Sandbox Code Playgroud)

程序结束后你会收到这个警告:

Task was destroyed but it is pending!
task: <Task pending coro=<async_foo() running at [...]
Run Code Online (Sandbox Code Playgroud)

为了防止您在事件循环完成后等待所有挂起的任务:

async def main():
    asyncio.ensure_future(async_foo())  # fire and forget

    print('Do some actions 1')
    await asyncio.sleep(0.1)
    print('Do some actions 2')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

    # Let's also finish all running tasks:
    pending = asyncio.Task.all_tasks()
    loop.run_until_complete(asyncio.gather(*pending))
Run Code Online (Sandbox Code Playgroud)

杀死任务而不是等待它们

有时您不希望等待完成任务(例如,可能会创建一些任务以永久运行).在这种情况下,你可以取消()而不是等待它们:

import asyncio
from contextlib import suppress


async def echo_forever():
    while True:
        print("echo")
        await asyncio.sleep(1)


async def main():
    asyncio.ensure_future(echo_forever())  # fire and forget

    print('Do some actions 1')
    await asyncio.sleep(1)
    print('Do some actions 2')
    await asyncio.sleep(1)
    print('Do some actions 3')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

    # Let's also cancel all running tasks:
    pending = asyncio.Task.all_tasks()
    for task in pending:
        task.cancel()
        # Now we should await task to execute it's cancellation.
        # Cancelled task raises asyncio.CancelledError that we can suppress:
        with suppress(asyncio.CancelledError):
            loop.run_until_complete(task)
Run Code Online (Sandbox Code Playgroud)

输出:

Do some actions 1
echo
Do some actions 2
echo
Do some actions 3
echo
Run Code Online (Sandbox Code Playgroud)

  • 请注意,“Task.all_tasks() 自 Python 3.7 起已弃用,请改用 asyncio.all_tasks()” (6认同)
  • @GilAllen此语法仅适用于Python 3.5+.Python 3.4需要旧语法(请参阅https://docs.python.org/3.4/library/asyncio-task.html).Python 3.3及更低版本根本不支持asyncio. (3认同)
  • 很好的答案。只是想指出,大多数时候不希望等待所有挂起的任务完成或取消所有挂起的任务。我认为需要有一种中间方法,理想情况下收集在“注册表”(列表、设置或其他)中关闭事件循环后要继续执行的任务,然后只需等待这些任务完成并取消所有任务其他的。 (2认同)

Ser*_*aev 7

这不是完全异步执行,但也许run_in_executor()适合您.

def fire_and_forget(task, *args, **kwargs):
    loop = asyncio.get_event_loop()
    if callable(task):
        return loop.run_in_executor(None, task, *args, **kwargs)
    else:    
        raise TypeError('Task must be a callable')

def foo():
    #asynchronous stuff here


fire_and_forget(foo)
Run Code Online (Sandbox Code Playgroud)

  • 好简洁的答案。值得注意的是,`executor` 将默认调用`concurrent.futures.ThreadPoolExecutor.submit()`。我提到是因为创建线程不是免费的;每秒 1000 次即发即忘可能会给线程管理带来很大压力 (4认同)

neh*_*iah 6

谢谢谢尔盖的简洁回答。这是相同的装饰版。

import asyncio
import time

def fire_and_forget(f):
    def wrapped(*args, **kwargs):
        return asyncio.get_event_loop().run_in_executor(None, f, *args, *kwargs)

    return wrapped

@fire_and_forget
def foo():
    time.sleep(1)
    print("foo() completed")

print("Hello")
foo()
print("I didn't wait for foo()")
Run Code Online (Sandbox Code Playgroud)

产生

>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed
Run Code Online (Sandbox Code Playgroud)

  • 在使用这种方法每秒创建约 5 个小型“即发即忘”任务后,我经历了大幅减速。不要在生产中使用它来执行长时间运行的任务。它会吃掉你的CPU和内存! (3认同)

neh*_*iah 6

出于某种原因,如果您无法使用,asyncio那么这里是使用普通线程的实现。检查我的其他答案和谢尔盖的答案。

import threading, time

def fire_and_forget(f):
    def wrapped():
        threading.Thread(target=f).start()

    return wrapped

@fire_and_forget
def foo():
    print("foo() started")
    time.sleep(1)
    print("foo() completed")

print("Hello")
foo()
print("I didn't wait for foo()")
Run Code Online (Sandbox Code Playgroud)

产生

>>> Hello
>>> foo() started
>>> I didn't wait for foo()
>>> foo() completed
Run Code Online (Sandbox Code Playgroud)

  • 如果我们只需要 fire_and_forget 功能而不需要 asyncio 的其他功能,那么使用 asyncio 是否会更好?有什么好处? (2认同)