Dask 因 freeze_support 错误而失败

Question

我尝试运行一个非常简单的 Dask 程序，如下所示：

# myfile.py
from dask.distributed import Client

client = Client()

但是当我运行这个程序时，我得到了这个奇怪的错误

    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

Answer 1

当调用Client()或LocalCluster()您在程序中启动一些新进程时。当模块或脚本启动这样的进程时，Python 不喜欢它。

要解决此问题，请将您的代码if __name__ == "__main__":包含在如下所示的块中：

# client = Client()

if __name__ == '__main__':
    client = Client()
    ...

或者，您可以选择安全地使用线程而不是进程。

client = Client(processes=False)

此外，如果您在 IPython 或 Jupyter 等交互式会话中运行，则不会出现此问题