为什么pytest-xdist让我的测试运行得更慢,而不是更快？

Question

我正在从鼻子到pytest移植一个~2000方法测试套件,因为django-nose不能很好地支持并行化.交换出去pytest鼻子似乎工作得很好,并添加后python_files,以pytest.ini它发现几乎所有的测试.

最大的缺点是,当我运行时-n 4,测试套件变得比没有-n标志的情况慢.在整个套件的~10%子集上运行,看起来大约有20-30%的减速,尽管我采取的时间相当嘈杂.这有点像开销,但无论我选择多少个进程,时间都不会下降.

运行--durations=20显示每个设置阶段每个进程需要几秒钟,而另一个测试稍微慢一点.

随着-vvv,因为它们可以运行,列出测试,输出的是几乎完全连载:

api/tests/VERSION_NUMBER.py::ATestCase::test_forbidden_methods <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::ATestCase::test_access_token <- api/testcases.py
api/tests/VERSION_NUMBER.py::ATestCase::test_create <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::ATestCase::test_create <- api/testcases.py
api/tests/VERSION_NUMBER.py::ATestCase::test_delete <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::ATestCase::test_delete <- api/testcases.py
api/tests/VERSION_NUMBER.py::ATestCase::test_patch <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::ATestCase::test_patch <- api/testcases.py
api/tests/VERSION_NUMBER.py::ATestCase::test_put <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::ATestCase::test_put <- api/testcases.py
api/tests/VERSION_NUMBER.py::ATestCase::test_retrieve <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::ATestCase::test_retrieve <- api/testcases.py
api/tests/VERSION_NUMBER.py::BTestCase::test_access_token <- api/testcases.py
[gw0] PASSED api/tests/VERSION_NUMBER.py::ATestCase::test_forbidden_methods <- api/testcases.py
api/tests/VERSION_NUMBER.py::ATestCase::test_list <- api/testcases.py
[gw0] PASSED api/tests/VERSION_NUMBER.py::ATestCase::test_list <- api/testcases.py
api/tests/VERSION_NUMBER.py::BTestCase::test_delete <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::BTestCase::test_access_token <- api/testcases.py
api/tests/VERSION_NUMBER.py::BTestCase::test_create <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::BTestCase::test_create <- api/testcases.py
api/tests/VERSION_NUMBER.py::BTestCase::test_list <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::BTestCase::test_list <- api/testcases.py
api/tests/VERSION_NUMBER.py::BTestCase::test_patch <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::BTestCase::test_patch <- api/testcases.py
api/tests/VERSION_NUMBER.py::BTestCase::test_put <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::BTestCase::test_put <- api/testcases.py
[gw0] PASSED api/tests/VERSION_NUMBER.py::BTestCase::test_delete <- api/testcases.py
api/tests/VERSION_NUMBER.py::BTestCase::test_forbidden_methods <- api/testcases.py
api/tests/VERSION_NUMBER.py::BTestCase::test_retrieve <- api/testcases.py
[gw0] PASSED api/tests/VERSION_NUMBER.py::BTestCase::test_forbidden_methods <- api/testcases.py
[gw1] PASSED api/tests/VERSION_NUMBER.py::BTestCase::test_retrieve <- api/testcases.py

除了少数例外,它几乎总是"开始测试,从工人那里获得通过"以获得整个日志.这让我相信某些东西正在序列化测试,但我很困惑.

我已经尝试禁用除pytest本身,pytest-xdist和pytest-django之外的所有pytest插件,没有任何变化.

Answer 1

阅读https://github.com/pytest-dev/pytest-xdist/blob/master/OVERVIEW.md你会猜到为什么在特定情况下它会慢得多。

并行化时可能会更慢：

• 总测试持续时间相当慢（不到 2 分钟） - 引导 pytest 工作人员会增加额外的时间，如果这比好处更大的话......
• 您的测试已经占用了共享的有限资源，例如磁盘或网络，因此并行运行可能会使其变慢