是否可以在 pytest 中并行执行相同的测试？

Question

我知道您可以参数化测试以使用不同的参数集重复测试。我也知道测试文件中的不同测试可以使用 并行运行-n，但我想并行执行同一组测试。在pytest中可能吗？

例如：

import pytest

@pytest.fixture()
def user_number(worker_id):
    return "user number : %s" %worker_id

def test_add(user_number):
    print("Adding 1+1 and returning the result and user number: {}".format(user_number))
    return 1+1

def test_subtract():
    print("subtracting 2-1 and returning the result and user number: {}".format(user_number))
    return 2-1

如果我运行以下命令：py.test -n 3 -s -v parallel_users.py

在结果中，test_add() 和 test_subtract() 并行运行，如下所示：

[gw1] PASSED parallel_users.py::test_subtract 
[gw0] PASSED parallel_users.py::test_add

如何让 test_add() 和 test_subtract() 运行两次，如下所示：

[gw1] PASSED parallel_users.py::test_add, test_subtract 
[gw0] PASSED parallel_users.py::test_add, test_subtract 

Answer 1

更新：

我想最接近 OP 正在寻找的是使用eachdistscope。使用它将执行测试选择n时间，每个进程一次：

$ pytest -n3 -v --dist=each
...
gw0 [2] / gw1 [2] / gw2 [2]
scheduling tests via EachScheduling

test_main.py::test_add 
[gw1] [ 50%] PASSED test_main.py::test_add 
[gw0] [ 50%] PASSED test_main.py::test_add 
test_main.py::test_subtract 
[gw2] [ 50%] PASSED test_main.py::test_add 
test_main.py::test_subtract 
[gw2] [100%] PASSED test_main.py::test_subtract 
[gw0] [100%] PASSED test_main.py::test_subtract 
[gw1] [100%] PASSED test_main.py::test_subtract 

双方test_add并test_subtract分别在每个工人执行一次gw0，gw1并且gw2，总结三个执行每个测试。

旧答案

要重复执行测试，请在您的conftest.py:

def pytest_collection_modifyitems(items):
    numrepeats = 2
    items.extend(items * (numrepeats - 1))

这将复制为执行numrepeats时间收集的每个测试。示例运行：

$ pytest -n3 -v --dist=each
...
gw0 [2] / gw1 [2] / gw2 [2]
scheduling tests via EachScheduling

test_main.py::test_add 
[gw1] [ 50%] PASSED test_main.py::test_add 
[gw0] [ 50%] PASSED test_main.py::test_add 
test_main.py::test_subtract 
[gw2] [ 50%] PASSED test_main.py::test_add 
test_main.py::test_subtract 
[gw2] [100%] PASSED test_main.py::test_subtract 
[gw0] [100%] PASSED test_main.py::test_subtract 
[gw1] [100%] PASSED test_main.py::test_subtract 

如果要使其可配置，请添加自定义 cli 参数：

import pytest

def pytest_addoption(parser):
    parser.addoption('--numrepeats', action='store', type=int, default=1)

def pytest_collection_modifyitems(items):
    numrepeats = pytest.config.getoption('--numrepeats')
    items.extend(items * (numrepeats - 1))

现在，你可以调用你的测试--numrepeats，例如pytest --numrepeats 5。

至于每个进程的批处理测试（问题的第二部分），pytest-xdist尚不支持，请参阅此问题以及与之相关的所有内容。最近，添加了一些基本支持，例如在单独进程中的单个模块或类中执行测试：

--dist=distmode       set mode for distributing tests to exec environments.
                      each: send each test to all available environments.
                      load: load balance by sending any pending test to any
                      available environment. loadscope: load balance by
                      sending pending groups of tests in the same scope to
                      any available environment. loadfile: load balance by
                      sending test grouped by file to any available
                      environment. (default) no: run tests inprocess, don't
                      distribute.

但是，如果您想根据某些自定义条件对测试进行负载平衡，除了编写自己的调度程序实现之外别无他法。

Answer 2

实现此目的的一种最简单的方法是欺骗 py.test 假设存在使用的测试的多种组合pytest.mark.parametrization。这里我只是使用虚拟参数count来欺骗 py.test 测试有多种变体。这是最简单的方法，如果您有更多的测试条件，您可以根据pytest_generate_tests(metafunc):您的需求进行微调。

import pytest

@pytest.fixture()
def user_number(worker_id):
    return "user number : %s" %worker_id

@pytest.mark.parametrize("count", [1, 2,3])
def test_add(user_number, count):
    print("Adding 1+1 and returning the result and user number: {}".format(user_number))
    return 1+1

@pytest.mark.parametrize("count", [1, 2 ,3])
def test_subtract(count):
    print("subtracting 2-1 and returning the result and user number: {}".format(user_number))
    return 2-1

$ py.test test.py -n 2 -s -vv
======================================== test session starts ========================================
platform darwin -- Python 2.7.13, pytest-2.9.2, py-1.5.3, pluggy-0.3.1 -- bin/python2.7
cachedir: .cache
rootdir: , inifile:
plugins: xdist-1.17.1, repeat-0.4.1, cov-1.8.1
[gw0] darwin Python 2.7.13 cwd: 
[gw1] darwin Python 2.7.13 cwd: 
[gw1] Python 2.7.13 (default, Dec 17 2016, 23:03:43)  -- [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
[gw0] Python 2.7.13 (default, Dec 17 2016, 23:03:43)  -- [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
gw0 [6] / gw1 [6]
scheduling tests via LoadScheduling

test.py::test_add[1]
test.py::test_add[2]
[gw1] PASSED test.py::test_add[2]
[gw0] PASSED test.py::test_add[1]
test.py::test_subtract[1]
[gw1] PASSED test.py::test_subtract[1]
test.py::test_add[3]
[gw0] PASSED test.py::test_add[3]
test.py::test_subtract[3]
test.py::test_subtract[2]
[gw0] PASSED test.py::test_subtract[3]
[gw1] PASSED test.py::test_subtract[2]