仅使用CMake / CTest运行更改或失败的测试？

Question

我使用的大型代码库具有将近400个测试可执行文件，运行时间在0.001秒至1800秒之间变化。当一些代码更改时，CMake只会智能地重建已更改的目标，而所需时间比实际测试运行时间要短许多倍。

我所知道的唯一方法是手动过滤您想运行的测试。我的直觉是，我想重新运行任何未成功存储运行的测试套件-要么是因为失败，要么是因为重新编译了它。

这可能吗？如果是这样，怎么办？

Answer 1

ctest命令接受几个参数，这些参数会影响要运行的测试集。例如，“-R”-按名称过滤测试，“-L”-按标签过滤测试。可能使用与仪表板相关的选项，您也可以选择要运行的测试。

至于根据更改的可执行文件为这些选项生成值，您可以编写程序或脚本，以检查可执行文件的修改时间和/或解析最后的日志文件以查找失败的测试。

仅运行更改的可执行文件的另一种方法是将测试包装到其他脚本中。仅当满足某些条件时，此脚本才会运行可执行文件。

对于Linux包装器脚本，可以如下实现：

test_wrapper.sh：

# test_wrapper.sh <test_name> <executable> <params..>
# Run executable, given as second argument, with parameters, given as futher arguments.
#
# If environment variable `LAST_LOG_FILE` is set,
# checks that this file is older than the executable.
#
# If environment variable LAST_LOG_FAILED_FILE is set,
# check that testname is listed in this file.
#
# Test executable is run only if one of these checks succeed, or if none of checks is performed.

check_succeed=
check_performed=
if [ -n $LAST_LOG_FILE ]; then
    check_performed=1
    executable=$2
    if [ ! ( -e "$LAST_LOG_FILE" ) ]; then
        check_succeed=1 # Log file is absent
    elif [ "$LAST_LOG_FILE" -ot "$executable" ]; then
        check_succeed=1 # Log file is older than executable
    fi
fi

if [ -n "$LAST_LOG_FAILED_FILE" ]; then
    check_performed=1
    testname=$1
    if [ ! ( -e "$LAST_LOG_FAILED_FILE" ) ]; then
        # No failed tests at all
    elif grep ":${testname}\$" "$LAST_LOG_FAILED_FILE" > /dev/null; then
        check_succeed=1 # Test has been failed previously
    fi
fi

if [ -n "$check_performed" -a -z "$check_succeed" ]; then
    echo "Needn't to run test."
    exit 0
fi

shift 1 # remove `testname` argument
eval "$*"

CMake宏用于添加包装测试：

CMakeLists.txt：

# Similar to add_test(), but test is executed with our wrapper.
function(add_wrapped_test name command)
    if(name STREQUAL "NAME")
        # Complex add_test() command flow: NAME <name> COMMAND <command> ...
        set(other_params ${ARGN})
        list(REMOVE_AT other_params 0) # COMMAND keyword
        # Actual `command` argument
        list(GET other_params 0 real_command)
        list(REMOVE_AT other_params 0)
        # If `real_command` is a target, need to translate it to path to executable.
        if(TARGET real_command)
            # Generator expression is perfectly OK here.
            set(real_command "$<TARGET_FILE:${real_command}")
        endif()
        # `command` is actually value of 'NAME' parameter
        add_test("NAME" ${command} "COMMAND" /bin/sh <...>/test_wrapper.sh
            ${command} ${real_command} ${other_params}
        )
    else() # Simple add_test() command flow
        add_test(${name} /bin/sh <...>/test_wrapper.sh
            ${name} ${command} ${ARGN}
        )
    endif()
endfunction(add_wrapped_test)

如果您只想运行那些测试，自上次运行以来更改了哪些可执行文件或上次失败了的测试，请使用

LAST_LOG_FILE=<build-dir>/Testing/Temporary/LastTest.log \
LAST_FAILED_LOG_FILE=<build-dir>/Testing/Temporary/LastTestsFailed.log \
ctest

所有其他测试将自动通过。