在测试中,有没有办法防止来自未定义行为的“正确”结果?

Enr*_*lis 7 c++ unit-testing googletest undefined-behavior

前言

我知道 UB 是什么,所以我不是在问如何避免它,但是是否有办法使单元测试对它更具抵抗力,即使它是一种概率方法,也只会使 UB 更有可能变得明显而不是默默无闻成功通过测试。

问题

假设我想为一个函数编写一个测试,但我做错了,就像这样:

#include <gtest/gtest.h>
#include <vector>

int main()
{
    std::vector<int> v{0};
    for (auto i = 0; i != 100; ++i) {
      v.push_back(3);     // push a 3
      v.pop_back();       // ops, popping the value I just pushed
      EXPECT_EQ(v[1], 3); // UB
    }
}
Run Code Online (Sandbox Code Playgroud)

在我的机器上,它一直通过;也许程序是如此简单,以至于没有理由真正将 3 从它之前所在的内存区域中抹去pop_back

因此,该测试显然是不可靠的。

有没有办法防止这种意外成功的测试,即使是在统计上(“在EXPECT_EQ你减少 UB 刺痛你的机会之前调用这样的函数”)?


上面的代码只是一个例子(我不愿意测试STL);我知道std::vector<T>::at作为 bound-safe std::vector<T>::operator[],但这是首先防止未定义行为的一种方法,而我正在徘徊如何防御它。

例如,通过*(&v[0] + 1) = 10;在 之后添加 来利用 UB 本身v.pop_back();将使测试的不正确性变得明显,至少在我的机器上是这样。

所以我有点在考虑一个工具/库/任何东西,比如说,v在每个可执行行之后将不保留的内存设置为随机值。

Mar*_*ink 8

Clang with Adress Sanitizer ( https://clang.llvm.org/docs/AddressSanitizer.html ) 捕获此错误:

$ clang++ -Wall -std=c++11 -o test test.cpp
$ ./test # program runs without errors

$ clang++ -fsanitize=address -Wall -std=c++11 -o test test.cpp
$ ./test
=================================================================
==94146==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000000f4 at pc 0x00010ebcbf54 bp 0x7ffee10362d0 sp 0x7ffee10362c8
READ of size 4 at 0x6020000000f4 thread T0
    #0 0x10ebcbf53 in main+0x393 (test:x86_64+0x100002f53)
    #1 0x7fff204c3f3c in start+0x0 (libdyld.dylib:x86_64+0x15f3c)

0x6020000000f4 is located 4 bytes inside of 8-byte region [0x6020000000f0,0x6020000000f8)
allocated by thread T0 here:
    #0 0x10ec38c9d in wrap__Znwm+0x7d (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x54c9d)
    #1 0x10ebcdb38 in std::__1::__libcpp_allocate(unsigned long, unsigned long)+0x18 (test:x86_64+0x100004b38)
    #2 0x10ebcdaa9 in std::__1::allocator<int>::allocate(unsigned long)+0x49 (test:x86_64+0x100004aa9)
    #3 0x10ebcd4cc in std::__1::allocator_traits<std::__1::allocator<int> >::allocate(std::__1::allocator<int>&, unsigned long)+0x1c (test:x86_64+0x1000044cc)
    #4 0x10ebcfbc0 in std::__1::__split_buffer<int, std::__1::allocator<int>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<int>&)+0x180 (test:x86_64+0x100006bc0)
    #5 0x10ebcf68c in std::__1::__split_buffer<int, std::__1::allocator<int>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<int>&)+0x2c (test:x86_64+0x10000668c)
    #6 0x10ebceec4 in void std::__1::vector<int, std::__1::allocator<int> >::__push_back_slow_path<int>(int&&)+0x154 (test:x86_64+0x100005ec4)
    #7 0x10ebcc480 in std::__1::vector<int, std::__1::allocator<int> >::push_back(int&&)+0xd0 (test:x86_64+0x100003480)
    #8 0x10ebcbedd in main+0x31d (test:x86_64+0x100002edd)
    #9 0x7fff204c3f3c in start+0x0 (libdyld.dylib:x86_64+0x15f3c)

SUMMARY: AddressSanitizer: heap-buffer-overflow (test:x86_64+0x100002f53) in main+0x393
Shadow bytes around the buggy address:
  0x1c03ffffffc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c03ffffffd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c03ffffffe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c03fffffff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c0400000000: fa fa fd fd fa fa 00 00 fa fa 00 06 fa fa 00 fa
=>0x1c0400000010: fa fa 00 00 fa fa 00 06 fa fa fd fa fa fa[04]fa
  0x1c0400000020: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0400000030: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0400000040: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0400000050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0400000060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==94146==ABORTING
[1]    94146 abort      ./test
 ? ? ? /tmp ?
Run Code Online (Sandbox Code Playgroud)

  • 考虑到问题的范围,值得指出的是,clang 还有一个更通用的未定义行为消毒剂:https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html (2认同)