Dra*_*rro 1491 c++ floating-point performance compilation visual-studio-2010
为什么这段代码,
const float x[16] = { 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8,
1.9, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6};
const float z[16] = {1.123, 1.234, 1.345, 156.467, 1.578, 1.689, 1.790, 1.812,
1.923, 2.034, 2.145, 2.256, 2.367, 2.478, 2.589, 2.690};
float y[16];
for (int i = 0; i < 16; i++)
{
y[i] = x[i];
}
for (int j = 0; j < 9000000; j++)
{
for (int i = 0; i < 16; i++)
{
y[i] *= x[i];
y[i] /= z[i];
y[i] = y[i] + 0.1f; // <--
y[i] = y[i] - 0.1f; // <--
}
}
运行速度比后续运行速度快10倍以上(除非另有说明,否则相同)？
const float x[16] = { 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8,
1.9, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6};
const float z[16] = {1.123, 1.234, 1.345, 156.467, 1.578, 1.689, 1.790, 1.812,
1.923, 2.034, 2.145, 2.256, 2.367, 2.478, 2.589, 2.690};
float y[16];
for (int i = 0; i < 16; i++)
{
y[i] = x[i];
}
for (int j = 0; j < 9000000; j++)
{
for (int i = 0; i < 16; i++)
{
y[i] *= x[i];
y[i] /= z[i];
y[i] = y[i] + 0; // <--
y[i] = y[i] - 0; // <--
}
}
使用Visual Studio 2010 SP1进行编译时.(我没有和其他编译器一起测试过.)
Mys*_*ial 1583
欢迎来到非规范化浮点世界!他们可以对性能造成严重破坏!
非正规(或次正规)数字是一种破解,可以从浮点表示中获得非常接近零的一些额外值.非规范化浮点运算比标准化浮点运算慢几十到几百倍.这是因为许多处理器无法直接处理它们,必须使用微码捕获并解决它们.
如果你以后10,000次重复打印出来的数字,你会看到,他们已经收敛到不同的值取决于是否0或0.1使用.
这是在x64上编译的测试代码:
int main() {
double start = omp_get_wtime();
const float x[16]={1.1,1.2,1.3,1.4,1.5,1.6,1.7,1.8,1.9,2.0,2.1,2.2,2.3,2.4,2.5,2.6};
const float z[16]={1.123,1.234,1.345,156.467,1.578,1.689,1.790,1.812,1.923,2.034,2.145,2.256,2.367,2.478,2.589,2.690};
float y[16];
for(int i=0;i<16;i++)
{
y[i]=x[i];
}
for(int j=0;j<9000000;j++)
{
for(int i=0;i<16;i++)
{
y[i]*=x[i];
y[i]/=z[i];
#ifdef FLOATING
y[i]=y[i]+0.1f;
y[i]=y[i]-0.1f;
#else
y[i]=y[i]+0;
y[i]=y[i]-0;
#endif
if (j > 10000)
cout << y[i] << " ";
}
if (j > 10000)
cout << endl;
}
double end = omp_get_wtime();
cout << end - start << endl;
system("pause");
return 0;
}
输出:
#define FLOATING
1.78814e-007 1.3411e-007 1.04308e-007 0 7.45058e-008 6.70552e-008 6.70552e-008 5.58794e-007 3.05474e-007 2.16067e-007 1.71363e-007 1.49012e-007 1.2666e-007 1.11759e-007 1.04308e-007 1.04308e-007
1.78814e-007 1.3411e-007 1.04308e-007 0 7.45058e-008 6.70552e-008 6.70552e-008 5.58794e-007 3.05474e-007 2.16067e-007 1.71363e-007 1.49012e-007 1.2666e-007 1.11759e-007 1.04308e-007 1.04308e-007
//#define FLOATING
6.30584e-044 3.92364e-044 3.08286e-044 0 1.82169e-044 1.54143e-044 2.10195e-044 2.46842e-029 7.56701e-044 4.06377e-044 3.92364e-044 3.22299e-044 3.08286e-044 2.66247e-044 2.66247e-044 2.24208e-044
6.30584e-044 3.92364e-044 3.08286e-044 0 1.82169e-044 1.54143e-044 2.10195e-044 2.45208e-029 7.56701e-044 4.06377e-044 3.92364e-044 3.22299e-044 3.08286e-044 2.66247e-044 2.66247e-044 2.24208e-044
请注意,在第二次运行中,数字非常接近于零.
非规范化数字通常很少,因此大多数处理器不会尝试有效地处理它们.
为了证明这与非规范化数字有关,如果我们将非正规数通过将其添加到代码的开头来将其刷新为零:
_MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);
然后版本0不再慢10倍,实际上变得更快.(这要求在启用SSE的情况下编译代码.)
这意味着我们不是使用这些奇怪的低精度几乎为零的值,而是将其舍入为零.
时间:Core i7 920 @ 3.5 GHz:
// Don't flush denormals to zero.
0.1f: 0.564067
0 : 26.7669
// Flush denormals to zero.
0.1f: 0.587117
0 : 0.341406
最后,这与它是整数还是浮点无关.的0或0.1f转换/存储到寄存器两个环路的外部.所以这对性能没有影响.
mvd*_*vds 412
使用gccdiff并将diff应用于生成的程序集只会产生这种差异:
73c68,69
< movss LCPI1_0(%rip), %xmm1
---
> movabsq $0, %rcx
> cvtsi2ssq %rcx, %xmm1
81d76
< subss %xmm1, %xmm0
那个cvtsi2ssq确实慢了10倍.
显然,该float版本使用从内存加载的XMM寄存器,而int版本将实际int值0 转换为float使用该cvtsi2ssq指令,花费了大量时间.传递-O3给gcc没有帮助.(gcc版本4.2.1.)
(使用double而float不是无关紧要,除了它改变cvtsi2ssq为cvtsi2sdq.)
更新
一些额外的测试表明它不一定是cvtsi2ssq指令.一旦消除(使用int ai=0;float a=ai;和a代替0),速度差异仍然存在.所以@Mysticial是正确的,非规范化的花车是有区别的.这可以通过测试0和之间的值来看出0.1f.0.00000000000000000000000000000001当循环突然占用10倍时,上述代码中的转折点大约为.
更新<< 1
这个有趣现象的一个小的可视化:
当非规范化设置时,您可以清楚地看到指数(最后9位)更改为其最低值.此时,简单加法变得慢20倍.
0.000000000000000000000000000000000100000004670110: 10111100001101110010000011100000 45 ms
0.000000000000000000000000000000000050000002335055: 10111100001101110010000101100000 43 ms
0.000000000000000000000000000000000025000001167528: 10111100001101110010000001100000 43 ms
0.000000000000000000000000000000000012500000583764: 10111100001101110010000110100000 42 ms
0.000000000000000000000000000000000006250000291882: 10111100001101110010000010100000 48 ms
0.000000000000000000000000000000000003125000145941: 10111100001101110010000100100000 43 ms
0.000000000000000000000000000000000001562500072970: 10111100001101110010000000100000 42 ms
0.000000000000000000000000000000000000781250036485: 10111100001101110010000111000000 42 ms
0.000000000000000000000000000000000000390625018243: 10111100001101110010000011000000 42 ms
0.000000000000000000000000000000000000195312509121: 10111100001101110010000101000000 43 ms
0.000000000000000000000000000000000000097656254561: 10111100001101110010000001000000 42 ms
0.000000000000000000000000000000000000048828127280: 10111100001101110010000110000000 44 ms
0.000000000000000000000000000000000000024414063640: 10111100001101110010000010000000 42 ms
0.000000000000000000000000000000000000012207031820: 10111100001101110010000100000000 42 ms
0.000000000000000000000000000000000000006103515209: 01111000011011100100001000000000 789 ms
0.000000000000000000000000000000000000003051757605: 11110000110111001000010000000000 788 ms
0.000000000000000000000000000000000000001525879503: 00010001101110010000100000000000 788 ms
0.000000000000000000000000000000000000000762939751: 00100011011100100001000000000000 795 ms
0.000000000000000000000000000000000000000381469876: 01000110111001000010000000000000 896 ms
0.000000000000000000000000000000000000000190734938: 10001101110010000100000000000000 813 ms
0.000000000000000000000000000000000000000095366768: 00011011100100001000000000000000 798 ms
0.000000000000000000000000000000000000000047683384: 00110111001000010000000000000000 791 ms
0.000000000000000000000000000000000000000023841692: 01101110010000100000000000000000 802 ms
0.000000000000000000000000000000000000000011920846: 11011100100001000000000000000000 809 ms
0.000000000000000000000000000000000000000005961124: 01111001000010000000000000000000 795 ms
0.000000000000000000000000000000000000000002980562: 11110010000100000000000000000000 835 ms
0.000000000000000000000000000000000000000001490982: 00010100001000000000000000000000 864 ms
0.000000000000000000000000000000000000000000745491: 00101000010000000000000000000000 915 ms
0.000000000000000000000000000000000000000000372745: 01010000100000000000000000000000 918 ms
0.000000000000000000000000000000000000000000186373: 10100001000000000000000000000000 881 ms
0.000000000000000000000000000000000000000000092486: 01000010000000000000000000000000 857 ms
0.000000000000000000000000000000000000000000046243: 10000100000000000000000000000000 861 ms
0.000000000000000000000000000000000000000000022421: 00001000000000000000000000000000 855 ms
0.000000000000000000000000000000000000000000011210: 00010000000000000000000000000000 887 ms
0.000000000000000000000000000000000000000000005605: 00100000000000000000000000000000 799 ms
0.000000000000000000000000000000000000000000002803: 01000000000000000000000000000000 828 ms
0.000000000000000000000000000000000000000000001401: 10000000000000000000000000000000 815 ms
0.000000000000000000000000000000000000000000000000: 00000000000000000000000000000000 42 ms
0.000000000000000000000000000000000000000000000000: 00000000000000000000000000000000 42 ms
0.000000000000000000000000000000000000000000000000: 00000000000000000000000000000000 44 ms
关于ARM的等价讨论可以在Objective-C中的Stack Overflow问题Denormalized浮点中找到？.
fig*_*fig 34
这是由于非规范化的浮点使用.如何摆脱它和性能损失？在互联网上寻找杀死非正常数字的方法之后,似乎还没有"最好"的方法来做到这一点.我发现这三种方法在不同的环境中效果最好:
可能无法在某些GCC环境中工作:
// Requires #include <fenv.h>
fesetenv(FE_DFL_DISABLE_SSE_DENORMS_ENV);
可能无法在某些Visual Studio环境中工作:1
// Requires #include <xmmintrin.h>
_mm_setcsr( _mm_getcsr() | (1<<15) | (1<<6) );
// Does both FTZ and DAZ bits. You can also use just hex value 0x8040 to do both.
// You might also want to use the underflow mask (1<<11)
似乎在GCC和Visual Studio中都有效:
// Requires #include <xmmintrin.h>
// Requires #include <pmmintrin.h>
_MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);
_MM_SET_DENORMALS_ZERO_MODE(_MM_DENORMALS_ZERO_ON);
英特尔编译器可以选择在现代英特尔CPU上默认禁用非正规.更多细节在这里
编译器开关.-ffast-math,-msse或者-mfpmath=sse将禁用非正规并更快地做一些其他事情,但遗憾的是还会做很多其他可能会破坏你的代码的近似值.仔细测试!相当于Visual Studio编译器的快速数学,/fp:fast但我无法确认这是否也会禁用非正规数.1
Ger*_*cia 19
在gcc中,您可以使用以下命令启用FTZ和DAZ:
#include <xmmintrin.h>
#define FTZ 1
#define DAZ 1
void enableFtzDaz()
{
int mxcsr = _mm_getcsr ();
if (FTZ) {
mxcsr |= (1<<15) | (1<<11);
}
if (DAZ) {
mxcsr |= (1<<6);
}
_mm_setcsr (mxcsr);
}
也使用gcc开关:-msse -mfpmath = sse
(Carl Hetherington的相应学分[1])
[1] http://carlh.net/plugins/denormals.php
丹·尼利的评论应扩展为一个答案：
归零0.0f化或导致速度降低的不是零常数，而是循环的每次迭代接近零的值。随着它们越来越接近于零，它们需要更高的精度来表示，并且它们变得规范化了。这些是y[i]值。（它们接近零，因为x[i]/z[i]所有值均小于1.0 i。）
慢速和快速版本之间的关键区别在于语句y[i] = y[i] + 0.1f;。在循环的每次迭代中执行此行后，浮点数中的额外精度就会丢失，并且不再需要代表该精度的非规范化。之后，y[i]由于没有进行非规范化，因此浮点运算仍然保持快速状态。
为什么添加时会失去额外的精度0.1f？因为浮点数只有很多有效数字。假设您有足够的存储空间来存储三个有效数字，然后是0.00001 = 1e-5，0.00001 + 0.1 = 0.1至少对于本示例的float格式而言，因为没有足够的空间来存储中的最低有效位0.10001。
简而言之，y[i]=y[i]+0.1f; y[i]=y[i]-0.1f;这不是您可能想的那样。
神秘主义者也这样说：浮点数的内容很重要，而不仅仅是汇编代码。