Numpy中零的性能

Question

我只是注意到这个zeros函数numpy有一个奇怪的行为:

%timeit np.zeros((1000, 1000))
1.06 ms ± 29.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit np.zeros((5000, 5000))
4 µs ± 66 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

另一方面,ones似乎有一个正常的行为.有人知道为什么用这个zeros函数初始化一个小的numpy数组需要比一个大数组更多的时间？

(Python 3.5,numpy 1.11)

Answer 1

这看起来像是calloc达到一个阈值,它会对操作系统请求内存归零,而不需要手动初始化它.查看源代码,numpy.zeros最终委托calloc获取归零内存块,如果比较numpy.empty,则不执行初始化:

In [15]: %timeit np.zeros((5000, 5000))
The slowest run took 12.65 times longer than the fastest. This could mean that a
n intermediate result is being cached.
100000 loops, best of 3: 10 µs per loop

In [16]: %timeit np.empty((5000, 5000))
The slowest run took 5.05 times longer than the fastest. This could mean that an
 intermediate result is being cached.
100000 loops, best of 3: 10.3 µs per loop

你可以看到np.zeros5000x5000阵列没有初始化开销.

实际上,在您尝试访问它之前,操作系统甚至没有"真正"分配该内存.数TB的数组请求在没有太字节的机器上成功:

In [23]: x = np.zeros(2**40)  # No MemoryError!