dask的非确定性结果

Question

我用dask获得了一些矩阵计算的非确定性结果.我把它缩小到这个简单的例子:

import numpy as np
import dask.array as da
seed = 1234
np.random.seed(seed)

N = 1000
p = 10
X = np.random.random((N, p + 1))
X = da.from_array(X, chunks=(N / 4, p + 1))
beta = np.random.random(p+1)
y = X.dot(beta)
test = X.T.dot(y)
for i in range(5):
    print(test.compute()[0])

对于N = 1000,这是我得到的:

1468.52247693
1468.52247693
1468.52247693
1468.52247693
1468.52247693

但是如果我提高N,例如N = 100000,则运行时的值不一样!

132623.076746
107791.947661
108065.532822
108228.788587
108065.532822

知道发生了什么事吗？

Answer 1

从dask版本开始2021.11.2，该错误不再重现。运行不同的 N 值会得到相同的结果。

# N = 1000
1468.522476925747
1468.522476925747
1468.522476925747
1468.522476925747
1468.522476925747

# N = 100000
108065.53282187709
108065.53282187709
108065.53282187709
108065.53282187709
108065.53282187709

# N = 10000000
15246946.969951011
15246946.969951011
15246946.969951011
15246946.969951011
15246946.969951011