为什么在Julia 0.5.0中将大矩阵的索引速度减慢170倍,比0.4.7慢？

Question

对大型矩阵进行索引似乎比FAR更长,0.5和0.6比0.4.7更长.

例如:

x = rand(10,10,100,4,4,1000)   #Dummy array

tic()
r = squeeze(mean(x[:,:,1:80,:,:,56:800],(1,2,3,4,5)),(1,2,3,4,5))
toc()

朱莉娅0.5.0 - >经过时间:176.357068283秒

Julia 0.4.7 - >经过时间:1.19991952秒

---

编辑:根据要求,我已更新基准使用BenchmarkTools.jl并将代码包装在函数中:

using BenchmarkTools
function testf(x)
    r = squeeze(mean(x[:,:,1:80,:,:,56:800],(1,2,3,4,5)),(1,2,3,4,5));
end

x = rand(10,10,100,4,4,1000)   #Dummy array
@benchmark testf(x)

在0.5.0中,我得到以下内容(内存使用量很大):

BenchmarkTools.Trial: 
  samples:          1
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%
  memory estimate:  23.36 gb
  allocs estimate:  1043200022
  minimum time:     177.94 s (1.34% GC)
  median time:      177.94 s (1.34% GC)
  mean time:        177.94 s (1.34% GC)
  maximum time:     177.94 s (1.34% GC)

在0.4.7我得到:

BenchmarkTools.Trial: 
  samples:          11
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%
  memory estimate:  727.55 mb
  allocs estimate:  79
  minimum time:     425.82 ms (0.06% GC)
  median time:      485.95 ms (11.31% GC)
  mean time:        482.67 ms (10.37% GC)
  maximum time:     503.27 ms (11.22% GC)

---

编辑:已更新以sub在0.4.7和view0.5.0中使用

using BenchmarkTools
function testf(x)
    r = mean(sub(x, :, :, 1:80, :, :, 56:800));
end

x = rand(10,10,100,4,4,1000)   #Dummy array
@benchmark testf(x)

在0.5.0中它运行了> 20分钟并给出:

BenchmarkTools.Trial: 
  samples:          1
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%
  memory estimate:  53.75 gb
  allocs estimate:  2271872022
  minimum time:     407.64 s (1.32% GC)
  median time:      407.64 s (1.32% GC)
  mean time:        407.64 s (1.32% GC)
  maximum time:     407.64 s (1.32% GC)

在0.4.7我得到:

BenchmarkTools.Trial: 
  samples:          5
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%
  memory estimate:  1.28 kb
  allocs estimate:  34
  minimum time:     1.15 s (0.00% GC)
  median time:      1.16 s (0.00% GC)
  mean time:        1.16 s (0.00% GC)
  maximum time:     1.18 s (0.00% GC)

---

这在其他机器上似乎是可重复的,因此已经打开了一个问题:https://github.com/JuliaLang/julia/issues/19174

Answer 1

编辑2017年3月17日此回归在Julia v0.6.0中修复.如果使用旧版Julia,讨论仍然适用.

尝试在Julia v0.4.7和v0.5.0中运行这个粗略的脚本(更改sub为view):

using BenchmarkTools

function testf()
    # set seed
    srand(2016)

    # test array
    x = rand(10,10,100,4,4,1000)

    # extract array view
    y = sub(x, :, :, 1:80, :, :, 56:800)   # julia v0.4
    #y = view(x, :, :, 1:80, :, :, 56:800)  # julia v0.5

    # wrap mean(y) into a function
    z() = mean(y)

    # benchmark array mean
    @time z() 
    @time z() 
end

testf()

我的机器:

julia> versioninfo() 
Julia Version 0.4.7 
Commit ae26b25 (2016-09-18 16:17 UTC) 
Platform Info: 
  System: Darwin (x86_64-apple-darwin13.4.0) 
  CPU: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz 
  WORD_SIZE: 64 
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell) 
  LAPACK: libopenblas64_ 
  LIBM: libopenlibm 
  LLVM: libLLVM-3.3 

我的输出,Julia v0.4.7:

1.314966 seconds (246.43 k allocations: 11.589 MB)
1.017073 seconds (1 allocation: 16 bytes)

我的输出,Julia v0.5.0:

417.608056 seconds (2.27 G allocations: 53.749 GB, 0.75% gc time)
410.918933 seconds (2.27 G allocations: 53.747 GB, 0.72% gc time)

您可能已经发现了性能回归.考虑提交问题.