并行化R脚本

Question

我的R脚本的问题是它花了太多时间,我考虑的主要解决方案是并行化它.我不知道从哪里开始.

我的代码看起来像这样:

n<- nrow (aa) 
output <- matrix (0, n, n)

akl<- function (dii){
        ddi<- as.matrix (dii)
        m<- rowMeans(ddi)
        M<- mean(ddi)
        r<- sweep (ddi, 1, m)
        b<- sweep (r, 2, m)
        return (b + M)  
        }
for (i in 1:n)
{
A<- akl(dist(aa[i,]))

dVarX <- sqrt(mean (A * A))

for (j in i:n)
{
    B<- akl(dist(aa[j,]))
        V <- sqrt (dVarX * (sqrt(mean(B * B))))

        output[i,j] <- (sqrt(mean(A * B))) / V        
}
}   

我想在不同的cpu上并行化.我怎样才能做到这一点？我看到了SNOW包,它适合我的目的吗？感谢您的建议,Gab

Answer 1

有两种方法可以使您的代码运行得更快,我能想到:

First:正如@Dwin所说(有一点点扭曲),你可以precomputeakl(是的,不是必要的dist,而是整个akl).

# a random square matrix
aa <- matrix(runif(100), ncol=10)
n <- nrow(aa)
output <- matrix (0, n, n)

akl <- function(dii) {
    ddi <- as.matrix(dii)
    m   <- rowMeans(ddi)
    M   <- mean(m) # mean(ddi) == mean(m)
    r   <- sweep(ddi, 1, m)
    b   <- sweep(r, 2, m)
    return(b + M)
}

# precompute akl here
require(plyr)
akl.list <- llply(1:nrow(aa), function(i) {
    akl(dist(aa[i, ]))
})

# Now, apply your function, but index the list instead of computing everytime
for (i in 1:n) {
    A     <- akl.list[[i]]
    dVarX <- sqrt(mean(A * A))

    for (j in i:n) {
        B <- akl.list[[j]]
        V <- sqrt (dVarX * (sqrt(mean(B * B))))
        output[i,j] <- (sqrt(mean(A * B))) / V        
    }
}

这应该已经让你的代码比以前运行得更快(因为你在内部循环中每次都计算akl)在更大的矩阵上.

Second: 除此之外,您可以通过以下方式并行化来加快速度:

# now, the parallelisation you require can be achieved as follows
# with the help of `plyr` and `doMC`.

# First step of parallelisation is to compute akl in parallel
require(plyr)
require(doMC)
registerDoMC(10) # 10 Cores/CPUs
    akl.list <- llply(1:nrow(aa), function(i) {
    akl(dist(aa[i, ]))
}, .parallel = TRUE)

# then, you could write your for-loop using plyr again as follows
output <- laply(1:n, function(i) {
    A     <- akl.list[[i]]
    dVarX <- sqrt(mean(A * A))

    t <- laply(i:n, function(j) {
        B <- akl.list[[j]]
        V <- sqrt(dVarX * (sqrt(mean(B*B))))
        sqrt(mean(A * B))/V
    })
    c(rep(0, n-length(t)), t)
}, .parallel = TRUE)

请注意,我只添加.parallel = TRUE了外部循环.这是因为,您将10个处理器分配给外部循环.现在,如果将它添加到外部和内部循环中,则处理器的总数将为10*10 = 100.请注意这一点.