R:Render_site不与Dopar合作

Gee*_*eek 7 parallel-processing yaml r r-markdown doparallel

我使用foreach的dopar来解决Render_site的问题.使用简单的代码完美地运行代码.虽然我在使用dopar时遇到问题,因为在YAML文件中使用dopar之后输出会重复.

请注意,如果我使用Render而不是Render_site,则dopar可以正常工作.

您可以在下面看到我的代码:

pacman::p_load(data.table)
pacman::p_load(data.table,rmarkdown)
pacman::p_load(rmarkdown,knitr,kableExtra,dplyr)

info.rmd=readLines("test.rmd",encoding = "UTF-8")
yml=readLines("site.yml",encoding = "UTF-8")

fun<-function(r){
 out <- file( description=paste0(r,"test_.Rmd"), open="w", encoding="UTF-8") 
 write(info.rmd,file=out )
 close(con=out)
 render_site(paste0(r,"test_.Rmd"),encoding = "UTF-8",quiet = T)
 file.remove(paste0(r,"test_.Rmd"))
}
Run Code Online (Sandbox Code Playgroud)

该功能正在为简单而工作

# Working with for

for (r in 1:10){
 fun(r)

}
Run Code Online (Sandbox Code Playgroud)

我有使用dopar的问题(在这种情况下在YAML文件中重复输出)

# Not working for dopar
library(doParallel)
no_cores<-3
cl <- makeCluster(no_cores, type="SOCK")
registerDoParallel(cl)

res<-foreach (r=1:10,.combine = c,.inorder = T) %dopar% {
 library(rmarkdown)
 fun(r)
}
print(ret)
stopCluster(cl)
Run Code Online (Sandbox Code Playgroud)

回溯消息如下:

Error in { : task 10 failed - "Duplicate map key: 'output'"
4.stop(simpleError(msg, call = expr))
3.e$fun(obj, substitute(ex), parent.frame(), e$data)
2.foreach(r = 1:10, .combine = c, .inorder = T) %dopar% { library(rmarkdown) out <- file(description = paste0(r, "test_.Rmd"), open = "w", encoding = "UTF-8") ...
1.system.time(foreach(r = 1:10, .combine = c, .inorder = T) %dopar% { library(rmarkdown) out <- file(description = paste0(r, "test_.Rmd"), open = "w", ...
Timing stopped at: 0.39 0.36 7.9
Run Code Online (Sandbox Code Playgroud)

然后我们可以看到_site.yml文件由render_site函数自动修改.