I would like to repeatedly use a function that calls on clusters from Rmpi, but I am having issues where the first function call hangs indefinitely.
Here is a simple code example:
library(parallel)
library(snow)
library(Rmpi)
run_rmpi_test <- function(
X, fun, iter, numer,filepath) {
n.cores <- mpi.universe.size()-1
cl <- makeMPIcluster(n.cores)
varlist <- c(ls(envir = parent.frame()))
parallel::clusterExport(cl, varlist = varlist,envir=environment())
parallel::clusterSetRNGStream(cl, 20)
# run parallel computation
computed <- parallel::parLapply(cl=cl, X = X, fun=fun, iter=iter, numer=numer)
save(list=varlist,
file=filepath,
envir=environment())
# stop the cluster
snow::stopCluster(cl)
mpi.finalize()
# return result
return(computed)
}
fun_function <- function(i,iter,numer){
l1 <- rnorm(iter)
l2 <- rnorm(numer)
return(list(l1,l2))
}
funct1 <- run_rmpi_test(X=1:10,fun_function,iter=5,numer=8,filepath="test1.RData")
funct2 <- run_rmpi_test(X=1:10,fun_function,iter=3,numer=2,filepath="test2.RData")
Thus far, it seems that the function run_rmpi_test() hangs right after save(). Do you have any suggestions on how to fix the above? My OpenMPI version is 4.1.4. Ideally, I would like to have both function calls (funct1 and funct2) in the same script, and I should have both outputs test1.RData and test2.RData after the code is ran.