Running parallel in R problems

Question

I tried to test parallel in R with parallel package. But in my example (as the code below), the time for parallel task is greater than that for single task. Anybody can give me some advice?

Thanks so much!

##parSquareNum.R
strt <- Sys.time()
workerFunc <- function(n) { return(n^2) }
values <- 1:1000000
library(parallel)
## Number of workers (R processes) to use:
cores <- detectCores()
## Set up the ’cluster’
cl <- makeCluster(cores-1)
## Parallel calculation (parLapply):
res <- parLapply(cl, values, workerFunc)
## Shut down cluster
write(Sys.time()-strt, 'parallel.txt')
stopCluster(cl)

##singleSquareNum.R

## The worker function to do the calculation:
strt <- Sys.time()
workerFunc <- function(n) { return(n^2) }
## The values to apply the calculation to:
values <- 1:1000000
## Serial calculation:
res <- lapply(values, workerFunc)
##print(unlist(res))
write(Sys.time() -strt, 'single.txt')

Parallelism is unlikely to be of benefit for simple tasks at the core because the dispatch and reassembly process takes time as well. Parceling out complex tasks will be needed to demonstrate benefit. — IRTFM
– IRTFM, Commented Jun 1, 2015 at 17:26

Oscar · Accepted Answer · 2015-06-01 17:36:36Z

The main reason you are seeing this is because loading the library plus making the cluster takes some time. Move the strt <- Sys.time() to right before res and you will see a difference, especially if you increase the value of values

##parSquareNum.R
workerFunc <- function(n) { return(n^2) }
values <- 1:1000000
library(parallel)
## Number of workers (R processes) to use:
cores <- detectCores()
## Set up the ’cluster’
cl <- makeCluster(cores-1)
## Parallel calculation (parLapply):
strt <- Sys.time()
res <- parLapply(cl, values, workerFunc)
write(Sys.time()-strt, 'parallel.txt')
## Shut down cluster
stopCluster(cl)

##singleSquareNum.R

## The worker function to do the calculation:
workerFunc <- function(n) { return(n^2) }
## The values to apply the calculation to:
values <- 1:1000000
## Serial calculation:
strt <- Sys.time()
res <- lapply(values, workerFunc)
##print(unlist(res))
write(Sys.time() -strt, 'single.txt')

When I run I get 0.6941409 sec for parallel and 1.117002 sec for single. 1.6X speedup. I am running on an i7 chip.

Collectives™ on Stack Overflow

Running parallel in R problems

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related