Rmpi mpi.remote.exec() can't access user-defined function

Sorry if this is not suitable but I think I can help with the error. There are two problems I can see here.

First is that you have to explicitly pass the custom function to the slaves, the function mpi.bcast.Robj2slave will do this for you. See the documentation for a more detailed explanation.

So the code will run with this:

library(Rmpi)
mpi.spawn.Rslaves()

df1 <- data.frame("x" = 1:2, "y" = 1:2)
df2 <- data.frame("x" = 2:3, "y" = 2:3)
df3 <- data.frame("x" = 3:4, "y" = 3:4)
df4 <- data.frame("x" = 4:5, "y" = 4:5)

fun <- function(x) {
  input <- get(paste0("df", x), envir = .GlobalEnv) 
  return(input)
}

array <- 1:4
MPIresults <- mpi.remote.exec(cmd = fun, array)

But now we see the slave can not find the dataframes. One way around this is to explicitly pass them to the slave and use a method to pick the ones each slave will work on. So for four slaves, you could do:

library(Rmpi)
mpi.spawn.Rslaves(nslaves=4)

df1 <- data.frame("x" = 1:2, "y" = 1:2)
df2 <- data.frame("x" = 2:3, "y" = 2:3)
df3 <- data.frame("x" = 3:4, "y" = 3:4)
df4 <- data.frame("x" = 4:5, "y" = 4:5)

fun <- function(x,df_list) {
  slave_num <- mpi.comm.rank() #give slave number
  input <- df_list[[slave_num]] 
  return(input)
}

array <- 1:4
df_list <- list(df1,df2,df3,df4)

mpi.bcast.Robj2slave(fun)
MPIresults <- mpi.remote.exec(cmd = fun, array, df_list)

Which will return your desired output

MPIresults
$slave1
  x y
1 1 1
2 2 2

$slave2
  x y
1 2 2
2 3 3

$slave3
  x y
1 3 3
2 4 4

$slave4
  x y
1 4 4
2 5 5