Memory problem in R "Error:cannot allocate vector of size 344 kb"

Question

My memory.limit() is 3583,I have a 64-bit machine with 8G RAM at home,and just remote access to my computer in the office then found it was also 8G RAM.So I can't run the R codes below successfully,should I reset the memory limit?But someone thinks it's a dangerous approach, could anyone tell me how to solve this problem? Thanks in advance!

loop<-1000;T<-45
bbb<-list()
for(i in 1:loop)
{
  bbb[[i]]<-list()
  bbb[[i]][[1]]<-matrix(rep(1,loop*(T-1)),loop,T-1)
  bbb[[i]][[2]]<-matrix(rep(0,loop*(T-1)),loop,T-1) 
  bbb[[i]][[3]]<-matrix(rep(0,loop*(T-1)),loop,T-1) 
  bbb[[i]][[4]]<-matrix(rep(0,loop*(T-1)),loop,T-1) 
  bbb[[i]][[5]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[6]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[7]]<-matrix(rep(0,loop*(T-1)),loop,T-1) 
  bbb[[i]][[8]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[9]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[10]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[11]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[12]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[13]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[14]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[15]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[16]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[17]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[18]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[19]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[20]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[21]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[22]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[23]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[24]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[25]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[26]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[27]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[28]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[29]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[30]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[31]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[32]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
  bbb[[i]][[33]]<-matrix(rep(0,loop*(T-1)),loop,T-1)
}

nniloc · Accepted Answer · 2020-04-15 00:33:43Z

1

I suppose it depends what you doing with the matrix list, but maybe you could break your task into smaller chunks? Or you can try using lapply, which runs much faster on my machine but ultimately creates an object of exactly the same size. I think lapply has some memory saving advantages when repeating data.

If this doesn't work, try looking into the Matrix package and sparse matrices.

create_bbb <- function(loop = 1000, T = 45){

  inner.list <- lapply(1:33, FUN = function(x){

    if(x == 1) fill <- 1
    else fill <- 0

    return(matrix(rep(fill, loop * (T-1)), loop, T-1))

  })

  bbb <- lapply(1:loop, function(.) inner.list)

  return(bbb)
}


bbb_test <- create_bbb()

# Check
all.equal(bbb, bbb_test)
# TRUE

answered Apr 15, 2020 at 0:33

nniloc

4,2432 gold badges16 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

nniloc Over a year ago

Did it work on your machine then? I think this is the best review of memory usage I found here. It states "a subtlety of the size of an object is that components can be shared across multiple objects" which is my best guess as to why lapply would help here

XUN ZHANG Over a year ago

Yes, it really works! Lapply is better than for-loops. Thanks again!

Collectives™ on Stack Overflow

Memory problem in R "Error:cannot allocate vector of size 344 kb"

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related