how to use set functions of data table inside an environment

Question

My task is to assign column names and group id to data inside an environment.

I have two cases below. The first one has data created using data.frame() and in case-2, the data is created using data.table().

The first case is showing error, but the second one works perfectly well. Why is the error occurring in case-1, but not in case-2? Is there a better way to use set functions of data.table inside an environment?

library('data.table')

Case 1:

my_env <- new.env()
my_env$d1 <- data.frame(a = 1:5, b = 1:5)
my_env$d2 <- data.frame(a = 1:5, b = 1:5)
my_env$d3 <- data.frame(a = 1:5, b = 1:5)

# set column names and value as group id
for(i in ls(my_env)){
  with(my_env, setDT(get(i))) # convert to data table by reference
  with(my_env, setnames( x = get(i), c('x', 'y')))  # assign column name by reference
  with(my_env, set( x = get(i), j = 'group', value = '0_0')) # assign group column with a value
}

Error:

 Error in set(x = get(i), j = "group", value = "0_0") : 
  Internal error, please report (including result of sessionInfo()) to datatable-help: oldtncol (0) < oldncol (2) but tl of
  class is marked.

Case 2:

my_env2 <- new.env()
my_env2$d1 <- data.table(a = 1:5, b = 1:5)
my_env2$d2 <- data.table(a = 1:5, b = 1:5)
my_env2$d3 <- data.table(a = 1:5, b = 1:5)

# set column names and value as group id
for(i in ls(my_env2)){
  # with(my_env, setDT(get(i))) # convert to data table by reference
  with(my_env2, setnames( x = get(i), c('x', 'y')))  # assign column name by reference
  with(my_env2, set( x = get(i), j = 'group', value = '0_0')) # assign group column with a value
}

Session Information

sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252 
[2] LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods  
[7] base     

other attached packages:
[1] data.table_1.9.7

loaded via a namespace (and not attached):
[1] tools_3.3.2

Don't think it's related to environments. You can repro this with setDT(get("mtcars")) ; set(x = get("mtcars"), j = "test", value = 1). It's probably has something to do with get. I'm also not sure how does setDT(get("mtcars")) even works as it doesn't assign attr(*, ".internal.selfref")=<externalptr> as a normal setDT will do (not to mention that just setDT(mtcars) won't work in this case). Either way, I think you should follow the advice of the error and file a bug report on GH — David Arenburg
– David Arenburg, Commented Apr 23, 2018 at 21:14
@DavidArenburg setDT(get("mtcars")) works because it is called inside with() in my example, I guess — Sathish
– Sathish, Commented Apr 23, 2018 at 21:19
Not sure what you mean. setDT(get("mtcars")) works inside global environment too — David Arenburg
– David Arenburg, Commented Apr 23, 2018 at 21:21
I think it is because setDT uses assign to make the in-place replacement but doesn't cover this input case, so that step doesn't go through. If it's passed as a symbol it works, though ex = quote(with(my_env, {setDT(d1); set(d1, j = "group", v = "0_0")})); eval(ex); my_env$d1 Fwiw, besides get("d1") it also fails with my_env[["d1"]]. — Frank
– Frank, Commented Apr 23, 2018 at 21:27
So I found the error here. The problem seem to be when running setDT(get(i)) there is no memory allocated (no attr(*, ".internal.selfref")=<externalptr>), hence truelength returns zero and hence the error. Can be rpeorduced with dt <- data.frame(a = 1) ; setDT(get("dt")) ; str(dt) ; truelength(dt) ; setDT(dt) ; str(dt) ; truelength(dt) — David Arenburg
– David Arenburg, Commented Apr 23, 2018 at 21:56

vpz · Accepted Answer · 2019-12-10 11:40:15Z

1

Well, I just changed the way to create the group column and it worked fine to me. I also used names() instead of ls():

# set column names and value as group id
for(i in names(my_env)){
  with(my_env, setnames( x = my_env[[i]], c('x', 'y')))  # assign column name by reference
  my_env[[i]][,"group"] <- "0_0" # assign group column with a value
}

answered Dec 10, 2019 at 11:40

vpz

1,0521 gold badge19 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

how to use set functions of data table inside an environment

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related