How to loop through data sets to graph particular columns only?

Question

The graph, I have down. The challenge is I have the _exact_same_code_ for graphing multiple data sets (rather, subsets of one LARGE data set), but I can't seem to get the looping code right to substitute the $ correctly.

Data sets, df1, df2, df3... of the form:

OBSDATE     REGION  AVG_RESP  P10  P90
2012-02-01  APAC    1.276     0.78 3.45
2012-02-01  EMEA    2.341     1.23 5.67
2012-02-02  APAC    1.343     0.89 3.21
2012-02-02  EMEA    2.473     1.37 5.98

The graph is more complex, but like this:

avgMx <- quantile(df1$P90,0.95)
ggplot(df1,aes(x=OBSDATE,y=AVG_RESP))+coord_cartesian(ylim=c(0,avgMx))+geom_ribbon(aes(ymin=P10,ymax=P90),fill="gray60",alpha=0.33)+geom_line(aes(x=OBSDATE,y=AVG_RESP),color="#007DB1",size=0.5)+facet_wrap(~REGION)

if I define a vector or list (both seem to fail with the same error messages) with the data set names I can't get the loop to work to find any descriptive values (like the quantile above or even a max!)

filenames <- c("df1","df2","df3")

I would like to get something like this to work

for (i in filenames) {
   quantile(i$AVG_RESP,0.95)
   max(i$AVG_RESP)
}

But I get errors about $ is invalid for atomic vectors. Upon investigation, that doesn't seem to yield any usable results.

So, I can get this to work:

max(df1$AVG_RESP) or max(df1['AVG_RESP'])

they both would output 2.473 from above. However, this doesn't fly:

for (i in pagesC) max(i['AVG_RESP'])

It does nothing. Changing it to this:

for (i in pagesC) print(max(i['AVG_RESP']))

Gives instances of NA.

I'm completely stuck. Any help would be tremendously appreciated!

EDIT: I fixed the data that was causing errors - should be reproducible now.

Brian Diggs · Accepted Answer · 2012-04-11 21:57:11Z

3

i is a character string; you want the object which has the name that is held in i. That is the get() function. (untested since what you gave was not reproducible.)

for (filename in filenames) {
   i <- get(filename)
   quantile(i$AVG_RESP,0.95)
   max(i$AVG_RESP)
}

This is probably not the best way to solve your problem, though. Putting all the data frames in a list and looping over that list with lapply might be a better approach (what Tyler described in his answer). Furthermore, if these are subsets you've made of a bigger, single data frame you have, then an even better approach would be to use something from the plyr package to define how to split the big data frame up and what to do with each part.

answered Apr 11, 2012 at 21:57

Brian Diggs

59.1k14 gold badges169 silver badges189 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

BenH Over a year ago

Brian, thank you for this! That's exactly what I was looking for, but didn't realize the shortcomings in both my understanding and the available functions. I'm a recovering SAS user making headway with our new implementation migrating to R. So, it looks like I won't be able to do what I wanted in this way - for some reason ggplot isn't firing within the loop. I will dig into plyr and see if that might get the job done.

Chase Over a year ago

@BenH - are you making sure you print() the ggplot object? Both ggplot and lattice objects must be printed when used in functions...I'm 99% sure this is a FAQ somewhere, maybe someone will recall for us.

BenH Over a year ago

You are right, Chase. Must have overlooked that portion of the FAQ.

Tyler Rinker · Accepted Answer · 2012-04-11 22:03:45Z

3

Your code isn't reproducible so this is my best guess at what you want:

df1 <- df2 <- df3 <- read.table(text="OBSDATE     REGION  AVG_RESP  P10  P90
2012-02-01  APAC    1.276     0.78 3.45
2012-02-01  EMEA    2.341     1.23 5.67
2012-02-02  APAC    1.343     0.89 3.21
2012-02-02  EMEA    2.473     1.37 5.98
2012-02-01  APAC    1.276     0.78 3.45
2012-02-01  EMEA    2.341     1.23 5.67
2012-02-02  APAC    1.343     0.89 3.21
2012-02-02  EMEA    2.473     1.37 5.98
2012-02-01  APAC    1.276     0.78 3.45
2012-02-01  EMEA    2.341     1.23 5.67
2012-02-02  APAC    1.343     0.89 3.21
2012-02-02  EMEA    2.473     1.37 5.98", header=TRUE)

info <- function(dataframe){
    c(quantile(dataframe$AVG_RESP,0.95), max(dataframe$AVG_RESP))
}

LIST <- list(df1, df2, df3)
lapply(LIST, info)   
#Or you may want to use sapply if you want it to return a matrix
sapply(LIST, info)

R can use loops but this really isn't the R way of doing things.

edited Apr 11, 2012 at 22:03

answered Apr 11, 2012 at 21:48

Tyler Rinker

111k74 gold badges335 silver badges535 bronze badges

4 Comments

BenH Over a year ago

I'm very curious why it isn't "reproducible" - the only thing I changed was the dataframe name and included less data. Everything else is code that runs for me - minus the loop, of course.

Tyler Rinker Over a year ago

Yeah ben I figured out what was wrong. I thought it was too little data that was giving me an error and it's that there's a small tick mark at the end of your data that barely shows up when I cut and paste but causes an error to be thrown up. That's why I pasted a bunch of your data sets together but this was not necessary.

Tyler Rinker Over a year ago

PS I know you want to use a loop but I really would recommend you become comfortable with the apply family of functions.

BenH Over a year ago

So, the tick was an artifact of trying to get my stuff to display correctly here. I'm trying to learn R, but I'm also trying to get work done in the way that makes sense to me. I'll start digging into the apply family.

Collectives™ on Stack Overflow

How to loop through data sets to graph particular columns only?

2 Answers 2

3 Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related