Importing multiple .csv files into R and adding a new column with file name

Question

I have 80 separate .csv files that have the same columns and headers that I was able to import and rbind as one dataframe using the following commands:

 file_names <- dir("~/Desktop/data") 
 df <- do.call(rbind,lapply(file_names,read.csv))

But I would like to add a new variable ("name") that identifies from which .csv file each observation came from. So for example, this variable "name" would be "NY" for all the observations from the 'NY.csv' file and "DC" for all observations from the 'DC.csv' file, etc... Is there any way to do this without adding this new column manually on each .csv? Thanks!

mpjdem · Accepted Answer · 2016-12-07 01:41:49Z

3

This should do it:

file_names <- dir("~/Desktop/data") 
df <- do.call(rbind, lapply(file_names, function(x) cbind(read.csv(x), name=strsplit(x,'\\.')[[1]][1])))

answered Dec 7, 2016 at 1:41

mpjdem

1,5449 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Agustín Indaco Over a year ago

Hey @mpjdem I like where you are going with this, but I get the following error. Any idea why/how to solve it? Thanks! "Error in !header : invalid argument type"

mpjdem Over a year ago

Does it help explicitly set header=TRUE in the read.csv() call? (supposing you do have a header; otherwise header=FALSE)

Agustín Indaco Over a year ago

No, I still get the same error. I have a header; can't figure it out. @mpjdem

Rodrigo Zepeda · Accepted Answer · 2023-10-14 19:27:22Z

2

With readr >= 2.0 just add the id option:

library(readr)
read_csv(file_names, id = "name")

If you would like to remove the csv at the end:

read_csv(file_names, id = "name") %>%
   mutate(name = str_remove_all(name, ".csv"))

See this thread for more options.

edited Oct 14, 2023 at 19:27

answered Oct 30, 2022 at 16:54

Rodrigo Zepeda

2,0632 gold badges17 silver badges27 bronze badges

Comments

DanY · Accepted Answer · 2023-06-30 04:18:27Z

0

Use the idcol argument from data.table's rbindlist() function:

# get a vector of all file names
myfiles <- list.files("path/to/directory/")

# loop over files names, reading in and saving each data.frame as an element in a list
n <- length(myfiles )
datalist <- vector(mode="list", length=n)
for(i in 1:n) {
    cat("importing file", i, ":", myfiles[i], "\n")
    datalist[[i]] <- read.csv(myfiles[i])
}

# assign list elements the file names
names(datalist) <- myfiles 

# combine all data.frames in datalist, use idcol argument to assign original file name
all_data <- data.table::rbindlist(datalist, idcol=TRUE)

edited Jun 30, 2023 at 4:18

answered Jun 30, 2023 at 4:12

DanY

6,1331 gold badge17 silver badges36 bronze badges

Collectives™ on Stack Overflow

Importing multiple .csv files into R and adding a new column with file name

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related