How to import multiple csv files at once

Question

I have 30 csv files of wind speed data on my computer- each file represents data at a different location. I have written code to calculate the statistics I need to run for each site; however, I am currently pulling in each csv file individually to do so(see code below):

from google.colab import files
data_to_load = files.upload()

import io
df = pd.read_csv(io.BytesIO(data_to_load['Downtown.csv']))

Is there a way to pull in all 30 csv files at once so each file is run through my statistical analysis code block and spits out an array with the file name and the statistic calculated?

I'm not sure if your code is dependent on using pandas but if not you could use csv and do something like for filename in data_to_load: with open filename as f: reader = csv.reader(f...) — 7koFnMiP
– 7koFnMiP, Commented Aug 11, 2020 at 15:42
data_to_load has the program ask me to select a file from my computer. I don't believe it allows me to select all the csv files within my project folder. Do you know of any function that allows you to select an entire folder to upload? — Dominic Grasso
– Dominic Grasso, Commented Aug 11, 2020 at 15:46

noobmaster69 · Accepted Answer · 2021-02-14 00:39:03Z

4

use a loop

https://intellipaat.com/community/17913/import-multiple-csv-files-into-pandas-and-concatenate-into-one-dataframe

import glob

import pandas as pd

# get data file names

local_path = r'/my_files'

filenames = glob.glob(local_path + "/*.csv")

dfs = [pd.read_csv(filename)) for filename in filenames]


# if needed concatenate all data into one DataFrame

big_frame = pd.concat(dfs, ignore_index=True)

Also you can try put data online: github or google drive and read from there https://towardsdatascience.com/3-ways-to-load-csv-files-into-colab-7c14fcbdcb92

edited Feb 14, 2021 at 0:39

noobmaster69

1921 gold badge3 silver badges14 bronze badges

answered Aug 11, 2020 at 15:43

Serge

3,8453 gold badges20 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to import multiple csv files at once

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related