Read and save multiple csv files from a for-loop

Question

I am trying to read multiple csv files from a list of file paths and save them all as separate pandas dataframes.

I feel like there should be a way to do this, however I cannot find a succinct explanation.

import pandas as pd

data_list = [['df_1','filepath1.csv'],
             ['df_2','filepath2.csv'],
             ['df_3','filepath3.csv']]

for name, filepath in data_list:
    name = pd.read_csv(filepath)

I have also tried:

data_list = [[df_1,'filepath1.csv'],[df_2,'filepath2.csv'],
             [df_3,'filepath3.csv']]

for name, filepath in data_list:
    name = pd.read_csv(filepath)

I would like to be able to call each dataframe by its assigned name.

Ex):

df_1.head()

Sesquipedalism · Accepted Answer · 2019-07-11 14:04:49Z

1

df_dct = {name:pd.read_csv(filepath) for name, filepath in data_list}

would create a dictionary of DataFrames. This may help you organize your data.

You may also want to look into glob.glob to create your list of files. For example, to get all CSV files in a directory:

file_paths = glob.glob(my_file_dir+"/*.csv")

answered Jul 11, 2019 at 14:04

Sesquipedalism

1,75315 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ali Hakim Taşkıran · Accepted Answer · 2019-07-11 14:03:41Z

0

I recommend you numpy. Read the csv files with numpy.

from numpy import genfromtxt
my_data = genfromtxt('my_file.csv', delimiter=',')

You will get nd-array's. After that you can include them into pandas.

answered Jul 11, 2019 at 14:03

Ali Hakim Taşkıran

10912 bronze badges

Comments

Parthasarathy Subburaj · Accepted Answer · 2019-07-11 14:03:42Z

0

You can make sure of dictionary for this...

import pandas as pd
data_list = ['filepath1.csv', 'filepath2.csv', 'filepath3.csv']
d = {}
for _, i in enumerate(data_list):
    file_name = "df" + str(_)
    d[file_name] = pd.read_csv(filepath)

Here d is the dictionary which contains all your dataframes.

answered Jul 11, 2019 at 14:03

Parthasarathy Subburaj

4,2942 gold badges12 silver badges25 bronze badges

Collectives™ on Stack Overflow

Read and save multiple csv files from a for-loop

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related