Reading multiple CSV files with different names using python dictionary in a for loop

Question

I have a list of filenames and filepaths that I have stored in a python dictionary. I am trying to read the files using pandas' read_csv and assign the dataframe names from the dictionary. I can read and print the dataframes while running the for loop but I cannot call the dataframes after the loop is finished. I could append all the dataframes in a list but that way I am not able to assign these dataframes different names which are also stored in the dictionary.

I checked various forums but none of them explain why the for loop with pd.read_csv doesn't work and why I am not able to assign the names to the dataframes in the for loop to later use them.

import pandas as pd
files_dict = {"Filter":"OUTTYPENM.csv","previous":"previous.csv"}
for key, value in files_dict.items():
    key = pd.read_csv(value)
Filter.head()

I expect to see the first five lines from the Filter dataframe as if I have read the dataframe as following.

Filter = pd.read_csv("OUTTYPENM.csv")

All the csv files are in the current working directory.

When I run the for loop code, and run the Filter.head(), I get an error saying - NameError: name 'Filter' is not defined

Simply use a dict of data frames and not separate named objects. — Parfait
– Parfait, Commented May 15, 2019 at 21:10
When I print(key) within the for loop, it prints all the dataframes one by one as the loop iterates. If I print it outside the for loop, it prints the last dataframe in the for loop. I am using spyder from Anaconda to do this. — Bhavin Solanki
– Bhavin Solanki, Commented May 16, 2019 at 6:56

ArkF · Accepted Answer · 2019-05-15 21:08:22Z

1

This doesn't exactly answer your question, but I think it gets you to a similar place, without involving any exec() or locals() calls.

Instead of creating variables named after your dictionary keys, you can just have a second dictionary where the keys are the same and the values are now the DFs you read in.

import pandas as pd
files_dict = {"Filter":"OUTTYPENM.csv","previous":"previous.csv"}
df_dict = {}
for key, value in files_dict.items():
    df_dict[key] = pd.read_csv(value)
df_dict['Filter'].head()

answered May 15, 2019 at 21:08

ArkF

12410 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Parfait Over a year ago

Or use dictionary comprehension: df_dict = {key: pd.read_csv(value) for key, value in files_dict.items()}

lyang · Accepted Answer · 2019-05-15 18:51:06Z

0

Try this:

for key, value in files_dict.items():
    locals()[key] = pd.read_csv(value)

This method is not recommended though. See the link here:https://www.linuxquestions.org/questions/programming-9/python-create-variables-from-dictionary-keys-859776/

answered May 15, 2019 at 18:51

lyang

162 bronze badges

Collectives™ on Stack Overflow

Reading multiple CSV files with different names using python dictionary in a for loop

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related