Reading CSV files into separate data frames using FOR Loop

Question

I have 600 csv files that I want to read to separate data frames using pd.read.csv. I have tried the code below.

for stock in stock_list:
    df_stock = pd.read_csv("D:/Stocks/CSV/" + stock + ".csv")

I want every dataframe to be stored separately as df_the respective stock name in stock_list (for example df_pfizer, df_ICBC and so on) but instead my code is reading it into a single dataframe. The stock names are in the list stock_list.

Whole Brain · Accepted Answer · 2021-06-18 12:32:08Z

2

Having 600 variables is not handleable, you should use a dictionary.

df_dict = {}
for stock in stock_list:
    df_dict[stock] = pd.read_csv("D:/Stocks/CSV/" + stock + ".csv")

... or:

df_dict = {stock: pd.read_csv("D:/Stocks/CSV/" + stock + ".csv") for stock in stock_list}

answered Jun 18, 2021 at 12:32

Whole Brain

2,1972 gold badges13 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Umar.H Over a year ago

what is stock_list in the dictionary comp?

Whole Brain Over a year ago

@Umar.H Isn't it your list of stock names ? You can then retrieve any dataframe from the dictionary with a name, like this: df_dict["some_name"]. Please check the python dictionaries documentation.

Red · Accepted Answer · 2021-06-18 12:32:33Z

1

You can add the dataframes into a python dict:

df_stocks = {}
for stock in stock_list:
    df_stock = pd.read_csv("D:/Stocks/CSV/" + stock + ".csv")
    df_stocks[stock] = df_stock

And to access any dataframe:

df_stocks['df_pfizer']

will return the 'df_pfizer' dataframe,

df_stocks['df_ICBC']

will return the df_stocks['df_ICBC'] dataframe, and so on.

answered Jun 18, 2021 at 12:32

Red

27.7k8 gold badges44 silver badges63 bronze badges

Comments

Umar.H · Accepted Answer · 2021-06-18 12:31:47Z

0

you need to first create a single dataframe then seperate them based on the stockname with a dictionary comp.

from pathlib import Path
import pandas as pd

dfs = pd.concat(
                [pd.read_csv(f) for f in Path(r"D:/Stocks/CSV/").glob('*.csv')]
              )

stock_dfs = {stock : frame for stock,frame in dfs.groupby('stock_list')}

then you can call each df by its key

stock_dfs['df_pfizer']

answered Jun 18, 2021 at 12:31

Umar.H

23.1k8 gold badges50 silver badges94 bronze badges

2 Comments

Washnaek Baloch Over a year ago

Thanks Umar for such a prompt reply. But i want separate dataframes. Isnt there a way to create separate dataframes with respective names?

Umar.H Over a year ago

creating n number of variables is not recommended. use a container like a dictionary I wasn't sure if you wanted to seperated by the stock_list column or the file name. that said, the above method will handle that for you.

Collectives™ on Stack Overflow

Reading CSV files into separate data frames using FOR Loop

3 Answers 3

2 Comments

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related