Pandas send multiple DataFrames to_csv

Question

I have three symbols with price data and successfully imported/modified that price data as shown below:

def read_files(): 
    assets = ['ES','ZN','VX']
    path = r"C:\Users\cost9\OneDrive\Documents\PYTHON\Exported_Data\%s\*.csv"

    files = []
    for a in assets:
        files.extend(glob.iglob(path % a))

    df_list = [pd.read_csv(f) for f in files]

    for b in df_list:
        b['Returns'] = b['Close'].pct_change()

The sample data now looks like this after creating the 'Returns' column above:

Ticker  Date/Time   Close   Returns
ES U7   3/14/2017 5:29  2365.5  
ES U7   3/14/2017 6:29  2362.25 -0.001373917
ES U7   3/14/2017 8:29  2355    -0.003069108
ES U7   3/14/2017 9:29  2359.25 0.001804671
ES U7   3/14/2017 10:29 2356    -0.001377556
ES U7   3/14/2017 11:29 2358    0.000848896
ES U7   3/14/2017 12:29 2358.5  0.000212044
ES U7   3/14/2017 13:29 2360.25 0.000741997
ES U7   3/14/2017 14:29 2360.5  0.000105921
ES U7   3/14/2017 21:29 2361.5  0.000423639
ES U7   3/14/2017 22:29 2363.5  0.000846919
ES U7   3/14/2017 23:29 2363.75 0.000105775
ES U7   3/15/2017 0:29  2364.5  0.000317292
ES U7   3/15/2017 1:29  2363.5  -0.000422922

Each of the three symbols' CSV files would look like this. Now I want to move each of these three DataFrames into separate folders:

end_path_to_csv = "C:\\Users\\cost9\\OneDrive\\Documents\\PYTHON\\Daily Tasks\\Individual Trading\\%s\CSV\\15M\\Trend_Identifier\\blah.csv" 

for c in df_list:
    c.to_csv(path_or_buf = end_path_to_csv % assets)

I get this error:

IOError: [Errno 2] No such file or directory: "C:\\Users\\cost9\\OneDrive\\Documents\\PYTHON\\Daily Tasks\\Individual Trading\\['ES', 'ZN', 'VX']\\CSV\\15M\\Trend_Identifier\\blah.csv"

Unfortunately I can't convert the 'assets' and 'df_list' (Dataframe) values into a dictionary - if I do that then the DataFrames won't be called and there's no csv data available.

I have three folders, each the exact same except for the symbol in the file directory, for example:

"C:\\Users\\cost9\\OneDrive\\Documents\\PYTHON\\Daily Tasks\\Individual Trading\\ES\\CSV\\15M\\Trend_Identifier\\blah.csv"

This would be for the 'ES' symbol, the 'VX' would replace 'ES' if I was referring to that DataFrame and so on.

How can I save each dataframe into the corresponding folder?

is this a folder on your computer? C:\\Users\\cost9\\OneDrive\\Documents\\PYTHON\\Daily Tasks\\Individual Trading\['ES', 'ZN', 'VX']\\CSV\\15M\\Trend_Identifier\\ — Paul H
– Paul H, Commented Nov 14, 2017 at 1:08
@PaulH no, but each of the three folders is present as follows: C:\\Users\\cost9\\OneDrive\\Documents\\PYTHON\\Daily Tasks\\Individual Trading\ES\CSV\\15M\\Trend_Identifier\\ — Cole Starbuck
– Cole Starbuck, Commented Nov 14, 2017 at 1:08
You made the same error from your last question I told you not to, — cs95
– cs95, Commented Nov 14, 2017 at 1:12
You need to iterate over assets, rather than insert assets into a string. — cs95
– cs95, Commented Nov 14, 2017 at 1:13
@cᴏʟᴅsᴘᴇᴇᴅ understood, the issue is 'a' isn't defined in the for loop being referenced, and I'm not sure how to define 'a' for 'assets' in the loop without creating a dictionary without eliminating the dataframe quality of 'df_list'. — Cole Starbuck
– Cole Starbuck, Commented Nov 14, 2017 at 1:15

cs95 · Accepted Answer · 2017-11-14 01:14:54Z

2

Just zip assets and df_list together and iterate over them.

assets = ['ES','ZN','VX']
dst_path = ...

for p, c in zip(assets, df_list):
    c.to_csv(dst_path % p)

answered Nov 14, 2017 at 1:14

cs95

406k106 gold badges745 silver badges798 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Mohammed Elmahgiubi · Accepted Answer · 2017-11-14 01:16:22Z

1

You are looping over df_list, yet the path is the same in every loop-iteration.Try this:

end_path_to_csv = "C:\\Users\\cost9\\OneDrive\\Documents\\PYTHON\\Daily Tasks\\Individual Trading\\%s\CSV\\15M\\Trend_Identifier\\blah.csv" 

for idx, c in enumerate(df_list):
    c.to_csv(path_or_buf = end_path_to_csv % assets[idx])

answered Nov 14, 2017 at 1:16

Mohammed Elmahgiubi

6514 silver badges11 bronze badges

2 Comments

cs95 Over a year ago

Why enumerate indices when you could just zip assets with df_list?

Mohammed Elmahgiubi Over a year ago

df_list is initially created from looping over assets (they have the same length), that was what led me to use enumerate @cᴏʟᴅsᴘᴇᴇᴅ

Collectives™ on Stack Overflow

Pandas send multiple DataFrames to_csv

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related