How to loop over multiple DataFrames and produce multiple csv?

Question

making a change from R to Python I have some difficulties to write multiple csv using pandas from a list of multiple DataFrames:

import pandas
from dplython import (DplyFrame, X, diamonds, select, sift, sample_n,
                  sample_frac, head, arrange, mutate, group_by, summarize,
                  DelayFunction)

diamonds = [diamonds, diamonds, diamonds]
path = "/user/me/" 

def extractDiomands(path, diamonds):
    for each in diamonds:
    df = DplyFrame(each) >> select(X.carat, X.cut, X.price) >> head(5)
    df = pd.DataFrame(df) # not sure if that is required
    df.to_csv(os.path.join('.csv', each))

extractDiomands(path,diamonds)

That however generates an errors. Appreciate any suggestions!

Max Power · Accepted Answer · 2017-05-03 08:45:24Z

Welcome to Python! First I'll load a couple libraries and download an example dataset.

import os
import pandas as pd

example_data =  pd.read_csv("http://www.ats.ucla.edu/stat/data/binary.csv")
print(example_data.head(5))

first few rows of our example data:

   admit  gre   gpa  rank
0      0  380  3.61     3
1      1  660  3.67     3
2      1  800  4.00     1
3      1  640  3.19     4
4      0  520  2.93     4

Now here's what I think you want done:

# spawn a few datasets to loop through
df_1, df_2, df_3 = example_data.head(20), example_data.tail(20), example_data.head(10)
list_of_datasets = [df_1, df_2, df_3]

output_path = 'scratch'
# in Python you can loop through collections of items directly, its pretty cool.
# with enumerate(), you get the index and the item from the sequence, each step through
for index, dataset in enumerate(list_of_datasets):

    # Filter to keep just a couple columns
    keep_columns =   ['gre', 'admit']
    dataset = dataset[keep_columns]

    # Export to CSV
    filepath = os.path.join(output_path, 'dataset_'+str(index)+'.csv')
    dataset.to_csv(filepath)

At the end, my folder 'scratch' has three new csv's called dataset_0.csv, dataset_1.csv, and dataset_2.csv

Collectives™ on Stack Overflow

How to loop over multiple DataFrames and produce multiple csv?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related