how to create unique pandas dataframes while iterating over a list

Question

With the following list and dataframe I would like to iterate over the list of dates and create a new dataframe for each date that is in the list and dataframe.

list:

['12/12/2017', '12/13/2017', '12/14/2017']

dataframe:

source  time        date
0      a  8:00  12/12/2017
1      b  9:00  12/13/2017
2      c  8:30  12/12/2017
3      b  8:05  12/14/2017
4      a  8:10  12/13/2017
5      a  8:15  12/12/2017

Desired output:

source  time        date
0      a  8:00      12/12/2017
2      c  8:30      12/12/2017
5      a  8:15      12/12/2017

...and so on for each date.

code:

import pandas as pd

unique_dates=['12/12/2017','12/13/2017','12/14/2017']

data=[['a','8:00','12/12/2017'],['b','9:00','12/13/2017'],['c','8:30','12/12/2017'],
      ['b','8:05','12/14/2017'],['a','8:10','12/13/2017'],['a','8:15','12/12/2017']]
headers=['source','time','date']
df=pd.DataFrame(data,columns=headers)

for item in unique_dates:
    if item in df:
        new_df=df[df['date']==item]
        print(new_df)

This gives me no output. I'm at a loss for how to do this and any help would be appreciated.

cs95 · Accepted Answer · 2018-01-28 13:30:34Z

2

Solving this becomes simple, once you identify it as a good use case for a groupby operation. All you need to do is pre-filter with isin, and then group and iterate.

df_list = [g for _, g in df[df.date.isin(unique_dates)].groupby('date')]

If you want a dictionary indexed by date instead, call dict on the groupby object -

df_dict = dict(list(df[df.date.isin(unique_dates)].groupby('date')))

Or, using a dict comprehension -

df_dict = {i : g for i, g in df[df.date.isin(unique_dates)].groupby('date')}

print(*df_list, sep='\n\n')

  source  time        date
0      a  8:00  12/12/2017
2      c  8:30  12/12/2017
5      a  8:15  12/12/2017

  source  time        date
1      b  9:00  12/13/2017
4      a  8:10  12/13/2017

  source  time        date
3      b  8:05  12/14/2017

answered Jan 28, 2018 at 13:30

cs95

406k106 gold badges745 silver badges798 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

how to create unique pandas dataframes while iterating over a list

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related