pandas groupby-apply missing columns

Question

I have a DataFrame and i want to do a "groupby-apply" style operation, however, the "key" columns were missing after the operation:

df = pd.DataFrame({
'col1':['A1', 'A2', 'A3'],
'col2':['B1', 'B1', 'B2'],
'col3':[1, 2, 3,]
})  
b1 = df.groupby(['col1', 'col2'], as_index=False)[['col3']].apply(lambda x: x+10)
b2 = df.groupby(['col1', 'col2'], as_index=False)['col3'].apply(lambda x: x+10)
print(b1)
print(b2)

b1 will print as:

b2 will print as:

b
0  0    11
1  1    12
2  2    13
Name: col3, dtype: int64

how can i make use the group-key columns(['col1','col2'] also be printed, as

col1  col2
A1    B1      11
A2    B1      12
A3    B2      13

It doesn't make sense to have the group keys output. Your calculation returns a Series of the original shape of the DataFrame, so the output is a Series indexed like the original DataFrame. Your output would be indexed with the group keys when there is an aggregation that gives you one value per group. I.e. the difference between groupby.sum() and groupby.cumsum() — ALollz
– ALollz, Commented Sep 3, 2019 at 14:44

BENY · Accepted Answer · 2019-09-03 14:53:29Z

1

When you print the type of each groupby object you will see , that is Series, which will have the index ahead of it . Github Open issue

df.groupby(['col1', 'col2'], as_index=False)['col3'].apply(lambda x: type(x) )
Out[11]: 
col1  col2
A1    B1      <class 'pandas.core.series.Series'>
A2    B1      <class 'pandas.core.series.Series'>
A3    B2      <class 'pandas.core.series.Series'>
dtype: object

To get the expected output

df.groupby(['col1', 'col2']).apply(lambda x: x['col3']+10).reset_index(level=-1,drop=True)
Out[32]: 
col1  col2
A1    B1      11
A2    B1      12
A3    B2      13
Name: col3, dtype: int64

answered Sep 3, 2019 at 14:53

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

pandas groupby-apply missing columns

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related