how to groupby-apply-aggregate back to a dataframe in Python Pandas?

Question

def my_per_group_func(temp):

    # apply some tricks here
    return a, b, c, d

output =  dataframe.groupby('group_id').apply(my_per_group_func)

my question here is how to aggregate "output" back to a dataframe with some column names (obviously the index of the dataframe is group_id)?

normally what i do is to use the aggregate function

but the problem here is that the my_per_group_func is very complicated here, it cannot be done using the normally 'aggregate' function syntax

does anyone have a clue?

Thanks

jezrael · Accepted Answer · 2017-02-10 15:29:56Z

It seems need return DataFrame or Series - check flexible apply docs:

dataframe = pd.DataFrame({'group_id':[1,1,3],
                   'B':[4,5,6],
                   'C':[7,8,9],
                   'D':[1,3,5],
                   'E':[5,3,6],
                   'F':[7,4,3]})

print (dataframe)
   B  C  D  E  F  group_id
0  4  7  1  5  7         1
1  5  8  3  3  4         1
2  6  9  5  6  3         3

def my_per_group_func(x):
    #print (x)
    #some sample operations
    a = x.B + x.C
    b = x.E + x.B
    c = x.D + x.F
    d = x.F + x.E
    return pd.DataFrame({'group_id': x.group_id, 'a':a, 'b':b, 'c':c, 'd':d})

output =  dataframe.groupby('group_id').apply(my_per_group_func)
print (output)
    a   b  c   d  group_id
0  11   9  8  12         1
1  13   8  7   7         1
2  15  12  8   9         3

def my_per_group_func(x):
    #print (x)
    #some sample aggregations
    a = (x.B + x.C).mean()
    b = (x.E + x.B).sum()
    c = (x.D + x.F).median()
    d = (x.F + x.E).std()
    return pd.Series([a,b,c,d], index=['a','b','c','d'])

output =  dataframe.groupby('group_id').apply(my_per_group_func)
print (output)
             a     b    c         d
group_id                           
1         12.0  17.0  7.5  3.535534
3         15.0  12.0  8.0       NaN

A.Kot · Accepted Answer · 2017-02-10 14:33:49Z

1

dataframe['new column name'] = dataframe.groupby('group_id').transform(my_per_group_func)

answered Feb 10, 2017 at 14:33

A.Kot

7,9732 gold badges24 silver badges24 bronze badges

1 Comment

sadmicrowave Over a year ago

maybe you can provide some context about your answer and why this would fix the issue?

Collectives™ on Stack Overflow

how to groupby-apply-aggregate back to a dataframe in Python Pandas?

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related