pandas groupby apply returning a dataframe

Question

Consider the following code:

>>> df = pd.DataFrame(np.random.randint(0, 4, 16).reshape(4, 4), columns=list('ABCD'))
... df
...
   A  B  C  D
0  2  1  0  2
1  3  0  2  2
2  0  2  0  2
3  2  1  2  0
>>> def grouper(frame):
...     return frame
...     
... df.groupby('A').apply(grouper)
...
   A  B  C  D
0  2  1  0  2
1  3  0  2  2
2  0  2  0  2
3  2  1  2  0

As you can see, the results are identical. Here is the documentation of apply:

The function passed to apply must take a dataframe as its first argument and return a DataFrame, Series or scalar. apply will then take care of combining the results back together into a single dataframe or series. apply is therefore a highly flexible grouping method.

Groupby will divide group into small dataframes like this:

   A  B  C  D
2  0  2  0  2

   A  B  C  D
0  2  1  0  2
3  2  1  2  0

   A  B  C  D
1  3  0  2  2

apply documentation says that it combines the dataframes back into a single dataframe. I am curious how it combined them in a way that the final result is the same as the original dataframe. If it had used concat, the final dataframe would have been equal to:

   A  B  C  D
2  0  2  0  2
0  2  1  0  2
3  2  1  2  0
1  3  0  2  2

I am curious how this concatenation has been done.

keiv.fly · Accepted Answer · 2019-12-12 22:59:34Z

4

If you look at the source code you will see that there is a parameter not_indexed_same that checks if the index remains the same after groupby. If it is the same then groupby does reindexing of the dataframe before returning results. I do not know why this was implemented.

The change was made on Aug 21, 2011 and Wes made no comments on the change: https://github.com/pandas-dev/pandas/commit/00c8da0208553c37ca6df0197da431515df813b7#diff-720d374f1a709d0075a1f0a02445cd65

edited Dec 12, 2019 at 22:59

answered Dec 12, 2019 at 22:01

keiv.fly

4,2408 gold badges30 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

pandas groupby apply returning a dataframe

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related