How to apply a concat function to a group by data frame using pandas?

Question

Code 1:

df = pd.read_csv("example.csv", parse_dates=['d'])
df2 = df.set_index(['d', 'c'])
df3 = df2.groupby(level=['c'])

def function(x):
    a = pd.rolling_mean(x, 3).rename(columns = {'b':'rm'})
    c = pd.rolling_std(x, 3).rename(columns = {'b':'rsd'})
    pd.concat([x, a, c], axis=1)

df4 = df3.apply(lambda x: function(x))

Code 2:

df = pd.read_csv("example.csv", parse_dates=['d'])
df2 = df.set_index(['d', 'c'])
df3 = df2.groupby(level=['c'])

def function(x):
    x.assign(rm = lambda x: pd.rolling_mean(x, 3))

df4 = df3.apply(lambda x: function(x))

Output of df4.head() in both of the above code1 AND code 2 is a square in iPython?? I can't figure out why.

Output:

What df3 looks like:

What df looks like:

what error? Are you using df.plot() to get that square? Can you show example of your dataframe. — Anand S Kumar
– Anand S Kumar, Commented Oct 19, 2015 at 1:56
Sorry no error, just the same square. No I am not using df.plot. I try just df4 or df4.head and get the square as my output. — pr338
– pr338, Commented Oct 19, 2015 at 1:57
It would be a lot easier if you simply print the df into the terminal in ipython and then copy the data (rather than the image) , that way i can directly copy it and use pd.read_clipboard() to reproduce your issue. — Anand S Kumar
– Anand S Kumar, Commented Oct 19, 2015 at 2:00

Andy Hayden · Accepted Answer · 2015-10-19 02:14:04Z

You're missing a return statement:

In [11]: def function(x):
             a = pd.rolling_mean(x, 3).rename(columns = {'bookings':'rm'})
             c = pd.rolling_std(x, 3).rename(columns = {'bookings':'rsd'})
             return pd.concat([x, a, c], axis=1)

In [12]: df3.apply(lambda x: function(x))
Out[12]:
                   bookings          rm        rsd
ds         city
2013-01-01 City_2        69         NaN        NaN
2013-01-02 City_2       101         NaN        NaN
2013-01-03 City_2       134  101.333333  32.501282
2013-01-04 City_2       155  130.000000  27.221315
2013-01-05 City_2       104  131.000000  25.632011
2013-01-06 City_2       121  126.666667  25.967929
2013-01-07 City_2       143  122.666667  19.553346
2013-01-08 City_2       173  145.666667  26.102363
2013-01-09 City_2       142  152.666667  17.616280
2013-01-10 City_2       154  156.333333  15.631165
2013-01-11 City_2       139  145.000000   7.937254

Without the return function was returning None, hence the empty DataFrame (which was rendered by ipython as a square - which may be a bug).

In [13]: df3.apply(lambda x: None)
Out[13]:
Empty DataFrame
Columns: []
Index: []

Note: In some languages (e.g. Ruby, Julia, Scala) the last line is returned without being explicit with the return. In Python if you miss out the return statement the function returns None.

In [21]: def foo():
             1

In [22]: foo() == None
Out[22]: True

Collectives™ on Stack Overflow

How to apply a concat function to a group by data frame using pandas?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related