When I use apply function to a single column returns error

Question

I'm learning Python and want to use the "apply" function. Reading around the manual I found that if a I have a simple dataframe like this:

df = pd.DataFrame([[4, 9]] * 3, columns=['A', 'B'])

   A  B
0  4  9
1  4  9
2  4  9

and then I use something like this:

df.apply(lambda x:x.sum(),axis=0)

output works because according to theory x receives every column and apply the sum to each so the the result is correctly this:

A    12
B    27
dtype: int64

When instead I issue something like:

df['A'].apply(lambda x:x.sum())

result is: 'int' object has no attribute 'sum'

question is: why is that working on a dataframe by column, it's ok and working on a single column is not ? In the end the logic should be the same. x should receive in input one column instead of two.

I know that for this simple example I should use other functions like df.agg or even df['A'].sum() but the question is to understand the logic of apply.

When you use apply on Series, the argument is row element in that Series. — Ynjxsjmh
– Ynjxsjmh, Commented May 1, 2022 at 8:44
@AndreaGrianti note that you should always avoid to use apply, here df.sum() and df['A'].sum() should be used. Always check whether your can use a vectorial alternative and only fallback to apply if there is really no other way (apply is slow). — mozway
– mozway, Commented May 1, 2022 at 9:13
Yes I read about that, but considering that I was used to work with for loops which is even slower, apply in general is a step ahead anyway afai can see. but thanks for your suggestion. — Andrea Grianti
– Andrea Grianti, Commented May 1, 2022 at 9:21
@Andrea actually, python loops (for or list comprehensions), are usually faster that apply ;) — mozway
– mozway, Commented May 1, 2022 at 10:35

Max · Accepted Answer · 2022-05-01 10:56:19Z

1

if you look at a specific column of a pandas.DataFrame object, you working with a pandas.Series with (in your case) integers as values. Well and integers don't have a sum() method. (Run type(df['A']) to see that you are working with a series and not a data frame anymore when slicing a single column).

The irritating part is that if you work with an actual pandas.DataFrame object, every column is a pandas.Series object and they have a sum() method.

So there are two ways to fix your problem

Work with a pandas.DataFrame and not with a pandas.Series: df[['A']]. The additional brackets force pandas to return a pandas.DataFrame object. (Verify by type(df[['A']])) and use the lambda function just as you did before
use a function rather than a method when using lambda: df['A'].apply(lambda x: np.sum(x)) (assuming that you have imported numpy as np)

I would recommend to go with the second option as it seems to me the more generic and clearer way

However, this is only relevant if you want to apply a certain function to ever element in a pandas.Series or pandas.DataFrame. In your specific case, there is no need to take the detour that your are currently using. Just use df.sum(axis=0). The approach with apply is over complicating things. The reason why this works is that every element of a pandas.DataFrame is a pandas.Series, which as a sum method. But so does a pandas.DataFrame has, so you can use this right away.

The only way, where you actually need to take the way with apply is if you had arrays in every cell of the pandas.DataFrame

edited May 1, 2022 at 10:56

answered May 1, 2022 at 8:53

Max

4,1072 gold badges12 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Andrea Grianti Over a year ago

I tried your second example with df['A'].apply(lambda x: np.sum(x)) but it returns me just the column and not the sum. In any case the first example works and it's interesting.

Max Over a year ago

@AndreaGrianti sorry, I focused too much on explaining your "bug". Actually, you don't need the apply() method and a lambda function. Just use the sum(axis=0) method of a pandas.DataFrame =P

Collectives™ on Stack Overflow

When I use apply function to a single column returns error

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related