using dataframe.apply to call unique function on each column

Question

I've been wanting to make the following work so as to have a simple story for executing pandas.DataFrame.someColumnName.unique() function on each column within a pandas.DataFrame.

df.apply(func=unique, axis=0)  # error NameError: name 'unique' is not defined

Is there some trick i'm overlooking to get this working or an alternative solution given the following to do something similar but using type() function on each column in pandas.DataFrame works.

df.apply(func=lambda x: type(x[0]), axis=0)

Note that i have been able to make the following work but doesn't seem to be a way in python to make single line for loops and i find the apply statement a better self documenting implementation.

for col in df.columns: 
    df[col].unique()

Motivation is when doing exploratory data analysis [eda] on a new dataset I want to not only output types associated with each column but also a listing of the unique values that exist in each column. This will define next steps needed to implement data wrangling code that deals with holes/NaN values and garbage values. — myusrn
– myusrn, Commented Jan 23, 2018 at 19:57
Yeah pandas does not like it when the result of the function passed to apply is a different size for different columns in the dataframe.... which I assume is very likely happening with your data. — Tasko Olevski
– Tasko Olevski, Commented Jan 23, 2018 at 19:59

akuiper · Accepted Answer · 2018-01-23 19:57:31Z

9

unique is not a registered function in global environment, you can use set for this purpose:

df.apply(set)

Or if using unique, refer it from pandas, also you'd better convert the result to a list, as there is no guarantee all columns contain same number of unique elements:

df.apply(lambda x: pd.unique(x).tolist())

answered Jan 23, 2018 at 19:57

akuiper

216k33 gold badges362 silver badges379 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Allen Qin · Accepted Answer · 2018-01-23 20:34:54Z

3

If you need a one liner loop, you can do:

{e:df[e].unique() for e in df.columns}

answered Jan 23, 2018 at 20:34

Allen Qin

20k9 gold badges55 silver badges68 bronze badges

1 Comment

myusrn Over a year ago

This approach prints the df['colname'].unique() output for each column all merged together vs each output separated by a crlf. The "for col in df.columns: df[col].unique()" one liner loop syntax, and two line syntax with the call to execute on each loop indented, both I find require executing any line of python script code following it in order for it to execute the loop. I found I could use the no-op "pass" call to satisfy that need. Is that a well known for loop expected matter?

Collectives™ on Stack Overflow

using dataframe.apply to call unique function on each column

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related