Find a null value and drop from a dataframe in Pandas

Question

Hi I have a dataframe like this with 500+ rows.

    company_url company tag_line    product data
0   https://angel.co/billguard  BillGuard   The fastest smartest way to track your spendin...   BillGuard is a personal finance security app t...   New York City · Financial Services · Security ...
1   https://angel.co/tradesparq Tradesparq  The world's largest social network for global ...   Tradesparq is Alibaba.com meets LinkedIn. Trad...   Shanghai · B2B · Marketplaces · Big Data · Soc...
2   https://angel.co/sidewalk   Sidewalk    Hoovers (D&B) for the social era    Sidewalk helps companies close more sales to s...   New York City · Lead Generation · Big Data · S...
3   https://angel.co/pangia Pangia  The Internet of Things Platform: Big data mana...   We collect and manage data from sensors embedd...   San Francisco · SaaS · Clean Technology · Big ...
4   https://angel.co/thinknum   Thinknum    Financial Data Analysis Thinknum is a powerful web platform to value c...   New York City · Enterprise Software · Financia...

What I want to do is that I want to find null in the "data" column and drop the row from the dataframe. I wrote my code for it but I believe it didn't work as expected since the number of rows didn't change. Could someone help me on this?

My code:

for item in bigdata_comp_dropped.iterrows():
    if item[1][4] == "":
        bigdata_comp_dropped.drop(item[1])

This example tsv doesn't have any NaN values in the data column... :( — Andy Hayden
– Andy Hayden, Commented May 7, 2015 at 2:48
Both of the solutions below would also be much faster than using iterrows. — Alexander
– Alexander, Commented May 7, 2015 at 3:33

Andy Hayden · Accepted Answer · 2015-05-07 02:47:01Z

1

You can keep only the notnull values using a boolean mask:

df = df[df["data"].notnull()]

answered May 7, 2015 at 2:47

Andy Hayden

378k110 gold badges640 silver badges546 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

EdChum Over a year ago

This is the more sensible/readable mask as the ~ operator is not so readable +1

Kathirmani Sukumar · Accepted Answer · 2015-05-07 02:44:37Z

1

Try

bigdata_filtered = bigdata_comp_dropped[~bigdata_comp_dropped['data'].isnull()]

answered May 7, 2015 at 2:44

Kathirmani Sukumar

11k5 gold badges36 silver badges35 bronze badges

Collectives™ on Stack Overflow

Find a null value and drop from a dataframe in Pandas

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related