Excluding pandas rows based on a String value

Question

I have a pandas table that contains a column which has a String datatype. What I need is to exclude any rows that have "Not found" as a string inside them from the data frame. I am currently trying:

df[df.some_column != "Not found"] but that is not working

Looking forward to replies.

Sample data:

card_number effective_date  expiry_date grouping_name       Ac. Year code
0       1206090    28 Sep 2012  21 Aug 2013    Dummy no.1  201213
1       1206090    21 Feb 2013  21 Aug 2013   Dummy no.2   201213
2       1206090    28 Sep 2012  30 Nov 2012    Dummy no.3  201213
3       1206090    03 Dec 2012  21 Aug 2013    Dummy no.3  201213
4       1206090    23 Apr 2013  31 Aug 2013   Dummy no.4   201213
5       1206090    28 Sep 2012  21 Aug 2013    Dummy no.5  201213
6       1206090    28 Sep 2012  21 Aug 2013    Dummy no.6  201213
7       1206090    24 Oct 2012  07 Aug 2013     Not found  201213
8       1206090    08 Jan 2013  08 Jan 2013     Not found  201213
9       1206090    08 Jan 2013  31 Aug 2013     Not found  201213
10    Not found    03 Jul 2013  21 Aug 2013    Dummy no.1  201213
11    Not found    03 Jul 2013  21 Aug 2013   Dummy no.2   201213

Extra note: My string matching must be extremely weird... When running df[grouping_name] != "Not found" I get true for 7,8,9... does anyone know why?

you need to use str.contains; see here; as in df.some_column.str.contains('Not Found', na=False, regex=False) — behzad.nouri
– behzad.nouri, Commented Mar 4, 2014 at 14:47
TypeError: contains() got an unexpected keyword argument 'regex'... besides that would find only the columns with the value... I want the column without such values — Maciej Musialek
– Maciej Musialek, Commented Mar 4, 2014 at 14:56
add ~ to the beginning and drop regex=False; i guess that is added in 13.1; — behzad.nouri
– behzad.nouri, Commented Mar 4, 2014 at 15:04
This is weird... when matching and seeing whether some rows will return true for this check... rows 7,8,9 are still false regardless of the string matching. — Maciej Musialek
– Maciej Musialek, Commented Mar 4, 2014 at 15:16

Amit · Accepted Answer · 2014-03-04 15:45:33Z

1

Try:

df[df['some_column'] != "Not found"]

Solution With sample data:

df = pd.read_csv("data.csv")
df

    card_number effective_date  expiry_date grouping_name   Ac. Year code
0    1206090     28 Sep 2012     21 Aug 2013     Dummy no.1  201213
1    1206090     21 Feb 2013     21 Aug 2013     Dummy no.2  201213
2    1206090     28 Sep 2012     30 Nov 2012     Dummy no.3  201213
3    1206090     03 Dec 2012     21 Aug 2013     Dummy no.3  201213
4    1206090     23 Apr 2013     31 Aug 2013     Dummy no.4  201213
5    1206090     28 Sep 2012     21 Aug 2013     Dummy no.5  201213
6    1206090     28 Sep 2012     21 Aug 2013     Dummy no.6  201213
7    1206090     24 Oct 2012     07 Aug 2013     Not found   201213
8    1206090     08 Jan 2013     08 Jan 2013     Not found   201213
9    1206090     08 Jan 2013     31 Aug 2013     Not found   201213
10   Not found   03 Jul 2013     21 Aug 2013     Dummy no.1  201213
11   Not found   03 Jul 2013     21 Aug 2013     Dummy no.2  201213


df[df['grouping_name'] != 'Not found']

card_number effective_date  expiry_date grouping_name   Ac. Year code
0    1206090     28 Sep 2012     21 Aug 2013     Dummy no.1  201213
1    1206090     21 Feb 2013     21 Aug 2013     Dummy no.2  201213
2    1206090     28 Sep 2012     30 Nov 2012     Dummy no.3  201213
3    1206090     03 Dec 2012     21 Aug 2013     Dummy no.3  201213
4    1206090     23 Apr 2013     31 Aug 2013     Dummy no.4  201213
5    1206090     28 Sep 2012     21 Aug 2013     Dummy no.5  201213
6    1206090     28 Sep 2012     21 Aug 2013     Dummy no.6  201213
10   Not found   03 Jul 2013     21 Aug 2013     Dummy no.1  201213
11   Not found   03 Jul 2013     21 Aug 2013     Dummy no.2  201213

edited Mar 4, 2014 at 15:45

answered Mar 4, 2014 at 14:50

Amit

20.6k7 gold badges51 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Maciej Musialek Over a year ago

No luck unfortunately

Amit Over a year ago

Can you provide sample data?

Maciej Musialek Over a year ago

There you go... grouping_name can't have Not found in it.

Amit Over a year ago

I am getting expected output

Maciej Musialek Over a year ago

My string comparisons are unfortunately thinking that the Not found in the table doesn't equal to regular Not found string for some reason

|

Collectives™ on Stack Overflow

Excluding pandas rows based on a String value

1 Answer 1

9 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

9 Comments

Your Answer

Sign up or log in

Post as a guest

Related