Search for multiple substrings when looping through a series. (Python/Pandas)

Question

Is there a more efficient way to do the following? Ideally, by using only one if statement?

Suppose there is a dataframe with an "author" series, a "comedy" series (default = True), and a "horror" series (default = False). I want to search the author series for "stephen king" and "lovecraft" and in those cases change the value of "comedy" from True to False and change the value of "horror" from False to True.

for count,text in enumerate(df.loc[0:, "author"]):
    if "stephen king" in str(text):
        df.loc[count, "comedy"] = False
        df.loc[count, "horror"] = True
        continue
    elif "lovecraft" in str(text):
        df.loc[count, "comedy"] = False
        df.loc[count, "horror"] = True
        continue

When I try using str.contains(), I get the error "str' object has no attribute 'str'".

James · Accepted Answer · 2021-03-08 21:25:40Z

1

Don't enumerate a data frame, index and slice it.

ix = df.author.str.contains('stephen king')
df.loc[ix, 'comedy'] = False
df.loc[ix, 'horror'] = True

ix = df.author.str.contains('lovecraft')
df.loc[ix, 'comedy'] = False
df.loc[ix, 'horror'] = True

answered Mar 8, 2021 at 21:25

James

37k4 gold badges54 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Nick Over a year ago

In the first example, is ix the index of those that do contain stephen king?

James Over a year ago

Yes. Stephen King for the first block, and Lovecraft for the second.

Arkadiusz · Accepted Answer · 2021-03-08 21:43:07Z

1

You can assign these values with df.loc. If string contains 'stephen king' or 'lovecraft', put False in column 'comedy' and True in column 'horror':

df.loc[df['author'].str.contains('stephen king|lovecraft'), 
       ['comedy', 'horror']] = [False, True]

edited Mar 8, 2021 at 21:43

answered Mar 8, 2021 at 21:34

Arkadiusz

1,8751 gold badge10 silver badges15 bronze badges

Comments

Boskosnitch · Accepted Answer · 2021-03-08 21:50:31Z

1

You can check that the column contains of several values by using

df['Author'].str.contains('|'.join(list_of_authors)

then assign the values using loc.

Ex;

>>> df
          Author  Comedy  Horror
0   stephen king    True   False
1      lovecraft    True   False
2  jonathan ames    True    True
3   stephen king   False    True
4          oprah    True   False
>>> df.loc[df['Author'].str.contains('|'.join(['stephen king','lovecraft']),case=False,na=False),('Comedy','Horror')]=False,True
>>> df
          Author  Comedy  Horror
0   stephen king   False    True
1      lovecraft   False    True
2  jonathan ames    True    True
3   stephen king   False    True
4          oprah    True   False

answered Mar 8, 2021 at 21:50

Boskosnitch

7744 silver badges8 bronze badges

Collectives™ on Stack Overflow

Search for multiple substrings when looping through a series. (Python/Pandas)

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related