pandas find strings in common among Series and return keywords

Question

I would like to improve this previous question about searching strings in pandas Series based on a Series of keywords. My question now is how to get the keywords found in the DataFrame rows as a new column. The Keywords Series "w" is:

Skilful
Wilful
Somewhere
Thing
Strange

and the DataFrame "df" is:

User_ID;Tweet
01;hi all
02;see you somewhere
03;So weird
04;hi all :-)
05;next big thing
06;how can i say no?
07;so strange
08;not at all

The following solution worked well for masking the DataFrame:

import re
r = re.compile(r'.*({}).*'.format('|'.join(w.values)), re.IGNORECASE)
masked = map(bool, map(r.match, df['Tweet']))
df['Tweet_masked'] = masked

and return this:

   User_ID              Tweet Tweet_masked
0        1             hi all        False
1        2  see you somewhere         True
2        3           So weird        False
3        4         hi all :-)        False
4        5     next big thing         True
5        6  how can i say no?        False
6        7         so strange         True
7        8         not at all        False

Now I'm looking for a result like this:

User_ID;Tweet;Keyword
01;hi all;None
02;see you somewhere;somewhere
03;So weird;None
04;hi all :-);None
05;next big thing;thing
06;how can i say no?;None
07;so strange;strange
08;not at all;None

Thanks in advance for your support.

eumiro · Accepted Answer · 2015-01-28 15:12:06Z

1

How about replacing

masked = map(bool, map(r.match, df['Tweet']))

with

masked = [m.group(1) if m else None for m in map(r.match, df['Tweet'])]

answered Jan 28, 2015 at 15:12

eumiro

214k36 gold badges307 silver badges264 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Fabio Lamanna Over a year ago

Thanks! It works! I wonder, how may I find multiple keywords in one string? I mean, if the df is updated by a new row containing: "09; such a strange thing" how can I get from the mask both "strange" and "thing"? I tried the r.search() method to search in the all string but with no results...

eumiro Over a year ago

@fblamanna - something like: df['Tweet'].map(lambda x: tuple(re.findall(r'({})'.format('|'.join(w.values)), x))) which returns tuples?

Collectives™ on Stack Overflow

pandas find strings in common among Series and return keywords

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related