Use regular expression to extract elements from a pandas data frame

Question

From the following data frame:

d = {'col1':['a-1524112-124', 'b-1515', 'c-584854', 'a-15154']}

df = pd.DataFrame.from_dict(d)

My ultimate goal is to extract the letters a, b or c (as string) in a pandas series. For that I am using the .findall() method from the re module, as shown below:

# import the module
import re
# define the patterns
pat = 'a|b|c'

# extract the patterns from the elements in the specified column
df['col1'].str.findall(pat)

The problem is that the output i.e. the letters a, b or c, in each row, will be present in a list (of a single element), as shown below:

Out[301]: 
0    [a]
1    [b]
2    [c]
3    [a]

While I would like to have the letters a, b or c as string, as shown below:

I know that if I combine re.search() with .group() I can get a string, but if I do:

df['col1'].str.search(pat).group()

I will get the following error message:

AttributeError: 'StringMethods' object has no attribute 'search'

Using .str.split() won't do the job because, in my original dataframe, I want to capture strings that might contain the delimiter (e.g. I might want to capture a-b)

Does anyone know a simple solution for that, perhaps avoiding iterative operations such as a for loop or list comprehension?

have you tried this: pandas.pydata.org/pandas-docs/stable/generated/… ? — Stael
– Stael, Commented Jan 7, 2019 at 15:17

Dani Mesejo · Accepted Answer · 2019-01-07 15:19:11Z

1

Use extract with capturing groups:

import pandas as pd

d = {'col1':['a-1524112-124', 'b-1515', 'c-584854', 'a-15154']}

df = pd.DataFrame.from_dict(d)

result = df['col1'].str.extract('(a|b|c)')

print(result)

Output

answered Jan 7, 2019 at 15:19

Dani Mesejo

62.2k6 gold badges57 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

BENY · Accepted Answer · 2019-01-07 15:19:51Z

0

Fix your code

pat = 'a|b|c'
df['col1'].str.findall(pat).str[0]
Out[309]: 
0    a
1    b
2    c
3    a
Name: col1, dtype: object

answered Jan 7, 2019 at 15:19

BENY

324k22 gold badges176 silver badges250 bronze badges

Comments

A l w a y s S u n n y · Accepted Answer · 2019-01-07 15:20:26Z

0

Simply try with str.split() like this- df["col1"].str.split("-", n = 1, expand = True)

import pandas as pd
d = {'col1':['a-1524112-124', 'b-1515', 'c-584854', 'a-15154']}
df = pd.DataFrame.from_dict(d)
df['col1'] = df["col1"].str.split("-", n = 1, expand = True) 
print(df.head())

Output:

  col1
0    a
1    b
2    c
3    a

answered Jan 7, 2019 at 15:20

A l w a y s S u n n y

38.8k9 gold badges68 silver badges120 bronze badges

4 Comments

BCArg Over a year ago

In fact this does the job for this sample example, though, in my original df, splitting the observations won't do the job because what I want to capture might contain the - symbol. i.e. I also want to capture something like a-b

A l w a y s S u n n y Over a year ago

@BCArg then edit your question and let us know more about what are the possible value of your col1?

A l w a y s S u n n y Over a year ago

@BCArg how does df['col1'].str.findall(pat).str[0] capture a-b?

BCArg Over a year ago

it will in case I specify that I want to capture it. In my original data frame I have a handful of parameters which I want to capture, hence I don't need to be sophisticated with regular expressions.

Collectives™ on Stack Overflow

Use regular expression to extract elements from a pandas data frame

3 Answers 3

Comments

Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related