Removing Partial Strings from Dataframe Column

Question

Similar question to : Replacing part of string in python pandas dataframe

However it wont work!?

Panas 23.4

Given the following df column:

    Expression
    XYZ&(ABC|DEF)
   (HIJ&FTL&JKK)&(ABC|DEF)
   (FML|AXY|AND)&(ABC|DEF)

I want to strip a substring that may be in each column.

flag = '(ABC|DEF)'
andFlag = '&' + flag #the reasoning for doing this is that 'flag' may change


#Below are all different ways I have tried to achieve this, none have worked. 
df['Expression'] = df['Expression'].replace(andFlag, '', regex=True)
df['Expression'] = df['Expression'].apply(lambda x: re.sub(andFlag, '', x))
df['Expression'] = df['Expression'].replace(to_replace=andFlag, value= '', regex=True)
df['Expression'] = df['Expression'].str.replace(andFlag, '')
df['Expression'] = df['Expression'].str.replace(andFlag, '', regex=True)

I have tried all of these functions with and without regex=True to no avail.

Expected output:

    Expression
    XYZ
   (HIJ&FTL&JKK)
   (FML|AXY|AND)

I'm going slightly crazy trying to figure this out, it seems so simple and straightforward.

Ben.T · Accepted Answer · 2019-07-16 17:12:36Z

2

Parenthesis and the vertical bar are special character in regex, so if you want to match these character, you can add a backslash '\' before such as:

flag = '\(ABC\|DEF\)' #see this is changed
andFlag = '&' + flag
print (df['Expression'].replace(andFlag, '', regex=True))

0              XYZ
1    (HIJ&FTL&JKK)
2    (FML|AXY|AND)
Name: Expression, dtype: object

answered Jul 16, 2019 at 17:12

Ben.T

29.7k6 gold badges39 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Zachary Oldham Over a year ago

Regardless of that, shouldn't it have worked when they tried df['Expression'] = df['Expression'].str.replace(andFlag, '')?

Ben.T Over a year ago

@ZacharyOldham no, but you are right, by setting the paramater regex to False (by default it is True) then you get the right answer df['Expression'].str.replace(andFlag, '', regex=False)

Zachary Oldham Over a year ago

Ah, so is regex=True the default?

Ben.T Over a year ago

@ZacharyOldham apparently :)

Erfan · Accepted Answer · 2019-07-16 17:16:12Z

2

Use str.replace and more important, set regex=False for literal matching:

df['Expression'] = df['Expression'].str.replace(andFlag, '', regex=False)

      Expression
0            XYZ
1  (HIJ&FTL&JKK)
2  (FML|AXY|AND)

answered Jul 16, 2019 at 17:16

Erfan

43.3k10 gold badges75 silver badges86 bronze badges

Collectives™ on Stack Overflow

Removing Partial Strings from Dataframe Column

2 Answers 2

4 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related