replacing part of a string with value from another column

Question

A pandas DataFrame contains a column with descriptions and placeholders in curly braces:

descr                        replacement
This: {should be replaced}   with this

The task is to replace the text in the curly braces with text from another column in the same row. It's unfortunately not as easy as:

df["descr"] = df["descr"].str.replace(r"{*?}", df["replacement"])

~/anaconda3/lib/python3.6/site-packages/pandas/core/strings.py in replace(self, pat, repl, n, case, flags, regex)
   2532     def replace(self, pat, repl, n=-1, case=None, flags=0, regex=True):
   2533         result = str_replace(self._parent, pat, repl, n=n, case=case,
-> 2534                              flags=flags, regex=regex)
   2535         return self._wrap_result(result)
   2536 

~/anaconda3/lib/python3.6/site-packages/pandas/core/strings.py in str_replace(arr, pat, repl, n, case, flags, regex)
    548     # Check whether repl is valid (GH 13438, GH 15055)
    549     if not (is_string_like(repl) or callable(repl)):
--> 550         raise TypeError("repl must be a string or callable")
    551 
    552     is_compiled_re = is_re(pat)

TypeError: repl must be a string or callable

Community · Accepted Answer · 2020-06-20 09:12:55Z

5

Your code is using the Pandas.Series.str.replace() and it expects two strings to perform the replacement operation, but the second parameter is a Series.

Series.str.replace(pat, repl, n=-1, case=None, flags=0, regex=True)[source]

Replace occurrences of pattern/regex in the Series/Index with some other string. Equivalent to str.replace() or re.sub(). Parameters:

pat : string or compiled regex

repl : string or callable ...

You can correct it using directly the Pandas.Series.replace() method:

df = pd.DataFrame({'descr': ['This: {should be replaced}'],
                   'replacement': 'with this'
                  })
>> df["descr"].replace(r"{.+?}", df["replacement"], regex = True)
0    This: with this

Observation:

I changed a bit of your regexp.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Mar 17, 2019 at 13:54

Daniel Labbe

2,0193 gold badges20 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Wiktor Stribiżew Over a year ago

You'd better use a r"{.+?}" or r"{[^{}]*}" pattern.

Daniel Labbe Over a year ago

Thanks, @WiktorStribiżew, you are correct! I haven't given so much emphasis on the regexp part. Just edited.

jezrael · Accepted Answer · 2019-03-17 14:02:51Z

4

Use list comprehension with re.sub, especially if performance is important:

import re

df['new'] = [re.sub(r"{.*?}", b, a) for a, b in zip(df['descr'], df['replacement'])]
print (df)
                        descr replacement              new
0  This: {should be replaced}   with this  This: with this
1                This: {data}         aaa        This: aaa

edited Mar 17, 2019 at 14:02

answered Mar 17, 2019 at 13:56

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

2 Comments

clstaudt Over a year ago

Does leaving pandas for list comprehension have better performance than using pandas.Series.replace ?

jezrael Over a year ago

@clstaudt - sure, best test it, str operation in pandas are slow.

Collectives™ on Stack Overflow

replacing part of a string with value from another column

2 Answers 2

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related