Stripping all trailing empty spaces in a column of a pandas dataframe

Question

I have a pandas DF that has many string elements that contains words like this:

'Frost                              '

Which has many leading white spaces in front of it. When I compare this string to:

'Frost'

I realized that the comparison was False due to the leading spaces.

Although I can solve this by iterating over every element of the pandas DF, the process is slow due to the large number of records I have.

This other approach should work, but it is not working:

rawlossDF['damage_description'] = rawlossDF['damage_description'].map(lambda x: x.strip(''))

So when I inspect an element:

rawlossDF.iloc[0]['damage_description']

It returns:

'Frost                              '

What's going on here?

Anton Protopopov · Accepted Answer · 2015-12-24 06:54:39Z

25

Alternatively you could use str.strip method:

rawlossDF['damage_description'] = rawlossDF['damage_description'].str.strip()

answered Dec 24, 2015 at 6:54

Anton Protopopov

31.9k13 gold badges93 silver badges96 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

I tried this on a 5M rows dataset and it takes twice the time compared to map+lambda

Luis Miguel · Accepted Answer · 2015-12-24 03:39:42Z

24

Replace your function with this:

rawlossDF['damage_description'] = rawlossDF['damage_description'].map(lambda x: x.strip())

You almost had it right, you needed to get rid off the '' inside strip()

answered Dec 24, 2015 at 3:13

Luis Miguel

5,1458 gold badges46 silver badges77 bronze badges