Pandas does not fill nan values with empty string

Question

I am trying to replace the Nan values with empty strings. This is the code I have

df.replace({r'[^\x00-\x7F]+': ''}, regex=True, inplace=True)
col_buyername = basicinfo.get('buyer_name')
col_product = basicinfo.get('product_name')
col_quantity = basicinfo.get('quantity')
col_price = basicinfo.get('price')
print(df[col_buyername])

df.loc[:, [col_buyername, col_product]].fillna("", inplace=True)

print('after')
print(df[col_buyername])

the output is

0          NaN
1          Roy Thomas
2          NaN
3          NaN

Name: buyer name, dtype: object

after

0          NaN
1          Roy Thomas
2          NaN
3          NaN

Name: buyer name, dtype: object

Why is the fillna nothing setting it to be blank strings?

NaN means Not a Number. it is float, not str. You can't put string to a float — Karina
– Karina, Commented Sep 17, 2021 at 21:00
it is showing as dtype:object?? how can I put blank str there ? — Derik0003
– Derik0003, Commented Sep 17, 2021 at 21:01
@Karina when i did df.fillna('', inplace=True) it was working — Derik0003
– Derik0003, Commented Sep 17, 2021 at 21:02
oh I think this is your mistake: df = df[[col_buyername, col_product, col_address]].fillna("", inplace=True). Add df = and problem is solved (probably) — Karina
– Karina, Commented Sep 17, 2021 at 21:07
@MichaelO. will this work df[df[[col_buyername, col_product, col_address]].isna()] = "" I got error TypeError: Cannot do inplace boolean setting on mixed-types with a non np.nan value — Derik0003
– Derik0003, Commented Sep 17, 2021 at 21:09

MaxNoe · Accepted Answer · 2021-09-17 21:47:34Z

3

Accessing with square brackets and a list of columns creates a copy, so you modify a temporary object, not the original dataframe.

You have three possible solutions, either pass a dict of column -> replacement for each column, assign or loop over the columns.

Looping

for col in (col_buyername, col_product):
    df[col].fillna('', inplace=True)

Assignment

df[[col_buyername, col_product]] = df[[col_buyername, col_product]].fillna('')

dict

df.fillna({col_buyername: '', col_product: ''}, inplace=True)

The loop and the dict approach should be a little more efficient than the reassignment.

For more info on when pandas created copies and when not, see https://stackoverflow.com/a/53954986/3838691

edited Sep 17, 2021 at 21:47

answered Sep 17, 2021 at 21:05

MaxNoe

15.1k3 gold badges44 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Derik0003 Over a year ago

This still has not fixed my issue.

Derik0003 Over a year ago

Added my question with latest code

MaxNoe Over a year ago

Sorry, yes, even with the loc, it does not work (surprised me). I updated my answer.

Collectives™ on Stack Overflow

Pandas does not fill nan values with empty string

1 Answer 1

Looping

Assignment

dict

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Looping

Assignment

dict

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related