Replace NaN in pandas DataFrame with random strings without using fillna

Question

I have a pandas DataFrame like below

      NAME      EMAIL      HEIGHT      WEIGHT

1     jlka       NaN        170          70

2     qwer     eee@ttt      180          80

3     ioff       NaN        175          75

4     iowu     iou@add      170          60

And I want to replace NaN in 'EMAIL' column with random strings with no duplicates, which does not necessarily contain @.

I have tried to make a def which generates random strings but NaNs were replaced with the same random string since I used 'fillna' method after all.

It seems like, and as I saw other Q$As, def in fillna works only once and replace all the NaN with the same values or strings came out from the def.

Should I try 'for' sentence to replace them one by one?

Or is there a more Pythonic way to replace them?

I'm curious, what's the problem with using None when there isn't email? — Guilhem L.
– Guilhem L., Commented Feb 20, 2020 at 9:38

andre · Accepted Answer · 2020-02-20 02:51:02Z

3

you could try something like this:

import pandas as pd
from numpy import nan
import random
import string

df = pd.DataFrame({
    'Name': ['aaa','bbb','CCC'],
    'Email': [nan,'ddd',nan]})

def processNan (x):
    return ''.join(random.choice(string.ascii_uppercase + string.digits) for x in range(10))

df['Email'] = df['Email'].apply(lambda x: processNan(x) if x is nan else x)

answered Feb 20, 2020 at 2:51

andre

4471 gold badge3 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user12929266 Over a year ago

Sorry I am too new here. I marked it to your answer!

sacuL · Accepted Answer · 2020-02-20 02:58:01Z

1

You could use pd.util.testing.rands_array, passing it the length of your desired string as the first (nchars) argument and the number of NaNs as the second (size) argument:

df.loc[df.EMAIL.isna(), "EMAIL"] = pd.util.testing.rands_array(10, sum(df.EMAIL.isnull()))      

>>> df                                                                                              

   NAME       EMAIL  HEIGHT  WEIGHT
1  jlka  YxzVaC38uw     170      70
2  qwer     eee@ttt     180      80
3  ioff  33kyDArtip     175      75
4  iowu     iou@add     170      60

pd.util.testing.rand_array could be replaced with any function that returns a list or array with a specific size.

answered Feb 20, 2020 at 2:58

sacuL

51.6k9 gold badges88 silver badges115 bronze badges

Collectives™ on Stack Overflow

Replace NaN in pandas DataFrame with random strings without using fillna

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related