Replace certain columns in Dataframe with null

Question

I want to remove a certain columns based on high null values. In few columns there is a value(in this case "Select) which is equivalent to null. I want to replace this with null so that i can calculate the null % and removes columns accordingly.

Lead Profile    City
Select          Select
Select          Select
Potential Lead  Mumbai
Select          Mumbai
Select          Mumbai

Tried using replace function as well as map function.

leads['Specialization'] = leads['Specialization'].replace('Select', "NaN")

This Code just replaces the string with string and doesnt actually impute null values

def colmap(x):
     return x.map({"Select": "Nan"})

df[['Lead Profile']] = df[['Lead Profile']].apply(colmap)

This code replaces all the values with NAN

pandas requires numpy, so you can safely add an import numpy as np statement. — Leporello
– Leporello, Commented Jun 6, 2019 at 9:14
Thanks guys for the insights. Will keep these in mind for all the null value situations going forward. cheers! — Mohd Abdul Mukheem
– Mohd Abdul Mukheem, Commented Jun 6, 2019 at 9:16

Olivier Cruchant · Accepted Answer · 2019-06-06 08:18:43Z

1

to replace value with nulls:

df['col'] = df['col'].replace('value', np.nan)

otherwise to directly return only columns which have less than N times the Select values, you can use this:

df2 = df[[col for col in df.columns if len(df[df[col] == 'Select']) < N]]

edited Jun 6, 2019 at 8:18

answered Jun 6, 2019 at 8:11

Olivier Cruchant

4,06718 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

pmarcol · Accepted Answer · 2019-06-06 08:13:44Z

1

Besides Olivier's answer, in case you import data with read_csv or read_excel, these methods have na_values argument:

df = pd.read_csv('file.csv', na_values=['Select'])

answered Jun 6, 2019 at 8:13

pmarcol

4632 silver badges9 bronze badges

Collectives™ on Stack Overflow

Replace certain columns in Dataframe with null

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related