How to convert object data type to float in pandas

Question

I have a data frame wherein a column is of "object" data type. I use pd.to_numeric() with errors = 'coerce' to convert this to "float" data type. However, the converted column appears as NaN for all entries. If I let errors = 'ignore', none of the entries are converted to float. Is there something I am missing? The following is the code snippet:

pd.to_numeric(df['gender'],errors = 'coerce')

The column df['gender'] comprises 'Male' and 'Female' entries. I would like to convert these to 'float' data type.

Thank you!

tdy · Accepted Answer · 2021-12-27 05:12:59Z

2

to_numeric can only convert numeric-ish things. For example it can convert the string '10' into the number 10, but it can't convert something like 'Male' into a number.

Instead use pd.factorize:

df['gender'] = pd.factorize(df['gender'])[0].astype(float)

Or Series.factorize:

df['gender'] = df['gender'].factorize()[0].astype(float)

The first element of factorize contains the integer codes, so then we convert them astype(float).

Or as you commented, Series.map also works:

df['gender'] = df['gender'].map({'Male': 0, 'Female': 1}).astype(float)

edited Dec 27, 2021 at 5:12

answered Dec 27, 2021 at 5:06

tdy

42k42 gold badges124 silver badges125 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Nanda Over a year ago

Thank you! One quick comment: is this not the same as df.gender.replace({'Male': 1, 'Female': 0})? Nonetheless, it works fine.

tdy Over a year ago

Yep map is also fine, just a bit more manual. I also added an explanation for why to_numeric doesn't work here.

Nanda Over a year ago

There is something glaring with pd.factorize(). The NaN entries in a given column are converted to '-1'. I tried including "na_sentinel=None", but it did not work. Doing so could be unfavorable in certain applications. Perhaps it is better to employ 'map' method, wherein NaN entries are retained and could be filled/replaced using imputation methods.

tdy Over a year ago

Good point if there are NaNs. In this case with only 2 categories + NaN, map would be simplest. If you had a ton of categories + NaN, you could factorize automatically and chain a replace, e.g. pd.Series(df['gender'].factorize()[0]).replace(-1, np.nan).

Collectives™ on Stack Overflow

How to convert object data type to float in pandas

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related