Detect Floating Point values in Pandas Dataframe

Question

I have a dataframe column containing integers, floating numbers and strings. I want to process this column depending on what type of data is present in a particular record.

Now the problem is that, I am able to separate out integer records by Series.str.isnumeric() call, but floating numbers return False here. How can I separate ints & floats together. Here is a basic code:

import numpy as np
import pandas as pd

d = {'A' : ['1234', '12.16', '1234m']}
df= pd.DataFrame(d)
df.A.str.isnumeric()

I get [True False False] as of now, I expect to get [True, True, False].

Erfan · Accepted Answer · 2019-10-16 13:25:39Z

5

Use pd.to_numeric with argument errors="coerce" and check which values come out not NaN:

pd.to_numeric(df['A'],errors='coerce').notna()

0     True
1     True
2    False
Name: A, dtype: bool

If you want to use str.isnumeric, pandas does not automatically recognizes the . as a decimal, so we have to replace it:

df['A'].str.replace('\.', '').str.isnumeric()

0     True
1     True
2    False
Name: A, dtype: bool

If I think ahead and what you want to do, you can write a try except to convert each element to it's type without losing any rows to NaN:

def convert_numeric(x):
    try:
        return pd.to_numeric(x)
    except:
        return x

df['A'].apply(convert_numeric)

0     1234
1    12.16
2    1234m
Name: A, dtype: object

If we then check the types per value, we see it's mixed type now:

df['A'].apply(convert_numeric).apply(type)

0      <class 'numpy.int64'>
1    <class 'numpy.float64'>
2              <class 'str'>
Name: A, dtype: object

edited Oct 16, 2019 at 13:25

answered Oct 16, 2019 at 13:18

Erfan

43.3k10 gold badges75 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Himanshu Gautam Over a year ago

I would have preferred a method where i am not forcing string to become numeric, thereby introducing nan. But I guess this will work as of now.

vb_rises · Accepted Answer · 2019-10-16 13:20:19Z

0

def my_func(x):
    try:
        float(x)
    except ValueError:
        return False
    return True

df['A'].apply(my_func)

0     True
1     True
2    False

answered Oct 16, 2019 at 13:20

vb_rises

1,9071 gold badge11 silver badges14 bronze badges

Collectives™ on Stack Overflow

Detect Floating Point values in Pandas Dataframe

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related