Python Pandas read_csv dtype fails to covert "string" to "float64"

Question

I have a csv file with header = "col1" and 5 values

I intended to set this as a numeric col in pandas so i wrote

import pandas as pd
data = pd.read_csv(r'\test1.csv', dtype = {'col1': 'float64'})

but error message ValueError: could not convert string to float: '/N'

Above code works fine without the slash and last row will turn into "Nan". But without changing my original data value, is there any way to suppress the "slash" and make the code run?

hd1 · Accepted Answer · 2020-09-26 16:46:14Z

1

data = pd.read_csv(r'\test1.csv', dtype = {'col1': 'float64'}, na_values=[r'/N'])

According to the docs, the na_values parameter is a list-like structure of strings that can be recognised as NaN.

answered Sep 26, 2020 at 16:46

hd1

34.9k5 gold badges83 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

gtomer · Accepted Answer · 2024-02-15 17:31:01Z

1

Try with error_bad_lines=False:

data = pd.read_csv(r'\test1.csv', dtype = {'col1': 'float64'}, on_bad_lines='skip')

edited Feb 15, 2024 at 17:31

answered Sep 26, 2020 at 16:41

gtomer

6,6041 gold badge14 silver badges29 bronze badges

2 Comments

Aaron Bramson Over a year ago

'TypeError: read_csv() got an unexpected keyword argument 'error_bad_lines''

Nat Over a year ago

@AaronBramson it is now on_bad_lines='skip'

NYC Coder · Accepted Answer · 2020-09-26 16:53:15Z

0

You can use converters, use errors='coerce' to convert to NaN:

def convert_float(val):
    return pd.to_numeric(val, errors='coerce')

df = pd.read_csv('test.csv', converters={'col1': convert_float})
print(df)

     col1
0   398.0
1  5432.0
2  5986.0
3  8109.0
4     NaN

answered Sep 26, 2020 at 16:53

NYC Coder

7,6443 gold badges14 silver badges25 bronze badges

Collectives™ on Stack Overflow

Python Pandas read_csv dtype fails to covert "string" to "float64"

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related