Pandas.read_csv(,parse_dates,) forces to make a column of float data into string

Question

I tried to read a csv file

Test.csv

using

df = pd.read_csv('Test.csv',index_col=0,parse_dates=[0],header=None,names=['Open','Close','High','Low','Vol','Mon'])
df['Open']

But for unknown reason the result of column 'Open' is a list of string data. The other columns such as 'High' 'Low' are fine, just the 'Open' is wrong. So I tried add dtype=floatat the end.

Still the data of 'Open' are string. And I figured out it's the problem of parse_dates cause without it it's fine.

without parse_dates

But I still need to make date's type datetime64 so I have to do

df = pd.read_csv('Test.csv',index_col=[0],header=None,names=['Open','Close','High','Low','Vol','Mon'])
df.index = pd.to_datetime(df.index)
df['Open']

to make things right. But I wonder what causes it? it seems to be a bug to me. So I post it out see if anyone had seen the same problem.

Please don't post images of code, text, data. They should be listed inside the question itself. — thuyein
– thuyein, Commented Jun 12, 2020 at 5:02
Try parse_dates=True. This way pandas will try to parse index by default and see if that help at all. — NotAName
– NotAName, Commented Jun 12, 2020 at 5:14
@ThuYeinTun Sorry I didn't know that. Will try that next time — Abraham Reinhardt
– Abraham Reinhardt, Commented Jun 12, 2020 at 7:10
@pavel thanks, if using parse_dates=True it woks. tho sometimes I may need to specify which column to be converted to datetime64 type, But i think in most cases ==True would just do the work, thanks — Abraham Reinhardt
– Abraham Reinhardt, Commented Jun 12, 2020 at 7:15

Chrstfer CB · Accepted Answer · 2020-06-12 05:50:29Z

1

Perhaps using df = pd.read_csv('Test.csv',infer_date_format=True,...) from the pandas docs. Also, in the same section under the dtype option it looks like you can pass converter functions, so perhaps also try:

df = pd.read_csv('Test.csv', dtype={'Open': pd.to_datetime},...)`

Not sure the underlying reason, without the text pasted to somewhere like bpaste.net i cant test it.

answered Jun 12, 2020 at 5:50

Chrstfer CB

759 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Abraham Reinhardt Over a year ago

I tried but unfortunately still not working. I'm new to python so I'm still learning. Sometimes things are tricky to me. I just tried type df = pd.read_csv('Test.csv',names=['Open','Close','High','Low','Vol','Mon']) turns out i don't nee to type index_col=0 and header=None then I tried df = pd.read_csv('Test.csv',parse_dates=True,names=['Open','Close','High','Low','Vol','Mon']) which works fine so I guess i will be using this if nothing wrong happens in the future. Thank you very much for your help

Abraham Reinhardt Over a year ago

sorry I didn't know how to accept answer so i come back and late check it

Collectives™ on Stack Overflow

Pandas.read_csv(,parse_dates,) forces to make a column of float data into string

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related