CSV Date Parsing in Pandas

Question

I am trying to parse dates together from the following sample set of data

No,year,month,day,hour,pm2.5,DEWP,TEMP,PRES,cbwd,Iws,Is,Ir
1,2010,1,1,0,NA,-21,-11,1021,NW,1.79,0,0
2,2010,1,1,1,NA,-21,-12,1020,NW,4.92,0,0
3,2010,1,1,2,NA,-21,-11,1019,NW,6.71,0,0
4,2010,1,1,3,NA,-21,-14,1019,NW,9.84,0,0

My code is as follows:

dateparser = lambda x: pd.datetime.strptime(x, "%Y %m %d %H")`
dataset = pd.read_csv("raw.csv", parse_dates=['year', 'month', 'day', 'hour'], index_col = 0,date_parser=mydateparser)

It's throwing this error:

ValueError: Missing column provided to 'parse_dates': 'day, hour, month, year'

Can someone help me understand why I am getting this error

Dishin Goyani · Accepted Answer · 2020-10-24 16:53:43Z

1

Try it passing as dict or list of list

dataset = pd.read_csv("raw.csv", parse_dates={'date':['year', 'month', 'day',
'hour']}, index_col = 1, date_parser=dateparser)

Or

dataset = pd.read_csv("raw.csv", parse_dates=[['year', 'month', 'day',
'hour']], index_col = 1, date_parser=dateparser)

PS: Was not able to reproduce the same error, but the proposed solution should work fine.

answered Oct 24, 2020 at 16:53

Dishin Goyani

7,7533 gold badges33 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Alex22 Over a year ago

Thanks for your response passing as a list of list works fine but for whatever reason I had to reference the columns by their index number and no the col name.

Collectives™ on Stack Overflow

CSV Date Parsing in Pandas

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related