Why Spark is not recognizing this time format?

Question

I get null for the timestamp 27-04-2021 14:11 with this code. What mistake am I doing? Why is the timestamp format string DD-MM-yyyy HH:mm not correct here?

df = spark.createDataFrame([('27-04-2021 14:11',)], ['t'])
df = df.select(to_timestamp(df.t, 'DD-MM-yyyy HH:mm').alias('dt'))
display(df)

vladsiv · Accepted Answer · 2021-11-17 22:37:56Z

3

D is for day of the year, and d is for day of the month.

Try this:

df = df.select(F.to_timestamp(df.t, "dd-MM-yyyy HH:mm").alias("dt"))

edited Nov 17, 2021 at 22:37

answered Nov 17, 2021 at 22:35

vladsiv

2,9841 gold badge13 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Why Spark is not recognizing this time format?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related