Replace a value of a row by another if null in Apache Spark Java

Question

I use Apache Spark 1.6.2 in Java.

I have a DataFrame containing:

If the business is not closed, then the value in close_date is null.

I would like to:

Can I ask Spark to do it or should I do it manually?

zero323 · Accepted Answer · 2016-07-08 22:37:18Z

8

All you need here is a coalesce:

import static org.apache.spark.sql.functions.*;

df.withColumn("last_date_business", coalesce(col("close_date"), current_date()));

answered Jul 8, 2016 at 22:37

zero323

331k108 gold badges982 silver badges958 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

What is the equivalent in java?

@zero323 , I am using spark-sql -2.4.1v , i dont find coalesce() function ... what is the alternative I can use ? please suggest.