How we pass variable which is a floating-point value to spark expr?
myVariable=0.50;
expr("data.mark = myVariable")
This is what I expect after substituting myVariable.
expr("data.mark = 0.50"). --- value 0.50 should be used during execution .
Add a comment
|
2 Answers
String interpolation? Like so:
expr(String.format("data.mark = %s", myVariable))
3 Comments
Neethu Lalitha
Does java support adding s? I get a compilation error.
Lars Skaug
My answer was for Scala. I have updated it for Java now.
Neethu Lalitha
No it is java unfortunately
This is one way I found to do it, making f string. Two input variables within an expression.
beginDate = '2000-01-01'
endDate = '2050-12-31'
df = spark.createDataFrame([(1,)], ["id"])
df1 = df.withColumn(
"date",
explode(expr(f"sequence(to_date('{beginDate}'), to_date('{endDate}'), interval 1 day)"))
)
