How to bind dynamic variable inside pyspark selectExpr?

Question

I am trying to bind dynamic variable in PySpark selectExpr.

Code:

name = "random_string"

df.selectExpr("variable_name as $name")

And this throws error.

Thanks.

Alper t. Turker · Accepted Answer · 2018-04-09 09:52:01Z

2

Use standard string formatting:

df.selectExpr(f"variable_name as {name}")

in Python 3.6 or later, or

df.selectExpr("variable_name as {}".format(name))

before.

answered Apr 9, 2018 at 9:52

Alper t. Turker

35.3k9 gold badges89 silver badges118 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

this solution also worked for me inside a F.expr() which was great! I.e. F.expr(f"LEFT( CAST( {dynamic_column_name} AS STRING), 3 )")

Pang · Accepted Answer · 2018-04-09 10:07:45Z

0

I tried

df.selectExpr(f"variable_name as " + name)

and it is working.

Pang

10.2k146 gold badges87 silver badges126 bronze badges

answered Apr 9, 2018 at 9:59

Immanuel Fredrick

5583 gold badges9 silver badges20 bronze badges