Fetching columns dynamically from dataframe , column name would come in variable

Question

I am not able to fetch values for given dynamic columns. Any help ?

var dynamicColumns = "col(\"one\"),col(\"two\"),col(\"three\")"

dataFrame.select(dynamicColumns)

Alper t. Turker · Accepted Answer · 2018-05-18 20:34:50Z

1

Just use names alone:

val dynamicColumns = Seq("one", "two", "three")
dataFrame.select(dynamicColumns map col: _*)

and if you don't have control over the format, use regexp to extract names first

val dynamicColumns = "col(\"one\"),col(\"two\"),col(\"three\")"
val p = """(?<=col\(").+?(?="\))""".r

dataFrame.select(p.findAllIn(dynamicColumns) map col toSeq: _*)

answered May 18, 2018 at 20:34

Alper t. Turker

35.3k9 gold badges89 silver badges118 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Fetching columns dynamically from dataframe , column name would come in variable

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related