Spark: DataFrame renaming columns and updating row values

Question

I want to rename a column using spark data frame and add a condition.

For example: I have a column "Code" which I want to rename to "Source Code" and I want to add a condition. When the values in the column equal "A" it should be renamed to "Agent" and when the value is "O" it should be renamed to "Other".

Check this conversation http://stackoverflow.com/questions/35592917/renaming-column-names-of-a-data-frame-in-spark-scala — Noman Khan
– Noman Khan, Commented Jan 5, 2017 at 6:24

Natalia · Accepted Answer · 2017-01-04 22:04:50Z

2

renaming column

df.withColumnRenamed("oldName", "newName")

updating value

val newColumnValue = df("c")... //some calculation
df.withColumn("updatedValue", newColumn)

answered Jan 4, 2017 at 22:04

Natalia

4,5322 gold badges27 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Spark: DataFrame renaming columns and updating row values

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related