explode column with comma separated string in Spark SQL

Question

I have below dataset

+----+-----------+
|col1|       col2|
+----+-----------+
|   1|val1, val2 |
|   2|val3, val4 |
+----+-----------+

Consider all values as String Now i want to convert it into below dataset

+----+-----------+
|col1|       col2|
+----+-----------+
|   1|val1       |
|   1|val2       |
|   2|val3       |
|   2|val4       |
+----+-----------+

How can I achieve this?

How can we write the same for redshift. We need to create a udf for it. Can someone help me to create the udf — Etisha
– Etisha, Commented May 30, 2021 at 2:50

Kombajn zbożowy · Accepted Answer · 2020-10-05 08:29:11Z

8

Use split to parse comma-separated values as an array, then explode to rearrange array elements into separate rows.

df.withColumn("col2", explode(split($"col2", ","))).show

answered Oct 5, 2020 at 8:29

Kombajn zbożowy

10.8k5 gold badges33 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Praveenrajan27 Over a year ago

I am using SQL editor and this worked for me SELECT explode(split(col1, ",")) FROM table

Collectives™ on Stack Overflow

explode column with comma separated string in Spark SQL

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related