Replace null character in a string in Spark sql

Question

I need to replace a null character in a spark sql string.

I can't find an equivalent of COLLATE in spark sql.

Can you help?

Do you mean coalesce?

Andrew
– Andrew

2020-01-29 15:37:05 +00:00
Commented Jan 29, 2020 at 15:37 — Andrew
– Andrew, Commented Jan 29, 2020 at 15:37
show some sample data

Karthik
– Karthik

2020-01-29 16:03:21 +00:00
Commented Jan 29, 2020 at 16:03 — Karthik
– Karthik, Commented Jan 29, 2020 at 16:03

mpSchrader · Accepted Answer · 2020-10-14 07:28:24Z

0

If you are looking for a way to replace every NULL character in a string you can use regexp_replace. Depending on the encoding and programming language you use the NULL character can be different: \000, \x00, \z, or \u0000. [Wikipedia]

null_character = u'\u0000'
replacement = ' '
df = df.withColumn('e', F.regexp_replace(F.col('columnX'), null_character, replacement))

Related response: https://stackoverflow.com/a/41152572/14338716

answered Oct 14, 2020 at 7:28

mpSchrader

9525 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Replace null character in a string in Spark sql

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related