How to get max length of string column from dataframe using scala?

Question

This can be a really simple question. I am using Spark 1.6 with scala

var DF=hivecontext.sql("select name from myTable")
val name_max_len =DF.agg(max(length($"name"))) // did not work

println(name_max_len)

How can I get max length?

eliasah · Accepted Answer · 2019-09-06 12:26:24Z

12

You should collect result:

import org.apache.spark.sql.functions.max

val df = Seq("foo", "bar", "foobar").toDF("name")
df.agg(max(length($"name"))).as[Int].first
// res0: Int = 6

edited Sep 6, 2019 at 12:26

eliasah

40.5k12 gold badges128 silver badges159 bronze badges

answered Dec 21, 2016 at 19:22

user7327360

1361 silver badge2 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jhon Over a year ago

df.agg(max(length($"name"))).as[Int].first did not work, ".as" became red. Do I need to import any library ?

Collectives™ on Stack Overflow

How to get max length of string column from dataframe using scala?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related