WrappedArray to Array in HiveQL

Question

I'm using collect_list to group some column like:

val res = hiveContext.sql("SELECT date, time, collect_list(id) AS id_list FROM table1 GROUP BY date, time")

The id_list returns as a WrappedArray:

WrappedArray(1,2,1,2)
WrappedArray(4,3,4)
WrappedArray(6,7,6,7,6)

However I'm passing the id_list into myFunc that takes an Array[Double] as input:

def myFunc(xs: Array[Double]) {...}

My question is how can I call myFunc correctly to parse the id_list. I'm having something like:

res.collect.foreach(x => myFunc(x(2)))

but it's giving me an type mismatch; found : Any required: Array[Double] error.

What is the correct way to implicitly convert the WrappedArray into an Array or how can I call myFunc in an optimized way?

Thanks!

scala.collection.mutable.WrappedArray has a def array: Array[T], so invoke like this res.collect.foreach(x => myFunc(x(2).array)) may solve you problem. — Allen Chou
– Allen Chou, Commented Aug 12, 2016 at 2:05
@AllenChou guess the main question here is how to let x(2) in type Any be recognized as an Array first instead of converting it directly. — Rock
– Rock, Commented Aug 12, 2016 at 5:02

Rock · Accepted Answer · 2016-08-12 16:54:28Z

1

The main question here is rather to explicitly convert Any to WrappedArray[Double] that you may do as:

x(2).asInstanceOf[mutable.WrappedArray[Double]].toArray

in your case. Or you may use pattern matching for type conversion that you may refer How to use a Scala match expression instead of isInstanceOf (to match types) for details.

answered Aug 12, 2016 at 16:54

Rock

2,9778 gold badges38 silver badges47 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Thank you. I did try your solution and it worked. Initially it worked without the mutable too. Solved my issue.