How does mapValues translate to parallel collections in Scala

Question

I'm trying to do a simple aggregation in a collection I've grouped. Like this:

case class Foo(key:String, value:Int)
val minPerKey = lotsOfFoos.groupBy(_.key).mapValues(_.minBy(_.value))

Now I want to do the same in parallel but ParMap doesn't have a mapValues function.

Why is this so, and how should I do it alternatively?

derekjw · Accepted Answer · 2013-01-24 20:48:53Z

3

Scala 2.10 has mapValues for ParMap.

But there isn't a whole lot of benefit to using mapValues with a ParMap. The mapValues method just gives you a new Map that applies your function when reading values, it doesn't actually change the values in the Map. So you would need to do some other action that would force the creation of a new Map with the mapped values.

answered Jan 24, 2013 at 20:48

derekjw

3862 silver badges3 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

2 revs, 2 users 91% · Accepted Answer · 2013-01-24 20:42:22Z

1

Since mapValues is not present on ParMapLike, there are two options left:

Just use map: .map{case (key, value) => (key->value.minBy(_.value)) }
Get out of the parallel universe: .toMap.mapValues(_.minBy(_.value))

I don't know why mapValues isn't just implemented on ParMapLike

edited Jan 24, 2013 at 20:42

community wiki

2 revs, 2 users 91%
iwein

1 Comment

om-nom-nom Over a year ago

I would use .seq instead of .toMap because it has better documenting intent.

Collectives™ on Stack Overflow

How does mapValues translate to parallel collections in Scala

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related