How to control the number of parallel processes with .par in Scala?

Question

I am using the par.map expression to execute processes in parallel in Scala (SBT).

Consider list.par.map(function(_)) (I am preparing an MWE). This means that function(_) should be applied to all the elements of the list in a parallel fashion. In my example, list has 3 elements. But Scala executes only function(list(1)) and function(list(2)) in parallel, and only afterwards function(list(3)).

Is there a reason for this behaviour? Is there a relation with the fact that the programme is executed on a two-core processor? Or how could you impose to execute all three things in parallel?

scala-lang.org/api/current/scala/collection/parallel/immutable/… — 余杰水
– 余杰水, Commented Nov 11, 2016 at 1:45
Possible duplicate of scala parallel collections degree of parallelism — Andrey
– Andrey, Commented Nov 11, 2016 at 2:01

Community · Accepted Answer · 2017-05-23 11:53:35Z

5

This question has been asked before:

and is well documented:

http://docs.scala-lang.org/overviews/parallel-collections/configuration.html

what you want is something like:

var parallelList = list.par
parallelList.tasksupport = new ForkJoinTaskSupport(
     new scala.concurrent.forkjoin.ForkJoinPool(parlevel))
parallelList.map(function(_))

That said if your running on a 2 core processor you only have two threads (unless the cores are hyper threaded of course) meaning you can't have more than 2 parallel operations at once.

edited May 23, 2017 at 11:53

CommunityBot

11 silver badge

answered Nov 11, 2016 at 1:53

Danny

5412 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Danny Over a year ago

@Karlo are you satisfied with this answer?

Collectives™ on Stack Overflow

How to control the number of parallel processes with .par in Scala?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related