How can I achieve reduction using OpenMP Tasks?

Question

I have this OpenMP code that performs a simple reduction:

for(k = 0; k < m; k++) 
{
      #pragma omp parallel for private(i) reduction(+:mysum) schedule(static) 
      for (i = 0; i < m; i++)
      {
           mysum += a[i][k] * a[i][k];
      }
}

I want to create a code equivalent to this one, but using OpenMP Tasks. Here is what I tried by following this article:

for(k = 0; k < m; k++) 
{
    #pragma omp parallel reduction(+:mysum)
    {
         #pragma omp single 
         {
                  for (i = 0; i < m; i++) 
                  {
                        #pragma omp task private(i) shared(k)
                        {
                                partialSum += a[i][k] * a[i][k];
                        }
                   }
         }

         #pragma omp taskwait
         mysum += partialSum;
     }
 }

The variable partialSum is declared as threadprivate and it's also a global variable:

int partialSum = 0;
#pragma omp threadprivate(partialSum)

a is a simple array of ints (m x m).

The problem is that when I run the code above (the one with tasks) multiple times, I get different results.

Do you have an idea on what should I change to make this work?

Thank you respectfully

In your second code, partialSum is shared among all your threads. The reduction handles making private copies of mysum and combining them at the end, but the same treatment is not extended to partialSum, which therefore is the subject of a data race. The slide deck you linked uses a threadprivate() directive to address that problem. I'm not certain that would be sufficient for you, but it would at least resolve the data race. — John Bollinger
– John Bollinger, Commented Nov 24, 2018 at 16:08
I don't think that partialSum is shared among all threads because I also declare it as threadPrivate, exactly as in that article — Cosmin Ioniță
– Cosmin Ioniță, Commented Nov 25, 2018 at 17:17
I guess I overlooked that at the end of the question. Please, present a minimal reproducible example exhibiting the problem. Not only will that reduce the likelihood of such misunderstandings, but the additional context may prove important. — John Bollinger
– John Bollinger, Commented Nov 25, 2018 at 17:43
What misunderstandings? I stated the fact that partialSum is threadPrivate from the beginning. I think that you should have read the entire question from the beginning. — Cosmin Ioniță
– Cosmin Ioniță, Commented Nov 26, 2018 at 8:23

Zulan · Accepted Answer · 2018-11-27 08:00:47Z

2

private variables are uninitialized (at least not initialized by their outside value). i should be firstprivate.

If you just get rid of private(i) shared(k) everything is correct by default. k comes from outside of the parallel section and thus is implicitly shared in the parallel section. This also makes it implicitly shared in the task generating construct. Right now i is also shared/shared. If you define it locally instead, (for (int i...), it becomes implicitly private to the parallel section and thus implicitly firstprivate in the task generating construct.

You should also add

#pragma omp atomic
mysum += partialSum;

On the other hand, you don't necessarily need the taskwait (see this answer)

Note that the talk uses firstprivate correctly.

edited Nov 27, 2018 at 8:00

answered Nov 26, 2018 at 12:27

Zulan

22.8k7 gold badges57 silver badges117 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Cosmin Ioniță Over a year ago

You say about k that is shared in the task generating construct but then you say that it's firstprivate. It's a bit unclear to me what's happening with k

Zulan Over a year ago

Sorry there was a typo and I misread your code regarding i. The first sentence (implicitly shared/shared applies to k). You should define i locally then it becomes private/firstprivate implicitly which is what you want. You may also specify the data-sharing attributes manually (just with firstprivate(i) instead.

Cosmin Ioniță Over a year ago

Understood. Thanks a lot for your answer!

Zulan Over a year ago

@CosminIoniță please note that the recently released OpenMP 5.0 natively supports task_reduction, but it is a bit different and I have no experience with it yet. AFAIK it is only supported in the Intel 18.0 compiler yet.

Cosmin Ioniță Over a year ago

Great to know! Thanks, @Zulan!

Collectives™ on Stack Overflow

How can I achieve reduction using OpenMP Tasks?

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related