Parallel.Foreach loop gets different result than For loop?

Question

I've written simple for loop iterating through array and Parallel.ForEach loop doing the same thing. However, resuls I've get are different so I want to ask what the heck is going on? :D

  class Program
  {
    static void Main(string[] args)
    {
      long creating = 0;
      long reading = 0;
      long readingParallel = 0;
      for (int j = 0; j < 10; j++)
      {
        Stopwatch timer1 = new Stopwatch();
        Random rnd = new Random();
        int[] array = new int[100000000];

        timer1.Start();
        for (int i = 0; i < 100000000; i++)
        {
          array[i] = rnd.Next(5);
        }
        timer1.Stop();

        long result = 0;
        Stopwatch timer2 = new Stopwatch();

        timer2.Start();
        for (int i = 0; i < 100000000; i++)
        {
          result += array[i];
        }
        timer2.Stop();

        Stopwatch timer3 = new Stopwatch();
        long result2 = 0;
        timer3.Start();
        Parallel.ForEach(array, (item) =>
          {
            result2 += item;
          });

        if (result != result2)
        {
          Console.WriteLine(result + " - " + result2);
        }

        timer3.Stop();

        creating += timer1.ElapsedMilliseconds;
        reading += timer2.ElapsedMilliseconds;
        readingParallel += timer3.ElapsedMilliseconds;
      }



      Console.WriteLine("Create : \t" + creating / 100);
      Console.WriteLine("Read: \t\t" + reading / 100);
      Console.WriteLine("ReadP: \t\t" + readingParallel / 100);

      Console.ReadKey();
    }
  }

So in the condition I get results: result = 200009295; result2 = 35163054;

Is there anything wrong?

Kirill Shlenskiy · Accepted Answer · 2016-04-27 08:46:17Z

4

The += operator is non-atomic and actually performs multiple operations:

load value at location that result is pointing to, into memory
add array[i] to the in-memory value (I'm simplifying here)
write the result back to result

Since a lot of these add operations will be running in parallel it is not just possible, but likely that there will be races between some of these operations where one thread reads a result value and performs the addition, but before it has the chance to write it back, another thread grabs the old result value (which hasn't yet been updated) and also performs the addition. Then both threads write their respective values to result. Regardless of which one wins the race, you end up with a smaller number than expected.

This is why the Interlocked class exists.

Your code could very easily be fixed:

Parallel.ForEach(array, (item) =>
{
    Interlocked.Add(ref result2, item);
});

Don't be surprised if Parallel.ForEach ends up slower than the fully synchronous version in this case though. This is due to the fact that

the amount of work inside the delegate you pass to Parallel.ForEach is very small
Interlocked methods incur a slight but non-negligible overhead, which will be quite noticeable in this particular case

edited Apr 27, 2016 at 8:46

answered Apr 27, 2016 at 8:35

Kirill Shlenskiy

9,60730 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mastenka Over a year ago

Thank you for wide explanation. Of course you're absolutely right!

Collectives™ on Stack Overflow

Parallel.Foreach loop gets different result than For loop?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related