Problems implementing properly a Parallel.ForΕach loop and continue other work in the mean time

Question

We are using the .Net Framework 4.8.2. For some part of the app I want to execute many long running CPU and IO bound processes (illustrated by Thread.Sleep()) in parallel. That's why I use the Parallel.ForΕach loop. These results are not needed for some time. That's why I want to continue with different work and "suspend" this task of tasks (so to speak) to a different thread whereas each Parallel.ForΕach is executed again on a different thread. That's why I return the Parallel.ForΕach encapsulated in a Task.Run() statement. The code looks as follows:

internal class Program
{
    static async Task Main(string[] args)
    {
        Console.WriteLine($"Before ParallelForeach on thread: {Environment.CurrentManagedThreadId}"); //Main debug 1
        await TaskContainingParallelForeach();
        Console.WriteLine($"After  ParallelForeach on thread: {Environment.CurrentManagedThreadId}"); //Main debug 2
        int counter = 0;
        while (counter < 7)
        {
            Console.WriteLine($"{counter}. one second gone on thread:  {Environment.CurrentManagedThreadId} at {DateTimeOffset.Now.ToUnixTimeMilliseconds()}");
            Thread.Sleep(1000);
            counter++;
        }

        Console.ReadKey();
    }

    public static Task TaskContainingParallelForeach()
    {
        return Task.Run(() =>
        {
            Console.WriteLine($"Inside ParallelForeach on thread: {Environment.CurrentManagedThreadId}");
            int[] numbers = { 1, 3, 2 };

            Parallel.ForEach(numbers, (num) =>
            {
                Console.WriteLine($"{num} is running on thread: {Environment.CurrentManagedThreadId}");
                Thread.Sleep(num * 1000);
                Console.WriteLine($"{num}    stopped on thread: {Environment.CurrentManagedThreadId}        at {DateTimeOffset.Now.ToUnixTimeMilliseconds()}");
            });
        });
    }}

Which gives me the following output:

Console Output without await:
Before TaskContainingParallelForeach on thread: 1
Inside TaskContainingParallelForeach on thread: 3  // why is -below- first done all the work from the ParallelForeach? 
1 is running on thread: 3
3 is running on thread: 4
2 is running on thread: 6
1    stopped on thread: 3        at 1734365740995
2    stopped on thread: 6        at 1734365741996
3    stopped on thread: 4        at 1734365742996
After  TaskContainingParallelForeach on thread: 3 // why is it the thread with id 3 here - expected was 1
0. one second gone on thread:  3 at 1734365742996
1. one second gone on thread:  3 at 1734365743997
2. one second gone on thread:  3 at 1734365745008
3. one second gone on thread:  3 at 1734365746023
4. one second gone on thread:  3 at 1734365747035
5. one second gone on thread:  3 at 1734365748047
6. one second gone on thread:  3 at 1734365749061

The Parallel.ForEach is run and only after it finished - the while loop starts. The await-operator should behave like: (Original docs)The await operator suspends evaluation of the enclosing async method until the asynchronous operation. My async method here is the TaskContainingParallelForeach. Or is it not?

And why is the thread id for After TaskContainingParallelForeach on thread 3 and not 1? And why is the while loop not beeing executed on thread 1? However, removing the await operator from await TaskContainingParallelForeach(); gets the desired result:

Console output without await:
Before TaskContainingParallelForeach on thread: 1
After  TaskContainingParallelForeach on thread: 1
Inside TaskContainingParallelForeach on thread: 3
1 is running on thread: 3
3 is running on thread: 4
2 is running on thread: 5
0. one second gone on thread:  1 at 1734364096009
1    stopped on thread: 3        at 1734364097021
1. one second gone on thread:  1 at 1734364097036
2    stopped on thread: 5        at 1734364098027
2. one second gone on thread:  1 at 1734364098043
3    stopped on thread: 4        at 1734364099022
3. one second gone on thread:  1 at 1734364099053
4. one second gone on thread:  1 at 1734364100064
5. one second gone on thread:  1 at 1734364101077
6. one second gone on thread:  1 at 1734364102078

This looks great!

all work The Parallel.Foreach loop is started
In the mean-time the while loop started its work
The whileloop is on the expected thread 1.

But thats not how i think await with Task.Run() and Parallel.Foreach should be used together. How can i iprove this code?

Thanks in advance!

You basically don't want to mix Parallel.ForEach with Task. That's what ForEachAsync is for. — Fildor
– Fildor, Commented Dec 16, 2024 at 16:37
@fildor we are currently on .Net Framework 4.8. Parallel.ForeachAsync is not available for this platform — theDrifter
– theDrifter, Commented Dec 16, 2024 at 18:16
Ah, that would have been a valuable piece of info to put into the question. — Fildor
– Fildor, Commented Dec 16, 2024 at 19:21
I'd say the observed behavior is to be expected. The await in the first example makes the execution sequential. That is what it is supposed to do. It just does not block, while doing that. But in the end, if this is going to be production code, I'd still suggest to reconsider the whole ordeal. And not use the two (P.FE and Tasks) together and intertwined. — Fildor
– Fildor, Commented Dec 16, 2024 at 19:29
"Why is the thread id 3 and not 1?" -- You might find answers in this question: What thread runs the code after the await keyword? — Theodor Zoulias
– Theodor Zoulias, Commented Dec 16, 2024 at 21:40

Ivan Petrov · Accepted Answer · 2024-12-16 20:59:20Z

2

I think what you want to do is this:

Console.WriteLine($"Before ParallelForeach on thread: {Environment.CurrentManagedThreadId}"); //Main debug 1
var parallelTask =  TaskContainingParallelForeach();
Console.WriteLine($"After  ParallelForeach on thread: {Environment.CurrentManagedThreadId}"); //Main debug 2
int counter = 0;
while (counter < 7) {
    Console.WriteLine($"{counter}. one second gone on thread:  {Environment.CurrentManagedThreadId} at {DateTimeOffset.Now.ToUnixTimeMilliseconds()}");
    Thread.Sleep(1000);
    counter++;
}

await parallelTask; // want to join here

answered Dec 16, 2024 at 20:59

Ivan Petrov

7,3592 gold badges15 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

theDrifter Dec 17, 2024 at 10:43

Nice one. This syntax var parallelTask = TaskContainingParallelForeach(); is task that is implicilty starting a task? So would it be best to use the Task.Run not in TaskContainingParallelForeach but directly in Main encapsulating this method call?

Ivan Petrov Dec 17, 2024 at 10:44

@theDrifter yes, you can do that too. Just save the task that Task.Run() gives you in a variable and await it later in the code.

Collectives™ on Stack Overflow

Problems implementing properly a Parallel.ForΕach loop and continue other work in the mean time

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related