How to implement multi-threading and parallel execute several tasks?

Question

I am new to threaded programming. I have to run few tasks in PARALLEL and in Background (so that main UI execution thread remain responsive to user actions) and wait for each one of them to complete before proceeding further execution.

Something like:

foreach(MyTask t in myTasks)
{
  t.DoSomethinginBackground(); // There could be n number of task, to save 
                               // processing time I wish to run each of them 
                               // in parallel
}

// Wait till all tasks complete doing something parallel in background


Console.Write("All tasks Completed. Now we can do further processing");

I understand that there could be several ways to achieve this. But I am looking for the best solution to implement in .Net 4.0 (C#).

Well, MyTask could be anything. It doesn't matter here. In my actual solution I have Array of URL and I have to fetch the HTML(Web Scrapping) for each one of them. — Aakash
– Aakash, Commented Aug 14, 2013 at 9:10

Nolonar · Accepted Answer · 2013-08-14 09:35:33Z

8

To me it would seem like you want Parallel.ForEach

Parallel.ForEach(myTasks, t => t.DoSomethingInBackground());

Console.Write("All tasks Completed. Now we can do further processing");

You can also perform multiple tasks within a single loop

List<string> results = new List<string>(myTasks.Count);
Parallel.ForEach(myTasks, t =>
{
    string result = t.DoSomethingInBackground();
    lock (results)
    { // lock the list to avoid race conditions
        results.Add(result);
    }
});

In order for the main UI thread to remain responsive, you will want to use a BackgroundWorker and subscribe to its DoWork and RunWorkerCompleted events and then call

worker.RunWorkerAsync();
worker.RunWorkerAsync(argument); // argument is an object

edited Aug 14, 2013 at 9:35

answered Aug 14, 2013 at 9:15

Nolonar

6,2043 gold badges39 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Aakash Over a year ago

Thanks Nolonar, seems a neat solution so far. But what if each tasks returns a value(say string value) and I have to consolidate return string from each task before proceeding further? In other words, can each task return value on completion while other tasks are still busy?

Aakash Over a year ago

Nolonar, the second code block seems a nice solution to me. For running the task in background can I use BackgroundWorker with PLINQ?

Nolonar Over a year ago

@Aakash I'm not sure what you mean with using a BackgroundWorker with PLINQ. You can use PLINQ inside a BackgroundWorker, if that's what you're asking.

Aakash Over a year ago

Yes that's exactly what I meant. I created a BackGroundWorker Instance and in DoWork() method I used PLINQ to run each task in parallel. Each task returns a Collection of String, which I stored in a global collection of string using AddRange() method.

Aakash Over a year ago

Wondering if DoWork could send a value as Argument to RunWorkerCompleted event. Like if I declare a local variable in DoWork and pass that in RunWorkerCompleted ?

|

cuongle · Accepted Answer · 2013-08-14 09:21:56Z

1

You can use Task library to complete:

 string[] urls = ...;
 var tasks = urls.Select(url => Task.Factory.StartNew(() => DoSomething(url)));

To avoid locking UI Thread, you can use ContinueWhenAll in .NET 4.0:

Task.Factory.ContinueWhenAll(tasks.ToArray(), _ => 
    Console.Write("All tasks Completed. Now we can do further processing");
);

If you are in the latest version of .NET, you can use Task.WhenAll instead

edited Aug 14, 2013 at 9:21

answered Aug 14, 2013 at 9:12

cuongle

75.5k30 gold badges155 silver badges212 bronze badges

1 Comment

Aakash Over a year ago

Still I think that the second block of code should be run on other thread than Main.

Huy Do · Accepted Answer · 2013-08-14 09:17:04Z

0

If you use Net 4.0 or up, refer to the Parallel class and Task class. Joseph Albahari wrote very clear book about that: http://www.albahari.com/threading/part5.aspx#_Creating_and_Starting_Tasks

answered Aug 14, 2013 at 9:17

Huy Do

446 bronze badges

1 Comment

Aakash Over a year ago

Thanks Huy, surely will do, seems it is time to dive into multi-threading. Sooner or later one have to do it. ;-)

Collectives™ on Stack Overflow

How to implement multi-threading and parallel execute several tasks?

3 Answers 3

7 Comments

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

7 Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related