Using the Task Parallel Library

Question

Im just starting out writing and using the TPL in .NET 4.5 and was wondering if you could help me.

Basically, my WPF program performs heavy duty work over a set of database records which could be in the range of 5 to 10 million. I want to make use of the TPL library to efficiently manage this work while keeping the UI responsive.

My scenario is as follows, I was planning on using 3 separate tasks to retrieve data of say 3 million each, and feed them to my "worker".

And I want a separate task that takes each item from ALL the items from the above tasks and does some work with it.

So i want a global List of items that all 3 tasks can put items in, my fourth task, should then pick items from this master list one at a time and do some work with it, it should wait if the list is empty for it to be filled up again etc.

Could you please provide some guidance on how to do this with TPL? I am a newbie with the TPL. Some sample code would be good.

This sounds very complicated. What about a Parallel.ForEach loop running over the results stream from the database? — usr
– usr, Commented Nov 17, 2014 at 17:48
It depends on the type of work you need to handle. For CPU intensive computations Parallel.ForEach() can be the right choice. For I/O bound (e.g. files and database connections) operations awaiting async methods will be more efficient. — alexm
– alexm, Commented Nov 17, 2014 at 18:12

littlerufe · Accepted Answer · 2014-11-17 17:45:38Z

1

Consider storing your items from each Task in a global, thread-safe list object, like a ConcurrentStack or ConcurrentQueue.

For more info, see: http://msdn.microsoft.com/en-us/library/system.collections.concurrent(v=vs.110).aspx

answered Nov 17, 2014 at 17:45

littlerufe

37.2k5 gold badges33 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Peter Duniho · Accepted Answer · 2014-11-17 22:07:48Z

If I understand the problem description correctly, you have a producer/consumer scenario where there are three producers, each retrieving independent records from a database, and a single consumer.

If this is the case, then it seems to me that you want to start with BlockingCollection<T>, where T is the type of object the producers are retrieving from the database. The underlying collection to use depends on how you want to process the data. Most common scenarios are FIFO and unordered, represented by ConcurrentQueue<T> and ConcurrentBag<T>, respectively.

Here's a simple code example that demonstrates how that would work:

BlockingCollection<int> consumeFrom = new BlockingCollection<int>();
int producerCount = 3;

for (int i = 0; i < producerCount; i++)
{
    int taskValue = i;

    // Make dummy task for example
    Task.Run(() =>
    {
        for (int j = 0; j < 10; j++)
        {
            Thread.Sleep(1000);
            consumeFrom.Add(taskValue * 10 + j);
        }

        if (Interlocked.Decrement(ref producerCount) == 0)
        {
            consumeFrom.CompleteAdding();
        }
    });
}

foreach (int i in consumeFrom.GetConsumingEnumerable())
{
    Console.WriteLine(i);
}

Note that this uses the default underlying data source of ConcurrentQueue<T>. You can provide other implementations of IProducerConsumerCollection<T> to the BlockingCollection<T> constructor, e.g.:

BlockingCollection<int> consumeFrom =
    new BlockingCollection<int>(new ConcurrentBag<int>());

Collectives™ on Stack Overflow

Using the Task Parallel Library

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related