how to run concurrent tasks in c#

Question

I have written a console application to process image files and match them to database records.

The image files are all located in a file store in different folders one folder per day so the structure of the filestore would be

FileStore -> Day1 -> Images -------------------> Indexfile.csv

So my program will open each folder grab the indexfile and match the image to a record in the database.

The folders are quite large some of them having 90000+ images, so I would like to run up to 5 different task each grabbing a different folder and processing them in parallel, I've tried creating 5 tasks and that work fine except that I don't know how to start a new task once one of the 5 tasks has finish.

Any pointers would be appreciated.

Thanks.

I'm voting to close this question as off-topic because you need to read the docs yourself, put your effort first and come back with specific questions. — Matías Fidemraizer
– Matías Fidemraizer, Commented Dec 16, 2015 at 10:19
Alternatively, show some actual code about which you are doubtful. — Codor
– Codor, Commented Dec 16, 2015 at 10:20
This is quite broad for Stack Overflow. However - are all these folders on the same physical disk? If so, you're unlikely to be able to read more than one folder at a time, and if the processing is mostly disk bound there's little point in running up parallel tasks. — James Thorpe
– James Thorpe, Commented Dec 16, 2015 at 10:20
who's to say I haven't put the effort in... the two answers provided have given me the solution... — franklores
– franklores, Commented Dec 16, 2015 at 10:33
Please make sure you post your code with your question. If you have a minimal reproducible example then we can answer easily. — Enigmativity
– Enigmativity, Commented Dec 16, 2015 at 11:20

Rovak · Accepted Answer · 2015-12-16 10:21:55Z

The easy way is to create a list with all the files and use Parallel.ForEach, checkout the docs https://msdn.microsoft.com/en-us/library/dd460720%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396

Example Code:

// A simple source for demonstration purposes. Modify this path as necessary.
String[] files = System.IO.Directory.GetFiles(@"C:\Users\Public\Pictures\Sample Pictures", "*.jpg");
String newDir = @"C:\Users\Public\Pictures\Sample Pictures\Modified";
System.IO.Directory.CreateDirectory(newDir);

// Method signature: Parallel.ForEach(IEnumerable<TSource> source, Action<TSource> body)
// Be sure to add a reference to System.Drawing.dll.
Parallel.ForEach(files, (currentFile) => 
    {
        // The more computational work you do here, the greater 
        // the speedup compared to a sequential foreach loop.
        String filename = System.IO.Path.GetFileName(currentFile);
        var bitmap = new Bitmap(currentFile);

        bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone);
        bitmap.Save(Path.Combine(newDir, filename));

        // Peek behind the scenes to see how work is parallelized.
        // But be aware: Thread contention for the Console slows down parallel loops!!!

        Console.WriteLine("Processing {0} on thread {1}", filename, Thread.CurrentThread.ManagedThreadId);
        //close lambda expression and method invocation
    });


// Keep the console window open in debug mode.
Console.WriteLine("Processing complete. Press any key to exit.");
Console.ReadKey();

@franklores You can't, but you can upvote as many as you like

M.kazem Akhgary · Accepted Answer · 2015-12-16 10:32:13Z

1

You should use WhenAny(non-blocking) or WaitAny(blocking) in list of tasks. then remove any completed task from list and add new ones.

List<Task<...>> tasks = ... // put your 5 tasks inside this list.

while(condintion)
{
    await Task.WhenAny(tasks);
    tasks.RemoveAll(t => t.IsCompleted);
    while(tasks.Count < 5 && condintion)
    {
        Task<...> newTask = ... // run your task
        tasks.Add(newTask);
    }
}

put the appropriate condition so that is only false when all folders are taken.

edited Dec 16, 2015 at 10:32

answered Dec 16, 2015 at 10:25

M.kazem Akhgary

19.4k8 gold badges68 silver badges126 bronze badges

1 Comment

M.kazem Akhgary Over a year ago

note that this is just the general idea. you need to modify this for your needs ;) @franklores

Collectives™ on Stack Overflow

how to run concurrent tasks in c#

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related