Delphi Parallel programming - multi threading slow

Question

Good day,

A webcam class has about 30 frames per second, and all of these frames will be saved in a vector(like a queue). Then 3 asynchronous threads will read the queue, and will try to do their jobs(to save these images). Why the queue is overflowing? So the problem is that these threads are slower than the webcam.

Procedure TSaveThread.Execute;
begin
   while not terminated  do
   begin
      elElement:=NIL;

      EnterCriticalSection(CritSect);
         if iElementsLength>=0 then
         begin
            elElement:=vElements[iElementsLength];
            Dec(iElementsLength);
         end;
      LeaveCriticalSection(CritSect);

      if elElement<>NIL then
      begin
         JpegImg.Assign(elElement.bmWebcam) ;
         JpegImg.SaveToFile('Save\'+elElement.sTime+'.jpg') ;
         elElement.Free;
      end;

      Sleep(20);
   end;
end;

Images added to the queue.

//------------------------------------------------------------------------------
Procedure TWebcam.OnSave(Sender:TObject; bmWebcam:TBitmap);
begin
   EnterCriticalSection(CritSect);
      inc(iElementsLength);
      vElements[iElementsLength]:=TElement.Create(bmWebcam);
   LeaveCriticalSection(CritSect);
end;

Creating the threads.

for i:=0 to 2 do
    TSaveThread.Create(false);

The thing is that, these threads are not able to save all these images. Why? How can I improve my threads?

Delphi Version: Delphi XE2

Webcam frame size: 1280x760 or 960x600 Entire source code here: http://pastebin.com/8SekN4TE

General suggestion - get OmniThreadsLibrary - it outright contains parallel queue primitive exactly to make parallel pipelines — Arioch 'The
– Arioch 'The, Commented Dec 12, 2012 at 13:20
2nd suggestion: what is elElement.bmWebcam and JpegImg ? if those are TBitmap - wrappers around windows GDI - ypu'd better avoid them in speed-crytical paths. Use Graphics32 or Vampyre Imaging or some another self-contained library, that does not jumps into Windows GDI layers on every occasion — Arioch 'The
– Arioch 'The, Commented Dec 12, 2012 at 13:23
You can't use Sleep this way and expect decent performance. You need real synchronization. — David Schwartz
– David Schwartz, Commented Dec 12, 2012 at 13:30
depending on your disk and cache it takes an amount of time to create the file and write the content and close it. More threads (parallel read/write) will increase this time using the same physical drive. You should also use sleep(20) only if you did not find an element. — Sir Rufo
– Sir Rufo, Commented Dec 12, 2012 at 13:37
If you're going to be doing a lot of this kind of video processing, you might want to look into the Mitov library: www.mitov.com. We've been VERY happy with it for the past 3 years. It's highly optimized, multi-threaded, easy-to-use and FAST. If you are unable to resolve your speed issues, you could use Mitov or other tools to save the live video data to disk and then read the .avi file back at your own speed. And, even with live data, you can turn on threading in the components and enable blocking and it will handle a queue of the depth you specify. The author provides great support. — RobertFrank
– RobertFrank, Commented Dec 12, 2012 at 18:27

David Heffernan · Accepted Answer · 2012-12-12 14:18:29Z

15

I wrote the following program:

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils, Vcl.Graphics, Vcl.Imaging.jpeg, 
  System.IOUtils, System.Diagnostics;

var
  i: Integer;
  bmp: TBitmap;
  jpeg: TJPEGImage;
  Stopwatch: TStopwatch;

begin
  bmp := TBitmap.Create;
  bmp.SetSize(1280, 760);
  jpeg := TJPEGImage.Create;
  Stopwatch := TStopwatch.StartNew;
  for i := 1 to 100 do begin
    jpeg.Assign(bmp);
    jpeg.SaveToFile('C:\desktop\temp\'+TPath.GetRandomFileName);
  end;
  Writeln(Stopwatch.ElapsedMilliseconds);
  Readln;
end.

It converts 1280x760 pixel bitmaps to JPEG images, and then saves to disk. It does so 100 times. On my machine this takes 9 seconds. That's a throughput of 11 images per second. If I skip the convert to JPEG step and save the bitmap directly I can get a through put of 150 images per second. Clearly the conversion to JPEG is a bottleneck.

You are looking for 30 images per second. Whilst multi-threading helps, I suspect you have a quad core machine. One processor for the webcam, and three for the saving. So if you only have three threads available, then you are probably going to struggle to reach the required throughput of 30 frames per second. Theoretical peak on my machine would be 33. If you don't achieve 30 frames per second then your queue will obviously overflow.

The obvious conclusion is that you need to find a faster JPEG conversion library. I'm quite sure that such libraries exist. For example, I think that libjpeg should be a lot faster.

As for your existing code there are some obvious flaws:

Sleep should usually be avoided. In your case it is suicidal to sleep if the last attempt to pull an image succeeded. Don't do that. You should use a real threaded queue. One that allow proper blocking waits on synchronization objects. It's actually pretty trivial to make one yourself using event objects and your favourite non-threaded queue.
You are holding the lock whilst calling TElement.Create(bmWebcam). That is going to hinder scaling. Assign TElement.Create(bmWebcam) to a local variable, outside the lock. And then assign to the shared data inside the lock.

So, you could check these ideas out by first of all removing the call to Sleep. And then changing TWebcam.OnSave to look like this:

Procedure TWebcam.OnSave(Sender:TObject; bmWebcam:TBitmap);
var
  NewElement: TElement;
begin
  NewElement := TElement.Create(bmWebcam);
  EnterCriticalSection(CritSect);
    inc(iElementsLength);
    vElements[iElementsLength] := NewElement;
  LeaveCriticalSection(CritSect);
end;

These suggestions will help a little, but I think you'll need to tackle the fundamental problem, namely the JPEG conversion.

edited Dec 12, 2012 at 14:18

answered Dec 12, 2012 at 13:59

David Heffernan

616k46 gold badges1.1k silver badges1.5k bronze badges

Sign up to request clarification or add additional context in comments.

14 Comments

David Heffernan Over a year ago

You should be able to do the maths yourself. Theoretical peak throughput for 3 threads on my machine would be 33 frames per second. That perilously close to the minimum throughput you need to achieve. Your poor threading code (Sleep, locks that are held too long) means you can't achieve theoretical peak. Plus there will be a lack of scaling on disk IO, although that's not the main bottleneck.

Arioch 'The Over a year ago

@David - you'd better separate saving and converting tasks :-) PS: blins gueess: three cores - as many threads as he wanted to create :-)

Arioch 'The Over a year ago

@David - you source BMP is all-black - it is not natural target of JPG libraries! can you take some real-life picture for the source BMP ? faces or trees or something that JPEG algo was created for.

David Heffernan Over a year ago

I agree that a single IO thread would be a better design. But it won't help that much since converting is, by my estimate, around 10 times as big a task as saving.

Arioch 'The Over a year ago

i meant Atom. i don;'t remember if i5 3xxx CPUs exist with dual-core and HT, i thought HT starts at 4-cores i5 to i7 boundary. Maybe i need to check the facts though

|

Sir Rufo · Accepted Answer · 2012-12-12 13:57:50Z

4

MultiThreadding will not speed up your Media (HardDrive).

In fact it can slow down with parallel write access.

At first you have to measure if your Media (HardDisk) is able to store an image in less than 33ms - because every 33.333ms you will get a new image from webcam.

If not, you can't expect to get this running.

You should have to (and/or)

use more hard disks (e.g. one per thread)
use more cache (e.g. Cache Controller)
use faster hard disks (e.g. SSD)
use smaller images (decrease resolution)
drop some images

And don't loose time if you need to be fast

if elElement<>NIL then
  begin
     JpegImg.Assign(elElement.bmWebcam) ;
     JpegImg.SaveToFile('Save\' + elElement.sTime + '.jpg' );
     elElement.Free;
  end
else
  Sleep(20);

OTL will not help to get your media faster, but would be much cleaner :o)

So you should have a look at

edited Dec 12, 2012 at 13:57

answered Dec 12, 2012 at 13:52

Sir Rufo

19.3k3 gold badges47 silver badges79 bronze badges

17 Comments

David Heffernan Over a year ago

Disk access is not the bottleneck here.

Sir Rufo Over a year ago

@Arioch'The this theoretical performance can be reached using a filestream and storing all images into. It is a great difference to save 1024 files with 1KB or 1 file with 1MB

Arioch 'The Over a year ago

@user - you should not remove Sleep completely - you should at least get leave it as Sleep(0) aka Yield. As a polite way to let other threads run. Proper solution would be starting-stopping threads as required. Well, after you'd get reasonable speed. As of now, try to think about all the advices here (in backgrond) and go search for fast JPEG libraries, preferably those that can take plain byte pointer as input rather than TBitmap object.

Arioch 'The Over a year ago

@Sir Rufo - VM, antiviri, there are many ways to slow it down. but well, since his images are 150KB average - that is not the issue copletely.

Sir Rufo Over a year ago

@Arioch'The David H pointed out the main problem in his answer

|

Collectives™ on Stack Overflow

Delphi Parallel programming - multi threading slow

2 Answers 2

14 Comments

17 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

14 Comments

17 Comments

Your Answer

Sign up or log in

Post as a guest

Related