Delphi loop speed question

There are a couple obvious problems with your code.

First off, you're wasting a lot of CPU cycles computing the same values over and over again. The AA..ZZ values aren't going to change, so there's no need to build them over and over. Try something like this: Create a third TStringList. Go through and fill it with all possible AA..ZZ permutations with your double loop. Once that's over with, loop through and merge this list of precomputed strings with the values in slist1. You should see a pretty big boost from that.

(Or, if time is absolutely at a premium, write a minor little program that will compute the permutation list and save it to a textfile, then compile that into your app as a string resource which you can load at runtime.)

Second, and this is probably what's killing you, you shouldn't have the ProcessMessages and the Sleep calls in the innermost loop. Sleep(1); sounds like it means "sleep for 1 milisecond", but Windows doesn't offer that sort of precision. What you end up getting is "sleep for at least 1 milisecond". It releases the CPU until Windows gets back around to it, which is usually somewhere on the order of 16 miliseconds. So you're adding a delay of 16 msec (plus as long as ProcessMessages takes) into a very tight loop that probably takes only a few microseconds to execute the rest of its code.

If you need something like that to keep the UI responsive, it should be in the outermost loop, not an inner one, and you probably don't even need to run it every iteration. Try something like if ch mod 100 = 0 then //sleep and process messages here. Craig's suggestion to move this task to a worker thread would also help, but only if you know enough about threads to get it right. They can be tricky.

You should surround your code with slist2.BeginUpdate() and slist2.EndUpdate(), to stop TStringList from doing extra processing.

From my experience, you would get a very large improvement by using fewer ProcessMessages(); Sleep(1); statements, as suggested in other answers.

Try moving it to just below the first for loop, and see what improvement you get.

An example of how you might use a secundary thread to do the heavy work.

Note that for the 35 items you mention, it is really not worth it to start another thread. For a few thousand items the game changes. Processing 10.000 items takes 10 seconds on my desktop computer.

Some benefits of multithreading:

• the main thread stays responsive.
• the calculation can be stopped at will.

and offcourse some pitfalls:

• care must be taken (in its current implementation) to not mess with the passed stringlists while the seeding is running.
• multithreading adds complexity and are source for hard to find bugs.

paste below code in our favorite editor and you should be good to go.

procedure TForm1.btnStartClick(Sender: TObject);
var
  I: Integer;
begin
  //***** Fill the sourcelist
  FSource := TStringList.Create;
  FDestination := TStringList.Create;
  for I := 0 to 9999 do
    FSource.Add(Format('Test%0:d', [I]));

  //***** Create and fire Thread
  FSeeder := TSeeder.Create(FSource, FDestination);
  FSeeder.OnTerminate := DoSeederDone;
  FSeeder.Resume;
end;

procedure TForm1.btnStopClick(Sender: TObject);
begin
  if Assigned(FSeeder) then
    FSeeder.Terminate;
end;

procedure TForm1.DoSeederDone(Sender: TObject);
var
  I, step: Integer;
begin
  I := 0;
  step := 0;
  while I < FDestination.Count do
  begin
    //***** Don't show every item. OutputDebugString is pretty slow.
    OutputDebugString(PChar(FDestination[I]));
    Inc(step);
    Inc(I, step);
  end;
  FSource.Free;
  FDestination.Free;
end;

{ TSeeder }

constructor TSeeder.Create(const source, destination: TStringList);
begin
  //***** Create a suspended, automatically freed Thread object.
  Assert(Assigned(source));
  Assert(Assigned(destination));
  Assert(destination.Count = 0);
  inherited Create(True);
  FreeOnTerminate := True; //***** Triggers the OnTerminate event
  FSource := source;
  FDestination := destination;
end;

procedure TSeeder.Execute;
var
  I, J: Integer;
  AString: string;
begin
  FDestination.BeginUpdate;
  try
    FDestination.Capacity := FSource.Count * 26 * 26;
    for I := 0 to Pred(FSource.Count) do
    begin
      AString := FSource[I];
      for J := 0 to Pred(26 * 26) do
      begin
        FDestination.Add(AString + Char(J div 26 + $41) + Char(J mod 26 + $41));
        if Terminated then Exit;
      end;
    end;
  finally
    FDestination.EndUpdate;
  end;
end;

I would see if you can do it in one loop as per comment. Also try doing it in a thread so you can eliminate the Application.ProcessMessages and Sleep calls without blocking the UI.

I know this doesn't specifically answer your question, but if you are interested in Delphi algorithm's, Julian Bucknall (CTO of DevExpress) wrote the definitive Delphi algorithms book

Tomes of Delphi: Algorithms and Data Structures:

• Chapter 1: What is an algorithm?
• Chapter 2: Arrays
• Chapter 3: Linked Lists, Stacks, and Queues
• Chapter 4: Searching
• Chapter 5: Sorting
• Chapter 6: Randomized Algorithms
• Chapter 7: Hashing and Hash Tables
• Chapter 8: Binary Trees
• Chapter 9: Priority Queues and Heapsort
• Chapter 10: State Machines and Regular Expressions
• Chapter 11: Data Compression
• Chapter 12: Advanced Topics

You can also get his EZDSL (Easy Data Structures Library) for Delphi 2009 and Delphi 6-2007.

try this sample code - hope this will help a little (Delphi 5 Ent./WinXP)

procedure TForm1.Button1Click(Sender: TObject);
var
   i,k: Integer;
   sourceList, destList: TStringList;
   ch1, ch2: char;
begin
   destList := TStringList.Create;
   sourceList := TStringList.Create;

   //some sample data but I guess your list will have 1000+
   //entries?
   sourceList.Add('Element1');
   sourceList.Add('Element2');
   sourceList.Add('Element3');

   try
      i := 0;
      while i < (26*26) do
      begin
         if (i mod 100) = 0 then
            Application.ProcessMessages;

         ch1 := char(65 + (i div 26));
         ch2 := char(65 + (i mod 26));

         for k := 0 to sourceList.Count -1 do
            destList.Add(Format('%s-%s%s', [sourceList.Strings[k], ch1, ch2]));
         Inc(i);
      end;

      Memo1.Lines.AddStrings(destList);
   finally
      FreeAndNil(destList);
      FreeAndNil(sourceList);
   end;
end;    

--Reinhard

If you want events to be processed during your loop, such as the Cancel button being clicked, calling Application.ProcessMessages is sufficient. If you call that regularly but not too regularly, e.g. 50 times per second, then your application will remain responsive to the Cancel button without slowing down too much. Application.ProcessMessages returns pretty quickly if there aren't any messages to be processed.

This technique is appropriate for relatively short computations (a few seconds) that you would expect the user to wait on. For long computations a background thread is more appropriate. Then your application can remain fully responsive, particularly if the user has a multi-core CPU.

Calling Sleep in the main thread does not allow your application to process events. It allows other applications to do something. Calling Sleep really puts your application (the calling thread, actually) to sleep for the requested amount of time or the remainder of the thread's time slice, whichever is larger.

Use Delphi backgroundworker Component for this purpose can be better than thread.it is a easy and event based.features of backgroundworker(additional use Thread) :

• Use Event based code. no need create class
• Add Progress to process

Sample Code:

procedure TForm2.FormCreate(Sender: TObject);
var
  I: Integer;
begin
  FSource := TStringList.Create;
  FDestination := TStringList.Create;

end;
procedure TForm2.Button1Click(Sender: TObject);
var
  I: Integer;
begin
  try
    FSource.BeginUpdate;
    FSource.Clear;
    for I := 0 to 9999 do
      FSource.Add(Format('Test%0:d', [I]));
    BackgroundWorker1.Execute;
  finally
    FSource.EndUpdate;
  end;

end;



procedure TForm2.StopButtonClick(Sender: TObject);
begin
  BackgroundWorker1.Cancel;
end;



procedure TForm2.FormDestroy(Sender: TObject);
begin
 FreeAndNil(FSource);
 FreeAndNil(FDestination);
end;


procedure TForm2.BackgroundWorker1Work(Worker: TBackgroundWorker);
var
  I, J: Integer;
  AString: string;
begin
  FDestination.BeginUpdate;
  try
    FDestination.Capacity := FSource.Count * 26 * 26;
    for I := 0 to Pred(FSource.Count) do
    begin
      AString := FSource[I];
      for J := 0 to Pred(26 * 26) do
      begin
        FDestination.Add(AString + Char(J div 26 + $41) + Char(J mod 26 + $41));
        if Worker.CancellationPending then
          Exit;
      end;
      if I mod 10 = 0 then
        TThread.Sleep(1);
      Worker.ReportProgress((I * 100) div FSource.Count);
    end;
    Worker.ReportProgress(100); // completed

  finally
    FDestination.EndUpdate;
  end;
end;

procedure TForm2.BackgroundWorker1WorkProgress(Worker: TBackgroundWorker;
  PercentDone: Integer);
begin
  ProgressBar1.Position := PercentDone;
end;

if you are looking for pure speed just unroll the code into a single loop and write each line as a separate assignment. You could write a program to write the lines for you automatically then copy and past them into your code. This would essentially be about the fastest method possible. Also turn off all updates as mentioned above.

procedure Tmainform.btnSeederClick(Sender: TObject);
var
  ch,ch2:char;
  i:integer;
  slist1, slist2:TStrings;
begin
  slist1:= TStringList.Create;
  slist2:= TStringList.Create;

  slist1.Text :=queuebox.Items.Text;

  slist2.BeginUpdate() 
     for I := 0 to slist1.Count - 1 do
        begin
        application.ProcessMessages; // so it doesn't freeze the application in long loops.  Not 100% sure where this should be placed, if at all.
         if cancel then Break; 
         slist2.Add(slist1.Strings[i]+'AA');
         slist2.Add(slist1.Strings[i]+'AB');
         slist2.Add(slist1.Strings[i]+'AC');
         ...
         slist2.Add(slist1.Strings[i]+'ZZ');
        end;
slist2.EndUpdate()
insertsingle(slist2,queuebox);
freeandnil(slist1);
freeandnil(slist2);
end;