PHP - Multithread a function with pthreads

Question

I'm currently working with pthreads to implement multithreading on a very demanding function. So far I got this working:

class Operation extends Thread {
  public function __construct($arg) {
    $this->arg = $arg;
  }  
  public function run() {
    if ($this->arg) {
      $parameters = $this->arg;
      echo my_function($parameters[0],$parameters[1]);
    }
  }
}
$stack = array();
foreach ($work as $operation) { $stack[] = new Operation($operation); };
foreach ($stack as $t) { $t->start(); };

It outputs the results directly. I'd like to have my results stored one by one in an array (in the same order would be nice) but of course this does not work :

class Operation extends Thread {
  public function __construct($arg) {
    $this->arg = $arg;
  }  
  public function run() {
    if ($this->arg) {
      $parameters = $this->arg;
      $results[] = my_function($parameters[0],$parameters[1]);
    }
  }
}
$stack = array();
foreach ($work as $operation) { $stack[] = new Operation($operation); };
foreach ($stack as $t) { $t->start(); };
var_dump($results);

Any help would be appreciated.

Details:

my_function outputs an UTF-8 string.

Hardly. It's huge and all the variables are in French. I refactored this part of the code for the comfort of the reader. Do you have a specific idea maybe ? It outputs a string of UTF-8 text. — 760 mmHg
– 760 mmHg, Commented Jun 1, 2015 at 13:59
So what you're saying is that you want the worker threads to output results synchronously, one after another, and not as they finish? — Mjh
– Mjh, Commented Jun 1, 2015 at 14:28
That would be best, but not if that's at the expense of speed. I can output an index number with the result and then sort everything after. — 760 mmHg
– 760 mmHg, Commented Jun 1, 2015 at 14:31
Well, you can't force threads to synchronously return results. That makes them useless, they need to return results as they finish and you don't know when that happens. That's why you must give them an index and synchronize with all of them, collecting their output, their index and then sorting the result in main context that invoked them. — Mjh
– Mjh, Commented Jun 1, 2015 at 14:35

Joe Watkins · Accepted Answer · 2015-06-03 05:19:45Z

2

The basic problem is that arrays are not thread safe, pthreads provides array-like interfaces on all Threaded objects; This means you can use Threaded objects in place of arrays in a multi-threaded context.

<?php

function demanding(...$params) {
  /* you have parameters here */
  return array(rand(), rand());
}

class Task extends Collectable {
  public function __construct(Threaded $result, $params) {
    $this->result = $result;
    $this->params = $params;
  }

  public function run() {
    $this->result[] = 
      demanding(...$this->params);
  }

  protected $result;
  protected $params;
}

$pool = new Pool(16);

$result = new Threaded();

while (@$i++<16) {
  $pool->submit(
    new Task($result, $argv));
}

$pool->shutdown();

var_dump($result);
?>

There isn't a built-in way to do a multi-threaded sort, so the simplest thing to do is sort your result when the threads are finished.

answered Jun 3, 2015 at 5:19

Joe Watkins

17.2k5 gold badges45 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

760 mmHg Over a year ago

"I am the author of pthreads for PHP." Well, I guess I couldn't have a better answer, thank you very much for your help! I'm intending to use pthreads quite often from now on, would you have some online resources that you might recommend to help me learn how to use it in an efficient and elegant way? I studied the examples in the pthreads Github, that's what helped me to make my code above, but I have to say that it wasn't easy!

760 mmHg · Accepted Answer · 2015-06-02 11:35:39Z

1

I'm sure someone can do better but it's (apparently) working and it's quite a boost of performance. The "waiting for thread" part is very inefficient and inelegant, any help will be appreciated!

First of all, check if you have pthreads installed with phpinfo() or install it https://php.net/manual/en/pthreads.installation.php .

$key = 0;//We initialise the key to sort our results
foreach($iteration as $parameters) {//We make a loop to create the task list 
  $arguments[] = array($key,$parameters,$_POST['stuff'],$another_parameter,...);//We index ALL the parameters our function need for each task ($_POST, $_FILES, $_GET, user defined...) in a nice array
  ++$key;//We increment our key
};

class operation extends Thread {
  public $done = 0;//We initialize the "done" indicator
  public function __construct($arguments) {
    $this->arguments = $arguments;//We put our parameters in a construct
  }
  public function run() {
    if ($this->arguments)
    {
      $parameters = $this->arguments;//We extract the parameters for this worker
      $this->result = array($parameters[0], your_function($parameters[1],$parameters[2],...));//We launch our function and add the key to the result
      $this->done = 1;//This thread is done
    }
  }
}

$stack = array();//Lets initialize our stack
foreach ($arguments as $operation) { $stack[] = new operation($operation); };//We initialize the process
foreach ($stack as $t) { if($t->start()) { $t->join(); }; };//We launch it and wait until all the workers have completed their task

foreach($stack as $strata) {//We get each of the results
  while($strate->done == 0) {sleep(1);};//Inefficient way to wait for the thread to complete, ANY HELP WILL BE APPRECIATED
  $results[] = $strata->result;//Once it's done we add it to our results array
};
ksort($results);//We sort our array with our key
foreach($results as $line) { $results_sorted[] = $line[1]; };//We delete the key

Now you have your $results_sorted ! Enjoy !

edited Jun 2, 2015 at 11:35

answered Jun 2, 2015 at 9:34

760 mmHg

9110 bronze badges

8 Comments

Mjh Over a year ago

sleep will totally suspend the cpu core for 1 second. You also assume that your threads will finish in that second. Instead of manually blocking, you can use a while loop that checks whether thread(s) are finished, and if yes - it joins them. The approach that I personally like is using an event loop (libev, libuv) and using async approach - I notify the main context that a particular thread is done, and the waiting process is non-blocking which lets the CPU core to be used while the threads are doing their processing.

760 mmHg Over a year ago

You're right. I'm looking into php.net/manual/en/threaded.isterminated.php .

Mjh Over a year ago

This might share some insight too: php.net/manual/en/thread.isrunning.php

760 mmHg Over a year ago

If I'm not mistaken, the while loop should produce sleep() instructions until all the threads are closed. It's inelegant and inefficient, this is why I'm looking for a solution, but I think it's working...

Mjh Over a year ago

The problem is that in plain PHP, we need to waste CPU cycles since it doesn't have any type of code that can hook into OS's event interface (IOCP on Win, epoll / kqueue on *nix). However, with while($thread->isRunning()) you can at least know when it's finished. With sleep you are guessing that a second or so was sufficient time for thread(s) ti complete. Both while and sleep are inelegant because they're wasting resources. That's why I personally use libev event loop and I send notifications from within threads to the event loop that they're done. It's nonblocking.

|

Collectives™ on Stack Overflow

PHP - Multithread a function with pthreads

2 Answers 2

1 Comment

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related