PHP CURL multi-threaded and single-threaded function help. How do I do this?

Why don't you use a rudementary task scheduler to schedule your requests and followups, instead of running everything at once?

See it in action: https://ideone.com/suTUBS

<?php
class Task 
{
    protected $follow_up = [];
    protected $task_callback;

    public function __construct($task_callback) 
    {
        $this->task_callback = $task_callback;
    }

    public function addFollowUp(Task $follow_up) 
    {
        $this->follow_up[] = $follow_up;
    }

    public function complete() 
    {
        foreach($this->follow_up as $runnable) {
            $runnable->run();
        }
    }

    public function run() 
    {
        $callback = $this->task_callback;

        $callback($this);
    }
}



$provided_task_scheduler_from_somewhere = function() 
{
    $tasks = [];

    $global_message_thing = 'failed';

    $second_global_message_thing = 'failed';

    $task1 = new Task(function (Task $runner) 
    {
        $something_in_closure = function() use ($runner) {
            echo "running task one\n";
            $runner->complete();
        };
        $something_in_closure();
    });

    /**
     * use $global_message_thing as reference so we can manipulate it
     * This will make sure that the follow up on this one knows the status of what happened here
     */
    $second_follow_up = new Task(function(Task $runner) use (&$global_message_thing)
    { 
        echo "second follow up on task one.\n";
        $global_message_thing = "success";
        $runner->complete();
    });

    /**
     * Just doing things in random order to show that order doesn't really matter with a task scheduler
     * just the follow ups
     */
    $tasks[] = $task1;

    $tasks[] = new Task(function(Task $runner) 
    {
        echo "running task 2\n";
        $runner->complete();
    });

    $task1->addFollowUp(new Task(function(Task $runner) 
    { 
        echo "follow up on task one.\n";
        $runner->complete();
    }));

    $task1->addFollowUp($second_follow_up);

    /**
     * Adding the references to our "status" trackers here to know what to print
     * One will still be on failed because we did nothing with it. this way we know it works properly
     * as a control.
     */
    $second_follow_up->addFollowUp(new Task(function(Task $runner) use (&$global_message_thing, &$second_global_message_thing) {
        if($global_message_thing === "success") {
            echo "follow up on the second follow up, three layers now, w00007!\n";
        }
        if($second_global_message_thing === "success") {
            echo "you don't see this\n";
        }
        $runner->complete();
    }));
    return $tasks;
};
/**
 * Normally you'd use some aggretating function to build up your tasks
 * list or a collection of classes. I simulated that here with this callback function.
 */
$tasks = $provided_task_scheduler_from_somewhere();

foreach($tasks as $task) {
    $task->run();
}

This way you can have nesting of tasks that need to follow after each other, with some clever uses of closures you can pass parameters to the executing functions and the encompassing objects outside it.

In my example the Task object itself is passed to the executing function so the executing function can call complete when it's done with it's job.
When complete is called the Task determine if it has scheduled follow up tasks to execute and if so, those are automatically called and works itself down the chain like that.

It's a rudimentary task scheduler, but it should help you on the way getting steps planned in the order you want them to be executed.

Here's an easier to follow example, From : http://arguments.callee.info/2010/02/21/multiple-curl-requests-with-php/

curl_multi_init. This family of functions allows you to combine cURL handles and execute them simultaneously.

EXAMPLE

build the individual requests, but do not execute them

$ch_1 = curl_init('http://webservice.one.com/');
$ch_2 = curl_init('http://webservice.two.com/');
curl_setopt($ch_1, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch_2, CURLOPT_RETURNTRANSFER, true);

build the multi-curl handle, adding both $ch

$mh = curl_multi_init();
curl_multi_add_handle($mh, $ch_1);
curl_multi_add_handle($mh, $ch_2);

execute all queries simultaneously, and continue when all are complete

  $running = null;
  do {
    curl_multi_exec($mh, $running);
  } while ($running);

close the handles

curl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);
curl_multi_close($mh);

all of our requests are done, we can now access the results

$response_1 = curl_multi_getcontent($ch_1);
$response_2 = curl_multi_getcontent($ch_2);
echo "$response_1 $response_2"; // output results

If both websites take one second to return, we literally cut our page load time in half by using the second example instead of the first!

Referances : https://www.php.net/manual/en/function.curl-multi-init.php