How can I send GET data to multiple URLs at the same time using cURL?

Question

My apologies, I've actually asked this question multiple times, but never quite understood the answers.

Here is my current code:

while($resultSet = mysql_fetch_array($SQL)){            
$ch = curl_init($resultSet['url'] . $fullcurl); //load the urls and send GET data
            curl_setopt($ch, CURLOPT_TIMEOUT, 2);           //Only load it for two seconds (Long enough to send the data)
            curl_exec($ch);                                 //Execute the cURL
            curl_close($ch);                                //Close it off 
} //end while loop

What I'm doing here, is taking URLs from a MySQL Database ($resultSet['url']), appending some extra variables to it, just some GET data ($fullcurl), and simply requesting the pages. This starts the script running on those pages, and that's all that this script needs to do, is start those scripts. It doesn't need to return any output. Just the load the page long enough for the script to start.

However, currently it's loading each URL (currently 11) one at a time. I need to load all of them simultaneously. I understand I need to use curl_multi_, but I haven't the slightest idea on how cURL functions work, so I don't know how to change my code to use curl_multi_ in a while loop.

So my questions are:

How can I change this code to load all of the URLs simultaneously? Please explain it and not just give me code. I want to know what each individual function does exactly. Will curl_multi_exec even work in a while loop, since the while loop is just sending each row one at a time?

And of course, any references, guides, tutorials about cURL functions would be nice, as well. Preferably not so much from php.net, as while it does a good job of giving me the syntax, its just a little dry and not so good with the explanations.

EDIT: Okay zaf, here is my current code as of now:

        $mh = curl_multi_init(); //set up a cURL multiple execution handle

$SQL = mysql_query("SELECT url FROM urls") or die(mysql_error()); //Query the shell table
                    while($resultSet = mysql_fetch_array($SQL)){   

        $ch = curl_init($resultSet['url'] . $fullcurl); //load the urls and send GET data
        curl_setopt($ch, CURLOPT_TIMEOUT, 2);           //Only load it for two seconds (Long enough to send the data)
        curl_multi_add_handle($mh, $ch);
    } //No more shells, close the while loop

        curl_multi_exec($mh);                           //Execute the multi execution
        curl_multi_close($mh);                          //Close it when it's finished.

possible duplicate of stackoverflow.com/questions/2692704/… — John Conde
– John Conde, Commented Apr 26, 2010 at 13:43

zaf · Accepted Answer · 2010-04-26 13:52:53Z

3

In your while loop, you need to do the following for each URL:

create a curl resource by using curl_init()
set options for resource by curl_setopt(..)

Then you need to create a multiple curl handle by using curl_multi_init() and adding all the previous individual curl resources by using curl_multi_add_handle(...)

Then finally you can do curl_multi_exec(...).

A good example can be found here: http://us.php.net/manual/en/function.curl-multi-exec.php

answered Apr 26, 2010 at 13:52

zaf

23.4k12 gold badges69 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

13 Comments

Rob Over a year ago

Oh I don't know why I didn't think of that. How would I be able to add a multiple curl handle from the handles in the while loop?

Rob Over a year ago

Interesting. Would I be able to do this:

$mh = curl_multi_init();  while getting rows from db { curl_multi_add_handle($mh,$resultSet['url']); }  curl_multi_exec($mh);

zaf Over a year ago

Yes, try it, it should work. If not then post back and we'll look at it.

Rob Over a year ago

Thanks, will do. Give me a couple of minutes

Rob Over a year ago

Oh another question, will I need to remove the handles before closing the multi_exec?

|

Collectives™ on Stack Overflow

How can I send GET data to multiple URLs at the same time using cURL?

1 Answer 1

13 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

13 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related