PHP cURL delay the execution for few seconds

Question

I'm trying to get the Bing search count result using PHP cURL.

I tried cURL but the response is not returning any result. It is returning only the header and footer of the Bing search page.

My code:

$url = "https://www.bing.com/search?q=site:mysite.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
sleep(10);
$response = curl_exec($ch);
curl_close($ch);

Output HTML:

Then I tried with Python with sleep.

url = "https://www.bing.com/search?q=site:mysite.com";
browser.get(url)
time.sleep(1)
print(browser.page_source)  # results

And this returned the HTML with Bing search result count.

    ......
    <span class="sb_count" data-bm="4">465 results</span>
    .....

Similar to Python how can I sleep PHP cURL execution in between for few seconds and get the search result.

Any help would be appreciated. Thanks in advance.

cURL is returned fastly before the cURLed url prepares the result...this doesn't make any sense. cURL sends a HTTP request, and then it waits for the response. cURL won't return a response until the remote server sends back some data. That is a hard fact. So maybe explain to us what response you're seeing, and what response you expected? I'm going to hazard a guess that the remote site uses some JavaScript/AJAX to load extra content, and you've forgotten about the fact that cURL cannot execute Javascript. so all you get back is the HTML returned by the initial basic HTTP request. — ADyson
– ADyson, Commented Jul 8, 2022 at 8:51
@ADyson Thank you for the response. I have edited my question. Please see it. — user19153785
– user19153785, Commented Jul 8, 2022 at 10:22
I don't know python much, what is the browser variable? Does it represent a headless browser? If so it might have the capability to execute javascript, which may be the difference here. I don't know but I'd guess there is a headless browser library for php somewhere which you could try — ADyson
– ADyson, Commented Jul 8, 2022 at 10:30
It's a good point actually - I would echo the answer below: why aren't you using the API? — ADyson
– ADyson, Commented Jul 8, 2022 at 10:47
OK well then if this was working before I'd expect the issue was that previously Bing was generating results without the use of JavaScript and maybe they changed that. As mentioned before, please start using the API, which will make parsing the results easier and also not leave you at the whim of unexpected changes by the UI designers. — ADyson
– ADyson, Commented Jul 8, 2022 at 11:42

Martin Zeitler · Accepted Answer · 2022-07-08 10:41:40Z

1

This approach is a) not feasible and b) questionable.
There's a Bing Search API, which appears more likely.

answered Jul 8, 2022 at 10:41

Martin Zeitler

79.7k20 gold badges170 silver badges244 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

PHP cURL delay the execution for few seconds

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related