How to drop PHP cURL request when target URL is unresponsive?

Question

I have a website in shared hosting with the Entry Processes limit of 30. I fetch data periodically from another URL using cURL function on a PHP cron job. The relevant code is as below.

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "GET");
    curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);

Most of the time my website runs pretty well with only around 4-5 entry processes being used. Whenever this target $url is doesn't response for some reason (and it happens frequently). I quickly get into Entry Process limit and all further requests are denied.

The CURLOPT_CONNECTTIMEOUT doesn't seem to be working as expected. How can I avoid this situation ? I have checked other cURL options but none seem to be working.

Have you forgot to call curl_close($ch) ?

Eriks Klotins
– Eriks Klotins

2020-01-22 10:30:43 +00:00
Commented Jan 22, 2020 at 10:30 — Eriks Klotins
– Eriks Klotins, Commented Jan 22, 2020 at 10:30
I have closed curl. Just forgot to include on question.

rockfight
– rockfight

2020-01-22 10:44:58 +00:00
Commented Jan 22, 2020 at 10:44 — rockfight
– rockfight, Commented Jan 22, 2020 at 10:44

n10n · Accepted Answer · 2020-01-24 21:36:40Z

2

A 10 seconds timeout is very long. Depending on how many requests you serve and if all your requests trigger this call, all your available processes could be just waiting for answers from that server.

You could consider lowering that number.

There is also a second timeout for curl CURLOPT_TIMEOUT. Try setting this in addition. If the connection to the server is done within 10 seconds and then the server takes 60 seconds to serve your request, the current timeout doesn't help you because it only limits the connect time.

If you don't want to be dependent on what cURL is doing you could also set a time limit for the php process itself with set_time_limit(). If you set that to e. g. 30 seconds, php will stop the execution after that time no matter if cURL is done yet or not. This should be done before the curl calls.

edited Jan 24, 2020 at 21:36

answered Jan 22, 2020 at 10:30

n10n

365 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

rockfight Over a year ago

Thank you for the detailed answer. This script just runs once per minute in background. Web request doesn't trigger this. So even it is stuck it should take one more entry process. But the whole 30 entry process are filled if remote URL is non responsive. What can be the cause.

n10n Over a year ago

This sounds like the requests are pending for a very long time and then the processes waiting for requests pile up. Have you tried adding that second timeout? In theory cURL should have a default timeout and should not wait indefinitely but you never know. You could also set a time limit for the php process itself with set_time_limit(). If you set that to e. g. 30 seconds, php will stop the execution after that time no matter if cURL is done yet or not.

rockfight Over a year ago

Thank you for the second parameter. I have tested with it and seems like the connection is dropped if server doesn't respond. I am testing on live. Currently the remote URL is working. If may fail to response after some time than I will check. Thank you for your answer.

Collectives™ on Stack Overflow

How to drop PHP cURL request when target URL is unresponsive?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related