74

How can I have PHP 5.2 (running as apache mod_php) send a complete HTTP response to the client, and then keep executing operations for one more minute?

The long story:

I have a PHP script that has to execute a few long database requests and send e-mail, which takes 45 to 60 seconds to run. This script is called by an application that I have no control over. I need the application to report any error messages received from the PHP script (mostly invalid parameter errors).

The application has a timeout delay shorter than 45 seconds (I do not know the exact value) and therefore registers every execution of the PHP script as an error. Therefore, I need PHP to send the complete HTTP response to the client as fast as possible (ideally, as soon as the input parameters have been validated), and then run the database and e-mail processing.

I'm running mod_php, so pcntl_fork is not available. I could work my way around this by saving the data to be processed to the database and run the actual process from cron, but I'm looking for a shorter solution.

1

13 Answers 13

53

This code from the PHP manual comments might be helpful:

<?php
 ob_end_clean();
 header("Connection: close");
 ignore_user_abort(); // optional
 ob_start();
 echo ('Text the user will see');
 $size = ob_get_length();
 header("Content-Length: $size");
 ob_end_flush(); // Strange behaviour, will not work
 flush();            // Unless both are called !
 session_write_close(); // Added a line suggested in the comment
 // Do processing here 
 sleep(30);
 echo('Text user will never see');
?>

I actually use it in few places. And it totally makes sense there: a banklink is returning the request of a successful payment and I have to call a lot of services and process a lot of data when that happens. That sometimes takes more than 10 seconds, yet the banklink has fixed timeout period. So I acknowledge the banklink and show him the way out, and do my stuff when he is already gone.

Sign up to request clarification or add additional context in comments.

6 Comments

Interesting approach, but unfortunately it doesn't work if you're running behind varnish or, presumably, other proxies.
it doesnt work on php5 and chrome browser on linux, chrome waits 30 seconds before terminating the connection
The ignore_user_abort(); // optional would have no effect, without passing a value (Boolean) that function returns the current setting.
I tested this solution on my shared hosting, and "Text user will never see" was shown after 30 seconds waiting.
There should be ignore_user_abort(true); instead of ignore_user_abort();
|
29

Have the script that handles the initial request create an entry in a processing queue, and then immediately return. Then, create a separate process (via cron maybe) that regularly runs whatever jobs are pending in the queue.

2 Comments

This the solution I originally had in mind. On the other hand, setting up a processing queue for the sole purpose of working around a timeout in a third party application makes me feel a bit uneasy.
This solution suffers from lack of parallelism... or one will need to start a pool of worker-proccesses to serve the queue. I ended up posting and then disconnecting http requests to self-localhost (in a manner described by SomeGuy here) to utilize a pool of existing httpd workers as background processors.
8

What you need is this kind of setup

alt text

2 Comments

Umm, but, based on this diagram, status message gets sent back to client only when cron executes - 5-10 minutes maximum. Anyway, nice diagram!
status messages could be requested at any time :) the point was that there are two separate and independent processes going on here. But otherwise, thanks!
8

One can to use "http fork" to oneself or any other script. I mean something like this:

// parent sript, called by user request from browser

// create socket for calling child script
$socketToChild = fsockopen("localhost", 80);

// HTTP-packet building; header first
$msgToChild = "POST /sript.php?&param=value&<more params> HTTP/1.0\n";
$msgToChild .= "Host: localhost\n";
$postData = "Any data for child as POST-query";
$msgToChild .= "Content-Length: ".strlen($postData)."\n\n";

// header done, glue with data
$msgToChild .= $postData;

// send packet no oneself www-server - new process will be created to handle our query
fwrite($socketToChild, $msgToChild);

// wait and read answer from child
$data = fread($socketToChild, $dataSize);

// close connection to child
fclose($socketToChild);
...

Now the child script:

// parse HTTP-query somewhere and somehow before this point

// "disable partial output" or 
// "enable buffering" to give out all at once later
ob_start();

// "say hello" to client (parent script in this case) disconnection
// before child ends - we need not care about it
ignore_user_abort(1);

// we will work forever
set_time_limit(0);

// we need to say something to parent to stop its waiting
// it could be something useful like client ID or just "OK"
...
echo $reply;

// push buffer to parent
ob_flush();

// parent gets our answer and disconnects
// but we can work "in background" :)
...

The main idea is:

  • parent script called by user request;
  • parent calls child script (same as parent or another) on the same server (or any other server) and gives request data to them;
  • parent says ok to user and ends;
  • child works.

If you need to interact with child - you can use DB as "communication medium": parent may read child status and write commands, child may read commands and write status. If you need that for several child scripts - you should keep child id on the user side to discriminate them and send that id to parent each time you want to check status of respective child.

I've found that here - http://linuxportal.ru/forums/index.php/t/22951/

2 Comments

This approach (slightly modified) is the only working solution I found to create background task from apache's mod_php without overhead of starting separate OS process - this will occupy and use one of already existing httpd workers instead
In the parent script's fread($socketToChild, $dataSize), where does $dataSize come from? Do you need to know exactly how much data to expect out of the socket (including size of headers)? I must be missing something.
5

What about calling a script on the file server to execute as if it had been triggered at the command line? You can do this with PHP's exec.

2 Comments

+1, something like Gearman is already set up for it (but other / ones own solutions are of course equally valid).
exec() is often a problem in shared/hosted spaces. Plus a huge security risk.
5

You can use the PHP function register-shutdown-function that will execute something after the script has completed its dialog with the browser.

See also ignore_user_abort - but you shouldn't need this function if you use the register_shutdown_function. On the same page, set_time_limit(0) will prevent your script to time out.

1 Comment

Apparently, according to the docs, register_shutdown_function is called before the script completed the dialog since 4.1.0. Your other link, however, contains a promising comment: php.net/manual/en/features.connection-handling.php#89177 I'll try to delve deeper into this and report back here.
4

Using a queue, exec or cron would be an overkill to this simple task. There is no reason not to stay within the same script. This combination worked great for me:

        ignore_user_abort(true);
        $response = "some response"; 
        header("Connection: close");
        header("Content-Length: " . mb_strlen($response));
        echo $response;
        flush(); // releasing the browser from waiting
        // continue the script with the slow processing here...

read more in: How to continue process after responding to ajax request in PHP?

1 Comment

You may need to disable additional buffering which occurs in Apache: <?php apache_setenv('no-gzip', 1); ini_set('zlib.output_compression', 0); ini_set('implicit_flush', 1);?>
4

It is possible to use cURL for that, with a very short timeout. This would be your main file:

<?php>
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, "http://example.com/processor.php");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_TIMEOUT_MS, 10);      //just some very short timeout
    curl_exec($ch);
    curl_close($ch);
?>

And this your processor file:

<?php
    ignore_user_abort(true);                       //very important!
    for($x = 0; $x < 10; $x++)                     //do some very time-consuming task
        sleep(10);
?>

As you can see, the upper script will timeout after a short time (10 milliseconds in this case). It is possible that CURLOPT_TIMEOUT_MS will not work like this, in this case, it would be equivalent to curl_setopt($ch, CURLOPT_TIMEOUT, 1).

So when the processor file has been accessed, it will do its tasks no matter that the user (i.e. the calling file) aborts the connection.

Of course you can also pass GET or POST parameters between the pages.

2 Comments

I have been looking for a solution to this problem for quite a while now and this one works! Thanks a lot. The other solutions might work in specific scenarios, except if you have have limited control over your webserver only and can't fork new processes; a configuration I commonly find on commercial webservers. This solution still works! One important addition. For UNIX systems you need to add curl_setopt($ch, CURLOPT_NOSIGNAL, 1); for timeouts < 1 sec to work. Check here for the explanation.
My processor script didn't execute, so I increased CURLOPT_TIMEOUT_MS to 100. It works like a charm, and enables me to quickly respond when receiving events from our payment provider.
3

You can create an http request between server and server. (not browser is needed). The secret to create a background http request is setting a very small timeout, so the response is ignored.

This is a working function that I have used for that pupose:

MAY 31 PHP asynchronous background request Another way to create an asynchronous request in PHP (simulating background mode).

 /**
  * Another way to make asyncronous (o como se escriba asincrono!) request with php
  * Con esto se puede simpular un fork en PHP.. nada que envidarle a javita ni C++
  * Esta vez usando fsockopen
  * @author PHPepe
  * @param  unknown_type $url
  * @param  unknown_type $params
  */
 function phpepe_async($url, $params = array()) {
  $post_params = array(); 
     foreach ($params as $key => &$val) {
       if (is_array($val)) $val = implode(',', $val);
         $post_params[] = $key.'='.urlencode($val);
     }
     $post_string = implode('&', $post_params);

     $parts=parse_url($url);

     $fp = fsockopen($parts['host'],
         isset($parts['port'])?$parts['port']:80,
         $errno, $errstr, 30);

     $out = "POST ".$parts['path']." HTTP/1.1\r\n";
     $out.= "Host: ".$parts['host']."\r\n";
     $out.= "Content-Type: application/x-www-form-urlencoded\r\n";
     $out.= "Content-Length: ".strlen($post_string)."\r\n";
     $out.= "Connection: Close\r\n\r\n";
     if (isset($post_string)) $out.= $post_string;

     fwrite($fp, $out);
     fclose($fp);
 }

 // Usage:
 phpepe_async("http://192.168.1.110/pepe/feng_scripts/phprequest/fork2.php");

For more info you can take a look at http://www.phpepe.com/2011/05/php-asynchronous-background-request.html

Comments

0

I would recommend spawning a new async request at the end, rather than continuing the process with the user.

You can spawn the other request using the answer here: Asynchronous PHP calls?

Comments

0

In your Apache php.ini config file, make sure that output buffering is disabled:

output_buffering = off

Comments

0

You can split these functions into three scripts.

  1. Initiate process and call second one via exec or command, this is also possible to run via http call.
  2. second one will run database processing and at the end will start last one
  3. last one will email

Comments

0

Looks like your requirements are:

  • Script receives input from an external source you do not control
  • Script processes and validates the input, and lets the external app know if they're good or not and terminates the session.
  • Script kicks off a long-running process.

In this case, then yes, using an outside job queue and/or cron would work. After the input is validated, insert the job details into the queue, and exit. Another script can then run, pick up the job details from the queue, and kick off the longer process. Alex Howansky's answer has the right idea.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.