PHP cURL download progress using jquery

Question

I can't figure out how to push the progress out every time it updates using php. For the sake of clarity, I will write an example.

jQuery:

function uploadMovieDownload(link){
    $.post("php/downloadmovie.php", { source:link }, function(json){ console.log(json); });
}

uploadMovieDownload(url);

PHP (php/downloadmovie.php):

session_start();
ob_start();
date_default_timezone_set("Europe/Bucharest");
ini_set('display_errors',true);

require_once(dirname(__FILE__)."/functions.php");


$url = $_POST['source'];
$headers = getHeaders($url);
$url = $headers['url'];
$path = dirname(__FILE__)."/temp/test.mp4";

$fp = fopen ($path, 'w+');
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, false );
curl_setopt( $ch, CURLOPT_PROGRESSFUNCTION, 'progress' );
curl_setopt( $ch, CURLOPT_NOPROGRESS, false );
curl_setopt( $ch, CURLOPT_BINARYTRANSFER, true );
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, 100 );
curl_setopt( $ch, CURLOPT_FILE, $fp );
curl_exec( $ch );
curl_close( $ch );
fclose( $fp );


function progress($resource,$download_size, $downloaded, $upload_size, $uploaded){
    if($download_size > 0) echo $downloaded / $download_size  * 100;
    ob_flush();
    flush();
}

echo "Done";
ob_flush();
flush();

The problem I have is that it returns the progress after it completes, it isn't pushing it while downloading. Thanks in advance if you have any sugestions.

a few ideas: 1. load balancers can inadvertently save up gradual responses. 2. you need to ship a certain # of bytes right away to get some browsers to behave right. 3. you need to return HTML with the data methodically wrapped in <script> tags to use a simplistic COMET approach like this. 4. i would consider re-implementing using a temp file to store the percentage and pinging another simple script to fetch that value from the file, bypassing the above script. 5. "SSE", aka EventSource is cross-browser and easy to implement in PHP to replace the pinging in #4... — dandavis
– dandavis, Commented Dec 30, 2016 at 21:42

hanshenrik · Accepted Answer · 2019-03-12 12:58:27Z

11

+50

with that method, you'll run in to all kind of caching problems, is php output buffering? you'll have a problem. are you behind a web server, ala nginx/apache/lighthttp/anything? you'll have a problem. is the browser cacheing output? (all mainstream browsers do), you'll have a problem.

i suggest an alternative which will have none of those problems: using $_SESSION to store download percentage, and querying the percentage with XMLHttpRequests (actually querying the percentage over WebSockets would be optimal, lagless, use less bw, etc, but much harder to implement)

downloadmovie.php

<?php 
require_once(dirname(__FILE__)."/functions.php");


$url = $_POST['source'];
$headers = getHeaders($url);
$url = $headers['url'];
//after validation of input
session_start();
$_SESSION['download_percentage']=0.0;//initialize it
session_write_close();
fastcgi_finish_request();//or if you're not using fcgi, some equivalent..

$path = dirname(__FILE__)."/temp/test.mp4";

$fp = fopen ($path, 'w+');
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, false );
curl_setopt( $ch, CURLOPT_PROGRESSFUNCTION, 'progress' );
curl_setopt( $ch, CURLOPT_NOPROGRESS, false );
curl_setopt( $ch, CURLOPT_BINARYTRANSFER, true );
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, 100 );
curl_setopt( $ch, CURLOPT_FILE, $fp );
curl_exec( $ch );
curl_close( $ch );
fclose( $fp );


function progress($resource,$download_size, $downloaded, $upload_size, $uploaded){
    $percentage=$download_size==0? 0.0 : (($downloaded/$download_size)*100);
    session_start();
    $_SESSION['download_percentage']=$percentage;
    session_write_close();
}

getProgress.xhr.php

<?php 
if(""===session_id()){
session_start();
}
echo $_SESSION['download_percentage']??'?';

then monitoring the progress in the browser:

(function checkProgress() {
    "use strict";
    var xhr = new XMLHttpRequest();
    xhr.open("GET", "getProgress.xhr.php");
    xhr.addEventListener("readystatechange", function(ev) {
        var xhr = ev.target;
        if (xhr.readyState !== 4) {
            return;
        }
        console.log(xhr.responseText + " percent downloaded!");
        if (xhr.responseText === "100") {
            return; /*stop checking for progress when its finished*/
        }
        setTimeout(checkProgress, 1000); //<<check for progress every 1 second

    });
    xhr.send();
})();

important edit: as @drew010 pointed out, it won't work without session_write_close();session_start(); each time curl updates the value, fixed that.

edited Mar 12, 2019 at 12:58

answered Jan 2, 2017 at 7:48

hanshenrik

22.4k4 gold badges63 silver badges114 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

drew010 Over a year ago

I think this is the best solution, but don't forget you'll need to use session_write_close after each update and then session_start again before, otherwise the Ajax request will get stuck until the session closes since it'll be locked and block subsequent requests using the same session.

Barkermn01 Over a year ago

Why are you wrapping session_start in an if just call it the first thing it does it check for a session_id from the cookie if it's there knows to load it.

hanshenrik Over a year ago

@drew010 wow. i didn't know about that, thanks! ill update the code in a sec

hanshenrik Over a year ago

@MartinBarker i don't know what runs before the actual php file is called, but some sites begins sessions in a auto_prepend_file php.ini directive or the like x.x in that case, calling session_begin() without checking if its already begun, would cause an error (E_NOTICE i believe?) - just checked, yup: Notice: A session had already been started - ignoring session_start() in C:\WTServer\WWW\sesstest\2.php on line 4

Barkermn01 Over a year ago

Strange if i try to use $_SESSION without a session start even if session already in it does not work for me...

Community · Accepted Answer · 2017-05-23 12:24:18Z

Quote from cURL download progress in PHP

echo "<pre>";
echo "Loading ...";

ob_flush();
flush();

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://stackoverflow.com");
//curl_setopt($ch, CURLOPT_BUFFERSIZE,128);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, 'progress');
curl_setopt($ch, CURLOPT_NOPROGRESS, false); // needed to make progress function work
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
$html = curl_exec($ch);
curl_close($ch);


function progress($resource,$download_size, $downloaded, $upload_size, $uploaded)
{
    if($download_size > 0)
         echo $downloaded / $download_size  * 100;
    ob_flush();
    flush();
    sleep(1); // just to see effect
}

echo "Done";
ob_flush();
flush();

?>

Walery Strauch · Accepted Answer · 2015-10-23 21:02:11Z

0

Don't use ob_flush(), just flush(). (Remove all the ob_ functions). Also, consider modifying the buffersize:

curl_setopt($ch, CURLOPT_BUFFERSIZE, 16000);

Experiment with the size.

edited Oct 23, 2015 at 21:02

Walery Strauch

7,1288 gold badges56 silver badges61 bronze badges

answered Oct 23, 2015 at 20:37

Lee Nicholls

1

Comments

LF-DevJourney · Accepted Answer · 2016-12-31 21:41:25Z

0

PHP will show the results when all the scripts are done. This is the reason that you just the finish progress.

You can execute the curl downloading in the background, and write the progress to session, database, session or memory such as redis, memchche. Then in the client side read the process every very short time.

You also can use Iframe to do this.

edited Dec 31, 2016 at 21:41

answered Dec 31, 2016 at 21:34

LF-DevJourney

28.6k30 gold badges170 silver badges322 bronze badges

Collectives™ on Stack Overflow

PHP cURL download progress using jquery

4 Answers 4

5 Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

5 Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related