Downloading a CSV file using PHP from an API - with a URL that doesn't end with .csv

Question

I have been trying to download a file from an API in 2 different way without any success: https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today

*Notice the file doesn't end with .csv, it just pop out the download file for the file.

The file that gets downloaded is a .CSV file.

I tried using CURL:

// Date looks like this: 2016-01-31     
$today = date("Y-m-d");

    $output_filename = "test.csv";

    $host = "https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today";
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $host);
    curl_setopt($ch, CURLOPT_VERBOSE, 1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_AUTOREFERER, false);
    curl_setopt($ch, CURLOPT_REFERER, "https://www.example.com");
    curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    $result = curl_exec($ch);
    curl_close($ch);

    print_r($result); // prints the contents of the collected file before writing..


    // the following lines write the contents to a file in the same directory (provided permissions etc)
    $fp = fopen($output_filename, 'w');
    fwrite($fp, $result);
    fclose($fp);

With this code I got a black test.csv file.
I don't get anything printed on the screen after running the function (print_r($result))

and I tried using file_put_contents function:

$today = date("Y-m-d");
echo $today;
file_put_contents("", fopen("https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today", 'r'));
// I TRIED THIS ONE TOO: 
// file_put_contents("temp.csv", "https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today");

*I get a CSV file with the URL inside it on the first row (https://example.com/export/banana/by_date/v4?api_token=666&from=2016-01-31&to=2016-01-31).

Can someone help me by telling me if I am doing this right? (since this isn't a direct link to a file maybe I'm working the wrong way). And what is the right way doing this.

so print_r($results) shows the content, but nothing shows up in the saved file? — RST
– RST, Commented Jan 31, 2016 at 15:37
Does the print_r work properly? If so, it's a PHP issue; otherwise, it's a CURL issue. — BradzTech
– BradzTech, Commented Jan 31, 2016 at 15:47
@BradzTech actually no, I don't get anything printed on the screen after running the function. — Imnotapotato
– Imnotapotato, Commented Jan 31, 2016 at 15:53
Try var_dump(curl_error($ch)); where you have the print_r. — BradzTech
– BradzTech, Commented Jan 31, 2016 at 15:58

Professor Abronsius · Accepted Answer · 2016-01-31 16:20:29Z

1

The target url is https so perhaps you need to add certain ssl specific options

curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, FALSE );
curl_setopt( $ch, CURLOPT_SSL_VERIFYHOST, 2 );
curl_setopt( $ch, CURLOPT_CAINFO, realpath( '/path/to/cacert.pem' ) );

another common cause of curl requests failing is the lack of a useragent string.

curl_setopt( $ch, CURLOPT_USERAGENT, 'my useragent string' );

You can set similar options when using file_get_contents by setting options for the $context

Based on your last comment, add:

 curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, TRUE );

edited Jan 31, 2016 at 16:20

answered Jan 31, 2016 at 16:09

Professor Abronsius

33.9k5 gold badges36 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Imnotapotato Over a year ago

I'm not sure what is: curl_setopt( $ch, CURLOPT_CAINFO, realpath( '/path/to/cacert.pem' ) );

Imnotapotato Over a year ago

I added: curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, FALSE ); and curl_setopt( $ch, CURLOPT_SSL_VERIFYHOST, 2 ); and I get: and I get

The resource was found at https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today_KJAGS; you should be redirected automatically.

the _KJAGS in the end of the URL is added, and changes the string every time I refresh the page.

Professor Abronsius Over a year ago

you can download cacert.pem from the interwebs - easy to find. It is a collection of certificates for validation tasks. The realpath() - replace the path with the actual path the the file on your system if you intend to use it

Imnotapotato Over a year ago

I added this after print_r($result);:

$match = ''; preg_match_all('#\bhttps?://[^\s()<>]+(?:\([\w\d]+\)|([^[:punct:]\s]|/))#', $result, $match); print_r($match[0][0]); file_put_contents($match[0][0]);

Imnotapotato Over a year ago

and I got the right output. I noticed that the api URL generates a new URL so I just download it using file_put_contents.

|

Aleksey Ratnikov · Accepted Answer · 2016-01-31 16:39:56Z

0

This is well known (for me) problem with accesing https resources with cURL from php - it can't verify certificate in default configuration. So easiest thing to get this script works, you should add two additional lines for curl_config:

$today = date("Y-m-d");

$output_filename = "test.csv";

$host = "https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today";
$ch = curl_init();
$curl_config = [
    CURLOPT_URL => $host,
    CURLOPT_VERBOSE => 1,
    CURLOPT_RETURNTRANSFER => 1,
    CURLOPT_AUTOREFERER => false,
    CURLOPT_REFERER => "https://www.example.com",
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_HEADER => 0,
    CURLOPT_SSL_VERIFYHOST => 0, //do not verify that host matches one in certifica
    CURLOPT_SSL_VERIFYPEER => 0, //do not verify certificate's meta
];

curl_setopt_array($ch, $curl_config); //apply config

$result = curl_exec($ch);

if (empty($result)){
    echo  curl_error($ch); //show possible error if answer if empty and exit script
    exit;
}

curl_close($ch);

print_r($result); // prints the contents of the collected file before writing..

// the following lines write the contents to a file in the same directory (provided permissions etc)
file_put_contents($output_filename, $result);

edited Jan 31, 2016 at 16:39

answered Jan 31, 2016 at 16:08

Aleksey Ratnikov

5593 silver badges11 bronze badges

2 Comments

Imnotapotato Over a year ago

the two additional lines are CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER ? Am I right?

Aleksey Ratnikov Over a year ago

Yes. I've corrected answer adding line where config is applied.

Collectives™ on Stack Overflow

Downloading a CSV file using PHP from an API - with a URL that doesn't end with .csv

2 Answers 2

6 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related