1

Sometimes I get a 404 error when trying to access a webpage via curl but if I use file_get_contents to access the exact same URL it works, any idea why it's like that?

Here is an example: My curl settings:

curl_setopt($this->ch, CURLOPT_USERAGENT, $this->UserAgent);
curl_setopt($this->ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($this->ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($this->ch, CURLOPT_COOKIEFILE, $this->CookieFile);
curl_setopt($this->ch, CURLOPT_COOKIEJAR, $this->CookieFile);
curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($this->ch,CURLOPT_VERBOSE,true);
curl_setopt($this->ch,CURLOPT_HEADER,true);
curl_setopt($this->ch,CURLOPT_ENCODING, "gzip");
curl_setopt($this->ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($this->ch, CURLOPT_TIMEOUT, 30);

The request header with my firefox webbrowser:

GET /image.png HTTP/1.1
Host: ssl.website.net
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cookie: some cookie info
Cache-Control: max-age=0

Response header:

HTTP/1.1 200 OK
Date: Wed, 20 Jun 2012 13:38:43 GMT
Content-Type: image/png
Transfer-Encoding: chunked
Connection: close
Pragma: no-cache
Cache-Control: no-cache
Server: '; DROP TABLE servertypes; --
7
  • 1
    Hi, Bobby Tables Server: '; DROP TABLE servertypes; -- Commented Jun 20, 2012 at 13:58
  • 1
    I guess you'll have to ask the guys at reddit.com why they 404 your requests. Commented Jun 20, 2012 at 14:01
  • That is not the only site, I sometimes get 404s on some images from different sites. The strange thing is why it works with file_get_contents() Commented Jun 20, 2012 at 14:04
  • 1
    Define "sometimes". Do you mean that some URLs give 404 with Curl but work with file_get_contents, or that the same URL sometimes works and sometimes gives 404? Commented Jun 20, 2012 at 14:10
  • @lanzz I wonder if they've ever caught some little script kiddy out with that and eaten their database. One can only hope... Commented Jun 20, 2012 at 14:10

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.