Python Requests library returns wrong status code

Question

The Python code below returns '403':

import requests
url = 'http://bedstardirect.co.uk/star-collection-braemar-double-bedstead.html'
r = requests.get(url)
print r.status_code

But this page is valid and the script should return '200', as does the perl script below:

use WWW::Mechanize;
my $mech = WWW::Mechanize->new();
my $url = 'http://bedstardirect.co.uk/star-collection-braemar-double-bedstead.html';
$mech->get($url);
print $mech->status,"\n";

I have also checked with Firebug in Firefox and all requests have a '200' status code.

I use Python Requests v1.2.0.

David McKeone · Accepted Answer · 2013-04-24 21:27:36Z

7

Seems your particular server requires a User-Agent header.

Try:
r = requests.get('http://bedstardirect.co.uk/star-collection-braemar-double-bedstead.html', headers={'User-Agent': 'a user agent'})

Edit:
The default User-Agent on requests for my machine comes out as: python-requests/1.2.0 CPython/2.7.4 Darwin/12.3.0

After some testing I found that any User-Agent that contains the word python will fail on this server.

edited Apr 24, 2013 at 21:27

answered Apr 24, 2013 at 17:11

David McKeone

3,5931 gold badge17 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ian Stapleton Cordasco Over a year ago

That's odd since requests sends one by default.

Collectives™ on Stack Overflow

Python Requests library returns wrong status code

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related