Download a file providing username and password using Python

Question

My file named as 'blueberry.jpg' begins downloading, when I click on the following url manually provided that the username and password are typed when asked: http://example.com/blueberry/download

How can I make that happen using Python?

import urllib.request

url = 'http://example.com/blueberry/download'

data = urllib.request.urlopen(url).read()

fo = open('E:\\quail\\' + url.split('/')[1] + '.jpg', 'w')
print (data, file = fo)

fo.close()

However above program does not write the required file, how can I provide the required username and password?

What type of authorization scheme does it use? Basic, Kerberos, NTLM??If it is basic you can try using username:[email protected]/download — MinimalMaximizer
– MinimalMaximizer, Commented Mar 19, 2014 at 5:07
You'll have to look at the headers returned, take a look at this article en.wikipedia.org/wiki/List_of_HTTP_header_fields — MinimalMaximizer
– MinimalMaximizer, Commented Mar 19, 2014 at 5:12
If you are hell bent on learning/seeing alot more of the gnitty-gritty check out voidspace.org.uk/python/articles/authentication.shtml#id1 — MinimalMaximizer
– MinimalMaximizer, Commented Mar 19, 2014 at 5:19
I'm not seeing why @burhan's solution doesn't work - it looks fine to me. When you navigate to example.com/blueberry/download are you asked to enter your credentials in a pop-up window, or on a form in the actual webpage? — MinimalMaximizer
– MinimalMaximizer, Commented Mar 19, 2014 at 5:42

sniperd · Accepted Answer · 2020-07-26 07:59:10Z

7

Use requests, which provides a friendlier interface to the various url libraries in Python:

import os
import requests

from urlparse import urlparse

username = 'foo'
password = 'sekret'

url = 'http://example.com/blueberry/download/somefile.jpg'
filename = os.path.basename(urlparse(url).path)

r = requests.get(url, auth=(username,password))

if r.status_code == 200:
   with open(filename, 'wb') as out:
      for bits in r.iter_content():
          out.write(bits)

UPDATE: For Python3 get urlparse with: from urllib.parse import urlparse

edited Jul 26, 2020 at 7:59

sniperd

5,2846 gold badges32 silver badges49 bronze badges

answered Mar 19, 2014 at 5:11

Burhan Khalid

175k20 gold badges254 silver badges291 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user3410943 Over a year ago

got TypeError: 'str' does not support the buffer interface

user3410943 Over a year ago

for python 3, i am using the code as 'code' fo = open(filename, 'wb') for bits in r.iter_content(): print (bits, file = fo) fo.close()

user3410943 Over a year ago

in python 2.7 it wrote a file but not the required jpg file because the known url is only 'example.com/blueberry/download'

Burhan Khalid Over a year ago

Simply replace filename with whatever you want the filename to be.

SurpriseDog Over a year ago

For Python3 get urlparse with: from urllib.parse import urlparse

MinimalMaximizer · Accepted Answer · 2014-03-19 05:15:08Z

-1

I'm willing to bet you are using basic auth. So try doing the following:

import urllib.request

url = 'http://username:[email protected]/blueberry/download'

data = urllib.request.urlopen(url).read()

fo = open('E:\\quail\\' + url.split('/')[1] + '.jpg', 'w')
print (data, file = fo)

fo.close()

Let me know if this works.

answered Mar 19, 2014 at 5:15

MinimalMaximizer

3922 gold badges4 silver badges19 bronze badges

3 Comments

user3410943 Over a year ago

i got the error perhaps my password is combination of special characters like $/_. The error message tells:http.client.InvalidURL: nonnumeric port:

MinimalMaximizer Over a year ago

As much as the solution i provided is simple - @Burhan Khalid's is much better. It can handle potential error messages in a clean way.

thom_nic Over a year ago

urlopen does not appear to parse any username and password in front of the hostname like http://user:[email protected] - it sees the colon between user:pwd and attempts to parse out a port.

Collectives™ on Stack Overflow

Download a file providing username and password using Python

2 Answers 2

5 Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related