Python Download Using CSV file

Question

I am trying to figure out how to use Python to download files listed in a CSV file and use the CSV file to name the download. So my CSV file would look like this:

HTTP://www.example.com/filetodownload.jpg,mypicture1.jpg
HTTP://www.example.com/2ndfiletodownload.jpg,mypicture2.jpg

The script would read the CSV file, download it from the first field, and name it with the value in the second field. It would cycle through the CSV file until the end.

Does anyone have any suggestions?

EDIT: I didn't include what I have so far...sorry about that. This will download the files but does not rename them with the value after the comma.

import csv, sys
import requests
import urllib2
import os

filename = 'test.csv'
with open(filename, 'rb') as f:
    reader = csv.reader(f)
    try:
        for row in reader:
            if 'http' in row[0]:
                #print row
                rev  = row[0][::-1]
                i  = rev.index('/')
                tmp = rev[0:i]
                #print tmp[::-1]
                rq = urllib2.Request(row[0])
                res = urllib2.urlopen(rq)
                if not os.path.exists("./"+tmp[::-1]):                
                    pdf = open("./" + tmp[::-1], 'wb')
                    pdf.write(res.read())
                    pdf.close()
                else:
                    print "file: ", tmp[::-1], "already exist"
    except csv.Error as e:
        sys.exit('file %s, line %d: %s' % (filename, reader.line_num, e))

instead of for row in reader: write for url, filename in reader: Then use those variables in your loop. Also, I noticed you aren't using requests in your code. It might be easier if you used it. — Steven Rumbalski
– Steven Rumbalski, Commented Aug 25, 2022 at 21:00

score 1 · Accepted Answer · 2022-08-25 23:34:11Z

1

You can use the requests module to download the file, and the csv module to read csv files. Here is a very simple example:

import csv
import requests

filename = "file.csv"


with open(filename, 'r') as file:
  # Get list from csv file
  csvFile = csv.reader(file)

  # data of download
  dat = None
  for item in csvFile:
      # get the length of the list
      args = len(item)

      pairs = args // 2
      for i in range(pairs):
          
        try:
          dat = requests.get(item[i], timeout=0.5)
        except:
          continue
        dat = dat.content
        with open(item[i+1], "wb") as download:
          download.write(dat)

Collectives™ on Stack Overflow

Python Download Using CSV file

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related