How to download a csv file in Python

Question

I am trying to download a csv file from the url

https://qubeshub.org/publications/1220/supportingdocs/1#supportingdocs .

the file is Elephant Morphometrics and Tusk Size-originaldata-3861.csv

I have tried using using pd.read_csv()

and

import pandas as pd
import io
import requests
url="https://qubeshub.org/publications/1220/supportingdocs/1#supportingdocs/Elephant Morphometrics and Tusk Size-originaldata-3861.csv"
s=requests.get(url).content
c=pd.read_csv(io.StringIO(s.decode('utf-8')))

always put full error message (starting at word "Traceback") in question (not in comments) as text (not screenshot, not link to external portal). There are other useful information. — furas
– furas, Commented Apr 7, 2022 at 21:26
link to file is https://qubeshub.org/publications/1220/serve/1/3861?el=1&download=1 — furas
– furas, Commented Apr 7, 2022 at 21:30
Visit the URL you have in your code, it's still just an HTML page with links to the data. You need to do a little more digging to get the correct URL to the CSV. — Zach Young
– Zach Young, Commented Apr 7, 2022 at 23:58

Andrej Kesely · Accepted Answer · 2022-04-07 21:36:12Z

4

Try:

import requests

url = "https://qubeshub.org/publications/1220/serve/1/3861?el=1&download=1"

r = requests.get(url)
filename = r.headers["Content-Disposition"].split('"')[1]

with open(filename, "wb") as f_out:
    print(f"Downloading {filename}")
    f_out.write(r.content)

Prints:

Downloading Elephant Morphometrics and Tusk Size-originaldata-3861.csv

and saves the file.

answered Apr 7, 2022 at 21:36

Andrej Kesely

196k15 gold badges60 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Lucas Roberts · Accepted Answer · 2022-04-07 21:37:41Z

This should download the file and parse the rows and columns into a csv file

import requests
import csv
url = "https://qubeshub.org/publications/1220/serve/1/3861?el=1&download=1"
req=requests.get(url)
rows = req.content.decode('utf-8').split("\r\n")
rows.pop()
csv_local_filename = "test.csv"
with open(csv_local_filename, 'w') as fs:
    writer = csv.writer(fs, delimiter = ',')
    for row in rows:
        entries = row.split(',')
        b=writer.writerow(entries)

You'll likely want to convert those columns into the desired types before you start working with them. The example code above leaves everything as a string.

After I run the above code I see:

>tail test.csv 
2005-13,88,m,32.5,290,162.3,40
2005-13,51,m,37.5,270,113.2,40
2005-13,86,m,37.5,310,175.3,38

and

>head test.csv
Years of sample collection,Elephant ID,Sex,Estimated Age (years),shoulder Height in  cm,Tusk Length in cm,Tusk Circumference   in cm
1966-68,12,f,0.08,102,,
1966-68,34,f,0.08,89,,
1966-68,162,f,0.083,89,,
1966-68,292,f,0.083,92,,

furas · Accepted Answer · 2022-04-07 21:40:47Z

0

In Firefox after downloading file in browser you can check link to this file and it shows

https://qubeshub.org/publications/1220/serve/1/3861?el=1&download=1

and this link you should use in code

import pandas as pd

df = pd.read_csv('https://qubeshub.org/publications/1220/serve/1/3861?el=1&download=1')

print(df)

answered Apr 7, 2022 at 21:40

furas

149k12 gold badges121 silver badges171 bronze badges

Collectives™ on Stack Overflow

How to download a csv file in Python

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related