Select table element from scraped html

Question

I need scrape two tables. This table can scrape with this code:

url = requests.get('https://www.quebec.ca/sante/problemes-de-sante/a-z/coronavirus-2019/situation-coronavirus-quebec/')
c = url.content

# Create a soup object
soup = BeautifulSoup(c, 'html.parser')

my_table = soup.find('table', attrs = {'class': 'contenttable'})

But why this second table I cant save:

Code:

 my_table2 = soup.find('table', attrs = {'class': 'table table-condensed table-hover table-striped'})

my_table2 is empty...

Have you tried using other id and classes to locate the table? — Assad Ali
– Assad Ali, Commented Nov 9, 2020 at 20:40
@AssadAli I tried "soup.find('div', attrs = {'table': 'tableauOverFlow'})" — 314mip
– 314mip, Commented Nov 9, 2020 at 20:49
I try lxml too: from urllib.request import urlopen from lxml import etree url = "quebec.ca/sante/problemes-de-sante/a-z/coronavirus-2019/…" response = urlopen(url) htmlparser = etree.HTMLParser() tree = etree.parse(response, htmlparser) xpathselector = '//*[@id="csv-display-cas-par-region"]/table' tree.xpath(xpathselector) — 314mip
– 314mip, Commented Nov 9, 2020 at 20:59

MendelG · Accepted Answer · 2020-11-09 21:30:53Z

2

The second table is a actually a csv file loaded from https://cdn-contenu.quebec.ca/cdn-contenu/sante/documents/Problemes_de_sante/covid-19/csv/cas-region.csv?t=1604956500

import requests
from bs4 import BeautifulSoup

URL = "https://cdn-contenu.quebec.ca/cdn-contenu/sante/documents/Problemes_de_sante/covid-19/csv/cas-region.csv?t=1604956500"

soup = BeautifulSoup(requests.get(URL).content, "html.parser")

print(soup.prettify())

answered Nov 9, 2020 at 21:30

MendelG

20.6k5 gold badges38 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

JaSON · Accepted Answer · 2020-11-09 21:31:36Z

0

To get required data you can do

import requests
import time
import calendar

epoch_time = calendar.timegm(time.gmtime())    
data = requests.get('https://cdn-contenu.quebec.ca/cdn-contenu/sante/documents/Problemes_de_sante/covid-19/csv/cas-region.csv?t={}'.format(epoch_time)).content

for entry in data.split('\r\n')[1:]:
    print(entry)

Output:

01 - Bas-Saint-Laurent;3;558 
02 - Saguenay – Lac-Saint-Jean;94;2 124 
03 - Capitale-Nationale;82;9 422 
04 - Mauricie-et-Centre-du-Québec;81;5 303 
05 - Estrie;54;3 142 
06 - Montréal;285;44 716 
07 - Outaouais;32;2 802 
08 - Abitibi-Témiscamingue;0;252 
09 - Côte-Nord;0;173 
10 - Nord-du-Québec;1;24 
11 - Gaspésie – Îles-de-la-Madeleine;25;1 102 
12 - Chaudière-Appalaches;69;4 127 
13 - Laval;109;9 829 
14 - Lanaudière;164;8 601 
15 - Laurentides;46;6 866 
16 - Montérégie;123;16 830 
17 - Nunavik;0;28 
18 - Terres-Cries-de-la-Baie-James;0;15 
Hors Québec;1;72 
Région à déterminer;0;3
Total;1169;115 989

edited Nov 9, 2020 at 21:31

answered Nov 9, 2020 at 21:20

JaSON

4,8912 gold badges12 silver badges18 bronze badges

2 Comments

314mip Over a year ago

Thank you, but why cant use soup.find(...)?

JaSON Over a year ago

@MiroslavAdamek It seem that data comes from CSV file, so it can't be found in page source. The output can be adjusted to more appropriate format

Collectives™ on Stack Overflow

Select table element from scraped html

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related