How can I scrape web in Python

Question

I would like first time do web scraping some website that is created in JS. But I don't know how to find a table on developer tools. This is my website: https://www.money.pl/gielda/gpw/akcje/

And my code :

import requests
from bs4 import BeautifulSoup as soup

my_url = "https://www.money.pl/gielda/gpw/akcje/"
headers = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36")
page = requests.get(my_url, headers=headers, timeout=10)
page_soup = soup(page.text, 'lxml')

baduker · Accepted Answer · 2021-01-30 18:47:40Z

2

Try this:

import csv

import requests
from bs4 import BeautifulSoup
from tabulate import tabulate

page = requests.get("https://money.pl/gielda/gpw/akcje/")
soup = BeautifulSoup(
    page.content, "html.parser",
).find_all("div", class_="rt-tr-group")

akcje_gpw = [
    [
        i.getText(strip=True) for i in div.find("div") if i.getText()
    ] for div in soup
]

columns = [
    "Walor", "Kurs PLN", "Zmiana (%)", "Otwarcie",
    "Min", "Max", "Obrót (szt.)", "Obrót (PLN)",
    "Czas aktualizacji",
]

print(tabulate(akcje_gpw[:10], headers=columns))

with open("akcje_tabela.csv", "w") as f:
    w = csv.writer(f)
    w.writerow(columns)
    w.writerows(akcje_gpw)

Output:

Walor                         Kurs PLN    Zmiana (%)    Otwarcie    Min     Max     Obrót (szt.)    Obrót (PLN)    Czas aktualizacji
----------------------------  ----------  ------------  ----------  ------  ------  --------------  -------------  -------------------
11 bit studios SA             485,00      -2,61         492,00      485,00  499,50  4 512           2 213 838      2021-01-29
3R Games SA                   1,03        -0,96         1,07        1,03    1,07    11 757          12 482         2021-01-29
4Fun Media SA                 6,48        +2,86         6,76        6,30    6,76    6 618           42 777         2021-01-29
AB SA                         31,40       +0,64         31,10       31,10   31,80   10 633          335 017        2021-01-29
AC SA                         35,00       0,00          35,50       34,80   35,50   3 080           107 836        2021-01-29
Action SA w restrukturyzacji  5,92        -0,67         6,06        5,84    6,06    14 456          86 121         2021-01-29
Adiuvo Investments SA         4,90        +4,26         4,85        4,83    4,96    8 865           43 344         2021-01-29
Agora SA                      6,84        -0,87         6,86        6,84    6,94    13 001          89 663         2021-01-29
Agroton Plc                   7,16        +0,85         7,10        6,80    7,18    31 773          223 314        2021-01-29
Ailleron SA                   11,60       -0,85         11,55       11,30   11,80   7 487           86 225         2021-01-29
----------------------------  ------  -----  ------  ------  ------  ------  ---------  ----------

And here's what's inside the .csv file:

edited Jan 30, 2021 at 18:47

answered Jan 30, 2021 at 18:41

baduker

20.2k9 gold badges43 silver badges63 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Przemek Dabek Over a year ago

Thanks a lot. Please Let Me know how you find it on web this one ("div", class_="rt-tr-group"). Or where you find on Dev.tools on chrome?

baduker Over a year ago

Turn off JS for that site and inspect the source again. Or you can select the element with Ctrl+Shift+C.

Przemek Dabek Over a year ago

its looking strange but i see ModuleNotFoundError: No module named 'tabulate' and i can't pip install tabulate

baduker Over a year ago

Then just remove the import and this line: print(tabulate(akcje_gpw[:10], headers=columns))

Collectives™ on Stack Overflow

How can I scrape web in Python

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related