Web Scraping For Loop Issue

Question

i have an issue with collecting all the data in the website. when i run my code it only prints out the first entry. And it should print out every Song, Artist, and Rank. Also It doesn't show on Csv.

from bs4 import BeautifulSoup
import requests
import csv


    my_url = "https://www.billboard.com/charts/hot-100"
    headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) 
   Chrome/87.0.4280.141 Safari/537.36'
   }
   r = requests.get(my_url)
   page_soup = BeautifulSoup(r.content, 'lxml')

   filename = "Billboard100.csv"
   csv_writer = csv.writer(open(filename, 'w'))

   Chart = page_soup.findAll('ol', class_='chart-list__elements')
   BB = []
   for item in Chart:
         Song = item.find('span', class_='chart-element__information__song text--truncate color--primary').text.strip()
         Artist = item.find('span', class_='chart-element__information__artist text--truncate color--secondary').text.strip()
         Rank = item.find('span', class_='chart-element__rank__number').text.strip()
       Billboard = {
         'Song': Song,
         'Artist': Artist,
         'Rank': Rank,
}
       BB.append(Billboard)

       print(BB)

       with open("Billboard100.csv", "w",
          newline="") as infile:
       writer = csv.writer(infile)

Samsul Islam · Accepted Answer · 2021-01-10 11:51:09Z

In your code Char length was one. Use Chart = page_soup.find_all('li', {'class': 'chart-list__element display--flex'}) to select all the entity.

from bs4 import BeautifulSoup
import requests
import csv


my_url = "https://www.billboard.com/charts/hot-100"
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36'
   }
r = requests.get(my_url)
page_soup = BeautifulSoup(r.content, 'lxml')

filename = "Billboard100.csv"
csv_writer = csv.writer(open(filename, 'w'))

Chart = page_soup.find_all('li', {'class': 'chart-list__element display--flex'})

BB = []
for item in Chart:
   Song = item.find('span', class_='chart-element__information__song text--truncate color--primary').text.strip()
   Artist = item.find('span', class_='chart-element__information__artist text--truncate color--secondary').text.strip()
   Rank = item.find('span', class_='chart-element__rank__number').text.strip()
   Billboard = {
         'Song': Song,
         'Artist': Artist,
         'Rank': Rank,
}
   BB.append(Billboard)

   print(BB)

with open("Billboard100.csv", "w",newline="") as infile:
   writer = csv.writer(infile)
   for row in BB:
      writer.writerow([row])

Larsluph · Accepted Answer · 2021-01-10 11:15:14Z

Your code isn't well indented. In python, always check your indentation if some code doesn't run at all. official doc reference

for item in Chart:
         Song = item.find('span', class_='chart-element__information__song text--truncate color--primary').text.strip()
         Artist = item.find('span', class_='chart-element__information__artist text--truncate color--secondary').text.strip()
         Rank = item.find('span', class_='chart-element__rank__number').text.strip()
       Billboard = {

should be

for item in Chart:
       Song = item.find('span', class_='chart-element__information__song text--truncate color--primary').text.strip()
       Artist = item.find('span', class_='chart-element__information__artist text--truncate color--secondary').text.strip()
       Rank = item.find('span', class_='chart-element__rank__number').text.strip()
       Billboard = {

Collectives™ on Stack Overflow

Web Scraping For Loop Issue

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related