Python Webscraping Vue Components

Question

I am trying to get the value marked in the picture extracted to be a variable, but it seems that when it is within Vue Components, bs4 is not doing the searching like i am expecting. Can anyone point me in the general direction as to how i would be able to extract the value from this document in Python? Code is found below picture, thanks in advance.

import requests
from bs4 import BeautifulSoup

URL = 'https://api.tracker.gg/api/v2/rocket-league/standard/profile/steam/76561198060134880'

headers = {"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36'}

page = requests.get(URL, headers = headers)

soup = BeautifulSoup(page.content, 'html.parser')

#print(soup.prettify())

div_list = soup.findAll({"class":'value'})

print(div_list)

Challe · Accepted Answer · 2020-09-01 19:24:48Z

1

Since the page is returning a json response you don't need beautifulsoup to parse it.

import requests
import json

URL = 'https://api.tracker.gg/api/v2/rocket-league/standard/profile/steam/76561198060134880'

headers = {"User-Agent": 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36'}

response = requests.get(URL, headers = headers)

dict_of_response = json.loads(response.text)

obj_list = dict_of_response['data']['segments']

print(obj_list)

The obj_list variable now contains a list of dicts. Those dicts contain the data you want and now you only need to loop trough the list and do what you want with the data.

answered Sep 1, 2020 at 19:24

Challe

6385 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python Webscraping Vue Components

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related