I have managed to get the script tag using BeautifulSoup.Then I turned into a json object. The information that I want is within data['x'] but it is stuck between b tags.
Example :
<b>infoiwant</b><br>NA<br>infoinwant</br>columniwant: 123','<b>infoiwant</b><br>NA<br>columniwant: 123'</br>columniwant: 123
How would I go about getting the info out of these b elements
Before converting to json, can you use the BeautifulSoup get_text() method? Maybe something like
soup.find('b').get_text()
One method how to extract the data from <script> tag is using re module:
import re
from bs4 import BeautifulSoup
html_text = """
<script>
var data['x'] = '<b>infoiwant</b><br>NA<br>infoinwant</br>columniwant: 123';
</script>
"""
html_data = re.search(r"data\['x'\] = '(.*?)';", html_text).group(1)
soup = BeautifulSoup(html_data, "html.parser")
print(soup.find("b").get_text(strip=True))
Prints:
infoiwant
