I can't convert string type to dictionary using json.loads in python

Question

The target is to extract a MP4 video link on MLB website.

url ="https://www.mlb.com/video/jeremy-pena-s-solo-homer?t=most-popular"
content = requests.get(url).text

I have found the target dict.

soup = BeautifulSoup(content,"lxml")

all_script_label = soup.find_all(name ="script")

target = all_script_label[20].text.split("\n")[1].split("=")[1]

But I can't turn the target into dict type with json.loads, it's still a string.

json_ob = json.loads(target)
print(type(json_ob))

Which step I did wrong?

I have tried ast.literal_eval method but it doesn't work too.

Can you update your question with a relevant sample from target? — quamrana
– quamrana, Commented Nov 5, 2022 at 13:26

Andrej Kesely · Accepted Answer · 2022-11-05 11:10:48Z

0

You can apply json.loads second time to convert the str to dict:

import re
import json
import requests
from bs4 import BeautifulSoup

url = "https://www.mlb.com/video/jeremy-pena-s-solo-homer?t=most-popular"
content = requests.get(url).text
soup = BeautifulSoup(content, "lxml")
all_script_label = soup.find_all(name="script")
target = all_script_label[20].text


data = re.search(r"window\.__VIDEO_INIT_STATE__ = (.*)", target).group(1)
data = json.loads(json.loads(data))

print(type(data))

Prints:

<class 'dict'>

answered Nov 5, 2022 at 11:10

Andrej Kesely

196k15 gold badges60 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

I can't convert string type to dictionary using json.loads in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related