I am trying to scrape this webpage: http://animeheaven.eu/watch.php?a=My%20Hero%20Academia%203&e=2 and download the video. As you can see it loads the 720p video. I can download the video from here. But I don't know how to get the other video version i.e. the 480p version from the drop-down menu. How do I select the 480p link?
-
4show what you have done in the problem so far?Rachit kapadia– Rachit kapadia2018-04-14 16:11:32 +00:00Commented Apr 14, 2018 at 16:11
-
well, I'm not really satisfied with the download part either but that got my job done. I noticed the javascript function never changes only the variables, so I hardcoded the function in python and get the variable each time, basically, instead of rendering the js I wrote a python equivalent, that outputs the download link, because js rendering was pretty slowEular– Eular2018-04-14 16:19:31 +00:00Commented Apr 14, 2018 at 16:19
Add a comment
|
1 Answer
If you make a POST request with the parameter "rs" = "1" you get the data you want.
from bs4 import BeautifulSoup
import requests
link = "http://animeheaven.eu/watch.php?a=My%20Hero%20Academia%203&e=2"
html= requests.post(link, data = {'rs': '1'})
soup= BeautifulSoup(html.content,"lxml")
scripts= soup.findAll("script")
sc=scripts[4]
print (sc)
...
Outputs:
...
document.write("<a class='an' href='"+ pli +"'><div class='dl2'>Download 159 MB</div></a>");
...
Not:
...
document.write("<a class='an' href='"+ pli +"'><div class='dl2'>Download 255 MB</div></a>");
...
UPDATED in response to comment:
...
select = soup.find("select", {'name': 'rs'})
for option in select.find_all('option'):
print ("{} = {}".format(option.text, option['value']))
Outputs
720p = 0
480p = 1
4 Comments
Eular
so
rs will give resolution and sr will give the server. Is there a way to know the available numbers for rs, I mean 0 or 1 or 2 etc.?
Dan-Dev
In FireFox or Chrome use inspect, look at the network tab and submit the forms, then look at the data sent, this will tell you the corresponding value which you need to use.
Eular
I meant within the code without openning the browser. e.g. I can see the available resolution from the page sourse too, so I was thinking I can get the page source, read the avilable resolutions, then again send the request with appropriate
rs dataDan-Dev
I put some code to do this as an update to my answer.