0

I am trying to get a website's (habr.com) page_source using python3 and selenium, but there are some svg tags that contain use tags with xlink:href parameter and #shadow-root (closed) in them. That peace of html looks like this:

html_source

My python code is this:

from selenium import webdriver

driver = webdriver.Chrome()
driver.get('https://habr.com/en')
source = driver.page_source

I am doing this using just python interpreter, not from file, so there is enough time to load all the resources. So, the source variable will contain all the html except this #shadow-root (closed) part.

I tried this solution, but I guess it works only with #shadow-root (open).

What should I do in order to obtain the whole html source including those Shadow DOM parts?

UPDATE: The whole point of this is that I want to make a some kind of proxy server that will point to desired website and change all the links on the page to my localhost. In order to make proper tests I wanted to get the source html from target website and compare it tag by tag with my localhost's source html. But I cannot do that until I get this Shadow DOM content. I mean I can do that but it just will not be an objective test.

Improve this question
5
  • go to the think the xlink:href in the use tag points to and load that. Shadow roots (closed) are not directly accessible. Commented Apr 12, 2019 at 10:38
  • @RobertLongson This raises several questions. How can I know that a certain element has this shadow DOM in it? How do I load it and then how do I properly add the result to the html? If I paste the link from xlink:href in browser, it shows an empty page. Commented Apr 12, 2019 at 10:47
  • <use> elements have shadow DOM, other svg elements don't. You don't add the result to the html file, it's already there but inaccessible to selenium. I don't know what you're pasting or where you're putting it. I don't know why you want to do this or what you're trying to achieve here. Commented Apr 12, 2019 at 10:50
  • @RobertLongson I updated the question explaining the point. Commented Apr 12, 2019 at 11:27
  • grab what the use points to, put that somewhere and then change the use xlink:href to point to the local location of that content. Commented Apr 12, 2019 at 13:07
Related questions
58
Selenium cannot find SVG element in XPath
4
Selenium: SVG element embedded into Object tag is not been identified by Selenium IDE or xPath?
4
Get path in svg using Selenium (Python)
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.