How to extract CSS styles using Scrapy

Question

i am web scraping using scrapy and selenium webdrivers. But i find selenium webdriver to be very slow, but i find it easier to extract CSS properties of a webelement eg.

webElement.value_of_css_property('font-size')

how can this be achieved using scrapy only and no selenium webdriver.

Community · Accepted Answer · 2017-05-23 11:51:25Z

1

For this you have to have a rendered content in a real browser. Scrapy downloader is not a browser and has only the initial HTML page, there is no javascript engine built-in and it cannot download additional CSS or JS files.

With Scrapy you can only get the style attribute value of an element, but nothing more than that. selenium is the tool for the job.

Note that you can avoid having a real display as a dependency and automate phantomjs headless browser or other browsers in a virtual display.

edited May 23, 2017 at 11:51

CommunityBot

11 silver badge

answered Aug 10, 2015 at 12:55

alecxe

476k127 gold badges1.1k silver badges1.2k bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to extract CSS styles using Scrapy

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related