What is the alternative of Selenium for executing Js in Python-Scrapy?

Question

I want to execute JavaScript's function in Python-Scrapy, before, i used Selenium, but Selenium is too much slow for scraping the big sites.

I would like to know what's the best alternative for selenium for executing js in Scrapy?

Have you tried webdriver.js ?

Karthikeyan
– Karthikeyan

2016-12-26 10:55:37 +00:00
Commented Dec 26, 2016 at 10:55 — Karthikeyan
– Karthikeyan, Commented Dec 26, 2016 at 10:55
@Karthikeyan yes, it's not a good choice for the big site

parik
– parik

2016-12-26 11:33:41 +00:00
Commented Dec 26, 2016 at 11:33 — parik
– parik, Commented Dec 26, 2016 at 11:33

Community · Accepted Answer · 2020-06-20 09:12:55Z

15

Splash - A javascript rendering service

Splash is a javascript rendering service with an HTTP API. It's a lightweight browser with an HTTP API, implemented in Python using Twisted and QT.

It's fast, lightweight and state-less which makes it easy to distribute.

Bonus points for it being developed by the same guys who are developing Scrapy.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Dec 26, 2016 at 10:55

Granitosaurus

21.6k6 gold badges64 silver badges88 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

What is the alternative of Selenium for executing Js in Python-Scrapy?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related