0

I have a python script that I launched with the command scrapy crawl mynamefile.

I would like to run the python cli command scrapy crawl from a js file in my node js application, then display it in the console.

The problem is that I don't how to import the scrapy python library in node.

Thanks for your help!

Improve this question
1
  • Please, provide your code examples for both js/python, but anyway, you can check this guide medium.com/@HolmesLaurence/… Commented Mar 7, 2019 at 10:01

1 Answer 1

Reset to default
2

I don't think it is a good idea to run Python code from NodeJS. If you are for some reason forced to do that, you probably need to call Python script in a child process (see for example this article).

Alternatively, you might use something like Scrapyrt, which exposes HTTP API for your Scrapy project and provides on-demand scraping ability.

Yet the best option seems to be using some native JavaScript scraping solution, like Apify.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.