I'm looking forward to make a web-scraping app with Vue js.
What are the packages should I get for this project ??
Looking around in web I get the following:
cheerionode-fetchaxioshttps://cors-anywhere.herokuapp.com/Is there anything more I should get to get-started or any better options for above mentioned packages ?
I think a better option suited for this would be Pupeteer. You can setup some cloud functions with Firebase that return Promises with the fetched data. This has to be done as Pupeteer is server side only.
Pupeteer fetching looks like this:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({path: 'example.png'});
await browser.close();
})();
I would keep all the web scraping on the server and not on client side as it can reduce speed.
If you really want to do the web-scraping on the client-side, then use cheerio. You also need to find a way to do a CORS request from the client side.
If you want to do the web-scraping on the server-side, use cheerio or pupeteer, cheerio is enough for most use case, but if you are doing some advance scraping, sometimes cheerio isn't enough, so you need to use pupeteer since pupeteer is a headless browser, unlike cheerio which is just a parser. Use cheerio if it's possible, since its more lightweight than pupeteer.
