How to parse XML feed URL and store items in Firestore using cloud functions?

Question

I have been given an assignment to fetch a JSON API, and also parse an XML feed URL and store their responses inside separate Firestore collections. I am not really good at cloud functions, but after lots of research, I have written the cloud function code below for the JSON API and it works well.

const functions = require("firebase-functions");
const axios = require("axios");
const admin = require("firebase-admin");


const api_token = "XXXXXXX";
const includes = "XXXXXX";
const url = "https://XXXXXXXXXXXXXX.com/?api_token=" + api_token + includes;

exports.allLeagues = functions.region('europe-west1').https.onRequest(async (req, res) => {
    try {
        let response = await axios.get(url);
        var data = response.data.data;
            
            for (let leagueData of data) {
                await admin.firestore().collection("leagues").doc(leagueData.id.toString()).collection("all_data").doc(leagueData.id.toString()).set({
                    id : leagueData.id,
                    name : leagueData.name,
                    logo_path : leagueData.logo_path,
                    is_cup : leagueData.is_cup
                });
            }
            console.log("Table complete...");

        console.log("successful");
        return res.status(200).json({ message: "successful" });
    } catch(error) {
        console.log("Error encountered: "+error);
        return res.status(500).json({ error });
    }
});

I am through with the JSON API. But for the XML feed, I don't know where to start. I have done lots of research to no avail. I found this on Stackoverflow but it doesn't address my need. Assuming this is my feed: https://www.feedforall.com/sample.xml , please how do I parse it and save the items inside Firestore? Kindly help. Thank you.

Dharmaraj · Accepted Answer · 2022-09-08 16:07:46Z

1

You can use rss-parser that can be used to fetch data from RSS feeds or parse from XML strings as shown below:

// npm install rss-parser

const Parser = require("rss-parser");
const parser = new Parser();

exports.rssFeedParser = functions.https.onRequest(
  async (request, response) => {
    const rssUrl = "https://www.feedforall.com/sample.xml";
    const feed = await parser.parseURL(rssUrl);
    const { items } = feed;

    const batch = db.batch();

    items.forEach((item) => {
      const docRef = db.collection("rss").doc(); 
      // restructure item if needed
      batch.set(docRef, item);
    });

    await batch.commit();
    response.send("Done");
  }
);

Do note that you can add up to 500 documents only using Batched Writes as in the answer above. If your feed can return more than that, then you should create multiple batches of 500 or add them individually.

answered Sep 8, 2022 at 16:07

Dharmaraj

51.3k8 gold badges68 silver badges100 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

MartinTexy Over a year ago

Thank you for the response. I will try it and come back to accept the answer.

Collectives™ on Stack Overflow

How to parse XML feed URL and store items in Firestore using cloud functions?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related