How to make a function wait for data to appear in the DB? NodeJS

Question

I am facing a peculiar situation.

I have a backend system (nodejs) which is being called by FE (pretty standard :) ). This endpoint (nodejs) needs to call another system (external) and get the data it produces and return them to the FE. Until now it all might seem pretty usual but here comes the catch.

The external system has async processing and therefore responds to my request immediately but is still processing data (saves them in a DB) and I have to get those data from DB and return them to the FE.

And here goes the question: what is the best (efficient) way of doing it? It usually takes a couple of seconds only and I am very hesitant of making a loop inside the function and for the data to appear in the DB.

Another way would be to have the external system call an endpoint at the end of the processing (if possible - would need to check that with the partner) and wait in the original function until that endpoint is called (not sure exactly how to implement that - so if there is any documentation, article, tutorial, ... would appreciate it very much if you could share guys)

thx for the ideas!

You can use recursion or setImmediate/setInterval.

Bipin Shrestha
– Bipin Shrestha

2021-05-29 11:59:22 +00:00
Commented May 29, 2021 at 11:59 — Bipin Shrestha
– Bipin Shrestha, Commented May 29, 2021 at 11:59

Murat Colyaran · Accepted Answer · 2021-05-29 11:54:56Z

1

I can give you an example that checks the Database and waits for a while if it can't find a record. And I made a fake database connection for example to work.

// Mocking starts
ObjectID = () => {};
const db = {
    collection: {
        find: () => {
            return new Promise((resolve, reject) => {
                // Mock like no record found
                setTimeout(() => { console.log('No record found!'); resolve(false) }, 1500);
            });
        }
    }
}
// Mocking ends

const STANDBY_TIME = 1000; // 1 sec
const RETRY = 5; // Retry 5 times


const test = async () => {

    let haveFound = false;
    let i = 0;
    while (i < RETRY && !haveFound) {

        // Check the database
        haveFound = await checkDb();
        // If no record found, increment the loop count
        i++
    }
}

const checkDb = () => {
    return new Promise((resolve) => {
        setTimeout(async () => {

            record = await db.collection.find({ _id: ObjectID("12345") });
            // Check whether you've found or not the record
            if (record) return resolve(true);
            resolve(false);

        }, STANDBY_TIME);
    });
}


test();

answered May 29, 2021 at 11:54

Murat Colyaran

2,2072 gold badges13 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mr.P Over a year ago

thx, I decided to put the main login on the FE side ... so the flow will be as follows: 1) FE will trigger the initial request.. I will process it and let FE know "hey, it's started" 2) FE will then trigger a progress bar and will start asking in intervals to check on the status (similar to what you wrote above just FE-driven) anyway, thx, tagging it as the correct answer :)

Collectives™ on Stack Overflow

How to make a function wait for data to appear in the DB? NodeJS

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related