NodeJS no code running after worker code finishes

Question

I'm having an issue with my worker code, currently i have some code that scans through two database tables and finds some matches and then adds some data from one table to the other creating a new table. This is a large set of data so i'm using worker threads to speed this up.

This is all working fine however once the worker threads are complete no other code runs i've tried adding the function LogData everywhere i can and it will not run i've even add the console.log("Finished building merge table") and that doesn't run either. Even the parentResolve does happen as i don't see the console.log("parentResolve") message.

if anyone can help me I would really appreciate it.

const calculateFactorialwithWorker = async () => {

    const SCCM = await ProgramDev.find({ "program name": { $not: { $regex: ".*[(]KB*[)]*" } } }).limit(8000)
    const sccmLength = SCCM.length

    mongoose.connection.close()

    return new Promise(async (parentResolve, parentReject) => {
        const numbers = [...new Array(sccmLength)].map((_, i) => i);

        const segmentSize = Math.ceil(sccmLength / userCPUCount);
        const segments = [];

        for (let segmentIndex = 0; segmentIndex < userCPUCount; segmentIndex++) {
            const start = segmentIndex * segmentSize;
            const end = start + segmentSize;
            const segment = numbers.slice(start, end)
            segments.push(segment);
        }
        try {
            const results = await Promise.all(
                segments.map(
                    segment =>
                        new Promise((resolve, reject) => {
                            const worker = new Worker(workerPath, {
                                workerData: segment,
                            });
                            worker.on('message', resolve);
                            worker.on('error', reject);
                            worker.on('exit', (code) => {
                                if (code !== 0)
                                    reject(new Error(`Worker stopped with exit code ${code}`));
                            });
                        })
                ));

            parentResolve(() => {

                console.log("parentResolve")

            })
        } catch (e) {
            parentReject(e)
        }
    });
};

calculateFactorialwithWorker().then(() => {

    console.log("Finished building merge table")
    LogData
})

ikhvjs · Accepted Answer · 2021-08-06 13:30:30Z

1

Add if else block in worker exit event. When exit fired with code === 0 , there is no resolve/reject to handle it. The promises will not be resolved/rejected.

Ref. https://nodejs.org/api/worker_threads.html#worker_threads_event_exit

Also, I rewrite your codes a bit because some promises wrapper is unnecessary.

const calculateFactorialwithWorker = async () => {
  try {
    const SCCM = await ProgramDev.find({
      "program name": { $not: { $regex: ".*[(]KB*[)]*" } },
    }).limit(8000);

    const sccmLength = SCCM.length;

    const numbers = [...new Array(sccmLength)].map((_, i) => i);

    const segmentSize = Math.ceil(sccmLength / userCPUCount);
    const segments = [];

    for (let segmentIndex = 0; segmentIndex < userCPUCount; segmentIndex++) {
      const start = segmentIndex * segmentSize;
      const end = start + segmentSize;
      const segment = numbers.slice(start, end);
      segments.push(segment);
    }

    const promises = segments.map(
      segment =>
        new Promise((resolve, reject) => {
          const worker = new Worker(workerPath, {
            workerData: segment,
          });
          worker.on("message", resolve);
          worker.on("error", reject);
          worker.on("exit", code => {
            if (code !== 0) {
              reject(new Error(`Worker stopped with exit code ${code}`));
            } else {
              resolve();
            }
          });
        })
    );

    await Promise.all(promises);
  } catch (err) {
    throw new Error(err);
  }
};

calculateFactorialwithWorker()
  .then(() => {
    console.log("Finished building merge table");
    LogData();
  })
  .catch(console.log)
  .finally(() => {
    mongoose.connection.close();
  });

edited Aug 6, 2021 at 13:30

answered Aug 6, 2021 at 9:36

ikhvjs

6,0874 gold badges24 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

adam Wadsworth Over a year ago

Thanks for the help i'm still getting the same problem with your code above also now i getting an Error: Error: Worker stopped with exit code 0, however it appears like the code is still working

ikhvjs Over a year ago

@adamWadsworth, I think I know why. The message event is never fired by the worker and the exit event is fired actually. I add a if else to handle the exit event instead. Please check my updates

adam Wadsworth Over a year ago

@ikhbjs well the console.log("Finished building merge table") is now working but it's still not firing off Logdata no error message it just ends and the first line of that function is a console.log("Starting logging data");

ikhvjs Over a year ago

@adamWadsworth, Maybe it should be LogData() instead? I just copy your codes with LogData. I don't see your LogData function in your codes anyway. If you can provide it, it may be more clear.

adam Wadsworth Over a year ago

@ikhbjs That sorted out the problem there was also an issue with the mongoose.connection.close(); being where it was as i needed to call the Database again inside logData so moved it and now everything is working thanks for you help

|

Collectives™ on Stack Overflow

NodeJS no code running after worker code finishes

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related