How to modify the same array using multi thread in node js?

Question

Hi does anyone know how to modify a same array by using 2 worker_threads in node js?

I add value in worker thread 1 and pop it in worker thread 2, but worker thread 2 can't see the value added by 1.

//in a.js
const {isMainThread, parentPort, threadId, MessageChannel, Worker} = require('worker_threads');

global.q = [1,2];

exports.setter_q= function(value){
    q.push(value);}

exports.getter_q=function(value){
  var v=q.pop()
  return v;
}

if(isMainThread) {
    var workerSche=new Worker("./w1.js")
    var workerSche1=new Worker("./w2.js")
}

//in w1.js
const {isMainThread, parentPort, threadId, MessageChannel, Worker} = require('worker_threads');

if(isMainThread){
    // do something
} else{
    var miniC1=require("./a.js")
    miniC1.setter_q(250);

    // do something
}

//in w2.js
const {isMainThread, parentPort, threadId, MessageChannel, Worker} = require('worker_threads');

if(isMainThread){
    // do something
} else{
    var miniC1=require("./a.js")
    var qlast=miniC1.getter_q();
    // do something
}

qlast variable in w2.js file is always value '2' instead of 250.

In node.js, you have to allocate something like a SharedArrayBuffer that you can then access from multiple threads. You will have to then manage concurrency properly so you aren't attempting to update the data simultaneously from more than one thread. — jfriend00
– jfriend00, Commented Jul 10, 2020 at 1:15

jfriend00 · Accepted Answer · 2020-07-10 04:09:13Z

3

In node.js, to share memory between threads, you have to allocate something like a SharedArrayBuffer that you can then access from multiple threads. The shared buffer objects are allocated differently that allows them to be accessed by multiple V8 threads in nodejs whereas regular arrays cannot.

You will then have to manage concurrency properly so you aren't attempting to update the data simultaneously from more than one thread (creating race conditions). In the cases where I've used shared memory in node.js WorkerThreads, I've designed the code so that only one thread ever had access to the shared memory at once and that is one way of solving concurrency issues. There are also Atomics in node.js that allow you to "control" access such that only one thread is accessing it at a time.

answered Jul 10, 2020 at 4:09

jfriend00

711k104 gold badges1.1k silver badges1k bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Muhammad Uzair Over a year ago

What if we only want to read data, for comparison with another value, only main thread write to array . Do we still have to implement Atomics ?

jfriend00 Over a year ago

@MuhammadUzair - You have to implement concurrency control if ANY thread (main or worker) is modifying the data. If the data is entirely constant during the time it can be accessed via multiple threads, then you do not need concurrency control.

jfriend00 Over a year ago

@MuhammadUzair - A SharedArrayBuffer holds raw binary data (individual bytes of data). You can't, for example, put a live Javascript object into the buffer. You could serialize a Javascript object to JSON and put the JSON bytes into the array and then the reader could get the JSON out of it or you could convert various primitive properties to binary and put that binary data in the buffer.

henry · Accepted Answer · 2020-07-10 03:33:29Z

0

EDIT: This is apparently outdated and wrong. See comment below.

You can't do that in javascript. Objects to worker threads are passed by value. This is by design so you don't have to deal with locking and all the problems that come when multiple threads can mutate an object.

The way to solve this problem in javascript is to send the work result via a message channel (again, by value).

So if you wanted to further process that object, you could pass it in a message channel from worker 1 to worker 2.

Or if you have a worker pool, you could pass messages to the main thread and add them to a result array there for example.

edited Jul 10, 2020 at 3:33

answered Jul 9, 2020 at 23:14

henry

1501 silver badge12 bronze badges

3 Comments

jfriend00 Over a year ago

This is no longer true now that we have WorkerThreads and shared memory in node.js. Any of the various forms of Shared memory such as SharedArrayBuffer can be accessed from multiple threads in node.js. And, if doing so, one has to use some form of concurrency control to make sure that multiple threads do not have conflicting access.

henry Over a year ago

Apologies, I was not aware of that and edit my answer to reflect that.

Muhammad Uzair Over a year ago

@jfriend00 What kinda of data we can share using SharedArrayBuffer ? Can we share only numeric data ? What about sharing b/w pool of threads ?

Collectives™ on Stack Overflow

How to modify the same array using multi thread in node js?

2 Answers 2

3 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related