122 questions
2
votes
0
answers
45
views
How to share a single WebAssembly.Memory instance between multiple ESM worker threads in Node.js 22?
I am working with Node.js 22 using native ESM and Worker Threads, I'm trying to share a single WebAssembly.Memory instance across multiple workers to avoid duplicating memory.
// main.mjs
import { ...
- 158
1
vote
1
answer
75
views
Typescript, Node, Two Worker Threads and one Main Thread with static Map
I have the following files, which result in an infinite loop through the workers if I even reference the static Map on the main thread - it doesn't even need to run the code referencing it.
Note: It ...
- 71
0
votes
0
answers
47
views
Question about how ffi-napi's async() works
I have a DLL that I built from C code. The DLL continuously runs a while loop that calls a JavaScript callback function every 3 seconds using sleep().
In my Electron project, I’m using the ffi-napi ...
- 1
0
votes
1
answer
52
views
How to increase `MaxListeners` on a EventTarget/BroadcastChannel?
I play a bit with worker threads and need to pass some data between them.
In my code i register for each component a new event listener, which results in a
(node:47363) MaxListenersExceededWarning: ...
- 4,049
1
vote
0
answers
302
views
How to use NodeJS worker_threads with typescript?
I need to use worker_threads in Nodejs (server side question only) + Typescript.
I tried the following without success:
scripts/worker.ts
import { parentPort, workerData } from 'worker_threads';
if (...
- 1,969
0
votes
0
answers
95
views
NextJS Route handler - wait for worker thread to finish
I have a NextJS application (App Router) which includes a Route Handler which triggers generation of a PDF. This process takes a while, which causes the main thread to stall.
The application runs as ...
- 1,185
1
vote
1
answer
90
views
How to make worker_threads in an express app deal with request and response object?
I want to make worker_threads in an express app deal with request and response object so that the main thread of the process is free to accept the next incoming request. I know clustering is a way ...
- 23
0
votes
0
answers
17
views
Passing passThroughStream using buffer from min thread to worker thread--File in worker thread upload issue
main thread
import { Worker } from "worker_threads";
import { PassThrough } from "stream";
export const deligateWork = (action: string, data: any): Promise<any> => {
...
0
votes
0
answers
85
views
Relationship between main thread and worker threads in Node.js: Context isolation and message exchange
I have tried implementing a test in my NodeJS service using Jest.
I've leveraged the auto mocks feature from Jest when mocking node_modules.
Since my application uses workers, I've noticed that the ...
0
votes
2
answers
457
views
How to test a worker thread in NodeJS?
I have a node script that uses a worker thread. I want to test it with Jest. The file is called, but the code is never executed.
Here is the function that calls the worker:
// index.ts
function foo(){...
- 6,427
0
votes
0
answers
42
views
Would it make sense performance wise to use workerpool for websocket client to parse incoming messages?
I am trying to understand how to make node ws client more performant and I have come across worker threads and worker pool. The use case is to parse incoming messages (done in the worker) and pass the ...
- 5
0
votes
0
answers
59
views
Mongoose cursor times out after a couple of hours even though noCursorTimeout flag is set
export const cacheRecommendations = async () => {
const userCursor = UserModel.find({})
.cursor()
.addCursorFlag("noCursorTimeout", true);
let counter = 0;
...
- 1,874
0
votes
1
answer
105
views
Nodejs worker threads implementation issue
I have a test1.ts file where I was trying to offload some CPU-intensive image manipulation work to worker threads. Below is the code shown for file test1.ts and also its invocation at another file. ...
- 21
1
vote
0
answers
409
views
Workerpool says my workers are terminated when I try to execute a function
I'm trying to add workers to my Node.JS application so that I can run jobs after an endpoint gets hit and eventually process files.
I am trying to use the npm package workerpool but when I call my ...
- 105
0
votes
0
answers
102
views
NODEJS - Call function between worker_threads and parent (or vice versa)
Nodejs supports communication between worker and parent using the postMessage method, however it only supports one-way communication.
If you want two-way communication by calling a worker function ...
- 1
0
votes
0
answers
76
views
Running worker file in a loop gives error for nodeJs application
When I am running a worker file inside a nodeJs application in a loop it gives error as :
Module did not self-register: '/server/node_modules/onnxruntime-node/bin/napi-v3/linux/x64/onnxruntime_binding....
- 313
1
vote
1
answer
281
views
Why worker threads aren't working when creating an class instance from other file?
I have this typescript file:
import { Worker, WorkerOptions, isMainThread, parentPort } from "worker_threads";
import path from "path";
export class SimpleWorker {
private ...
- 1,164
0
votes
1
answer
121
views
Worker thread in Twilio’s serverless function?
I’m building a voice application with Twilio’s serverless functions, I need a run a process after gathering user speech input form an incoming call that can take > 15 secs to execute thus resulting ...
1
vote
0
answers
212
views
How many threads Node creates per core when using worker threads?
I'm running node js worker threads on MacBook Air M1. There are 8 core. And using PISCINA to create threads. I'm getting 12 threads. Are those for one core or on all 8 core. How do I know if main ...
4
votes
0
answers
1k
views
How to get worker.threadId inside a worker using worker_threads?
I'm using the nodejs with worker_threads module.
I created several workers.
I need to get the id of each worker within the worker itself.
I know that in the cluster module, just call cluster.worker.id ...
- 93
1
vote
1
answer
2k
views
How to use ES6 import in worker thread in Node.js?
How to use ES6 import in worker thread?
I'm trying to utilize ES6 import syntax in a worker thread in a Node.js environment. However, I'm encountering issues with the import statement. Here's a ...
- 21
0
votes
0
answers
238
views
Looking for a design pattern to get Node.js worker thread results back into a stream on the main thread
I'm using worker threads in Node.js to process a massive ndjson file. I'm trying not to be too specific about what I currently have as far as code because that can be changed depending on the answer ...
- 1
1
vote
0
answers
124
views
Module not found when using typescript in worker_threads
const worker = new Worker('./counter.js')
my index.ts file
import { Worker, isMainThread } from "worker_threads";
if (isMainThread) {
const worker = new Worker('./counter.js')
...
- 19
1
vote
0
answers
76
views
Problems using sqlite3 while processing images with sharp
I'm downscaling images within a separate worker thread using the sharp library. The code snippet goes like this:
async function create_preview(path) {
return lib.sharp(path)
.resize({ width: ...
- 6,703
0
votes
0
answers
571
views
best way to use node-worker-threads-pool with for loop
I'm using node-worker-threads-pool to process a daily function on all the documents from one collection, for that I'm using a for loop to send the id of each document to the pool, I'm trying to get ...
1
vote
1
answer
661
views
Should I use worker threads for sending more than 5000 HTTP requests?
If I want to send messages to more than 5000 users in Node.js, which takes a long time, and I don't want it to be render blocking, is "Worker threads" what I need?
The documentation says:
...
- 797
2
votes
1
answer
1k
views
Worker Thread, other file than self, cannot find module
I want to have my worker thread as an external file from the runtime file.
My current folder structure is
src/
> service.ts // my 'main'
> thread/
>> test.js
inside my service.ts i have ...
0
votes
0
answers
77
views
How can I optimise a Node.js simulation that writes results to a MySQL database?
I have a monte carlo simulation (essentially a simulation of game involving random numbers that I run millions of times) that I've coded in Node.js. The simulation runs a batch of 100,000 games and ...
0
votes
0
answers
280
views
Why is a specific import not working in a worker thread?
I have a worker thread that successfully imports a lot of files using a pattern (TypeORM models, if relevant). Each of those files in return imports other files - each other and unrelated. That has ...
- 716
3
votes
1
answer
969
views
Facing problem of making node worker thread promises
I am trying to make a test of offloading task to worker thread.
The workflow is quite simple, I have 100 promises of array, each promise will create a worker thread.
The main thread will pass the ...
- 113
0
votes
0
answers
275
views
Is it possible to create a fully NodeJS multi-threaded server?
I am working on a server handling lot of requests and then making intensive tasks on database.
I am trying to create a thread to interact with the database using worker_threads but they seem more like ...
- 2,813
1
vote
0
answers
220
views
Cannot use worker_threads in electron main thread
In Electron v21.3.0 and electron-forge, when i try to run a node worker thread in the main process, i cannot require anything from the module 'worler_threads'
// myWorker.js
const { parentPort } = ...
- 425
0
votes
2
answers
942
views
How to specify a local file path in Firebase Functions using worker_threads?
I have a firebase function that uses worker_threads to off load some cpu intensive tasks. To initialize the workerThread I give it a local file path to the worker file. But when the firebase function ...
- 24
0
votes
1
answer
313
views
how to block all worker threads till the first worker thread not completed in node.js
More than one workers thread are started when running test in parallel using WebDriverIO and node.js. There is a method which fetch and lock the data in database so next time different set of data to ...
- 33
0
votes
1
answer
241
views
Custom MessageChannel bizarre behavior in Node.js 19
As the title says, I'm experiencing a behavior that I find pretty strange regarding a re-implementation of mine of the MessageChannel provided by Node.js.
The goal of my implementation is to provide ...
user11874489
0
votes
1
answer
509
views
Error trying to terminate worker thread using setTimeout()
When trying to terminate a worker thread using setTimeout() I get the following error:
node:internal/worker:361
this[kHandle].stopThread();
^
TypeError: Cannot read properties ...
0
votes
1
answer
231
views
Worker threads using 'node-worker-threads-pool' – can't find class at runtime
I may be demonstrating my deep, deep ignorance of threading in node, but this is my first attempt at using worker threads, and the documentation says I ought to use a worker pool.
So here we are. I am ...
- 12.8k
2
votes
0
answers
2k
views
When to use child process over worker threads?
I'm still trying to understand what is worker threads, and how it's different from child process so please bear with me.
So I'm currently building a desktop app with Node.JS + Electron. The app would ...
- 156
1
vote
1
answer
649
views
Worker thread postMessage() vs command line command
I recently learned about Worker threads in Node JS. I was trying to create a worker thread to run Stockfish chess engine in node js.
The npm package I am using for this is called stockfish. I tried ...
0
votes
0
answers
699
views
Node.js worker_threads memory overflow(may be)
When i use worker_threads to handle a lot of complex logic that irrelevant with the main thread, i found that memory on the server very high.
Below is part of my simplified code.
main.js
const worker =...
- 1
0
votes
0
answers
323
views
Parallel processing in Node.js
I'm trying to write a node code that will read multiple json files from directory and insert mapped data into mongodb collection. In order to speed up the process, I have follwed following steps:
Put ...
- 833
1
vote
0
answers
1k
views
How to wait for worker thread to complete in the worker thread itself to do some computations?
I have an existing app which uses worker concept to do audio download from an API. Now I would like to cut the audio once the download is complete in the same worker but the problem is before the full ...
- 11
1
vote
0
answers
441
views
Cannot load data url in Worker Thread
My current worker thread which works fine with file path that tries to console log in that JS file:
console.log("hit here");
Now I get the base64 of that file and use Data URL to replace JS ...
- 458
0
votes
0
answers
184
views
Node Worker thread is not working inside api call
I am using node version 16 and following all documented steps. Still worker thread is not working.
`
const addPaymentApi = async (req) => {
const { payload, adminId } = req.body;
logInfo(`...
- 39
0
votes
2
answers
1k
views
Getting 'Cannot use import statement outside a module' in my test
I've written a small program using Playwright and I'm trying to make it use worker_threads. Now I've written a test for it but for some reason I'm getting the above error in my test.
I tried tweaking ...
- 425
0
votes
1
answer
423
views
Unable to debug typescript project with node:worker_threads, cannot find module error
I have a typescript project which uses node:worker_threads. However, when I try to debug individual ts files, I keep getting error
Error: Cannot find module 'node:worker_threads'
It should be noted ...
- 493
0
votes
0
answers
168
views
How to prevent main process to lock UI after invoking multil
I am building an app which list a set of files. When the user performs an action to one of the files, the action is process intensive (performs multiple ajax requests, updates an offline sqlite ...
- 595
1
vote
0
answers
390
views
Is it possible to pass stdout to the nodejs worker?
I write a nodejs console application that does some logic that blocks the main thread.
I also want to use ora terminal spinner:
const sp1 = ora('runnning job 1').start();
job1();
sp1.succeed();
...
- 11.9k
2
votes
1
answer
2k
views
Node.js- share a heavy variable from parent to worker threads
I want to share a variable between parent and worker threads. The variable is heavy (100+ MB - arrays of JSON data) so if I pass it as an argument or re-read from data source, it drastically increases ...
- 363
0
votes
0
answers
301
views
How do i sync Data between Worker Threads / Clusters in Node.js
I have a question how do i "sync" data, for example userdata in cache for all workers / cluster in a node js Cluster Thread?
Like a static variable but for all threads.
for example a static ...
- 36