Is it possible to fork child processes and wait for them to return in Node/JS?

Question

I have to do a certain calculation many many times. I want to split it over multiple cores, so I want to use child_process.fork(), and then pass each child a chunk of the work.

This works fine, except that my main process (the one that does the fork()) just keeps going after fork()ing and has already terminated by the time the children complete their work.

What I really want to do is wait for the children to finish before continuing execution.

I can't find any good way to do this in Node. Any advice?

Look into the async module, which can be used for managing and synchronizing callbacks like this. — Joe
– Joe, Commented Oct 17, 2012 at 20:36
waitpid is the system call that you're looking for. Or you can use the close/end events emitted by child process objects. — Mark K Cowan
– Mark K Cowan, Commented Apr 9, 2017 at 13:02

jAndy · Accepted Answer · 2012-10-17 20:20:55Z

5

If you spawn a new V8 process via .fork(), it returns a new child object which implements a communication layer. For instance

var cp    = require( 'child_process' ),
    proc  = cp.fork( '/myfile.js' );

proc.on('message', function( msg ) {
    // continue whatever you want here
});

and within /myfile.js you just dispatch an event when you're done with the work

process.send({ custom: 'message' });

Be aware of the fact that this method indeed spawns a new V8 instance, which eats a good chunk of memory. So you should use that very thoughtfully. Maybe you don't even need to do it that way, maybe there is a more "node like" solution (using process.nextTick to calculate heavy data).

answered Oct 17, 2012 at 20:20

jAndy

237k57 gold badges313 silver badges363 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user844942 Over a year ago

Thanks for your answer. I've gotten this far. My problem is, the process which calls fork() continues execution after fork()ing, and by the time the child process responds (with process.send()), the initial process has already terminated so the response is useless. I'm looking for a way to force my parent process to wait for all children to finish execution before continuing along. Kind of like wait() in C.

jAndy Over a year ago

@user844942: well, that would entirely pervert the node event model. You seriously should seek for another solution which goes hand in hand with event looping and cooperative multitasking.

Gates VP Over a year ago

As some mentioned above, take a look at the async module, that should have the type of wait semantics you are looking for.

user844942 Over a year ago

@jAndy thanks for your answer. After giving it some thought I realized I WAS going about it all wrong.

Alexander Mills Over a year ago

process.nextTick is not the way to process heavy data! It just kicks the can down the road.

Collectives™ on Stack Overflow

Is it possible to fork child processes and wait for them to return in Node/JS?

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related