2

I am spawning 3 processes using multiprocessing.process and waiting for them to complete. If one of them fails then i want to stop all other processes and also the main program. But when i use sys.exit the execution is only stopping just the process and not the main program. Here is the snippet of the code.

proc1=process(function1)
proc2=process(function2)
proc3=process(function3)
proc1.start
proc2.start
proc3.start
proc1.join
proc2.join
proc3.join

. . .

I am running some tasks in functions 1,2 and 3. I have a condition in each function to check the returncode of the task and if the returncode is not success then i would like to stop proc1,proc2 and proc3 and stop execution of the main program. When i execute sys.exit inside the function it just comes out of that process and not the main program.

2 Answers 2

2

For this to work you need to have communication between the worker processes and the main process. Probably the simplest way is to use multiprocessing.Event.

Before starting the processes, create a pair of multiprocessing.Event. Give them meaningful names like stop_main and stop_workers. For portability, one should give add these Events to the arguments given for the Process target.

A worker process should call stop_main.set() when it wants the main program to exit. A worker process should also call stop_workers.is_set() regularly and exit when this returns True.

After the main process starts all the workers it should keep polling stop_main.is_set(). When that returns True it should call stop_workers.set(), join the workers and exit.

Updated:

Edited to make it shorter and hopefully make it work on ms-windows.

An example:

import multiprocessing as mp
import time


def worker(num, sw, sm):
    if num == 5:
        print('This is worker', num)
        time.sleep(1)
        print('Worker', num, 'signalling main program to quit')
        sm.set()
    while not sw.is_set():
        print('This is worker', num)
        time.sleep(0.7)
    else:
        print('Worker', num, 'signing off..')


if __name__ == '__main__':
    stop_worker = mp.Event()
    stop_main = mp.Event()

    workers = [mp.Process(target=worker, args=(n, stop_worker, stop_main))
               for n in range(1, 6)]
    for w in workers:
        w.start()
    while not stop_main.is_set():
        time.sleep(1)
    print('MAIN: Received stop event')
    print('MAIN: Sending stop event to workers')
    stop_worker.set()
    for c, w in enumerate(workers, start=1):
        w.join()
        print('worker', c, 'joined')

It runs like this:

This is worker 1
This is worker 2
This is worker 3
This is worker 4
This is worker 5
This is worker 2
This is worker 3
This is worker 1
This is worker 4
Worker 5 signalling main program to quit
This is worker 5
This is worker 2
This is worker 3
This is worker 1
This is worker 4
This is worker 5
MAIN: Received stop event
MAIN: Sending stop event to workers
Worker 3 signing off..
Worker 1 signing off..
Worker 2 signing off..
worker 1 joined
worker 2 joined
worker 3 joined
Worker 4 signing off..
worker 4 joined
Worker 5 signing off..
worker 5 joined
Sign up to request clarification or add additional context in comments.

2 Comments

dep_event = Event() proc1=process(function1, args= dep_event) proc1.start while True if dep_event.is_set() == "True": sys.exit(1) .... ..... .... .... function1(depp_event) .... ... .... depp_event.set() I tested the above code and still it doesn't end the main program
I am able to able to fix the issue and now able to stop the main program using the events. Thanks a lot for your timely help. But i can creating a subprocess(shell = True enabled) from this process, So if i terminate this process it is not killing the subprocess , So how do i make sure that i kill both the process and subprocess before exiting the main program.
-1

Here's a solution. You lose all communication to the launched process once __main__ exits, but that's ok for my use-case with matplotlib. Just close the window or, as you mentioned, close the shell.

The example script is located in detail here.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.