Correctly shutting down subprocesses in Python [duplicate]

Question

I have a Python server that starts multiple subprocesses via subprocess.Popen(). When I run a specific method on the server, I want to terminate all of these processes.

My approach to this was to keep references to all processes in a list ...

self.procs.append(subprocess.Popen(shlex.split(cmd)))

... and send interrupts to all of these to shut them down:

for i, p in enumerate(self.procs) :
    p.send_signal(signal.SIGINT)
    print 'interrupted ' + str(i)
self.procs = []

However, this only works for some of the subprocesses. For instance, one of the processes that does not stop after this call is a bash script that has a trap function for SIGINT signals. If I manually start this script and interrupt it via keyboard, it correctly executes the trap statement and closes . When the server sends the signal instead, the script seems to keep running.

What could be going wrong here?

edit: I have also tried using the SIGTERM signal instead, which did not work either.

The usual signal to send for a shutdown is SIGTERM. Often we might have a "grace-time" delay and send SIGKILL (like kill -9) if the process still has not closed. The interrupt might not work if the script is waiting on a child process to complete at the time. — cdarke
– cdarke, Commented Sep 9, 2015 at 12:20
I have also tried SIGTERM which still did not solve my problem. Also, I assume my problem has its root elsewhere since my bash script traps SIGINT specifically, but does not seem to execute the trap statement when getting the signal from the server. — zinfandel
– zinfandel, Commented Sep 9, 2015 at 12:24
Try to group all processes in a group and kill it. as mentionned here. stackoverflow.com/questions/4789837/…. — Ali SAID OMAR
– Ali SAID OMAR, Commented Sep 9, 2015 at 14:28
if you want to kill a process tree then use the solution from @AliSAIDOMAR 's comment (if you don't need to shutdown the processes gracefully; you could send SIGKILL immediately) — jfs
– jfs, Commented Sep 10, 2015 at 10:46

zinfandel · Accepted Answer · 2015-09-10 16:25:02Z

1

Thanks everyone for your contributions, what worked for me in the end was the approach that Ali linked. I started all subprocesses with this additional parameter:

preexec_fn=os.setpgrp

And then called

os.killpg(p.pid, signal.SIGTERM)

SIGTERM, SIGINT, kill() or terminate() alone did not work for my shell script. The script was internally starting two services (sudo service start) and was supposed to sudo service stop them in a trap function, which did not work as expected.

I actually had some more trouble with another process that wouldn't shut down - a Python server that was not deployed but simply started as a script. No idea why the killpg approach did not work with this one. I finally resolved the issue by having it write its pid to a file and later have the parent script read it out and kill that pid.

answered Sep 10, 2015 at 16:25

zinfandel

4581 gold badge5 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Correctly shutting down subprocesses in Python [duplicate]

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related