Where is a good place to see best practice for controlling processes.
Specifically I want to know how to give a shutdown command. I was thinking a global boolean that each process polls
Also a good way to pass data to and from and in-between active processes.
I have not found any good blog posts they are mostly how-tos
The idea of a global boolean is quite good, except that you can't share it between processes. But you can keep a copy of that boolean in each subprocess.
One way would be to use pipes and/or queues together with threads. Something like that:
from multiprocessing import Process, Pipe
from threading import Thread
Alive = True
def listener_thread(conn):
global Alive
while True:
data = conn.recv()
if data == "kill":
Alive = False
break
def subprocess(conn):
t = Thread(target=listener_thread, args=(conn,))
t.start()
while Alive:
# do some stuff here
if __name__ == '__main__':
parent_conn, child_conn = Pipe()
p = Process(target=subprocess, args=(child_conn,))
p.start()
# do some stuff
parent_conn.send("kill")
p.join()
Note that you will need a (parent_conn, child_conn) pair per each subprocess. That is a standard way of communicating between process and subprocess.
Or you could use a database (and by database I mean any database, even a simple file) to share data between processes. However you will need a polling thread for that which might not be efficient.
You could also use some pub/sub system (so you won't have to worry about polling efficiency) like Redis which could be the best depending on your needs.
The less processes share the better in general.
