0

Desired goal

pass information to other sub processes or main process

Concerns

-Is it a problem that the processes never finish?

-Is it a problem the class methods never return/finish?

will this cause stack overflow? ( I expect to run for hours).

Alternative

Would it be appropriate to have several different scripts and save to, load from a database/pickle?

What works

If I swap the multiprocessing for threading the desired output is achieved.

Have tried

-Tried switching the multiprocessing for threading which seems to give me the desired outcome, but would like to understand why multiprocessing is not giving the same output.

-Tried using Queue() instead or LifoQueue()

-Tried running and printing Queue.get() from another child subprocess instead of the main script

-Tried .join() all tasks

Desired output

I expect the color to be changed within a subprocess and passed to the Queue, I expect to retrieve the new color with Queue.get().

Example

import multiprocessing
import threading
import time

class MyClass:
    def __init__(self):
        self.color = 'gray'
    
    def task_blue(self,q):
        print('sleeping 5..')
        time.sleep(5)
        while True:     
            time.sleep(1)
            self.color = 'blue'
            q.put([self.color])

    def task_red(self,q):
        print('sleeping 5..')
        time.sleep(5)
        while True:
            time.sleep(3)
            self.color = 'red'
            q.put([self.color])
        
    def printer(self,q):
        while True:
            time.sleep(.1)
            if q.empty():
                print('<empty>')
            else:
    
                print(q.get())

from queue import LifoQueue, Queue        
q = LifoQueue()
my_class = MyClass()

p1 = multiprocessing.Process(target=my_class.task_blue,args=(q,),name='BLUE')
p2 = multiprocessing.Process(target=my_class.task_red,args=(q,),name='RED')
# p3 = multiprocessing.Process(target=my_class.printer,args=(q,),name='PRINTER')

tasks = []
tasks.append(p1)
tasks.append(p2)
# tasks.append(p3)

for task in tasks:
    task.start()

while True:
    time.sleep(.2)
    if q.empty():
        print(['empty'])
    else:
        print(q.get())

output

output

1 Answer 1

1

Is it a problem that the processes never finish?

It's OK for a process to run forever.

Is it a problem the class methods never return/finish?

No. If you want your program to run forever, there is going to be some section of code that doesn't return. It's OK if this area of code is inside a method.

I like the idea of the entire program being one script/class

You can do it many ways. There's no correct answer here, especially absent other requirements.

Tried using Queue() instead or LifoQueue() -Tried running and printing Queue.get() from another child subprocess instead of the main script -Tried .join() all the tasks

when using the multiprocessing package, you should use the Queue class from the multiprocessing package. E.g. queue = multiprocessing.Queue(). Docs here: https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue

Sign up to request clarification or add additional context in comments.

1 Comment

Thank you, you've cleared up some fundamental concerns, and the script is now working as I wanted by simply using multiprocessing.Queue(). It seems one is not able to use LIFO in such a case?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.