Executing a binary semaphore or mutex for context switching operation in python multiprocessing

Question

I'm trying to automate a sync relation between a win application and a java application. My criteria is :

start win and jav application
execute command in jav application
wait for response from jav application
use the response from jav application into windows application as input.
execute command in win application
wait for response from win application
use the response from win application into java application as input and repeat.

(Note we're executing commands within the console of the application i.e. both applications are required to be active processes.)

I'm having a hard time trying to implement a mutex or a binary semaphore.

Here is a code snippet I'm trying to complete:

import datetime
import time
import multiprocessing




class Win_App():
    def __init__(self):
        self.input_command = [] # list of string
        self.output_log = [] #list of string
        self.next_jav_command=''

    def execute_win_application(self):
        exe_path = 'path_ to_exe'
        init_command= 'command to execute '
        win_process = subprocess.Popen('cmd.exe', stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                                       stderr=subprocess.STDOUT, text=True)
        win_process.stdin.write(exe_path)
        win_process.stdin.write(init_command)
        
        
        print('Win Application Started !')
        while True:
            #TODO: mutex/semaphore acquire and start tasks
            win_process.stdin.write(self.input_command[-1]) # execute the last command in the list
            for lines in process.stdout():
                write_into_file("win_app_output.log",line)
                self.output_log.append(lines)
            
            self.next_jav_command = parse_lines_and_get_command()
            
            if (nex_win_command == '-1'):
                self.next_jav_command == 'exit'
                break:    
             
            # TODO: hold the process here and move to Java application
            # TODO: release mutex/ semaphore
        
         
class Java_App():
    def __init__(self):
        self.input_command = [] # list of string
        self.output_log = [] #list of string
        self.win_next_command = ''
        
    def execute_java_application(self):
        jav_path = 'path_ to_java_app'
        init_command= 'command to execute '
        jav_process = subprocess.Popen('cmd.exe', stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                                       stderr=subprocess.STDOUT, text=True)
        
        
        print('Java Application Started!')
        while True:
            #TODO: mutex/semaphore acquire and start tasks
            jav_process.stdin.write(self.input_command[-1]) # execute the last command in the list
            for lines in process.stdout():
                write_into_file("jav_app_output.log",line)
                self.output_log.append(lines)
            
            
                
            nex_win_command = parse_lines_and_get_command()
            
            if (nex_win_command == 'exit'):
                break:
            # TODO: hold the process here and move to Java application
            # TODO: release mutex/ semaphore
        jav_process.kill()    


##### MAIN PROGRAM #####

if __name__ == '__main__':
    # Initialize process given certain parameters
    win = Win_App()
    jav = Java_App()

    p1 = multiprocessing.Process(target=win.execute_win_application)
    p1.start()
    print(" win App started!")
    p2 = multiprocessing.Process(target=jav.execute_java_application)
    p2.start()
    
    while True:
        # TODO : implement context switch between process in execution
        #        like a binary semaphore or mutex

I'm able to handle the miniature functions/methods but the I'm unable to implement the #TODO tasks in above code.

Those are not Python comments. What's going on?

Hai Vu
– Hai Vu

2024-07-24 05:06:42 +00:00
Commented Jul 24, 2024 at 5:06 — Hai Vu
– Hai Vu, Commented Jul 24, 2024 at 5:06
This sounds like a job for Pexpect, to be honest :)

AKX
– AKX

2024-07-24 06:10:46 +00:00
Commented Jul 24, 2024 at 6:10 — AKX
– AKX, Commented Jul 24, 2024 at 6:10

Hai Vu · Accepted Answer · 2024-07-24 17:47:09Z

1

I have attempted to send input line by line to a subprocess.Popen in the past without any luck. The sure bet is to use pexpect as AKX pointed out.

Here is a toy example to get you started. The first thing to do is to create a virtual environment and within that virtual environment install pexpect.

As part of this toy example, I wrote 2 apps:

app1.py: Take a string and reverse it
app2.py: Take a sentense and reverse every words

Code:

#!/usr/bin/env python3
"""
app1.py
Print text in reversed
"""
while (input_text := input("> ")) != "q":
    print(input_text[::-1])

#!/usr/bin/env python3
"""
app2.py
Reverse each words
"""

while (text := input("> ")) != "q":
    words = [word[::-1] for word in text.split()]
    print(" ".join(words))

#!/usr/bin/env python3
"""
main.py
Spawn 2 different apps, send input to them and receiving output.
"""

import pexpect

PROMPT = "> "


def main():
    """App Entry"""
    app1 = pexpect.spawn("python3 app1.py", echo=False)
    app2 = pexpect.spawn("python3 app2.py", echo=False)

    text_input = [
        "The more you learn",
        "I love my dog",
    ]

    for text in text_input:
        app1.expect(PROMPT)
        app1.sendline(text)
        output = app1.readline()
        output = output.rstrip().decode()  # Convert bytes to str
        print(f"{text} -> {output} ", end="")

        app2.expect(PROMPT)
        app2.sendline(output)
        output2 = app2.readline()
        output2 = output2.rstrip().decode()  # Convert bytes to str
        print(f"-> {output2}")

    # Quit apps
    app1.expect(PROMPT)
    app1.sendline("q")

    app2.expect(PROMPT)
    app2.sendline("q")


if __name__ == "__main__":
    main()

Output:

The more you learn -> nrael uoy erom ehT -> learn you more The
I love my dog -> god ym evol I -> dog my love I

A few notes:

When spawning, I use echo=False to prevent the echo of the input text
Output are bytes, which include CR LF. That is why I have to convert to string and strip those CR LF off

edited Jul 24, 2024 at 17:47

answered Jul 24, 2024 at 14:42

Hai Vu

41.4k16 gold badges75 silver badges106 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Raj Sawant Over a year ago

Thanks for the update. The above programs work successfully using the note below, however if I'm trying to execute an exe application or even cmd.exe- execution fails. Note- I had to use pexpect.popen_spawn.PopenSpawn to make it work on windows.

Hai Vu Over a year ago

I am sorry. I don't have a Windows system to try out the .exe scenario.

Collectives™ on Stack Overflow

Executing a binary semaphore or mutex for context switching operation in python multiprocessing

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related