I have a process that creates several subprocesses with which it needs to coummunicate in Python 2.5. I use the subprocess module to start the processes, setting stdin and stdout to subprocess.PIPE. So far so good.
In a loop, I then, run select.select() on each subproccess's stdout stream waiting for them to tell me they're ready. I then write to their stdin to give them work and repeat the process.
At the moment, I am simply using stdin.write and stdout.readline to preform the communication. However, I'd like to be able to communicate complex messages between the processes. Simple newline terminated messages won't suffice (unless I take strides to escape new lines in the messages somehow). I am thinking I can prefix all messages with the length in bytes so that my messages look something like:
6:foobar
But this brings me to my question: does something like this already exist? I don't really want to reinvent the wheel here. I want a library that tells me when there's a complete message ready and hands it over to me. There's plenty of other protocols that do things like this in various ways (TCP, Message Queue Servers, HTTP, etc) but they're all overkill for my use case. What's the right way to do message passing between processes in Python?
