Execute two process in parallel in Python

Question

I am trying to execute two commands in parallel for 10 seconds using the following piece of code, but the whole process takes more than 10 seconds as you can see in the output. Would you please help me to better understand the reason and the best solution for this question.

stime = datetime.datetime.now()
print stime
commands = ("sudo /usr/local/bin/snort -v -u snort -g snort -c /usr/local/snort/etc/snort.conf -i eth0 &", "sudo gedit test")
for p in commands:
    p = subprocess.Popen(shlex.split(p), stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
    class Alarm(Exception):
        pass
    def alarm_handler(signum, frame):
        raise Alarm
    signal.signal(signal.SIGALRM, alarm_handler)
    signal.alarm(10) #in seconds
    try:
        stdoutdata, stderrdata = p.communicate()
        signal.alarm(0) #reset the alarm
    except Alarm:
        print 'Ooops, taking too long!!!!'
etime = datetime.datetime.now() 
print etime

And the output:

2013-01-08 03:30:00.836412
Ooops, taking too long!!!!
2013-01-08 03:30:16.548519

I like to execute two programs, Snort as a detection engine and Wireshark as a traffic capture tool. In my code I just put 'gedit test' instead of Wireshark. These two packages are analyzing the traffic packets received in a specified period of time (10 seconds). — samaneh
– samaneh, Commented Jan 11, 2013 at 2:08

mgilson · Accepted Answer · 2013-01-11 12:34:31Z

2

I feel like a threading.Timer might be more appropriate:

from threading import Timer
from subprocess import Popen,PIPE
import shlex
import datetime
import sys

jobs = ['sleep 100','sleep 200']

timers = []
processes = []
print datetime.datetime.now()
for job in jobs:
    p = Popen(shlex.split(job),stdout = PIPE)
    t = Timer(10,lambda p=p: p.terminate())
    t.start()
    timers.append(t)
    processes.append(p)

for t in timers:
    t.join()

stdout,stderr = processes[0].communicate()    
stdout,stderr = processes[1].communicate()
print datetime.datetime.now()

edited Jan 11, 2013 at 12:34

answered Jan 11, 2013 at 2:07

mgilson

312k70 gold badges656 silver badges722 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

samaneh Over a year ago

I guess this solution is working. I need to test my jobs which are two network traffic analyzers and then see if they are capturing traffic in parallel. BTW, are these two jobs running in the background?

mgilson Over a year ago

@samaneh -- What do you mean background? They're not blocking your main thread of execution if that's what you're wondering...

samaneh Over a year ago

I was thinking of if I can see the programs in the Terminal while they are running. I just checked your code and it seems that everything is working fine. Many thanks for your help.

samaneh Over a year ago

What I have noticed after running the code is that the process is not producing the output it should. Do we need p.communicate() somewhere in our code?

mgilson Over a year ago

Yeah, you can p.communicate() after you join the threads...I've updated with a solution that I think should work.

Suku · Accepted Answer · 2013-01-11 03:17:48Z

1

import multiprocessing
import subprocess
import shlex
import time

commands = ("echo -n HI-FIRST ", "echo -n HI-SECOND ")
def parallel():
    p = subprocess.Popen(shlex.split(cmd), stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
    stdoutdata, stderrdata = p.communicate()
    print stdoutdata + "\t" + time.ctime()
for cmd in commands:
    p = multiprocessing.Process(target=parallel)
    p.start()

Output:

$ python stack.py 
HI-FIRST    Fri Jan 11 08:47:18 2013
HI-SECOND   Fri Jan 11 08:47:18 2013

edited Jan 11, 2013 at 3:17

answered Jan 11, 2013 at 2:50

Suku

3,9101 gold badge24 silver badges24 bronze badges

1 Comment

samaneh Over a year ago

Is it possible to run two commands for a specified period of time like 10 seconds in parallel?

Collectives™ on Stack Overflow

Execute two process in parallel in Python

2 Answers 2

5 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related