0

I have a python program running. It is basically does an ssh connection and sustains that connection i.e the program does not enter bash shell again until terminated manually. Something like below,

bash-4.1$ python monitor.py
CONNECTION MADE
one done...
two done...
three done...
.
.
.

I want to same monitor.py to run in parallel for various such ssh connections. All of them are independent of each other and needn't exchange information. Can I achieve this using by Multithreading or Multiprocessing?

2
  • You can run monitor.py multiple times, or you can make it multi-threaded, or even you can fork multiple children. Commented May 12, 2016 at 7:45
  • multithreaded will be best for this scenario Commented May 12, 2016 at 7:47

2 Answers 2

1

Here is a similar example, that uses multiprocessing instead of multithreading (for documentation, see the official docs ). Mulitprocessing works very similarly to multithreading, but it circumvents the Global Interpreter Lock, therefore allowing your script to actually run different processes at the same time, and potentially making better use of limited computing resources.

import multiprocessing as mp

def my_function(*args):
    print("Arguments: {0}".format(args))


class MyProcess(mp.Process):
    def __init__(self, target, args):
        mp.Process.__init__(self, target=target, args=args)

def main():
    a1 = MyProcess(target=my_function, args=("1st Process...",))
    a2 = MyProcess(target=my_function, args=("2nd Process...",))
    a3 = MyProcess(target=my_function, args=("3rd Process...",))
    a4 = MyProcess(target=my_function, args=("4th Process...",))

    proclist = [a1, a2, a3, a4]
    for proc in proclist:
        proc.start()

    for proc in proclist:
        proc.join()

if __name__ == '__main__':
    main()

Output:

Arguments: ('1st Process...',)
Arguments: ('2nd Process...',)
Arguments: ('3rd Process...',)
Arguments: ('4th Process...',)

While these came in in what appears to be a set order, if you add a task that takes a non-deterministic time, they will come in in the order they finish. Just replace the contents of my_function with your code, and you should be set. (Note: this uses Python 2. The same works in Python 3 with very little modification--maybe none--but if you're in Python 3, you should also investigate concurrent.futures).

Sign up to request clarification or add additional context in comments.

1 Comment

I tried with my function making the ssh connection... This works too... Thanks...!!! :)
1

You can make multiple threads to achieve what you have mentioned in your question. Below code is an example of multithreading in Python. Try it with your file.

import thread
import time

# Define a function for the thread
def print_time( threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print "%s: %s" % ( threadName, time.ctime(time.time()) )

# Create two threads as follows
try:
   thread.start_new_thread( print_time, ("Thread-1", 2, ) )
   thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
   print "Error: unable to start thread"

while 1:
   pass

Output

Thread-1: Thu Jan 22 15:42:17 2009
Thread-1: Thu Jan 22 15:42:19 2009
Thread-2: Thu Jan 22 15:42:19 2009
Thread-1: Thu Jan 22 15:42:21 2009
Thread-2: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:25 2009
Thread-2: Thu Jan 22 15:42:27 2009
Thread-2: Thu Jan 22 15:42:31 2009
Thread-2: Thu Jan 22 15:42:35 2009

1 Comment

I tried it with ssh connections... This works... thank you:)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.