Multithreading in python

Question

I am trying to create AMI of EC2 instances using python. I need to implement multi threading so that AMI creation can run in parallel. But below code is not working. It is creating single AMI at a time.

module1

from Libpy import Libpy
import boto.ec2
import sys
if __name__=='__main__':

  a = B()
  Libpy().multithreading(ec2list1,a.create_amibkp(ec2listname,ec2list1))

module2

import threading
import time

class Libpy:

    def multithreading(l, function):
        threads = []
        for z,v in enumerate(l):
            dummy = z
            t = threading.Thread(target=function, args=(v,))
            threads.append(t)
            t.start()

ec2listname - contains the list of instance names for whom AMI creation should be done
ec2list1 - contains the list of instance ids for whom AMI creation should be done
create_amibkp - customized function which takes AMI for the filtered instances From module1, I am calling module2 to multithread the function but it is not working. It is creating AMI one at time. 4.B() - class will return ec2listname and ec2list1

BPL · Accepted Answer · 2016-09-06 10:33:02Z

1

I can't reproduce your example but I've created a dummy version:

import sys
import threading
import time
import random

random.seed(1)


class Libpy:

    def multithreading(self, lst, function):
        threads = []
        for v in lst:
            t = threading.Thread(target=function, args=(v,))
            t.start()
            threads.append(t)

        for t in threads:
            t.join()


def ami_creation(v):
    t = random.random() * 4
    print("creating ami {0}, sleeping {1}".format(v, t))
    time.sleep(t)
    print("ami {0} created ok".format(v))

ec2list1 = ["ec2-small", "ec2-medium", "ec2-large"]
Libpy().multithreading(ec2list1, ami_creation)

print("All boxes have been created...")

In the above example the main thread would wait till all boxes are created

answered Sep 6, 2016 at 10:33

BPL

10k12 gold badges69 silver badges135 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

BPL Over a year ago

In my answer I've showed you how to spawn multiple threads in parallel (no delay of triggering) and only when all threads have finished their business the main thread will continue executing, that's because of the line for t in threads: t.join(). I'd recommend reading about thread-objects, that will clarify much better than any short comment here

Dave Over a year ago

Hi, I am getting below error when trying to execute the AMI backups BotoServerError: BotoServerError: 503 Service Unavailable <?xml version="1.0" encoding="UTF-8"?> <Response><Errors><Error><Code>RequestLimitExceeded</Code><Message>Request limit exceeded.</Message></Error></Errors><RequestID>30694d63-d23e-4f28-a892-ab9d958ca0fa</RequestID></Response> Please help

Collectives™ on Stack Overflow

Multithreading in python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related