How to run multiple python files at the same time?

Question

How can I run multiple python files at the same time? There are 3 files: bot_1.py, bot_2.py, bot_3.py. I would like to run them at the same time. I attached the code. What should I write in the worker function to make this script work? I will appreciate any help.

import multiprocessing
import subprocess
def worker(file):
    #your subprocess code
    subprocess.Popen(['screen', './bot_1.py'])
    subprocess.Popen(['screen', './bot_2.py'])
    subprocess.Popen(['screen', './bot_3.py'])

if __name__ == '__main__':
    files = ["bot_1.py","bot_2.py","bot_3.py"]
    for i in files:
        p = multiprocessing.Process(target=worker(i))
        p.start()

Jon Betts · Accepted Answer · 2018-02-21 11:59:28Z

Assuming that your bot files execute something when you run them at the command line, we can load them and execute them by importing them into our python process (instead of the shell). As each python file defines a package we can do this as follows:

import bot_1, bot_2, bot_3

This however will run them one after another, and also prevent you from running the same one twice. To get them to run at once, we can use multiprocessing as you suggest:

import multiprocessing

for bot in ('bot_1', 'bot_2', 'bot_3'):
    p = multiprocessing.Process(target=lambda: __import__(bot))
    p.start()

Processes need a function to run, so we use an anonymous lambda to give it one, and then dynamically import the name.

It's not shown here, but as long as you don't import a module into the parent process, the children will be forced to load it, meaning you can run the same one over and over in the separate process if you need to.

F. Leone · Accepted Answer · 2018-02-21 11:54:48Z

1

From the docs, In multiprocessing, processes are spawned by creating a Process object and then calling its start() method.

So i guess the good way to do it is with the following:

bots = ['bot1','bot2','bot3']
modules = map(__import__,bots)

import multiprocessing,subprocess

multiprocessing.Process(target=modules)

In bot1,2,3 i used a simple print("bot1"), bot2 and bot3 and the output is as expected:

user@machine:~$ python mainscript.py 
bot1
bot2
bot3

answered Feb 21, 2018 at 11:54

F. Leone

5955 silver badges14 bronze badges

Collectives™ on Stack Overflow

How to run multiple python files at the same time?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related