Python run bash script with flags and pipes

Yes, you can use subprocess with pipe. i will ilustrate an exemple:

ps = subprocess.Popen(('whois', 'stackoverflow.com'), stdout=subprocess.PIPE)
output = subprocess.check_output(('grep', 'Registrar WHOIS'), stdin=ps.stdout)
ps.wait()

You can ajust as your's need

The easiest solution is to write the commands into a script file and execute that file.

If you don't want that, you can execute any command with

bash -c 'command'

This is covered in the Replacing Older Functions with the subprocess Module section of the docs.

The example there is this bash pipeline:

output=`dmesg | grep hda`

rewritten for subprocess as;

p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output = p2.communicate()[0]

Note that in many cases, you don't need to handle all of the same edge cases that the shell handles in exactly the same way. But if you don't know what you need, it's better to be fully general like this.

Your $() does the same thing as the backticks in that example, your pipes are the same as the example's pipes, and your arguments aren't anything special.

So:

whois = Popen(['whois', 'stackoverflow.com'], stdout=PIPE)
grep = Popen(['grep', 'Registrar WHOIS Server:'], stdin=whois.stdout, stdout=PIPE)
whois.stdout.close()
cut = Popen(['cut', '-f2-', '-d:'], stdin=grep.stdout, stdout=PIPE)
grep.stdout.close()
inneroutput, _ = cut.communicate()
whois = Popen(['whois', '-h', inneroutput, 'stackoverflow.com'], stdout=PIPE)
grep = Popen(['grep', 'Admin Email:', stdin=whois.stdout, stdout=PIPE)
whois.stdout.close()
cut = Popen(['cut', '-f2-', '-d:'], stdin=grep.stdout)
grep.stdout.close()
cut.communicate()

If this seems like a mess, consider that:

• Your original shell command is a mess.
• If you actually know exactly what you're expecting the pipeline to do, you can skip a lot of it.
• All of the stuff you're doing here could just be done directly in Python without the need for this whole mess.
• You may be happier using a third-party library like plumbum.

How could you write the whole thing in Python without all this piping? For example, instead of using grep, you could use Python's re module. Or, since you're not even using a regular expression at all, just a simple in check. And likewise for cut:

whois = subprocess.run(['whois', 'stackoverflow.com'], 
                       check=True, stdout=PIPE, encoding='utf-8').output
for line in whois.splitlines():
    if 'Registrar WHOIS Server:' in line:
        registrar = line.split(':', 1)[1]
        break
whois = subprocess.run(['whois', '-h', registrar, 'stackoverflow.com'],
                       check=True, stdout=PIPE, encoding='utf-8').output
for line in inner.splitlines():
    if 'Admin Email:' in line:
        admin = line.split(':', 1)[1]
        break