Correct way in Python to use subprocess.run when PIPE is in the command [duplicate]

Question

I am trying to write a python script that executes the following terminal command:

echo -n | openssl s_client -connect {host}:{port} | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > {host}_{port}.cert

If I try to break up the command into arguments to pass to the subprocess.run it does not work (something is run but it does not store the certificate as I would like it to.

Using the below sytax correctly executes the command, however I fear it is not best practice and wanted to understand the correct way for how this should be done:

store_certificate_command = f"echo -n | openssl s_client -connect {host}:{port} | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > {host}_{port}.cert"

subprocess.run(store_certificate_command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

Is there some reason you need to use echo and sed? Your echo is equivalent to passing subprocess.DEVNULL as the stdin. Your sed command is trivially implementable in Python, which can write the results to the file itself. All you really need to run is the openssl command. — ShadowRanger
– ShadowRanger, Commented Apr 7, 2021 at 1:16
The pipes are handled by the shell therefore you must either do it as shown by you or you would have to do the "plumbing" yourself, call the different commands in command line separately and feed the output of one command as input into the next. — Michael Butscher
– Michael Butscher, Commented Apr 7, 2021 at 1:17
@ShadowRanger I am actually just following someone else's solution for how to download a certificate. I agree that I can use python instead for parts of the command that require pipe (honestly can achieve the entire result with just python) but I would still like to understand the best practice for running commands that require pipe. — AnotherCourier
– AnotherCourier, Commented Apr 7, 2021 at 1:22
@MichaelButscher I did try the second solutio in this link: thomas-cokelaer.info/blog/2020/11/… (feeding p1 into p2, p2 into p3, etc) but got the error at the bottom. — AnotherCourier
– AnotherCourier, Commented Apr 7, 2021 at 1:24
Don't use run if you are doing your own plumbing. The object it returns has an attribute stdout which is however not a filehandle. — tripleee
– tripleee, Commented Apr 7, 2021 at 1:32

ShadowRanger · Accepted Answer · 2021-04-07 01:31:56Z

The correct way to do this is to limit yourself to the minimum amount of non-Python executables. In this case, echo isn't necessary, Python can do the work sed is doing, and can write the resulting file as well. A clean solution would be something like:

import subprocess

with open(f'{host}_{port}.cert', 'wb') as outf,\
     subprocess.Popen(['openssl', 's_client', '-connect', f'{host}:{port}'], stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) as proc:
    for line in proc.stdout:
        if b'-BEGIN CERTIFICATE-' in line:
            outf.write(line)
            break
    else:
        raise ValueError("BEGIN CERTIFICATE not found in output")

    for line in proc.stdout:
        outf.write(line)
        if b'-END CERTIFICATE-' in line:
            break
    else:
        raise ValueError("END CERTIFICATE not found in output")

I switched to subprocess.Popen instead since it allows you to process the output in a streaming fashion (the same way the shell pipes would work), but given the relatively small output, subprocess.run would likely work just fine too. The ValueErrors aren't strictly necessary, but I like having them there so you fail hard when the output isn't what you expect.

Collectives™ on Stack Overflow

Correct way in Python to use subprocess.run when PIPE is in the command [duplicate]

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related