Catching python subprocess exception

Question

I have a python script where i am calling another script using subprocess as below.

        sp = subprocess.Popen("script.py --arg1 --arg2', cwd=GIVEN_PATH, shell=True, stdout=subprocess.PIPE,  stderr=subprocess.STDOUT)

        while sp.poll() is None:
            for out in iter(sp.stdout.readline,''):
                self.log(out.rstrip())

This is working fine for me but i want to get any exception from the script.py. I know we can get the retcode but i actually need to get the full exception information.

If script.py raises

         raise IOERROR("got an exception")

then i need to know this information. simailar way we get the sys.exc_info() etc.

is there a way i can do this?

Thanks in Advance

In addition to the answer below, it's useful to note that Python can call other python modules directly, without the need for a subprocess, which will allow you to catch Exceptions. This is what you should do in this case. — VooDooNOFX
– VooDooNOFX, Commented Jan 6, 2015 at 2:50

Daniel Pryden · Accepted Answer · 2015-01-06 01:35:15Z

1

No, you can't get the sys.exc_info() from a subprocess -- by the time you're able to see the return code, objects in memory (including stack traces) of the child process are already gone.

What you can do is parse the text your subprocess writes to stderr, by providing a separate pipe for the stderr output and reading that. Assuming that your program never writes to stderr otherwise, the only text that will show up in that stream will be the error message and stack trace.

You'll want to be cautious with this approach, though, because if the process writes more text to stderr than you can buffer, you'll deadlock. The best solution is to have a separate thread to read from both stdout and stderr in parallel.

answered Jan 6, 2015 at 1:35

Daniel Pryden

61.3k16 gold badges102 silver badges137 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

jfs · Accepted Answer · 2015-01-11 15:31:51Z

0

If you want to get sys.exc_info() then you could import the module and run functions from it instead of using subprocess module. You could use multiprocessing module if you need to run in a different process.

Another possibility is to use execfile() to run the module as a script in the same process as the parent script. If you want to pass command-line arguments to the script then you could manipulate sys.argv global list before calling execfile().

Or you could combine subprocess + execfile methods: write a wrapper script that uses execfile to run the child script and serializes all exceptions (e.g., using pickle) and run the wrapper script using subprocess in your parent script. Capture stderr separately, to be able to unpickle child's exception info.

edited Jan 11, 2015 at 15:31

answered Jan 6, 2015 at 7:45

jfs

417k210 gold badges1k silver badges1.7k bronze badges

2 Comments

user2891472 Over a year ago

Thanks. The only problem is that i need to run the python file with arguments which is see is not possible on execfile()

jfs Over a year ago

@user2891472: I've updated the answer to mention how to pass command-line arguments to scripts started using execfile()

Collectives™ on Stack Overflow

Catching python subprocess exception

2 Answers 2

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related