How to save an error message from subprocess command to file

Question

When running bash command using subprocess, I might run into situation where the command is not valid. In this case, bash would return an error messsage. How can we catch this message? I would like to save this message to a log file. The following is an example, where I try to list files in a non-existed directory.

try:
    subprocess.check_call(["ls", "/home/non"]) 
    df = subprocess.Popen(["ls", "/home/non"], stdout=subprocess.PIPE)        
    output, err = df.communicate()
    # process outputs
except Exception as error:        
    print error
    sys.exit(1)

Bash would prints "ls: cannot access /home/non: No such file or directory". How can I get this error message? The error caught by the except line is clearly different, it says "Command '['ls', '/home/non']' returned non-zero exit status 2".

I think you need redirect both: i.e. stdout=subprocess.PIPE, stderr=subprocess.STDOUT then the error message will be in err. — martineau
– martineau, Commented Apr 11, 2015 at 17:13
@martineau Thanks for the suggestion. I just checked, the error message is in output if add "stderr=..." to the Popen. I prefer a way to catch this message by the excpet line, if there are any. Any ideas? — Luke
– Luke, Commented Apr 11, 2015 at 17:46
To do that you need interprocess communication. Since subprocess.Popen() can start any kind of other process that would be difficult. About your only general options are returncode and pipes. Consider using subprocess.check_output instead of Popen. — martineau
– martineau, Commented Apr 11, 2015 at 18:05

jfs · Accepted Answer · 2015-04-11 18:49:16Z

13

"ls: cannot access /home/non: No such file or directory" is generated by ls command, not bash here.

If you want to handle non-existing files using exception handling then use subprocess.check_output():

#!/usr/bin/env python
from subprocess import check_output, STDOUT, CalledProcessError

try:
    output = check_output(['ls', 'nonexistent'], stderr=STDOUT)
except CalledProcessError as exc:
    print(exc.output)
else:
    assert 0

Output

ls: cannot access nonexistent: No such file or directory

answered Apr 11, 2015 at 18:49

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Coolio2654 Over a year ago

Just for clarification purposes, this is a method of accessing some existing subprocess' stderr in the global Python session, correct? Instead of sending the error message(s) to a created text file, such as Cunningham's answer describes?

jfs Over a year ago

@Coolio2654 stderr parameter works as usual here. You can pass a file object (with a valid .fileno()) if you'd like to save stderr to a file.

AnukuL · Accepted Answer · 2016-12-13 02:16:06Z

8

You can redirect stderr to a file object:

from subprocess import PIPE, CalledProcessError, check_call, Popen

with open("log.txt", "w") as f:
    try:
        check_call(["ls", "/home/non"], stderr=f)
        df = Popen(["ls", "/home/non"], stdout=PIPE)
        output, err = df.communicate()
    except CalledProcessError as e:
        print(e)
        exit(1)

Output to log.txt:

ls: cannot access /home/non: No such file or directory

If you want the message in the except:

try:
    check_call(["ls", "/home/non"])
    df = Popen(["ls", "/home/non"], stdout=PIPE)
    output, err = df.communicate()
except CalledProcessError as e:
    print(e.message)

For python 2.6 the e.message won't work. You can use a similar version of python 2.7's check_output that will work with python 2.6:

from subprocess import PIPE, CalledProcessError, Popen

def check_output(*args, **kwargs):
    process = Popen(stdout=PIPE, *args, **kwargs)
    out, err = process.communicate()
    ret = process.poll()
    if ret:
        cmd = kwargs.get("args")
        if cmd is None:
            cmd = args[0]
        error = CalledProcessError(ret, cmd)
        error.out = out
        error.message = err
        raise error
    return out

try:
    out = check_output(["ls", "/home"], stderr=PIPE)
    df = Popen(["ls", "/home/non"], stdout=PIPE)
    output, err = df.communicate()
except CalledProcessError as e:
    print(e.message)
else:
    print(out)

edited Dec 13, 2016 at 2:16

AnukuL

6351 gold badge10 silver badges21 bronze badges

answered Apr 11, 2015 at 17:18

Padraic Cunningham

181k30 gold badges264 silver badges327 bronze badges

11 Comments

Luke Over a year ago

Is that possible to catch this message by the except line?

Padraic Cunningham Over a year ago

@Luke, yes just catch the CalledProcessError and print the message

Luke Over a year ago

Thanks a lot. Unfortunately, I got this" "DeprecationWarning: BaseException.message has been deprecated as of Python 2.6".

Padraic Cunningham Over a year ago

What are you catching? Also what version of python?

Luke Over a year ago

I use python 2.6.6, just got an error. test.py:15: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 print e.message

|

ryanjdillon · Accepted Answer · 2023-11-15 13:15:48Z

3

Per the subprocess docs, the command run is now prefered.

Example:

import logging
import subprocess

logger = logging.getLogger(__name__)

try:
    subprocess.run(["ls", "-l"], shell=True, check=True, capture_output=True)
except subprocess.CalledProcessError as err:
    logger.error(f"{err} {err.stderr.decode('utf8')}")

As others have mentioned, if you want to save to a file, you can use the stdout param to run; however, you may as well use logging to do that, and then just log the error in your method.

edited Nov 15, 2023 at 13:15

answered Apr 29, 2022 at 11:04

ryanjdillon

19.2k10 gold badges88 silver badges113 bronze badges

Collectives™ on Stack Overflow

How to save an error message from subprocess command to file

3 Answers 3

Output

2 Comments

11 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Output

2 Comments

11 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related