Python: subprocess.popen: read each line of output

Question

I'm having trouble reading my subprocess output by line. The subprocess simply greps the contents of a file against another file. The output, which should be a two column file, prints to stdout just fine. But when I try to read each line, it reads each char followed by \n:

#!/usr/bin/python    

import sys
import getopt
import os
import subprocess
from subprocess import Popen, PIPE, STDOUT

inputfile = ''
target = ''

try:
        opts, args = getopt.getopt(sys.argv[1:],"s:t:",['servers=', 'target='])
except getopt.GetoptError:
        print 'getopt failure: exit'
        sys.exit()

for opt, arg in opts:
        if opt in ("-s", "--servers"):
                inputfile = arg
        if opt in ("-t", "--target"):
                boxwin = arg

p1 = subprocess.Popen(["grep -f " + inputfile + " " + target + " | awk '{print $2, $1}'"], stdout=subprocess.PIPE, shell=True)

output, error = p1.communicate()

print output # this prints normally

for line in output:
        print line # this prints each char of output followed by \n???

Expected output after reading by line:

abc 123
def 456
ghi 789

^^ this will print if I just "print output"

Actual output when using for loop to read each line:

a
b
c

1
2
3

d
e
f

...

Any ideas? Thanks.

Community · Accepted Answer · 2017-05-23 11:33:16Z

10

for c in s: reads one character at a time from a string s (as it should). To get a list of lines from a string instead, you could use .splitlines() method:

lines = output.splitlines()

You don't need to call .communicate() to read the output line by line:

p = subprocess.Popen(cmd, stdout=PIPE)
for line in p.stdout:
    # do something with a line

You could modify the code to handle buffering differently or to enable universal newlines support.

edited May 23, 2017 at 11:33

CommunityBot

11 silver badge

answered Jan 30, 2014 at 4:17

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

corneria Over a year ago

Thanks for the tip. Cut about 5 lines of code out of my script.

jgritty · Accepted Answer · 2022-01-07 21:47:31Z

9

Try the following :

for line in output.splitlines():

instead of:

for line in output:

edited Jan 7, 2022 at 21:47

answered Jan 29, 2014 at 3:47

jgritty

12k3 gold badges41 silver badges62 bronze badges

Collectives™ on Stack Overflow

Python: subprocess.popen: read each line of output

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related