Redirecting stdout from a python script called in *nix environment does not work correctly

Question

I have a Python program in which I have some Print statements, followed by calls to some c++ executables from which I also get some output to the stdout. When I run this python script in a unix terminal I get output on the screen as expected (in the correct order i.e. first from the print an then from c++ executables). The problem is when I redirect this output to a file like

python test.py > out.txt

I get the output in the wrong order. I get the output of the c++ executables first and then the other.

Are you sure everything is being written to STDOUT sys.stdout? — James Mills
– James Mills, Commented Dec 13, 2013 at 10:22
I think it was going to the STDOUT ... I got the solution now... i should the stdout with the sys.stdout.flush() ... — zeeshan
– zeeshan, Commented Dec 13, 2013 at 15:23

David Zwicker · Accepted Answer · 2013-12-13 10:47:53Z

17

You can run python with unbuffered output using the command line switch -u, i.e. you can just call python -u myscript.py and the output to stdout should be synchronized thereafter.

answered Dec 13, 2013 at 10:47

David Zwicker

24.5k8 gold badges68 silver badges80 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

PePa Over a year ago

Or, when running with a shebang: #!/usr/bin/env -S python3 -u

user966588 · Accepted Answer · 2013-12-13 14:56:15Z

Python uses line-buffering when stdout is a tty device, hence the output is as per print statements order. In case of redirection python buffers the output of print statements. While c++ executable output buffering is not handled by python in both cases. So when the output is redirected, the print statements are getting buffered and doesn't output to file till the buffer is full or program ends.

sys.stdout.flush() will flush the output of print statements as below. Note it should follow the print statements

#!/usr/local/bin/python
import os
import sys

print("Hello1")
print("Hello2")
sys.stdout.flush()
os.system("/python/cppprog.o")
print("Bye1")
print("Bye2")

Output:

]# python script.py > o.txt
]# cat o.txt
Hello1
Hello2
Hello, world!
Bye1
Bye2

Mohammad Salehe · Accepted Answer · 2013-12-13 10:22:46Z

1

This is because python's stdout buffer and os' stdout buffer are not synced.

Try flushing stdout after your print statements and before executing c++ executables.

import sys
sys.stdout.flush()

answered Dec 13, 2013 at 10:22

Mohammad Salehe

3511 silver badge6 bronze badges

Collectives™ on Stack Overflow

Redirecting stdout from a python script called in *nix environment does not work correctly

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related