Asynchronous stdin read in Python

Question

A program continuously prints to standard output one line at a time.

I am trying to read and process one line at a time of this input without having to wait for the program to complete.

As an example, the below writeOutput.py writes one line at a time to stdout (waiting between each line between 1 and 3 seconds).

Calling ./writeOutput.py | ./processEachLine.py requires writeOutput.py to complete before processEachLine.py is able to start processing the first line.

Is there anyway to achieve this in python? Even by calling writeOutput.py directely within the python program instead of using a pipe?

Any help would be highly appreciated.

writeOutput.py

#!/usr/bin/env python
import random
import time

i = 0
while i < 5:  
    n = int(1 + (random.random()*10) % 3)
    i += 1
    time.sleep(n)
    print(str(n) + " test")

processEachLine.py

#!/usr/bin/env python    
import sys

while 1:
    line = sys.stdin.readline()
    if not line:
      break
    print(">>" + line)

uselpa · Accepted Answer · 2013-06-02 17:43:05Z

5

Instead of

#!/usr/bin/env python

use

#!/usr/bin/env python -u

answered Jun 2, 2013 at 17:43

uselpa

19k2 gold badges36 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

tartifletteblvd Over a year ago

Thank you. #!/usr/bin/env python –u would create an error: "/usr/bin/env: python -u: No such file or directory" An easy workaround is to use python -u writeOutput.py | python -u processEachLine.py Or add #!/usr/bin/python -u Many thanks for the help.

uselpa Over a year ago

That's strange, it works well for me (tried on OS X and Linux). What OS are you on?

Nitz Over a year ago

This is probably necro, but the dash is probably bad - managed to recreate the error by copying the dash from tartifletteblvd's comment

uselpa Over a year ago

@Nitz You're right, he does seem to use a character that looks like a dash but isn't.

tzot Over a year ago

Actually, the hashbang mechanism parses the rest of the first line as one program name (/usr/bin/env) and one argument (python -u, space included, which is not a program as you can understand). So #!/usr/bin/python -u works, while #!/usr/bin/env python -u doesn't. See also: stackoverflow.com/a/4304187/6899

|

Collectives™ on Stack Overflow

Asynchronous stdin read in Python

1 Answer 1

7 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related