How to implement a python REPL that nicely handles asynchronous output?

Question

I have a Python-based app that can accept a few commands in a simple read-eval-print-loop. I'm using raw_input('> ') to get the input. On Unix-based systems, I also import readline to make things behave a little better. All this is working fine.

The problem is that there are asynchronous events coming in, and I'd like to print output as soon as they happen. Unfortunately, this makes things look ugly. The "> " string doesn't show up again after the output, and if the user is halfway through typing something, it chops their text in half. It should probably redraw the user's text-in-progress after printing something.

This seems like it must be a solved problem. What's the proper way to do this?

Also note that some of my users are Windows-based.

TIA

Edit: The accepted answer works under Unixy platforms (when the readline module is available), but if anyone knows how to make this work under Windows, it would be much appreciated!

cdleary · Accepted Answer · 2009-01-12 21:46:10Z

8

Maybe something like this will do the trick:

#!/usr/bin/env python2.6

from __future__ import print_function

import readline
import threading

PROMPT = '> '

def interrupt():
    print() # Don't want to end up on the same line the user is typing on.
    print('Interrupting cow -- moo!')
    print(PROMPT, readline.get_line_buffer(), sep='', end='')

def cli():
    while True:
        cli = str(raw_input(PROMPT))

if __name__ == '__main__':
    threading.Thread(target=cli).start()
    threading.Timer(2, interrupt).start()

I don't think that stdin is thread-safe, so you can end up losing characters to the interrupting thread (that the user will have to retype at the end of the interrupt). I exaggerated the amount of interrupt time with the time.sleep call. The readline.get_line_buffer call won't display the characters that get lost, so it all turns out alright.

Note that stdout itself isn't thread safe, so if you've got multiple interrupting threads of execution, this can still end up looking gross.

edited Jan 12, 2009 at 21:46

answered Jan 12, 2009 at 21:33

cdleary

71.9k55 gold badges165 silver badges194 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

andy Over a year ago

Thanks, readline.get_line_buffer() is a good start. If anyone has any tips on making this work in Windows (which has no readline module), let me know.

Twisted AISTRIKE at 1687989253 Over a year ago

Is this still the way, or a close approximation thereof, to do it in 2021? Anything I should know that has changed? I'm making a discord bot using discord.py and am running into the same issue as the asker. I get non-blocking input using aioconsole, but I can't seem to get it without different input/output sources interleaving one another in visually unpleasing ways

4 revs, 2 users 90% · Accepted Answer · 2024-07-05 06:34:02Z

5

Why are you writing your own REPL using raw_input()? Have you looked at the cmd.Cmd class? Edit: I just found the sclapp library, which may also be useful.

Note: the cmd.Cmd class (and sclapp) may or may not directly support your original goal; you may have to subclass it and modify it as needed to provide that feature.

edited Jul 5, 2024 at 6:34

community wiki

4 revs, 2 users 90%
Teddy

2 Comments

andy Over a year ago

cmd.Cmd looks pretty awesome and I should probably start using it. However, it doesn't seem to accomplish my original goal, any tips?

Dilum Ranatunga Over a year ago

sclapp seems to be defunct. The link provided didn't work, and I've not found its new home.

nosklo · Accepted Answer · 2010-12-22 10:57:43Z

2

run this:

python -m twisted.conch.stdio

You'll get a nice, colored, async REPL, without using threads. While you type in the prompt, the event loop is running.

answered Dec 22, 2010 at 10:57

community wiki

nosklo

Comments

user250418 · Accepted Answer · 2010-12-09 02:15:57Z

0

look into the code module, it lets you create objects for interpreting python code also (shameless plug) https://github.com/iridium172/PyTerm lets you create interactive command line programs that handle raw keyboard input (like ^C will raise a KeyboardInterrupt).

answered Dec 9, 2010 at 2:15

community wiki

user250418

Comments

user202729 · Accepted Answer · 2024-07-05 06:33:48Z

0

Another method is to use the prompt_toolkit library (I believe this is what IPython uses internally).

Example from the linked answer:

from prompt_toolkit import prompt
from prompt_toolkit.patch_stdout import patch_stdout
import threading

def fn():
    print('async event')

with patch_stdout():
    threading.Timer(3, fn).start()
    a = prompt('Cli> ')

answered Jul 5, 2024 at 6:33

community wiki

user202729

Comments

orip · Accepted Answer · 2009-01-12 23:04:43Z

-1

It's kind of a non-answer, but I would look at IPython's code to see how they're doing it.

answered Jan 12, 2009 at 23:04

orip

76k21 gold badges120 silver badges150 bronze badges

Comments

Ali Afshar · Accepted Answer · 2009-01-13 15:36:14Z

-1

I think you have 2 basic options:

Synchronize your output (i.e. block until it comes back)
Separate your input and your (asyncronous) output, perhaps in two separate columns.

answered Jan 13, 2009 at 15:36

Ali Afshar

41.8k12 gold badges98 silver badges111 bronze badges

Collectives™ on Stack Overflow

How to implement a python REPL that nicely handles asynchronous output?

7 Answers 7

2 Comments

2 Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

7 Answers 7

2 Comments

2 Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related