35

I'm trying to implement something similar to git log, which will only page the output if the log is of a certain length. If you're not familiar with git, I'm essentially trying to achieve this:

python some_script.py | less

With some help from the paging implementation in python2.6/pydoc.py, I was able to come up with this:

import os
text = '...some text...'
pipe = os.popen('less', 'w')
pipe.write(text)
pipe.close()

which works great, but os.popen() is deprecated. I've considered writing to a temp file and calling less with its path, but that doesn't seem ideal. Is this possible with subprocess? Any other ideas?

EDIT:

So I've gotten subprocess working. I was able to give it the text variable with Popen.communicate(text), but since I really want to redirect print statements, I've settled on this:

  import os, sys, subprocess, tempfile

  page = True
  if page:
      path = tempfile.mkstemp()[1]
      tmp_file = open(path, 'a')
      sys.stdout = tmp_file
  print '...some text...'
  if page:
      tmp_file.flush()
      tmp_file.close()
      p = subprocess.Popen(['less', path], stdin=subprocess.PIPE)
      p.communicate()
      sys.stdout = sys.__stdout__

Of course, I'd end up wrapping it into functions. Does anyone see a problem with that?

Improve this question
2
  • 1
    A few remarks: (1) the temporary file name is unique: the opening mode should be 'w', not 'a' (impossible to append to file). (2) There is no need to close() the file before reading it. (3) There is no need to communicate with the pager process (a simple subprocess.call() suffices). (4) It is more explicit to not tamper with a global like sys.stdout; unless you really need to do this (like if you want to redirect the output of all the submodules that you use), it is a good idea to explicitly call a special printing function. Commented Jul 18, 2011 at 10:39
  • This is similar topic with nice and pithy Answer: stackoverflow.com/questions/37584717/… Commented Jun 2, 2016 at 10:04

6 Answers 6

Reset to default
55

How about this:

import pydoc
text = '... some text ... '
pydoc.pager(text)

This (on my opensuse linux box) sends the text to a pager ('less' in my case), and works the same as calling "help(... python command...)" within the Python interpreter.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

While I doubt that this function will not be available in the future, it is not documented, unfortunately, so in principles it is not fully guaranteed that it will remain available. Given how convenient it is, though, I guess that this is a risk that should often be worth it. Furthermore, the source code looks like it can easily be copied anyway.
It breaks my output when tried to print a formatted json data.
6 years later, this approach still works in Python 3.7.2 ;-)
10 years later, still works in Python 3.11.3
4

It is a good idea to be explicit in your code, so that it shows that you use a special print function printc() instead of the standard one. Using subprocess.call() is also sufficient (you don't need the pipe machinery). Furthermore, you can save a variable by not storing the name of the temporary file:

from __future__ import print_function

import subprocess, tempfile

page = True  # For tests

# Definition of a printc() function that prints to the correct output
if page:
    tmp_file = open(tempfile.mkstemp()[1], 'w')  # No need to store the name in a specific variable
    def printc(*largs, **kwargs):
        if 'file' not in kwargs:  # The code can still use the usual file argument of print()
            kwargs['file'] = tmp_file  # Forces the output to go to the temp file
        print(*largs, **kwargs)
else:
    printc = print  # Regular print

# Main program:

printc('...some text...', 'some more text', sep='/')  # Python3 syntax

# Paging of the current contents of the temp file:
if page:
    tmp_file.flush()  # No need to close the file: you can keep printing to it
    subprocess.call(['less', tmp_file.name])  # Simpler than a full Popen()

This way, you get the flexibility of Python 3's print function, with a code that explicitly shows that you're doing some fancy printing stuff. This scales better with larger programs than modifying the "global" sys.stdout variable in some locations of your code.

Improve this answer

1 Comment

Thanks for the advice! I've added your optimizations. I do need to redirect submodule output, like you mentioned in the comment, so it seems like I'm stuck with the sys.stdout approach. I've put this all into a class, so it can be called with pager = Pager() and pager.begin().
3

Use subprocess.Popen instead.

http://docs.python.org/library/subprocess.html#subprocess-replacements

http://docs.python.org/library/subprocess.html#subprocess.Popen

There is even a note about this in the os.popen docs.

http://docs.python.org/library/os.html#os.popen

Improve this answer

4 Comments

Yeah, that was the first thing I tried. Couldn't get it working at first, but it is now. Thanks.
Sure. I gave it to you. I was hoping for some more general implementation advice if you have any--see my edit.
Wish I could help you more but I'm on Windows right now and none of these work for me, even when supplied the full path of my MSYS supplied less.
@eryksun Less works fine for me, the MYS bin dir is also in my path; just neither of the above python scripts works for me. Do they work for you?
2

I didn't like executing external commands, so I wrote pager in pure Python. It still has a problem - piped input works only for Windows.

Improve this answer

1 Comment

@MichaelR, please open a bugreport here - bitbucket.org/techtonik/python-pager/issues so that it won't lost.
1

With one caveat, this works for me as a way to redirect process output to a pager on Linux (I don't have Windows handy to test there) when, for whatever reason, it's not feasible to write it all to a file or StringIO and then feed it to the pager all at once:

import os, sys

less = None
if os.isatty(sys.stdout.fileno()):
    less = subprocess.Popen(
        ['less', '-R', '--quit-if-one-screen'],
        stdin=subprocess.PIPE)
    os.dup2(less.stdin.fileno(), sys.stdout.fileno())

Now for the caveat:

less behaves as if -E has been passed and terminates when I reach the bottom of the output. I assume that's because, as the child of the script, it's dying when the script does, but adding the following to the end of the script just causes less to hang when it would otherwise have exited and I haven't had time to figure out why:

if less:
    less.stdin.close()  # no different with or without
    less.communicate()  # no different with less.wait()

(It wasn't an issue when I first came up with this hack because I originated it for making a PyGTK 2.x program pipe its output to grep to work around PyGTK not exposing the function from the GTK+ C API needed to silence some spurious log messages.)

Improve this answer

1 Comment

Doing both os.close(less.stdin.fileno()) and os.close(sys.stdout.fileno()) seems to work
0

In py2.7 I was able to do just

    sys.stdout = os.popen(pager, 'w')

but this stopped working in py3. Some of the above solutions didn't seem to work for me, but I wrote this decorator that does:

  def _get_pager_name():                                
      for p in (os.environ.get('PAGER', ''), '/usr/bin/less', '/bin/more', 'cat'):                                
          if Path(p).is_file():                                
              return p                                
      return None                                
                                  
                                  
  def paged(f):                                
      @wraps(f)                                
      def wrapper(*a, **kw):                                
          try:                                
              with subprocess.Popen([_get_pager_name()], stdin=subprocess.PIPE, text=True) as p:                                
                  sys.stdout = p.stdin                                
                  return f(*a, **kw)                                
          finally:                                
              sys.stdout = sys.__stdout__                                
                                  
      pager = _get_pager_name()                                
      return wrapper if sys.stdout.isatty() and pager else f
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.