Jupyter notebook show and capture output at the same time

Question

I'm trying to figure out a way to "save" the variables in a jupyter notebook but also capture the rich output at the same time. Mainly for reasons of ssh disconnecting, etc. Just assigning to variables doens't work since some of the libraries print information like progress bars.

So far the most promising way is to use %%capture however seems in recent versions that just captures the output and doesn't output while the browser session is active. Ideally I'm looking for a solution that does both, if the browser session is active print out the output, but if for whatever reason it gets interrupted, I can dump the captured output later.

Can you post example code of how you are doing this currently? — WMRamadan
– WMRamadan, Commented Apr 24, 2021 at 20:16
The documentation would suggest that the %%capture command takes an optional variable for saving the output to, and that variable can have .show() called on it to also display what it captured. — Mars Buttfield-Addison
– Mars Buttfield-Addison, Commented Jul 26, 2021 at 11:24
I checked it with jupyter-lab and it's true that no output is provided, but maybe we can do a little of cell magic — David Davó
– David Davó, Commented Jan 26, 2022 at 21:07

David Davó · Accepted Answer · 2022-01-26 21:52:18Z

2

We can create a custom magic with register_cell_magic, based on capture code from IPython, I could do this function that simulates Linux's tee command (hence the name). The only thing I did is extract it from the class and calling the io thing at the end.

from IPython import get_ipython
from IPython.core import magic_arguments
from IPython.core.magic import register_cell_magic
from IPython.utils.capture import capture_output

@magic_arguments.magic_arguments()
@magic_arguments.argument('output', type=str, default='', nargs='?',
    help="""The name of the variable in which to store output.
    This is a utils.io.CapturedIO object with stdout/err attributes
    for the text of the captured output.
    CapturedOutput also has a show() method for displaying the output,
    and __call__ as well, so you can use that to quickly display the
    output.
    If unspecified, captured output is discarded.
    """
)
@magic_arguments.argument('--no-stderr', action="store_true",
    help="""Don't capture stderr."""
)
@magic_arguments.argument('--no-stdout', action="store_true",
    help="""Don't capture stdout."""
)
@magic_arguments.argument('--no-display', action="store_true",
    help="""Don't capture IPython's rich display."""
)
@register_cell_magic
def tee(line, cell):
    args = magic_arguments.parse_argstring(tee, line)
    out = not args.no_stdout
    err = not args.no_stderr
    disp = not args.no_display
    with capture_output(out, err, disp) as io:
        get_ipython().run_cell(cell)
    if args.output:
        get_ipython().user_ns[args.output] = io
    
    io()

As you can see in the following image, It is working correctly

answered Jan 26, 2022 at 21:52

David Davó

8121 gold badge8 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Tim Kuipers Over a year ago

This only prints the cell output after its done computing. I would like to see the progress of my cell while it's going and save the progress afterwards.

Wayne Mar 20 at 15:22

By the way, jupyter_capture_output magic is another implementation, and perhaps more mature version, of this idea to 'teeing' output to both a separate file and inside the notebook. To more fully understand the context of the approach, I would suggest reading about the nuances of the implementation in the exchanges mainly here and here.

Collectives™ on Stack Overflow

Jupyter notebook show and capture output at the same time

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related