matplotlib-devel Mailing List for matplotlib

John Hunter schrieb:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
>     >> overridden.  I don't see any elegant way to do this.  Any
>     >> ideas?
> 
>     Fernando> In [2]: sys.__excepthook__ is sys.excepthook Out[2]:
> 
> Excellent.  So if the backends define a function excepthook_backend
> and matplotlib.matlab does
> 
> if sys.__excepthook__ is sys.excepthook:
>     sys.excepthook = excepthook_backend
> 
> would we be in reasonably good standing with the python gods and
> others?

You'd have to ask the gods directly, but it looks pretty good to me, a mere 
mortal :)

Best,

f

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    >> overridden.  I don't see any elegant way to do this.  Any
    >> ideas?

    Fernando> In [2]: sys.__excepthook__ is sys.excepthook Out[2]:

Excellent.  So if the backends define a function excepthook_backend
and matplotlib.matlab does

if sys.__excepthook__ is sys.excepthook:
    sys.excepthook = excepthook_backend

would we be in reasonably good standing with the python gods and
others?

JDH

John Hunter schrieb:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
>     >> Actually, it would be fine if matplotlib overrode
>     >> sys.except_hook and ipython later came along and overrode that.
>     >> Basically, ipython would be saying, "I know I've got a shell to
>     >> display errors in, so we don't need to GUI method".  I don't
>     >> think matplotlib would have a problem with that.  Ditto for
>     >> envisage.  Basically, we would be providing a default method to
>     >> get the message to the GUI which could be overriden by other
>     >> applications that want to (ipython, envisage, what-have-you).
> 
>     Fernando> I guess this is a good heads-up for me.  I know ipython
>     Fernando> does some of this, I'll just add a bit more such
>     Fernando> control.  That way, if during the running of user code
>     Fernando> they need sys.ehook for something, they'll get it.  But
>     Fernando> ipython will keep it for when it needs it.  I'm pretty
>     Fernando> sure the embeddable ipython has such control in it, I
>     Fernando> may just need to put it in the general case as well.
> 
>     Fernando> Ultimately I'm not 100% sure what a good solution for
>     Fernando> matplotlib is, I just wanted to make you aware of these
>     Fernando> issues, so that at least they are on your radar.
> 
> 
> We do have some more options.  For one, we could use the excepthook
> only in the matlab interface -- in this case matlab is being used more
> as an application rather than a library.  Folks using matplotlib as a
> library, eg embedding in a GUI, would be advised to do their own
> trapping.
> 
> The only exceptional case I see is basically the ipython (and friends)
> case.  Ie, someone wants to write a shell or otherwise that embeds
> matplotlib.matlab.  In this case it would be fine to override
> matplotlib.matlab's excepthook, as discussed.
> 
> To play really nicely, matplotlib.matlab would like to be able to
> override excepthook only if it hadn't been otherwise overridden.  I
> don't see any elegant way to do this.  Any ideas?

Yup:

planck[mayavi]> ip
In [1]: import sys

In [2]: sys.__excepthook__ is sys.excepthook
Out[2]: False

planck[mayavi]> python
 >>> import sys
 >>> sys.__excepthook__ is sys.excepthook
True

Cheers,

f

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    >> Actually, it would be fine if matplotlib overrode
    >> sys.except_hook and ipython later came along and overrode that.
    >> Basically, ipython would be saying, "I know I've got a shell to
    >> display errors in, so we don't need to GUI method".  I don't
    >> think matplotlib would have a problem with that.  Ditto for
    >> envisage.  Basically, we would be providing a default method to
    >> get the message to the GUI which could be overriden by other
    >> applications that want to (ipython, envisage, what-have-you).

    Fernando> I guess this is a good heads-up for me.  I know ipython
    Fernando> does some of this, I'll just add a bit more such
    Fernando> control.  That way, if during the running of user code
    Fernando> they need sys.ehook for something, they'll get it.  But
    Fernando> ipython will keep it for when it needs it.  I'm pretty
    Fernando> sure the embeddable ipython has such control in it, I
    Fernando> may just need to put it in the general case as well.

    Fernando> Ultimately I'm not 100% sure what a good solution for
    Fernando> matplotlib is, I just wanted to make you aware of these
    Fernando> issues, so that at least they are on your radar.


We do have some more options.  For one, we could use the excepthook
only in the matlab interface -- in this case matlab is being used more
as an application rather than a library.  Folks using matplotlib as a
library, eg embedding in a GUI, would be advised to do their own
trapping.

The only exceptional case I see is basically the ipython (and friends)
case.  Ie, someone wants to write a shell or otherwise that embeds
matplotlib.matlab.  In this case it would be fine to override
matplotlib.matlab's excepthook, as discussed.

To play really nicely, matplotlib.matlab would like to be able to
override excepthook only if it hadn't been otherwise overridden.  I
don't see any elegant way to do this.  Any ideas?

JDH

John Hunter schrieb:
>>>>>>"Fernando" == Fernando Perez <Fer...@co...> writes:
> 
> 
> 
>     Fernando> I realize that excepthook is a tempting tool to use, but
>     Fernando> I hope you guys reconsider this.  I really think it
>     Fernando> would cause many more headaches down the road than those
>     Fernando> it initially appears to solve.
> 
> OK, good to know.  That was news to me.  Now why is it that ipython
> and envisage get to mess around with it and we don't :-) ?
> 
> So what is the canonical way to funnel exceptions into GUI dialog
> boxes?  Isn't this what sys.except_hook is for?

Mmh, I don't really know for sure.  But here's a quick test:

planck[mayavi]> egrep -r sys.except *
planck[mayavi]>

This is on the mayavi sources.  Mayavi DOES show all VTK exceptions into a GUI 
(such as you get when you try to open the volume rendering module with 
floating point data, for example).  I imagine it just traps them locally, but 
I'm not sure.  It is possible that mayavi has an easier job because it's done 
in Tk, with no threading issues to worry about.  Threads make this problem 
MUCH worse, since python has no sensible way for an exception raised in one 
thread to be handled by another: all go to the sys.excepthook bucket.  There 
have been long, complicated threads recently on c.l.py on the cousin topics of 
signals and exceptions in threads, and it doesn't look pretty.

> Actually, it would be fine if matplotlib overrode sys.except_hook and
> ipython later came along and overrode that.  Basically, ipython would
> be saying, "I know I've got a shell to display errors in, so we don't
> need to GUI method".  I don't think matplotlib would have a problem
> with that.  Ditto for envisage.  Basically, we would be providing a
> default method to get the message to the GUI which could be overriden
> by other applications that want to (ipython, envisage, what-have-you).

I guess this is a good heads-up for me.  I know ipython does some of this, 
I'll just add a bit more such control.  That way, if during the running of 
user code they need sys.ehook for something, they'll get it.  But ipython will 
keep it for when it needs it.  I'm pretty sure the embeddable ipython has such 
control in it, I may just need to put it in the general case as well.

Ultimately I'm not 100% sure what a good solution for matplotlib is, I just 
wanted to make you aware of these issues, so that at least they are on your radar.

Best,

f

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:


    Fernando> I realize that excepthook is a tempting tool to use, but
    Fernando> I hope you guys reconsider this.  I really think it
    Fernando> would cause many more headaches down the road than those
    Fernando> it initially appears to solve.

OK, good to know.  That was news to me.  Now why is it that ipython
and envisage get to mess around with it and we don't :-) ?

So what is the canonical way to funnel exceptions into GUI dialog
boxes?  Isn't this what sys.except_hook is for?

Actually, it would be fine if matplotlib overrode sys.except_hook and
ipython later came along and overrode that.  Basically, ipython would
be saying, "I know I've got a shell to display errors in, so we don't
need to GUI method".  I don't think matplotlib would have a problem
with that.  Ditto for envisage.  Basically, we would be providing a
default method to get the message to the GUI which could be overriden
by other applications that want to (ipython, envisage, what-have-you).

So I don't really see a danger here, but please educate me!

JDH

John Hunter schrieb:
>>>>>>"Steve" == Steve Chaplin <ste...@ya...> writes:
> 
> 
>     Steve> By subclassing Verbose you group all the functions that
>     Steve> display messages to the user together into one class.  I
>     Steve> may have misunderstood some of the discussion - I agree
>     Steve> with doing all error handling with exceptions, but when you
>     Steve> catch the exception in a GUI I'm assuming you still want to
>     Steve> popup a message to inform the user.  Are you describing
>     Steve> using exceptions and tracebacks without any error messages
>     Steve> for GUI backends? I don't think you can assume a GUI
>     Steve> backend user will see a traceback since the terminal window
>     Steve> may be obscured, iconified or even closed.
> 
> For the GUI error handling, I was assuming we would use the
> except_hook, and do away with error reporting in the verbose class.

I'd like to strongly plead that you stay away from sys.excepthook.

That is the 'last resort' tool to manipulate exceptions, and it's typically 
used by frameworks which need to completely control the python process.  For 
example, ipython puts its internal crash handler in sys.excepthook, so that if 
all else fails, the crash handler generates a very detailed crash report.  But 
all 'normal' exception handling is done by the internal user loop, with manual 
control.

I haven't had time to look in detail, but I even think that ipython pretty 
aggressively reclaims sys.excepthook if user code messes with it.  I wouldn't 
be surprised if other frameworks (like envisage) also used sys.excepthook 
themselves.  Matplotlib is 'only' a plotting library :), and it should IMHO 
play nicely with other code running along with it.  If it gets into a 
foodfight over who owns sys.excepthook, or if it crashes because 
sys.excepthook is not what it thinks it is, I expect serious interoperability 
problems to pop up down the road.

I realize that excepthook is a tempting tool to use, but I hope you guys 
reconsider this.  I really think it would cause many more headaches down the 
road than those it initially appears to solve.

Regards,

f

>>>>> "Steve" == Steve Chaplin <ste...@ya...> writes:

    Steve> By subclassing Verbose you group all the functions that
    Steve> display messages to the user together into one class.  I
    Steve> may have misunderstood some of the discussion - I agree
    Steve> with doing all error handling with exceptions, but when you
    Steve> catch the exception in a GUI I'm assuming you still want to
    Steve> popup a message to inform the user.  Are you describing
    Steve> using exceptions and tracebacks without any error messages
    Steve> for GUI backends? I don't think you can assume a GUI
    Steve> backend user will see a traceback since the terminal window
    Steve> may be obscured, iconified or even closed.

For the GUI error handling, I was assuming we would use the
except_hook, and do away with error reporting in the verbose class.

For the regular verbose reporting, I'm not averse to plugging it into
a GUI dialog, but I'm not sure this is useful.  As I wrote before,
this will mainly be used in debug situations when we can probably
assume a user has access to a shell output.  They can always capture
it to a file using the regular rc mechanism of setting verbose.fileo.

If we did want to do it in a GUI, we would have to be fairly careful
about the implementation, so that the GUI cached sequential methods
and only displayed them if some time interval (eg 100ms) had lapsed
with no new messages.  This would be used to prevent the curse of 20
popups.  This could presumably be done in GTK with an idle handler and
a changed timestamp.

Alternatively, it could be done in the verbose base class itself,
implemented using threads, but I'm a little wary of the extra
complexity here.  

That said, I think that having the figure manager define a
popup_dialog method would be generally useful.  

    Steve> It does look like they are all non-fatal. I guess the fatal
    Steve> ones are the ones that matplotlib does not anticipate or
    Steve> catch, thinks like faulty installations, missing libraries
    Steve> etc. At the moment these would cause matplotlib to
    Steve> terminate but if we add a default exception handler we will
    Steve> start catching these also.  We could have a policy for
    Steve> matplotlib to catch all exceptions and always attempt to
    Steve> continue, and if it becomes unusable its up to the user to
    Steve> close the window. Also we could recognise some situations
    Steve> (if there are any) where we need to terminate, so we raise
    Steve> SystemExit and set the default exception handler to
    Steve> terminate on SystemExit and continue on all other cases.

Sounds right to me....  So in summary, if all agree and we've covered
all the bases, Verbose.report_error and error_msg disappear and are
replaced by regular exceptions.  matplotlib code can raise a
SystemExit for the relatively rare fatal errors.  GUIs define
sys.excepthook = exception_handler following the lead of Steve's
implementation in backend_gtk (in CVS).

verbose.report is left untouched for now but may be hooked into a GUI
reporting functionality if we can resolve the issues of whether this
is desirable and how to handle caching of many independent sequential
messages.

JDH

On Sat, 2004-11-20 at 17:26 -0600, John Hunter wrote:
> As for report_error, subclassing Verbose, or using figure manager for
> this as Jochen has suggested, are both workable solutions, but what
> does it ultimately buy us?  I am inclined to the logically cleaner
> solution of doing all error handling with exceptions, using a hook
> like you've provided for GUIs.
By subclassing Verbose you group all the functions that display messages
to the user together into one class.
I may have misunderstood some of the discussion - I agree with doing all
error handling with exceptions, but when you catch the exception in a
GUI I'm assuming you still want to popup a message to inform the user.
Are you describing using exceptions and tracebacks without any error
messages for GUI backends? I don't think you can assume a GUI backend
user will see a traceback since the terminal window may be obscured,
iconified or even closed.

> I grepped for all the current uses of report error (included below) --
> on quick inspection none of these appear fatal for a GUI.  I think
> simply informing the user of the error may suffice.  Can you provide
> an example of where we may need to exit (and would it suffice for the
> raiser to simply raise a SystemExit for this case?)

It does look like they are all non-fatal. I guess the fatal ones are the
ones that matplotlib does not anticipate or catch, thinks like faulty
installations, missing libraries etc. At the moment these would cause
matplotlib to terminate but if we add a default exception handler we
will start catching these also.
We could have a policy for matplotlib to catch all exceptions and always
attempt to continue, and if it becomes unusable its up to the user to
close the window. Also we could recognise some situations (if there are
any) where we need to terminate, so we raise SystemExit and set the
default exception handler to terminate on SystemExit and continue on all
other cases.

Steve