matplotlib-devel Mailing List for matplotlib

John Hunter wrote:

>>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:
>>>>>>           
>>>>>
>
>    Perry> My 2 cents is that I think Fernando is right on this
>    Perry> issue. I'd rather go with a solution that causes temporary
>    Perry> pain for matlab users rather than one that causes
>    Perry> lingering, long-term irritations.
>
> OK, looks like a consensus to me :-)
>  
>
I guess my opinion on this is already clear :).

> I'm happy with Fernando's proposed names amin, amax, around, etc.  If
> everyone else is too, I propose Andrew implement his patch, provide
> the compatibility names, and update the relevant docs to advertise
> this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
> Particularly in the latter two, I think we should warn people about
> the potential performance hit of using the builtin min, max and
> friends on large arrays.
>  
>
I'm happy to implement whatever the consensus is, but I'm quite busy 
this week and away this weekend, so it'll be next week until I can do 
anything.  If someone else wants to jump in and do it, I certainly won't 
mind.

Norbert Nemec wrote:

> There might be a solution that avoids the performance hit: there 
> should not be any problem with pylab offering an optimized set of min, 
> max, etc. as long as their signature is identical to the builtins and 
> the behavior only extends them. Something along the line of:
>

> def min(*args, **kwargs):
>  if args == ():
>   raise TypeError, "min() takes at least 1 argument (0 given)"
>  if len(args) == 1 and type(args[0]) is ArrayType:
>   axis=kwargs.pop('axis',0)
>   res = minimum.reduce(args[0],axis)
>  else:
>   res = __builtin__.min(*args)
>  if len(kwargs)>0:
>   raise TypeError, (
>    "min() got an unexpected keyword argument '%s'"
>    %kwargs.keys()[0]
>   )
>  return res


What do people think about Norbert's "best of both worlds" approach?  
Although it seems great in theory, I'm disinclined to use it simply 
because it does override the builtin.  Although he's doubtlessly 
constructed this with the greatest of care to perform exactly as the 
builtin, I wonder about obscure corner cases which won't behave exactly 
the same and may result in even more obscure bugs.  Maybe my misgivings 
are undue paranoia on my part, and his 3rd way really is best.  I 
suppose I'd want to throw lots of tests at it before I pronounce my 
final judgement on it, which I don't have time to do at the moment. 
(Next week, if need be...)

Just a thought for consideration: perhaps Norbert's code could actually 
be used by the underlying mlab.py modules?  I guess some code in the 
wild uses the axis argument not as a keyword, so there would be a 
backwards incompatible change in that regard...  Other than that, 
though, this kind of behavior from the mlab.py modules would probably 
have resulted in a less serious conundrum than what we now face.

Also, we must not forget about round, sum, and abs (and any others I 
have missed).  For example, abs() caught me because I use the cgkit 
quaternion type, which overrides the __abs__ method and thus fails to 
work properly with the mlab.py implementation of abs().

Cheers!
Andrew

John Hunter wrote:

>>>>>>"Perry" == Perry Greenfield <pe...@st...> writes:
>>>>>>            
>>>>>>
>
>    Perry> My 2 cents is that I think Fernando is right on this
>    Perry> issue. I'd rather go with a solution that causes temporary
>    Perry> pain for matlab users rather than one that causes
>    Perry> lingering, long-term irritations.
>
>OK, looks like a consensus to me :-)
>  
>
I guess my opinion on this is already clear :).

>I'm happy with Fernando's proposed names amin, amax, around, etc.  If
>everyone else is too, I propose Andrew implement his patch, provide
>the compatibility names, and update the relevant docs to advertise
>this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
>Particularly in the latter two, I think we should warn people about
>the potential performance hit of using the builtin min, max and
>friends on large arrays.
>  
>
I'm happy to implement whatever the consensus is, but I'm quite busy 
this week and away this weekend, so it'll be next week until I can do 
anything.  If someone else wants to jump in and do it, I certainly won't 
mind.

Norbert Nemec wrote:

> There might be a solution that avoids the performance hit: there should not be 
> any problem with pylab offering an optimized set of min, max, etc. as long as 
> their signature is identical to the builtins and the behavior only extends 
> them. Something along the line of:
> 

> def min(*args, **kwargs):
>  if args == ():
>   raise TypeError, "min() takes at least 1 argument (0 given)"
>  if len(args) == 1 and type(args[0]) is ArrayType:
>   axis=kwargs.pop('axis',0)
>   res = minimum.reduce(args[0],axis)
>  else:
>   res = __builtin__.min(*args)
>  if len(kwargs)>0:
>   raise TypeError, (
>    "min() got an unexpected keyword argument '%s'"
>    %kwargs.keys()[0]
>   )
>  return res

What do people think about Norbert's "best of both worlds" approach?  Although it seems great in theory, I'm disinclined to use it simply because it does override the builtin.  Although he's doubtlessly constructed this with the greatest of care to perform exactly as the builtin, I wonder about obscure corner cases which won't behave exactly the same and may result in even more obscure bugs.  Maybe my misgivings are undue paranoia on my part, and his 3rd way really is best.  I suppose I'd want to throw lots of tests at it before I pronounce my final judgement on it, which I don't have time to do at the moment. (Next week, if need be...)

Just a thought for consideration: perhaps Norbert's code could actually be used by the underlying mlab.py modules?  I guess some code in the wild uses the axis argument not as a keyword, so there would be a backwards incompatible change in that regard...  Other than that, though, this kind of behavior from the mlab.py modules would probably have resulted in a less serious conundrum than what we now face.

Also, we must not forget about round, sum, and abs (and any others I have missed).  For example, abs() caught me because I use the cgkit quaternion type, which overrides the __abs__ method and thus fails to work properly with the mlab.py implementation of abs().

Cheers!
Andrew

Norbert Nemec wrote:
> Am Montag, 10. Januar 2005 11:46 schrieb John Hunter:
> 
>>I'm happy with Fernando's proposed names amin, amax, around, etc.  If
>>everyone else is too, I propose Andrew implement his patch, provide
>>the compatibility names, and update the relevant docs to advertise
>>this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
>>Particularly in the latter two, I think we should warn people about
>>the potential performance hit of using the builtin min, max and
>>friends on large arrays.
> 
> 
> There might be a solution that avoids the performance hit: there should not be 
> any problem with pylab offering an optimized set of min, max, etc. as long as 
> their signature is identical to the builtins and the behavior only extends 
> them. Something along the line of:

Hmm.  Those extra checks in your code don't come for free... I'd rather leave 
the builtins alone (many of them are C-coded, hence quite fast), and just 
provide array versions where needed.

Just my 1e-2

Best,

f

On Mon, 2005-01-10 at 06:23, John Hunter wrote:
> >>>>> "John" == John Hunter <jdh...@ac...> writes:
> 
>     John> I'm happy with Fernando's proposed names amin, amax, around,
>     John> etc.  If everyone else is too, I propose Andrew implement
>     John> his patch, provide the compatibility names, and update the
>     John> relevant docs to advertise this prominently: API_CHANGES,
>     John> CHANGELOG, tutorial and users guide.  Particularly in the
>     John> latter two, I think we should warn people about the
>     John> potential performance hit of using the builtin min, max and
>     John> friends on large arrays.
> 
> Hmm, another thought.
> 
> If we are going to make this change for the pylab namespace, then it
> seems we might as well make the change in the numerix namespace as
> well, since all the same arguments apply.  I think doing different
> things in numerix and pylab *is* a recipe for confusion.  What do you
> think Todd, does this seem sensible?
> 
> Basically, the theme would be we use the underlying names from
> Numeric/numarray except when they clash with python builtins, in which
> case we'd use something like the a* names Fernando proposed.
> 
> Of course, changing the numerix names would mean the matplotlib code
> and examples will have to be updated as well.  The former should be
> relatively easy since I believe there are no uses of unqualified min
> and max from numerix in the base code.  Eg, we do in axes.py
> 
> from numerix import max as nxmax
> from numerix import min as nxmin
> 
> and in mlab.py we do MLab.max.  But we'll have to be careful to make
> sure all the names Andrew identified are handled thoughout the coude
> and examples...
> 
> JDH

This all sounds fine to me.  My understanding is that anywhere we "get
this wrong" the impact will be degraded performance.  As an aside,  I
avoid "from *" like the plague myself but understand that it's important
to optimize interactive use.

Unleash Andrew and I'll try to propagate the changes forward to Scipy.

Todd

On Jan 10, 2005, at 6:23 AM, John Hunter wrote:

>>>>>> "John" == John Hunter <jdh...@ac...> writes:
>
>     John> I'm happy with Fernando's proposed names amin, amax, around,
>     John> etc.  If everyone else is too, I propose Andrew implement
>     John> his patch, provide the compatibility names, and update the
>     John> relevant docs to advertise this prominently: API_CHANGES,
>     John> CHANGELOG, tutorial and users guide.  Particularly in the
>     John> latter two, I think we should warn people about the
>     John> potential performance hit of using the builtin min, max and
>     John> friends on large arrays.
>
> Hmm, another thought.
>
> If we are going to make this change for the pylab namespace, then it
> seems we might as well make the change in the numerix namespace as
> well, since all the same arguments apply.  I think doing different
> things in numerix and pylab *is* a recipe for confusion.  What do you
> think Todd, does this seem sensible?
>
I can't argue with that (though Todd may have some comments of his 
own). In fact it would seem silly if numerix wasn't consistent.

Am Montag, 10. Januar 2005 11:46 schrieb John Hunter:
> I'm happy with Fernando's proposed names amin, amax, around, etc.  If
> everyone else is too, I propose Andrew implement his patch, provide
> the compatibility names, and update the relevant docs to advertise
> this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
> Particularly in the latter two, I think we should warn people about
> the potential performance hit of using the builtin min, max and
> friends on large arrays.

There might be a solution that avoids the performance hit: there should not be 
any problem with pylab offering an optimized set of min, max, etc. as long as 
their signature is identical to the builtins and the behavior only extends 
them. Something along the line of:

def min(*args, **kwargs):
 if args == ():
  raise TypeError, "min() takes at least 1 argument (0 given)"
 if len(args) == 1 and type(args[0]) is ArrayType:
  axis=kwargs.pop('axis',0)
  res = minimum.reduce(args[0],axis)
 else:
  res = __builtin__.min(*args)
 if len(kwargs)>0:
  raise TypeError, (
   "min() got an unexpected keyword argument '%s'"
   %kwargs.keys()[0]
  )
 return res

Probably, one could even avoid separate amin, amax, etc. functions. The user 
just has to be aware that the axis can only be given as keyword argument.

-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

>>>>> "John" == John Hunter <jdh...@ac...> writes:

    John> I'm happy with Fernando's proposed names amin, amax, around,
    John> etc.  If everyone else is too, I propose Andrew implement
    John> his patch, provide the compatibility names, and update the
    John> relevant docs to advertise this prominently: API_CHANGES,
    John> CHANGELOG, tutorial and users guide.  Particularly in the
    John> latter two, I think we should warn people about the
    John> potential performance hit of using the builtin min, max and
    John> friends on large arrays.

Hmm, another thought.

If we are going to make this change for the pylab namespace, then it
seems we might as well make the change in the numerix namespace as
well, since all the same arguments apply.  I think doing different
things in numerix and pylab *is* a recipe for confusion.  What do you
think Todd, does this seem sensible?

Basically, the theme would be we use the underlying names from
Numeric/numarray except when they clash with python builtins, in which
case we'd use something like the a* names Fernando proposed.

Of course, changing the numerix names would mean the matplotlib code
and examples will have to be updated as well.  The former should be
relatively easy since I believe there are no uses of unqualified min
and max from numerix in the base code.  Eg, we do in axes.py

from numerix import max as nxmax
from numerix import min as nxmin

and in mlab.py we do MLab.max.  But we'll have to be careful to make
sure all the names Andrew identified are handled thoughout the coude
and examples...

JDH

>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:

    Perry> My 2 cents is that I think Fernando is right on this
    Perry> issue. I'd rather go with a solution that causes temporary
    Perry> pain for matlab users rather than one that causes
    Perry> lingering, long-term irritations.

OK, looks like a consensus to me :-)

I'm happy with Fernando's proposed names amin, amax, around, etc.  If
everyone else is too, I propose Andrew implement his patch, provide
the compatibility names, and update the relevant docs to advertise
this prominently: API_CHANGES, CHANGELOG, tutorial and users guide.
Particularly in the latter two, I think we should warn people about
the potential performance hit of using the builtin min, max and
friends on large arrays.

I'll put this on the new "News Flash" section of the web site with the
next release, which at least should get people's attention.

A-foolish-consistency-is-the-hobgobblin-of-a-small-mindly-yours,
JDH

Fernando Perez wrote:

> While pylab's mission is indeed is matlab compatibility, you 
> already point out 
> that this is not an 'at all costs' objective.  This is one case 
> where I really 
> think that breaking compatibility with the base python language 
> is a too high 
> price to pay.  I'm having a hard time justifying my position in a clear 
> manner, but I have a strong 'gut feeling' about it.  I'll try to 
> provide some 
> rational points, though:
> 
[...]

My 2 cents is that I think Fernando is right on this issue. I'd rather
go with a solution that causes temporary pain for matlab users rather
than one that causes lingering, long-term irritations.

Perry

John Hunter wrote:

> I'm weakly inclined to leave the situation as it is: it's compatible
> with matlab which is essentially the pylab mission, and it's worked
> for 10 or so years for Numeric's MLab.  Cautious users and power users
> have a clear alternative.  If we leave it as in, we can easily provide
> pymin, pymax, pyround, etc, for users who want the python version.  I
> am open to the proposal, but I think we should frame the argument as
> one of performance versus convenience versus
> least-likely-to-bite-your-ass versus matlab-compatibility rather than
> fixing a bug.

While pylab's mission is indeed is matlab compatibility, you already point out 
that this is not an 'at all costs' objective.  This is one case where I really 
think that breaking compatibility with the base python language is a too high 
price to pay.  I'm having a hard time justifying my position in a clear 
manner, but I have a strong 'gut feeling' about it.  I'll try to provide some 
rational points, though:

One of pylab's, objectives is to help matlab users move over to python.  While 
initially they will naturally only use the compatible functions, we hope they 
will grow out into using all the things python offers which matlab lacks (nice 
OO, listcomps, generator expressions, the great generic standard library, 
etc.).  This means that ultimately, we hope they will really use the python 
language to its fullest.  At that point, if they begin using pyton code from 
'the wild', they are very likely to be bitten by this kind of incompatibility 
(as we all have).

The result: a decision made to ease the initial stages of a transition, ends 
up causing an everlasting headache.  And it's not like we can guarantee 100% 
source compatibility, since they are after all different languages.  I think 
it's much better to add min, max & friends to the few things matlab users need 
to learn in the transition, rather than have everyone pay for this from now on.

You also need to keep in mind that pylab is likely to be used by _python 
users_ who have no matlab experience (I am such a person).  For this group, 
the change of a builtin in this manner is very unexpected, and the source of 
all sorts of problems.  As anecdotal evidence, it was precisely this 
particular problem with MLab which convinced me, a few years ago, to _never_ 
use 'from foo import *'.  Even though I was not a matlab user, I thought the 
MLab names were nice and short, and for a while imported it wholesale.  Until 
I wasted a lot of time tracking the min/max bug one day.  When I found it, I 
felt like screaming at the MLab writers, and decided never again to trust a 
third party library with a * import.  To this day, I use Numeric and Scipy 
always with qualified imports.

IMHO, MLab simply got this one wrong 10 years ago, and pylab should not repeat 
their mistake.  In my own code, I have often written simple a* routines: amap, 
amin, amax, around, short for arraymap, arraymin, etc.  I think it's short and 
clear, and provides a nice distinction of their functionality versus the 
builtins (it's quicker to type amin than nxmin, esp. on a qwerty keyboard 
where nx is an off-home-row chord).

Anyway, this is as much as I'll say on the topic.  It's ultimately your 
choice.  But if I had my way, pylab would just provide a set of a*foo routines 
as array-based counterparts to the builtins, and it would document such a 
feature very prominently.

Cheers,

f