matplotlib-users Mailing List for matplotlib

Hi,

I've found that the match_original=True option in PatchCollection  
causes matplotlib to crash. I have submitted two bug reports:

https://sourceforge.net/tracker/?func=detail&aid=2732455&group_id=80706&atid=560720
https://sourceforge.net/tracker/?func=detail&aid=2723527&group_id=80706&atid=560720

I was wondering whether these are problems that other people have  
encountered too? (description in the bug reports)

Thanks,

Thomas

hi Eric, then I misunderstood what the code does.... I thought it was 
transforming the data. In any case, my second figure in my previous post 
is wrong because the code should read:
plt.errorbar(cE,corrE*fluxes, yerr=corrE*unc_fluxes)
instead of
plt.errorbar(cE,corrE*fluxes, yerr=unc_fluxes)

and indeed this seems to behave correctly, so I guess I got completely 
confused while trying to understand what was happening. I will check out 
your patch and start from the beginning.

thanks a lot and sorry,
Johann

Eric Firing wrote:
> Cohen-Tanugi Johann wrote:
>> ok, maybe it is in scale.py :)
>> And what I see there confirms my initial fears : the log scale 
>> transform applies a log to the argument, which is incorrect for an 
>> error..... So first of all, another transform needs to be created so 
>> that erry is transformed into erry/y/log(base) where base was 10 in 
>> my example.
>
> Johann,
>
> I don't understand your objection here.  It seems to me that an 
> errorbar  should always be in the same units, and on the same scale, 
> as the point to which it applies.  Therefore when switching from 
> linear to log scales, one simply plots the same data and error ranges 
> on a log scale instead of a linear scale.  This is what mpl does.  If 
> what you want is for the error to be proportional to the value of the 
> point, then you need to specify the error bounds so that they have 
> that property.  For estimation of a power spectrum, for example, this 
> is often the case; the error bars are some fraction (depending on the 
> equivalent number of degrees of freedom) of the spectral level at each 
> frequency, so on a log plot they will have equal length.  It is not 
> the job of the errorbar function to figure that out, however.
>
> If I am misunderstanding, then please provide a very simple concrete 
> example that will make your point clear.
>
>>
>> past midnight here.... I will see tomorrow if I find time to try out 
>> a patch (not the easiest entry point for starting developer's 
>> activities in MPL I am afraid....)
>
> I have just committed a change (trunk r7023) that allows non-positive 
> numbers to be clipped to a small positive value.  To illustrate its 
> use with an errorbar, I added the example provided by Matthias to 
> examples/pylab_examples/log_demo.py, as a 4th subplot.
>
> Eric
>>
>> Johann

Cohen-Tanugi Johann wrote:
> ok, maybe it is in scale.py :)
> And what I see there confirms my initial fears : the log scale transform 
> applies a log to the argument, which is incorrect for an error..... So 
> first of all, another transform needs to be created so that erry is 
> transformed into erry/y/log(base) where base was 10 in my example.

Johann,

I don't understand your objection here.  It seems to me that an errorbar 
  should always be in the same units, and on the same scale, as the 
point to which it applies.  Therefore when switching from linear to log 
scales, one simply plots the same data and error ranges on a log scale 
instead of a linear scale.  This is what mpl does.  If what you want is 
for the error to be proportional to the value of the point, then you 
need to specify the error bounds so that they have that property.  For 
estimation of a power spectrum, for example, this is often the case; the 
error bars are some fraction (depending on the equivalent number of 
degrees of freedom) of the spectral level at each frequency, so on a log 
plot they will have equal length.  It is not the job of the errorbar 
function to figure that out, however.

If I am misunderstanding, then please provide a very simple concrete 
example that will make your point clear.

> 
> past midnight here.... I will see tomorrow if I find time to try out a 
> patch (not the easiest entry point for starting developer's activities 
> in MPL I am afraid....)

I have just committed a change (trunk r7023) that allows non-positive 
numbers to be clipped to a small positive value.  To illustrate its use 
with an errorbar, I added the example provided by Matthias to 
examples/pylab_examples/log_demo.py, as a 4th subplot.

Eric
> 
> Johann

ok, maybe it is in scale.py :)
And what I see there confirms my initial fears : the log scale transform 
applies a log to the argument, which is incorrect for an error..... So 
first of all, another transform needs to be created so that erry is 
transformed into erry/y/log(base) where base was 10 in my example.

past midnight here.... I will see tomorrow if I find time to try out a 
patch (not the easiest entry point for starting developer's activities 
in MPL I am afraid....)

Johann

Cohen-Tanugi Johann wrote:
> hello,
> for the sake of concreteness, here is an example without any limit 
> issues : the python script is attached and the 2 resulting figures as 
> well. The dirst one is drawn using log directly in the arguments, and 
> correctly transforming the y-errors into y-errors/y-values/log(10).  
> In the second figure, I use the log scaling in x and y. Clearly 
> something goes wrong with plotting the error bars, and the y-scale 
> limits also seem ill chosen....
>
> HTH debugging this. I looked again at the code, but I am decidedly 
> lost as to where the error bar plotting occurs, and where it gets 
> modified by the log scale request.
>
> Johann
>
>
> Cohen-Tanugi Johann wrote:
>> hello..... Anyone? I would very much love to see this fixed, and I am 
>> ready to help out, but I do not know how to browse through the code. 
>> Despite the fact that log(errors) should of course not be used, but 
>> rathter errors/values/log(10), Michael's point still remains : 
>> values- errors in log scale can be negative, so that the artist 
>> should just draw a bar until the lower limit of the vertical bar. I 
>> would say that this is the standard practice.
>> Sorry for my previous email beside the point.
>> Johann
>>
>> Cohen-Tanugi Johann wrote:
>>> I tried to look at the code (axes.py I presume) in order to attempt 
>>> a patch, but it defeated me, I do not have the  instructions to 
>>> navigate through this code :)
>>> Where is the actual transform of the error bars occurring?
>>> thanks,
>>> Johann
>>>
>>> Michael Droettboom wrote:
>>>  
>>>> I have to say I don't really have a lot of experience with error 
>>>> bars on log plots -- but the root cause here is that the lower 
>>>> bound of the error bar goes negative, and as we all know, the log 
>>>> of a negative number is undefined.  If you can suggest where the 
>>>> lower bound should be drawn or provide third-party examples, I'm 
>>>> happy to look into this further and resolve this "surprise".
>>>>
>>>> Mike
>>>>
>>>> Cohen-Tanugi Johann wrote:
>>>>   
>>>>> yes exactly....
>>>>> I should have provided a test case, thanks for following up!
>>>>> Johann
>>>>>
>>>>> Matthias Michler wrote:
>>>>>  
>>>>>     
>>>>>> Hello Johann,
>>>>>>
>>>>>> is the problem you are reporting the one I observe in the 
>>>>>> attached picture? Namely some vertical and horizontal lines are 
>>>>>> missing when using yscale="log". More precisely everything below 
>>>>>> y=1 seems to be missing.
>>>>>>
>>>>>> The picture was generated with the code below and
>>>>>> matplotlib.__version__ = '0.98.6svn'
>>>>>> matplotlib.__revision__ = '$Revision: 6887 $'
>>>>>>
>>>>>> best regards Matthias
>>>>>>
>>>>>> ###############################
>>>>>> import numpy as np
>>>>>> import matplotlib.pyplot as plt
>>>>>>
>>>>>> plt.subplot(111, xscale="log", yscale="log")
>>>>>> x = 10.0**np.linspace(0.0, 2.0, 20)
>>>>>> y = x**2.0
>>>>>> plt.errorbar(x, y, xerr=0.1*x, yerr=5.0+0.75*y)
>>>>>> plt.show()
>>>>>> ################################
>>>>>> On Friday 27 March 2009 16:12:12 Cohen-Tanugi Johann wrote:
>>>>>>            
>>>>>>> Hello, what is the best way to get log log plots with error bars? I
>>>>>>> tried putting log10() everywhere but as I was afraid results 
>>>>>>> look ugly....
>>>>>>> thanks,
>>>>>>> johann
>>>>>>>    
>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------ 
>>>>>>>
>>>>>>>     
>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Matplotlib-users mailing list
>>>>>>> Mat...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>>>>>>                     
>>>>> ------------------------------------------------------------------------------ 
>>>>>
>>>>> _______________________________________________
>>>>> Matplotlib-users mailing list
>>>>> Mat...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>>>>         
>>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> _______________________________________________
>>> Matplotlib-users mailing list
>>> Mat...@li...
>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>>   
>
> ------------------------------------------------------------------------
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> ------------------------------------------------------------------------
>
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

hello,
for the sake of concreteness, here is an example without any limit 
issues : the python script is attached and the 2 resulting figures as 
well. The dirst one is drawn using log directly in the arguments, and 
correctly transforming the y-errors into y-errors/y-values/log(10).  In 
the second figure, I use the log scaling in x and y. Clearly something 
goes wrong with plotting the error bars, and the y-scale limits also 
seem ill chosen....

HTH debugging this. I looked again at the code, but I am decidedly lost 
as to where the error bar plotting occurs, and where it gets modified by 
the log scale request.

Johann


Cohen-Tanugi Johann wrote:
> hello..... Anyone? I would very much love to see this fixed, and I am 
> ready to help out, but I do not know how to browse through the code. 
> Despite the fact that log(errors) should of course not be used, but 
> rathter errors/values/log(10), Michael's point still remains : values- 
> errors in log scale can be negative, so that the artist should just 
> draw a bar until the lower limit of the vertical bar. I would say that 
> this is the standard practice.
> Sorry for my previous email beside the point.
> Johann
>
> Cohen-Tanugi Johann wrote:
>> I tried to look at the code (axes.py I presume) in order to attempt a 
>> patch, but it defeated me, I do not have the  instructions to 
>> navigate through this code :)
>> Where is the actual transform of the error bars occurring?
>> thanks,
>> Johann
>>
>> Michael Droettboom wrote:
>>  
>>> I have to say I don't really have a lot of experience with error 
>>> bars on log plots -- but the root cause here is that the lower bound 
>>> of the error bar goes negative, and as we all know, the log of a 
>>> negative number is undefined.  If you can suggest where the lower 
>>> bound should be drawn or provide third-party examples, I'm happy to 
>>> look into this further and resolve this "surprise".
>>>
>>> Mike
>>>
>>> Cohen-Tanugi Johann wrote:
>>>    
>>>> yes exactly....
>>>> I should have provided a test case, thanks for following up!
>>>> Johann
>>>>
>>>> Matthias Michler wrote:
>>>>  
>>>>      
>>>>> Hello Johann,
>>>>>
>>>>> is the problem you are reporting the one I observe in the attached 
>>>>> picture? Namely some vertical and horizontal lines are missing 
>>>>> when using yscale="log". More precisely everything below y=1 seems 
>>>>> to be missing.
>>>>>
>>>>> The picture was generated with the code below and
>>>>> matplotlib.__version__ = '0.98.6svn'
>>>>> matplotlib.__revision__ = '$Revision: 6887 $'
>>>>>
>>>>> best regards Matthias
>>>>>
>>>>> ###############################
>>>>> import numpy as np
>>>>> import matplotlib.pyplot as plt
>>>>>
>>>>> plt.subplot(111, xscale="log", yscale="log")
>>>>> x = 10.0**np.linspace(0.0, 2.0, 20)
>>>>> y = x**2.0
>>>>> plt.errorbar(x, y, xerr=0.1*x, yerr=5.0+0.75*y)
>>>>> plt.show()
>>>>> ################################
>>>>> On Friday 27 March 2009 16:12:12 Cohen-Tanugi Johann wrote:
>>>>>             
>>>>>> Hello, what is the best way to get log log plots with error bars? I
>>>>>> tried putting log10() everywhere but as I was afraid results look 
>>>>>> ugly....
>>>>>> thanks,
>>>>>> johann
>>>>>>    
>>>>>> ------------------------------------------------------------------------ 
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------ 
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------ 
>>>>>>
>>>>>>     
>>>>>> ------------------------------------------------------------------------ 
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Matplotlib-users mailing list
>>>>>> Mat...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>>>>>                     
>>>> ------------------------------------------------------------------------------ 
>>>>
>>>> _______________________________________________
>>>> Matplotlib-users mailing list
>>>> Mat...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>>>         
>>
>> ------------------------------------------------------------------------------ 
>>
>> _______________________________________________
>> Matplotlib-users mailing list
>> Mat...@li...
>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>>   

Eric Firing wrote:
> Tobias Wood wrote:
>> Hi everyone,
>> After getting fed severely fed up with Matlab in recent months I 
>> downloaded Python, Numpy and Matplotlib to try out as an alternative. So 
>> far I'm pleasantly impressed, even if building from source on Mac OS X 
>> is an experience ;) However, I have discovered a couple of problems with 
>> Matplotlib's imread() function and, shall we say, 'esoteric' PNG files. 
>> My research group uses a 12-bit CCD controlled through Labview to 
>> capture high dynamic range image stacks. Often there are ~30 images in a 
>> single data set. These get read into Matlab in one go for processing as 
>> a stack. I tried converting my code over to Python but, after digging 
>> through the _png.cpp source file found the following that are problems 
>> from my point of view:
>>
>> 1) All .png files in imread() are converted to 4-plane RGBA files 
>> regardless of original format. I would prefer greyscale images to return 
>> a single plane.
>> 2) 16-bit PNGs are stripped to 8 bit, losing any extra precision.
>> 3) The significant bits option in the PNG header was not being checked. 
>> Our camera software will automatically save the PNGs at the maximum 
>> bit-depth required to cover the dynamic range in the image, and can sum 
>> images before saving, so pixels can be anywhere from 6- to 16-bits (at 
>> least those are the values I have observed whilst using the camera).
>>
>> I have attached the results of an svn diff after I made an attempt at 
>> correcting these issues. This is the first time I have contributed to an 
>> open source project, so am not sure of the etiquette here. Also, I have 
>> only had Python and Matplotlib for a fortnight so am still unfamiliar 
>> with them and haven't programmed with libpng before so I apologise in 
>> advance if there any stupid mistakes in my code. I am aware that 
>> imread() is a pretty important function in Matplotlib and hence any 
>> changes I suggest would need comprehensive testing. In brief, I made the 
>> following changes:
> 
> Tobias,
> 
> Thank you very much for the patch and the careful explanation.  I'm not 
> the right person to review it, but I expect someone familiar with libpng 
> use in mpl will do so soon.  Mike D. would be a candidate, but I think 
> he will be unavailable for several days.  If you have not gotten any 
> feedback within a week, *please* ping us with a reminder.  If it comes 
> to that, you could do it by forwarding your original message to 
> matplotlib-devel.

Tobias,

I would like to apply your patch, but the test in
examples/tests/pngsuite fails. If you can submit a new patch where this
test passes, and, even better, if a small example 12-bit PNG of yours is
added to the test, I will apply it.

Apart from that, I would echo Eric's thanks for the patch and explanation.

-Andrew

Tobias Wood wrote:
> Hi everyone,
> After getting fed severely fed up with Matlab in recent months I 
> downloaded Python, Numpy and Matplotlib to try out as an alternative. So 
> far I'm pleasantly impressed, even if building from source on Mac OS X 
> is an experience ;) However, I have discovered a couple of problems with 
> Matplotlib's imread() function and, shall we say, 'esoteric' PNG files. 
> My research group uses a 12-bit CCD controlled through Labview to 
> capture high dynamic range image stacks. Often there are ~30 images in a 
> single data set. These get read into Matlab in one go for processing as 
> a stack. I tried converting my code over to Python but, after digging 
> through the _png.cpp source file found the following that are problems 
> from my point of view:
> 
> 1) All .png files in imread() are converted to 4-plane RGBA files 
> regardless of original format. I would prefer greyscale images to return 
> a single plane.
> 2) 16-bit PNGs are stripped to 8 bit, losing any extra precision.
> 3) The significant bits option in the PNG header was not being checked. 
> Our camera software will automatically save the PNGs at the maximum 
> bit-depth required to cover the dynamic range in the image, and can sum 
> images before saving, so pixels can be anywhere from 6- to 16-bits (at 
> least those are the values I have observed whilst using the camera).
> 
> I have attached the results of an svn diff after I made an attempt at 
> correcting these issues. This is the first time I have contributed to an 
> open source project, so am not sure of the etiquette here. Also, I have 
> only had Python and Matplotlib for a fortnight so am still unfamiliar 
> with them and haven't programmed with libpng before so I apologise in 
> advance if there any stupid mistakes in my code. I am aware that 
> imread() is a pretty important function in Matplotlib and hence any 
> changes I suggest would need comprehensive testing. In brief, I made the 
> following changes:

Tobias,

Thank you very much for the patch and the careful explanation.  I'm not 
the right person to review it, but I expect someone familiar with libpng 
use in mpl will do so soon.  Mike D. would be a candidate, but I think 
he will be unavailable for several days.  If you have not gotten any 
feedback within a week, *please* ping us with a reminder.  If it comes 
to that, you could do it by forwarding your original message to 
matplotlib-devel.

Eric

> 
> 1) Removed the libpng 16- to 8-bit strip command
> 2) Added in the libpng calls to cope with variable bit-depth and 
> converting 16-bit pngs from big-endian to little-endian
> 3) Added a large if/else if stucture at the end to return different 
> PyArrays depending on the input data. RGBA images are 4 plane, RGB 3 
> plane and greyscale 1 plane. Numbers within these are still floats 
> scaled between 0 and 1, except 16-bit images which are doubles (Are 
> floats preferable to doubles?). The scaling factor is worked out from 
> the significant bits struct.
> 
> There are still a couple of issues with this code, mainly that I have 
> only tested it with PNGs I have lying to hand, all of which display 
> correctly with imshow() and I have not made much attempt at supporting 
> 1,2 and 4 bit pngs. I'm personally not a big fan of large if/else ifs 
> but in this case thought it was the clearest way to return the different 
> types.
> 
> I would finally like to point out that no software I have used so far 
> has been able to read the images produced by this camera completely 
> correctly. PIL interprets the variable bit-depth images as binary (?!), 
> and we had to write a wrapper round the Matlab imread() function using 
> iminfo() as Matlab ignores the significant bits setting as well.
> 
> Oh, almost forgot, I'm compiling on Mac OS X 10.5, Python 2.6.1 
> (r261:67515) and the latest Numpy and Matplotlib SVN checkouts.
> 
> Kind regards,
> Tobias Wood

Hi everyone,
After getting fed severely fed up with Matlab in recent months I  
downloaded Python, Numpy and Matplotlib to try out as an alternative. So  
far I'm pleasantly impressed, even if building from source on Mac OS X is  
an experience ;) However, I have discovered a couple of problems with  
Matplotlib's imread() function and, shall we say, 'esoteric' PNG files. My  
research group uses a 12-bit CCD controlled through Labview to capture  
high dynamic range image stacks. Often there are ~30 images in a single  
data set. These get read into Matlab in one go for processing as a stack.  
I tried converting my code over to Python but, after digging through the  
_png.cpp source file found the following that are problems from my point  
of view:

1) All .png files in imread() are converted to 4-plane RGBA files  
regardless of original format. I would prefer greyscale images to return a  
single plane.
2) 16-bit PNGs are stripped to 8 bit, losing any extra precision.
3) The significant bits option in the PNG header was not being checked.  
Our camera software will automatically save the PNGs at the maximum  
bit-depth required to cover the dynamic range in the image, and can sum  
images before saving, so pixels can be anywhere from 6- to 16-bits (at  
least those are the values I have observed whilst using the camera).

I have attached the results of an svn diff after I made an attempt at  
correcting these issues. This is the first time I have contributed to an  
open source project, so am not sure of the etiquette here. Also, I have  
only had Python and Matplotlib for a fortnight so am still unfamiliar with  
them and haven't programmed with libpng before so I apologise in advance  
if there any stupid mistakes in my code. I am aware that imread() is a  
pretty important function in Matplotlib and hence any changes I suggest  
would need comprehensive testing. In brief, I made the following changes:

1) Removed the libpng 16- to 8-bit strip command
2) Added in the libpng calls to cope with variable bit-depth and  
converting 16-bit pngs from big-endian to little-endian
3) Added a large if/else if stucture at the end to return different  
PyArrays depending on the input data. RGBA images are 4 plane, RGB 3 plane  
and greyscale 1 plane. Numbers within these are still floats scaled  
between 0 and 1, except 16-bit images which are doubles (Are floats  
preferable to doubles?). The scaling factor is worked out from the  
significant bits struct.

There are still a couple of issues with this code, mainly that I have only  
tested it with PNGs I have lying to hand, all of which display correctly  
with imshow() and I have not made much attempt at supporting 1,2 and 4 bit  
pngs. I'm personally not a big fan of large if/else ifs but in this case  
thought it was the clearest way to return the different types.

I would finally like to point out that no software I have used so far has  
been able to read the images produced by this camera completely correctly.  
PIL interprets the variable bit-depth images as binary (?!), and we had to  
write a wrapper round the Matlab imread() function using iminfo() as  
Matlab ignores the significant bits setting as well.

Oh, almost forgot, I'm compiling on Mac OS X 10.5, Python 2.6.1  
(r261:67515) and the latest Numpy and Matplotlib SVN checkouts.

Kind regards,
Tobias Wood

I am pretty happy with the default for how dates ticks
are updated with zooming the plot with the navigation
toolbar...until it gets down to the level of hours.

Hours are by default displayed in a format like:
"23:00:00 UTC".  How can I get it to display it in a
more common way, such as "11:00 pm", or better,
"11 pm, 3/15"?

Everything else should stay the same.

Thanks,
Che