matplotlib-users Mailing List for matplotlib

On Sun, 2006-06-18 at 00:05 +1000, John Pye wrote:
> Hi all,
> 
> A thought just occurred to me: I wonder if it would be useful to be able
> to 'pickle' Matplotlib plots, using the python cPickle library. This
> way, I could save my plots in a form that would allow me to load them
> back later (with just the necessary source data) and fiddle with things
> like titles and legends and so on. Would be useful perhaps in the
> context of preparing diagrams for an article or report.
> 
> Has anyone tried this? Would it be recommended/not recommended/not even
> possible?

I had a look at this a while back. It looks like the well thought out
structure of MPL should make this easy, although it would require a few
adjustments. To make a Figure object pickle-able, all the internal
objects in a Figure must also be pickle-able. Most of the innards of a
Figure are python objects which should pickle without problem. The only
parts which aren't are the "BinOps". These are custom C-coded objects
which implement 'lazy evaluation' of the transformation expressions.
They're defined in the _transforms.cxx/h files.

In theory, you can easily make these C-objects pickle-able using the
'copy_reg' module; you just register two functions, one to extracts the
object's state as a pickle-able object, the other to construct a new
instance of the object, initialised with the previously stored state.

However, I ran into a problem: there's some bug in either python or CXX
(the automatic C++ class wrapper which mpl uses for the compiled
components) which results in a segfault when I tried pickling copy_reg
enhanced BinOps. The templating techniques used by CXX are completely
beyond me so this is where things have stuck.

... but ... I just now tested this again with python-2.4.2 and
mpl-0.87.2 and it works! yeay. Thus, if every object in
matplotlib._transforms gets given a python reduction/construction
function pair and registers them with copy_reg, this *should* be enough
to make a Figure pickle-able. Unless I've missed something else...

I may try this out later this week, unless someone else tries it first.

Bryan

PS. copy_reg example follows >>>

import cPickle as pickle
import copy_reg

#let's test this on a simple 'Value' BinOp
from matplotlib._transforms import Value

def fcon(val):
    #constructor
    return Value(val)

def fred(o):
    #reduction functions
    val = o.get()
    return fcon, (val,)

#my starting object
a=Value(5)
print a, a.get()

copy_reg.pickle(type(a), fred)

data = pickle.dumps(a)

new = pickle.loads(data)
print "new", new, new.get()

> 
> Cheers
> JP

John Hunter wrote:
>>>>>>"Eric" == Eric Firing <ef...@ha...> writes:
> 
>     Eric> they will not be transparent.  If you need transparent
>     Eric> masked regions, then try pcolor instead of imshow.  Pcolor
>     Eric> plots nothing at all in masked cells.  Pcolormesh, on the
>     Eric> other hand, is like imshow in plotting the assigned bad
>     Eric> color and in using a single alpha for everything.
> 
> I'm confused about the comments about alpha not working on imshow --
> can you elaborate.  On the agg backend at least, the alpha channel is
> respected in imshow, eg examples/layer_images.py.  Is there a reason
> it does not (or should not) work in the masked example?

John,

I don't know why it doesn't work; I know only that in my example, it 
doesn't work as I perhaps naively think it should.  My interpretation of 
alpha is that if alpha is zero in any colored region, and if nothing 
else is drawn on top, then the background should show through; that is, 
the r,g,b values in the r,g,b,a tuple for a region should have no effect 
if a is zero.  If you uncomment
#cmap.set_bad((1,1,1,0)
in my example, you will find that the masked region is white; and if you 
change the rgb part of that tuple, it takes on that color, regardless of 
alpha.

The Colormap.__call__() is trying to respect the individual alphas of 
the special colors (bad, under, over) while setting a uniform alpha for 
the normal colors, but somewhere along the line this is not working as 
intended. I ran into this when I wrote the code for the special colors, 
and I left a comment to that effect, but apart from taking another look 
just now I have not tried to track it down.  The behavior is the same in 
imshow and in pcolormesh.  I tried to work around it by setting the 
default bad color to white.  The difference in pcolor is that it draws 
nothing at all in the masked region, so one sees the gray background.

Eric

John,

It sounds like you are not using the latest *released* version of mpl, 
which is 0.87.3; the only thing in my example that requires svn is the 
colorbar command.  If you did install 0.87.3, then you must have an 
earlier installation still in place and being found.

imshow requires a uniform rectangular grid. pcolor is much more general 
and colors rectangles specified by their boundaries, not their centers. 
  pcolormesh is similar (though not identical) but faster.

As you described it, your application seems most natural for imshow, 
except for the problem with masked areas not being transparent.  I 
suggested pcolor only as a workaround for that, if you really need that 
transparency.

Eric

John Pye wrote:
> Hi Eric,
> 
> Thanks for your suggestion. The colorbar(shrink) command throws me an
> error, as you said it would. But I get another error with the '0.3',
> '0.5', etc. I had to replace those with (0.3,0.3,0.3) etc -- RGB tuples.
> 
> Finally, my plot only shows white, grey red. I don't get any other
> colors -- do you? Is that because of the colorbar(shrink) thing, or is
> something else not working? Do I *need* your SVN changes?
> 
> Also, when should I be using pcolor versus imshow? If my image is
> constructed of colors at specified sample points, is it better that I
> use pcolor? The pcolor command isn't mentioned at all in the matplotlib
> user's guide...
> 
> Cheers
> JP
> 
> Eric Firing wrote:
> 
> 
>>John,
>>
>>Something like this might be what you want:
>>
>>from pylab import *
>>import matplotlib.numerix.ma as ma
>>import matplotlib.colors as colors
>>xx = rand(10,15)
>>xxx = (xx*15 - 5).astype(Int)
>>xxx = ma.masked_where(xxx < 0, xxx)
>>xxx.set_fill_value(-1) #(not necessary)
>>cmap = colors.ListedColormap(('r', 'g', 'b',
>>                                'c', 'y', 'm',
>>                                'k', '0.3', '0.5',
>>                                '0.7'))
>>#cmap.set_bad((1,1,1,0)) # setting alpha to zero does not work, at least
>>                        # for imshow
>>im = imshow(xxx, interpolation='nearest',
>>                 cmap=cmap, norm=colors.no_norm())
>>cb = colorbar(shrink=0.6)
>>
>>
>>What we are doing here is making a custom colormap from a list of
>>colors (using any valid mpl color specification), and then indexing
>>directly into it with values from a (masked) integer array.  Note the
>>use of "norm=colors.no_norm()" so that the array values are passed
>>directly to the colormap as integers.
>>
>>Caution: the colorbar command works correctly in this example only
>>with the modifications that I committed to svn a few minutes ago.
>>
>>As noted, the masked regions will have a specified color; they will
>>not be transparent.  If you need transparent masked regions, then try
>>pcolor instead of imshow.  Pcolor plots nothing at all in masked
>>cells. Pcolormesh, on the other hand, is like imshow in plotting the
>>assigned bad color and in using a single alpha for everything.
>>
>>Eric
>>
>>
>>John Pye wrote:
>>
>>
>>>Hi all
>>>
>>>I have some data with enumerated values in an array. Values are like
>>>1,2,7,9 spread around in the array. I want to plot those values so that
>>>I can see the 'regions' in my data, then I want to overlay this with
>>>some contour lines drawn from other data.
>>>
>>>I want to be able to specify the colors for each of the enumerated
>>>values, as they need to be consistent throughout my work. My array of
>>>enumerated values is *masked* so that there are areas where I don't want
>>>to show anything (this would plot as transparent pixels).
>>>
>>>Has anyone got suggestions on the best way of doing this? It seems that
>>>the technique in
>>>http://www.scipy.org/Cookbook/Matplotlib/Plotting_Images_with_Special_Values
>>>
>>>might be overkill, right? It also seemed that it had some problems with
>>>masked arrays.
>>>
>>>Cheers
>>>
>>>JP
>>>
>>
>>

Hi all,

A thought just occurred to me: I wonder if it would be useful to be able
to 'pickle' Matplotlib plots, using the python cPickle library. This
way, I could save my plots in a form that would allow me to load them
back later (with just the necessary source data) and fiddle with things
like titles and legends and so on. Would be useful perhaps in the
context of preparing diagrams for an article or report.

Has anyone tried this? Would it be recommended/not recommended/not even
possible?

Cheers
JP

Hi Eric,

Thanks for your suggestion. The colorbar(shrink) command throws me an
error, as you said it would. But I get another error with the '0.3',
'0.5', etc. I had to replace those with (0.3,0.3,0.3) etc -- RGB tuples.

Finally, my plot only shows white, grey red. I don't get any other
colors -- do you? Is that because of the colorbar(shrink) thing, or is
something else not working? Do I *need* your SVN changes?

Also, when should I be using pcolor versus imshow? If my image is
constructed of colors at specified sample points, is it better that I
use pcolor? The pcolor command isn't mentioned at all in the matplotlib
user's guide...

Cheers
JP

Eric Firing wrote:

> John,
>
> Something like this might be what you want:
>
> from pylab import *
> import matplotlib.numerix.ma as ma
> import matplotlib.colors as colors
> xx = rand(10,15)
> xxx = (xx*15 - 5).astype(Int)
> xxx = ma.masked_where(xxx < 0, xxx)
> xxx.set_fill_value(-1) #(not necessary)
> cmap = colors.ListedColormap(('r', 'g', 'b',
>                                 'c', 'y', 'm',
>                                 'k', '0.3', '0.5',
>                                 '0.7'))
> #cmap.set_bad((1,1,1,0)) # setting alpha to zero does not work, at least
>                         # for imshow
> im = imshow(xxx, interpolation='nearest',
>                  cmap=cmap, norm=colors.no_norm())
> cb = colorbar(shrink=0.6)
>
>
> What we are doing here is making a custom colormap from a list of
> colors (using any valid mpl color specification), and then indexing
> directly into it with values from a (masked) integer array.  Note the
> use of "norm=colors.no_norm()" so that the array values are passed
> directly to the colormap as integers.
>
> Caution: the colorbar command works correctly in this example only
> with the modifications that I committed to svn a few minutes ago.
>
> As noted, the masked regions will have a specified color; they will
> not be transparent.  If you need transparent masked regions, then try
> pcolor instead of imshow.  Pcolor plots nothing at all in masked
> cells. Pcolormesh, on the other hand, is like imshow in plotting the
> assigned bad color and in using a single alpha for everything.
>
> Eric
>
>
> John Pye wrote:
>
>> Hi all
>>
>> I have some data with enumerated values in an array. Values are like
>> 1,2,7,9 spread around in the array. I want to plot those values so that
>> I can see the 'regions' in my data, then I want to overlay this with
>> some contour lines drawn from other data.
>>
>> I want to be able to specify the colors for each of the enumerated
>> values, as they need to be consistent throughout my work. My array of
>> enumerated values is *masked* so that there are areas where I don't want
>> to show anything (this would plot as transparent pixels).
>>
>> Has anyone got suggestions on the best way of doing this? It seems that
>> the technique in
>> http://www.scipy.org/Cookbook/Matplotlib/Plotting_Images_with_Special_Values
>>
>> might be overkill, right? It also seemed that it had some problems with
>> masked arrays.
>>
>> Cheers
>>
>> JP
>>
>
>

>>>>> "Eric" == Eric Firing <ef...@ha...> writes:
    Eric> they will not be transparent.  If you need transparent
    Eric> masked regions, then try pcolor instead of imshow.  Pcolor
    Eric> plots nothing at all in masked cells.  Pcolormesh, on the
    Eric> other hand, is like imshow in plotting the assigned bad
    Eric> color and in using a single alpha for everything.

I'm confused about the comments about alpha not working on imshow --
can you elaborate.  On the agg backend at least, the alpha channel is
respected in imshow, eg examples/layer_images.py.  Is there a reason
it does not (or should not) work in the masked example?

JDH