matplotlib-users Mailing List for matplotlib

This is primarily a bug-fix release from 0.65, though a couple of
little features managed to sneak in

 - 4x image speedups for large images

 - figimage bug fixed

 - fixed some bugs which caused the colorbar not to update properly
   when changing colormap interactively

 - refactored axes management to support delaxes, which deletes, as
   opposed to clears, a specified axes.  Default to current axes

 - tkagg's classic and new-fangled toolbars are now embeddable.

 - extended the new set/get introspection features to more classes

 - fixed some tkagg flakiness on win32 regarding unusual uses of show.

 - new cross backend animation idiom in examples/anim.py - use
   interactive mode rather than timers/idle handlers.

 - deferred some initializations in dates and colors modules for
   faster load times.


http://sourceforge.net/projects/matplotlib

Enjoy!

JDH 

>>>>> "seberino" == seberino  <seb...@sp...> writes:

    seberino> I'm trying to make a 2D color plot where the z-axis
    seberino> values show up as different colors for each (x,y) point.
    seberino> I tried to plot the list a+b+c+d using the code snipped
    seberino> below.  I don't understand why this regular grid of
    seberino> points is giving me the weird color plot attached.  Any
    seberino> help would be greatly appreciated.

    seberino> WHY DON't I SEE **SQUARES** OF COLORS IN ATTACHED
    seberino> PLOT???

Because you are not filling your arrays properly.  Print xarray and
yarray in your example and you'll see the problem.  In your example,
xarray is

[[ 1  2  3  4]
 [ 5  6  7  8]
 [ 9 10  1  2]
 [ 3  4  5  6]
 [ 7  8  9 10]
 [ 1  2  3  4]
 [ 5  6  7  8]
 [ 9 10  1  2]
 [ 3  4  5  6]
 [ 7  8  9 10]]

is which is causing the weird plots you observe.

In your script, the problem line is

  xarray.shape = yarray.shape = zarray.shape = xsize, ysize

replace this with  

  xarray.shape = yarray.shape = zarray.shape = ysize, xsize

and you'll get the pcolor you are looking for, I suspect.


JDH

I'm trying to make a 2D color plot where the z-axis values show
up as different colors for each (x,y) point.  I tried to
plot the list a+b+c+d using the code snipped below.
I don't understand why this regular grid of points is giving
me the weird color plot attached.  Any help would be greatly
appreciated.

WHY DON't I SEE **SQUARES** OF COLORS IN ATTACHED PLOT???

a = [(1, 10, 1), (2, 10, 2), (3, 10, 3), (4, 10, 4),  (5, 10, 5),
     (6, 10, 4), (7, 10, 3), (8, 10, 2), (9, 10, 1), (10, 10, 0)]
b = [(1, 20, 1), (2, 20, 2), (3, 20, 3), (4, 20, 4),  (5, 20, 5),
     (6, 20, 4), (7, 20, 3), (8, 20, 2), (9, 20, 1), (10, 20, 0)]
c = [(1, 30, 1), (2, 30, 2), (3, 30, 3), (4, 30, 4),  (5, 30, 5),
     (6, 30, 4), (7, 30, 3), (8, 30, 2), (9, 30, 1), (10, 30, 0)]
d = [(1, 40, 1), (2, 40, 2), (3, 40, 3), (4, 40, 4),  (5, 40, 5),
     (6, 40, 4), (7, 40, 3), (8, 40, 2), (9, 40, 1), (10, 40, 0)]


        # Extracts arrays of values for each axis.

        xarray, yarray, zarray = zip(*plotdata)
        xarray = matplotlib.matlab.array(xarray)
        yarray = matplotlib.matlab.array(yarray)
        zarray = matplotlib.matlab.array(zarray)

        # Finds and sets shapes of arrays.

        ysize = len(sets.Set(yarray))
        xsize = len(yarray) / ysize
        xarray.shape = yarray.shape = zarray.shape = xsize, ysize

        # Create plot.

        matplotlib.matlab.pcolor(xarray, yarray, zarray)


Chris
--
_______________________________________

Christian Seberino, Ph.D.
SPAWAR Systems Center San Diego
Code 2872
49258 Mills Street, Room 158
San Diego, CA 92152-5385
U.S.A.

Phone: (619) 553-9973
Fax  : (619) 553-6521
Email: seb...@sp...
_______________________________________

On Thu, 23 Dec 2004, Steve Chaplin wrote:

[...]

> > P.S: we just looked at backend_gtk.py.
> > Couldn't one safely replace
> >
> >     def draw_lines(self, gc, x, y):
> >         x = x.astype(nx.Int16)
> >         y = self.height*ones(y.shape, nx.Int16) - y.astype(nx.Int16)
> >         self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))
> > by
> >
> >     def draw_lines(self, gc, x, y):
> >         x = x.astype(nx.Int16)
> >         y = self.height - y.astype(nx.Int16)
> >         self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))
> >
> > ? It might give a small improvement.
> I hope so because I made this change to the code a few weeks ago and it
> is now in the file backend_gdk.py in matplotlib 0.65!
> It looks like you are running an old version of matplotlib.

Well, 0.64 ;-) Last week, before we started with this, there were
no debian packages for 0.65. But now they are,
  http://anakonda.altervista.org/debian/
Sorry for the duplication ...

On Wed, 2004-12-22 at 15:43 +0100, Arnd Baecker wrote:
>From this we get for all **Agg backends that
>   - new_gc
>   - _draw_solid
>   - draw_text
> eat up a major part of the time.
> Another important part is spread in the draw chain
> (for example from 74.3% in  to 47.0 %+10.3 % in GTKAgg).

I've noticed that that new_gc() is called many times (60 times when running 
simple_plot.py) with each call creates a new GraphicsContext instance. I think
it would be more efficient to create just one GC instance and reuse it. It 
would suit the way that the Cairo, PS and possibly SVG and Agg backends 
do their drawing but that might mean completely changing the way the 
matplotlib frontend does its drawing.

> P.S: we just looked at backend_gtk.py.
> Couldn't one safely replace
> 
>     def draw_lines(self, gc, x, y):
>         x = x.astype(nx.Int16)
>         y = self.height*ones(y.shape, nx.Int16) - y.astype(nx.Int16)
>         self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))
> 
> 
> by
> 
>     def draw_lines(self, gc, x, y):
>         x = x.astype(nx.Int16)
>         y = self.height - y.astype(nx.Int16)
>         self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))
> 
> ? It might give a small improvement.
I hope so because I made this change to the code a few weeks ago and it
is now in the file backend_gdk.py in matplotlib 0.65!
It looks like you are running an old version of matplotlib.

Steve

>>>>> "Chris" == Chris Barker <Chr...@no...> writes:

    Chris> This trick here is that the binary format of a wxBitmap is
    Chris> both platform and instance dependent. The idea, as I
    Chris> understand it, is that a wxBitmap has the same binary
    Chris> format as is needed for the current display, so each
    Chris> platform is different, and it can also be different
    Chris> depending on the depth of the display. Given all this, I
    Chris> doubt you're going to be able to improve on the wx supplied
    Chris> methods for converting from a wxImage to a wxBitmap. (and
    Chris> if you do, contribute it to wx!)

But this might helpful - if we can detect what kind of binary format
wx is using, we can ask agg to convert itself to this format in a
python buffer object and pass this on directly to the wxBitmap.  Agg
has efficient conversion routines from just about any pixel format to
any other.  This would avoid one copy, because we could do (making up
the syntax for copying a buffer into the bitmap

        FigureCanvasAgg.draw(self)
        if display_format=='ZZZ': # made up pixel format
           buffer = self.to_zzz()
           self.bitmap.UpdateFromBuffer(buffer) # made up method
        else:   # fall back on old method
            s = self.tostring_rgb()  
            w = int(self.renderer.width)
            h = int(self.renderer.height)
            image = wxEmptyImage(w,h)
            image.SetData(s)
            self.bitmap = image.ConvertToBitmap()
            self.gui_repaint()

If we could handle the most common cases, it would be a win for most
users.  Any idea how to query the pixel format of the wx display?

    Chris> Does the Agg backend use a binary format compatible with
    Chris> wxImage? If not, that means there are two conversions
    Chris> required, which might be the source of the slowdown.

agg uses an array of unsigned chars r0, b0, g0, a0, r1, b1, g1, a1,
...

What does wxImage use?

JDH

Matt Newville wrote:
> I also agree that the best
> solution is likely to mean converting the Agg image (pixBuffer??)
> into the wx.bitmap in c++. 

This trick here is that the binary format of a wxBitmap is both platform
and instance dependent. The idea, as I understand it, is that a wxBitmap
has the same binary format as is needed for the current display, so each
platform is different, and it can also be different depending on the
depth of the display. Given all this, I doubt you're going to be able to
  improve on the wx supplied methods for converting from a wxImage to a
wxBitmap. (and if you do, contribute it to wx!)

Does the Agg backend use a binary format compatible with wxImage? If
not, that means there are two conversions required, which might be the
source of the slowdown.

-Chris





-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

>>>>> "Dominique" == Dominique Orban <Dom...@po...> writes:

    Dominique> Aha. I just managed to have the stem drawn. My silly
    Dominique> mistake; i thought that to instantiate a Line2D i
    Dominique> needed to pass it (x0, y0) and (x1, y1), but it rather
    Dominique> expects (x0, x1) and (y0, y1). The arrow looks cool
    Dominique> now.

Rather than a line and a polygon, it might be more flexible and
attractive to design the arrow simply as a polygon (you could then
have control of the linewidth, facecolor, and edgewidth, something
like
         
                p0
              /   \
             /     \
            /       \
         p6--p5   p2--p1
             |    |
             |    |
             |    |
             |    |
             |    |    
             p4--p3


    Dominique> My remaining problem is the coordinates. It seems that
    Dominique> matplotlib is positioning the arrow using pixels as
    Dominique> coordinates, from the bottom left corner of the figure
    Dominique> window.

    Dominique> Is my problem a 'transformation' issue?

Yes.  If you derive your class from Artist and add it to the axes with
ax.add_artist (or Patch if you use the polygon approach above and add
it with ax.add_artist), the axes will set the default data
transformation for you, iff and only if you haven't already set the
transform.  There are three default transforms you can choose from 

  fig.transFigure   # 0,0 is lower left of fig and 1,1 is upper right
  ax.transAxes      # 0,0 is lower left of axes and 1,1 is upper right
  ax.transData      # same coordinates as the data in the axes

You have a additional choices with custom transforms.  One approach
would be to set the coordinates of the polygon in points such that the
arrow tip is 0,0 and the width and height are both 1.  You could then
use a scaling and rotation affine where sx, sy are the x and y scales,
and theta is the angle.  If you apply this affine to the arrow, the
width of the arrow would be sx points, the height sy points, and the
angle would be theta and the sucker would still be pointing at 0,0.
One nice feature of transformations is that the let you combine two
coordinate systems by applying a an offset transformation.  In this
case you'd want to apply and offset in data coords and then the arrow
would be pointing at some data location x,y but would still have a
width and height specified in points.  

This is basically how the ticks work.  An x tick is located at an x
location in data coords, a y location in axes coords (eg 0 for bottom
ticks and 1 for top ticks) and a length in points.


Here's an example.  I'm not sure this is the best design.  It might be
more useful to specify a point for the base and a point for the arrowhead,
and draw the arrow between them.  But I am not sure what the best way
to specify the arrow width if you use that design.  In any case, this
will serve as an example you can study to get an idea of how the
transforms work, and you can go from there. It would also be nice to
have some intelligent labeling built it, eg at the arrow base

from pylab import *
from matplotlib.patches import Polygon
from matplotlib.transforms import Affine, Value, zero
import math

class Arrow(Polygon):
    zorder = 4 # these should generally above the things they mark
    def __init__(self, x, y, xytrans, width, height, theta,
                 tipx=2, tipy=0.2):
        """
        Create an arrow pointing at x,y with a base width and total
        height in points

        theta is the arrow rotation - 0 degrees is point up, 90 is
          pointing to the right, 180 is pointing down, 270 is pointing
          left.

        tipx is the tip width and is expressed as fraction of the base width.

        tipy is the tip height expressed as a fraction of the total
          height

        xytrans is the transformation of the x,y coordinate, eg
          ax.transData for data coords and ax.transAxes for axes coords

        """
         
        #        p0
        #      /   \
        #     /     \
        #    /       \
        # p6--p5   p2--p1
        #     |    |
        #     |    |
        #     |    |
        #     |    |
        #     |    |    
        #     p4--p3

        p0 = 0,0
        p1 = tipx*0.5, -tipy
        p2 = 0.5, -tipy
        p3 = 0.5, -1        
        p4 = -0.5, -1
        p5 = -0.5, -tipy
        p6 = -tipx*0.5, -tipy        
        
        verts = p0, p1, p2, p3, p4, p5, p6
        Polygon.__init__(self, verts)


        theta = math.pi*theta/180.
        a = width*math.cos(theta)
        b = -width*math.sin(theta)
        c = height*math.sin(theta)
        d = height*math.cos(theta)
        a,b,c,d = [Value(val) for val in (a,b,c,d)]
        trans  = Affine(a, b, c, d, zero(), zero())
        trans.set_offset((x,y), xytrans)
        self.set_transform(trans)
        


plot([0,1,2], [1,2,3], 'bo', ms=15)
axis([0,3, 0, 4])

ax = gca()
arrow = Arrow(1,2, ax.transData, 10, 100, 135)
set(arrow, fc='g', ec='r', lw=1)
ax.add_patch(arrow)


show()

On Wed, 22 Dec 2004, Dominique Orban apparently wrote:
> Based on John's advice in a previous post about designing an Arrow class
> (http://sourceforge.net/mailarchive/message.php?msg_id=9962785), i have
> restarted from scratch, defining the stem as a Line2D instance and the
> head as a RegularPolygon instance (for now, a triangle).

This will be great.
I hope you will make sure the arrow head is
filled but not stroked, so that the tip ends
up precisely where desired.

Thank you,
Alan Isaac

Hi Matt,

On Tue, 21 Dec 2004, Matt Newville wrote:

[...]

> I agree that the Agg rendering itself does not seem like the
> bottleneck for WXAgg.  Partly because of that, I'm assuming that
> the WXAgg will be good enough for my needs (as opposed to
> completely rewriting backend_wx) and that getting to GTKAgg level
> of performance would be the goal.  I also agree that the best
> solution is likely to mean converting the Agg image (pixBuffer??)
> into the wx.bitmap in c++.  I'm not sure I have a firm grasp on
> how exactly to do that, but it's worth trying.

We did some more profiling (see the other mail) which
confirms that the calls to ConvertTotBitmap and wxEmptyImage
are the main difference to GTKAgg.
The best would be to directly blit the Agg image
onto the wxFrame (however this assumes that these are
of the same type, which we don't know.
If not, the C++ route should be the fastest.
Unfortunately we both don't speak C++ and cannot
help further here).

Best,

Nikolai and Arnd

Hi John,

On Tue, 21 Dec 2004, John Hunter wrote:

> >>>>> "imaginee1" == imaginee1  <ima...@gm...> writes:
>
>     imaginee1> Hi, after spending a nice afternoon profiling the
>     imaginee1> dynamic examples and looking a bit through the code, we
>     imaginee1> can make a few comments on the performace of the wx
>     imaginee1> backends.  We have used kcachegrind to display the
>     imaginee1> results of hotshot - all files can be found under
>     imaginee1> http://www.physik.tu-dresden.de/~baecker/tmp/profiling/
>
> Hi Arnd, thanks for your profiling information - I very much like the
> hotshot graphs!

Nikolai and I were also quite impressed, in particular
because there is another window in which one
can see the corresponding lines of code, including timings
(even down to the wxpython level!).

However, we don't understand the output completely.
One reason certainly is that we don't know the matplotlib
code well enough. Another reason is that there might
be some glitches (either kcachegrind or the
conversion script).

We just did some more profiling,
for TkAgg, GTK, GTKAgg, WX and WXAgg, see:
  http://www.physik.tu-dresden.de/~baecker/tmp/profiling2/

From this we get for all **Agg backends that
  - new_gc
  - _draw_solid
  - draw_text
eat up a major part of the time.
Another important part is spread in the draw chain
(for example from 74.3% in  to 47.0 %+10.3 % in GTKAgg).

Best,

Nikolai and Arnd


P.S: we just looked at backend_gtk.py.
Couldn't one safely replace

    def draw_lines(self, gc, x, y):
        x = x.astype(nx.Int16)
        y = self.height*ones(y.shape, nx.Int16) - y.astype(nx.Int16)
        self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))


by

    def draw_lines(self, gc, x, y):
        x = x.astype(nx.Int16)
        y = self.height - y.astype(nx.Int16)
        self.gdkDrawable.draw_lines(gc.gdkGC, zip(x,y))

? It might give a small improvement.

P.P.S: Thanks for mentioning good experiences with GTK under
Windows - we will give it a try.

Aha. I just managed to have the stem drawn. My silly mistake; i thought 
that to instantiate a Line2D i needed to pass it (x0, y0) and (x1, y1), 
but it rather expects (x0, x1) and (y0, y1). The arrow looks cool now.

My remaining problem is the coordinates. It seems that matplotlib is 
positioning the arrow using pixels as coordinates, from the bottom left 
corner of the figure window.

Is my problem a 'transformation' issue?

Thanks,
Dominique

Hello John,

>Axel, try replacing the FigureManagerTkAgg code in
>site-packages/matplotlib/backends/backend_tkagg.py with the following
>
>  
>
Perfect. Problem solved !

Best wishes and a merry Christmas,

                               Axel

Hi,

Based on John's advice in a previous post about designing an Arrow class 
(http://sourceforge.net/mailarchive/message.php?msg_id=9962785), i have 
restarted from scratch, defining the stem as a Line2D instance and the 
head as a RegularPolygon instance (for now, a triangle).

I can correctly set the orientation of the arrow head, and from 
debugging information, the class seems to be basically working. The 
class is imported in axes.py and i have defined the new methods 
add_arrow() and arrow()  in Axes. Arrows are stored in a list called 
arrows (i know John advised to avoid the overhead of that---i am simply 
trying to get it to work first).

Now, i know i am missing something (and this must be at the heart of 
matplotlib): how are 'figure coordinates' transformed into 'axes 
corrdinates'? I guess it has to do with the 'transforms' module but.... 
For instance, if i say

    arrow( [0,1], [0,1] )

with the intent of drawing an arrow from (0,0) to (1,1), i can't see the 
graphical result. Now if i say

    arrow( [0,100], [0,100] )

I can see something (the arrow head). And this may or may not be the 
source of my 2nd problem: the arrow head is drawn, but not the stem (as 
in the thread linked to above). The draw() method of my Arrow class is 
very simple:

    self._stem.draw(renderer)
    self._head.draw(renderer)

Any help would be greatly appreciated. I'll be happy to add spiff to the 
Arrow class once i have a basic version of it working and contribute it 
to matplotlib.

I can post some code if you would like; i just thought it would be too 
long in a mailing list...

Thanks,
Dominique

ps: Thanks for the matplotlib manual!


-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.296 / Virus Database: 265.6.3 - Release Date: 12/21/2004

Hi Arnd, John, all,

Thanks for the profiling information on WX and WXAgg backends, and
sorry I haven't been able to participate in this conversation
more.  I've been running experiments for several weeks, and am
definitely looking forward to a break!  Hopefully I'll be able to
devote some time to this in January.

I'm really not a wx or matplotlib expert, buit I am definitely
interested in getting WX or WXAgg to go faster.  Re-drawing line
scans at 15Hz would be plenty fast enough for my needs but I would
like that to include rescaling the axes as well as updating the
line (which is what most of the current benchmarks test).  
Refreshing relatively small images at 1Hz would be OK for me - I
think that's already good enough for me.  Also, I'll have to admit
my timeframe is probably going to be slower than many people on
this list!  So if you or someone else wants to jump in, that would
be fine with me.
  
I agree that the Agg rendering itself does not seem like the
bottleneck for WXAgg.  Partly because of that, I'm assuming that
the WXAgg will be good enough for my needs (as opposed to
completely rewriting backend_wx) and that getting to GTKAgg level
of performance would be the goal.  I also agree that the best
solution is likely to mean converting the Agg image (pixBuffer??)
into the wx.bitmap in c++.  I'm not sure I have a firm grasp on
how exactly to do that, but it's worth trying.

--Matt