matplotlib-users Mailing List for matplotlib (Page 3)

Hello John,

>Axel, try replacing the FigureManagerTkAgg code in
>site-packages/matplotlib/backends/backend_tkagg.py with the following
>
>  
>
Perfect. Problem solved !

Best wishes and a merry Christmas,

                               Axel

Hi,

Based on John's advice in a previous post about designing an Arrow class 
(http://sourceforge.net/mailarchive/message.php?msg_id=9962785), i have 
restarted from scratch, defining the stem as a Line2D instance and the 
head as a RegularPolygon instance (for now, a triangle).

I can correctly set the orientation of the arrow head, and from 
debugging information, the class seems to be basically working. The 
class is imported in axes.py and i have defined the new methods 
add_arrow() and arrow()  in Axes. Arrows are stored in a list called 
arrows (i know John advised to avoid the overhead of that---i am simply 
trying to get it to work first).

Now, i know i am missing something (and this must be at the heart of 
matplotlib): how are 'figure coordinates' transformed into 'axes 
corrdinates'? I guess it has to do with the 'transforms' module but.... 
For instance, if i say

    arrow( [0,1], [0,1] )

with the intent of drawing an arrow from (0,0) to (1,1), i can't see the 
graphical result. Now if i say

    arrow( [0,100], [0,100] )

I can see something (the arrow head). And this may or may not be the 
source of my 2nd problem: the arrow head is drawn, but not the stem (as 
in the thread linked to above). The draw() method of my Arrow class is 
very simple:

    self._stem.draw(renderer)
    self._head.draw(renderer)

Any help would be greatly appreciated. I'll be happy to add spiff to the 
Arrow class once i have a basic version of it working and contribute it 
to matplotlib.

I can post some code if you would like; i just thought it would be too 
long in a mailing list...

Thanks,
Dominique

ps: Thanks for the matplotlib manual!


-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.296 / Virus Database: 265.6.3 - Release Date: 12/21/2004

Hi Arnd, John, all,

Thanks for the profiling information on WX and WXAgg backends, and
sorry I haven't been able to participate in this conversation
more.  I've been running experiments for several weeks, and am
definitely looking forward to a break!  Hopefully I'll be able to
devote some time to this in January.

I'm really not a wx or matplotlib expert, buit I am definitely
interested in getting WX or WXAgg to go faster.  Re-drawing line
scans at 15Hz would be plenty fast enough for my needs but I would
like that to include rescaling the axes as well as updating the
line (which is what most of the current benchmarks test).  
Refreshing relatively small images at 1Hz would be OK for me - I
think that's already good enough for me.  Also, I'll have to admit
my timeframe is probably going to be slower than many people on
this list!  So if you or someone else wants to jump in, that would
be fine with me.
  
I agree that the Agg rendering itself does not seem like the
bottleneck for WXAgg.  Partly because of that, I'm assuming that
the WXAgg will be good enough for my needs (as opposed to
completely rewriting backend_wx) and that getting to GTKAgg level
of performance would be the goal.  I also agree that the best
solution is likely to mean converting the Agg image (pixBuffer??)
into the wx.bitmap in c++.  I'm not sure I have a firm grasp on
how exactly to do that, but it's worth trying.

--Matt

>>>>> "imaginee1" == imaginee1  <ima...@gm...> writes:

    imaginee1> Hi, after spending a nice afternoon profiling the
    imaginee1> dynamic examples and looking a bit through the code, we
    imaginee1> can make a few comments on the performace of the wx
    imaginee1> backends.  We have used kcachegrind to display the
    imaginee1> results of hotshot - all files can be found under
    imaginee1> http://www.physik.tu-dresden.de/~baecker/tmp/profiling/

Hi Arnd, thanks for your profiling information - I very much like the
hotshot graphs!

I just have two comments.  

All of your suggestions are imminently reasonable.  The major problem
is that the wx backend has been mostly rudderless since Jeremy, the
author, stopped maintaining it, though I've filled in when I can.
Matthew Newville has recently signed on as the new maintainer and has
CVS commit privileges, but I don't know how much time he has to
address these issues right now.  I don't have any extra time to devote
to wx optimizations, currently.  If you would like to do some work
here, I would be happy to add you to the developers list.

The second point is that in your previous email you appeared to
indicate that GTK wasn't a good option for you because many of your
students use win32.  I use the gtk backend on win32 - you have to run
the GTK runtime installer and the pygtk installer, but it otherwise
works great, and the matplotlib gtk extension code is compiled into
the matplotlib win32 installer.  There are install instructions for
win32 at http://matplotlib.sourceforge.net/backends.html#GTK .

JDH

>>>>> "Michel" == Michel Sanner <sa...@sc...> writes:

    Michel> Hello, I finally got around to play with matplotlib and
    Michel> try to create node for using it within Vision (my visual
    Michel> programming environemnt).

Hi Michel,

It's excellent to hear that you are trying to incorporate matplotlib!
Only good things can come of stress testing matplotlib in such a
sophisticated environment.

    Michel> I was wondering if there is a way the clear the area used
    Michel> by a given subplot.  cla() seesm the clear the data, but
    Michel> the axis and background remain.  The reason I am asking is
    Michel> that I envisioned that in my networks I would like to be
    Michel> able to create a Figure using one node and then have say a
    Michel> Histogram node be placed in a subplot and a scatter plot
    Michel> in another subplot (in the same figure).  Now if the
    Michel> Histogram node has a subplot parameter which can be
    Michel> modified by the user to move this particular graph to
    Michel> another location in the picture I need to get ride of what
    Michel> I drew in the previous location, but since parts of the
    Michel> figure might be generated by other nodes in the network I
    Michel> would not want to clear the whole figure.

I see what you are trying to do and there is no support in the current
release for this.  However, it will be mostly trivial to add, and
definitely useful, so I can quickly put it in for the next release.

Before doing so, it would help for me to know if you are using the OO
interface (eg examples/embedding_in_tk.py) or the pylab (formerly
matplotlib.matlab).  The latter does a fair amount of magic under the
scenes managing the current figure and axes and so is not the best for
embedding in an application.  It thus requires more work to delete an
axes, since there is a separate figure/axes management layer.

Do you envision providing a scripting interface to your Vision users,
in which case the pylab interface probably makes sense, or will you
ultimately be maintaining control over the creation of figure windows
and axes and providing a GUI layer to your users, in which case the OO
embedded approach makes sense.

Note that if all you want to do is move the subplot in a figure, all
you need to do is call

  ax = subplot(211)
  ax.set_position((left,bottom,width,height))

where l,b,w,h are fractions of the figure width and height.  If you
want to drag and drop axes from one figure to another, which is
reasonable, then additional work will need to be done to remove the
axes from one and add to another.  Sharing a subplot in two figures
will probably remain unsupported, but with a little work you could
move one from one figure to another.  Note that
matplotlib.axes.Subplot is a special case of matplotlib.axes.Axes, and
each are placed with l,b,w,h.  The difference is that subplot does the
l,b,w,h computation for you -- see examples/axes_demo.py.

On an related note, the ticklabels in your screenshot appeared a
little crowded.  You may want to take a look at
http://matplotlib.sourceforge.net/faq.html#TEXTOVERLAP .

    Michel> Another question was whether it is possible to find out
    Michel> the figure number from the figure handle ?  If I pass the
    Michel> Figure instance between nodes to tell drawing node in
    Michel> which figure to place their graphical output I need a way
    Michel> to active the figure that comes as an input.  Currently I
    Michel> add a number attribute to the figure when I create it in
    Michel> the figure node.

The Figure class doesn't have a concept of number, but the
FigureManager does.  In the pylab interface, calls to make a new
figure create a new FigureManager (abstract class in
backend_bases.FigureManager, concrete classes in the various
matplotlib/backends/backend_something.py).  The figure manager has a
few public attributes you can access

    manager = get_current_fig_manager()
    manager.num    # what you are after
    manager.canvas # a backend_bases.FigureCanvasBase concrete impl
    manager.window # the GUI window, eg tk.Window

As I alluded to above though, depending on your ultimate goals, once
you get past the playing stage you *may* want to forgo the pylab
interface (which the figure manager is designed for) in exchange for
the control of the OO interface.  Note that a new
examples/embedding_in_tk2.py was recently added to CVS which shows how
to use tk matplotlib with the default toolbar in a tk app.

There is an unreleased users guide in progress, and it has some more
details and schematics on how matplotlib is organized.

  http://cvs.sourceforge.net/viewcvs.py/*checkout*/matplotlib/users_guide/users_guide.pdf?rev=1.3

    Michel> Thanks for any input .. and congratualtion on the very
    Michel> nice package !

High praise coming from you!  Thanks.

JDH

Hello,

I finally got around to play with matplotlib and try to create node for 
using it within Vision (my visual programming environemnt).

I was wondering if there is a way the clear the area used by a given 
subplot.  cla() seesm the clear the data, but the axis and background 
remain.  The reason I am asking is that I envisioned that in my networks 
I would like to be able to create a Figure using one node and then have 
say a Histogram node be placed in a subplot and a scatter plot in 
another subplot (in the same figure).  Now if the Histogram node has a 
subplot parameter which can be modified by the user to move this 
particular graph to another location in the picture I need to get ride 
of what I drew in the previous location, but since parts of the figure 
might be generated by other nodes in the network I would not want to 
clear the whole figure.

Another question was whether it is possible to find out the figure 
number from the figure handle ?  If I pass the Figure instance between 
nodes to tell drawing node in which figure to place their graphical 
output I need a way to active the figure that comes as an input.  
Currently I add a number attribute to the figure when I create it in the 
figure node.

Thanks for any input .. and congratualtion on the very nice package !

-- 

-----------------------------------------------------------------------
   o
  /    Michel F. Sanner Ph.D.            The Scripps Research Institute
o      Associate Professor               Department of Molecular Biology
  \                                      10550 North Torrey Pines Road
   o   Tel. (858) 784-2341               La Jolla, CA 92037
  /    Fax. (858) 784-2860
o     sa...@sc...                 http://www.scripps.edu/~sanner
-----------------------------------------------------------------------

>>>>> "John" == John Hunter <jdh...@ac...> writes:

    John> I made some small changes which helped here - eg, deferring
    John> the initialization of the LUTs until they are actually
    John> requested.  This shaved 0.3 s off startup time on my system.
    John> With Todd's help, I also made some changes in the core
    John> "fromarray" in extension code which delivered some speedups,
    John> and removed some extra checks in the colormapping code which
    John> are not needed for data that are properly normalized.  I
    John> also think I found and fixed redundant calls to draw in some
    John> backends due to improper event handling and hold handling
    John> that crept into 0.65.

Well, Xavier Gnata just pointed out to me off list that almost half
the cost of the default image handling was in the normalization calls
to min and max.  After a little poking around, I discovered we were
using python's min and max here, which means sequence API.  Ouch!

So we get another 2x speedup on top of the numbers I just posted using
default normalization and colormapping.

# GTKAgg default normalization and colormapping

   # 0.65
   matplotlib 0.65 figimage : 9.97s    
   matplotlib 0.65 imshow   : 9.91s    

   # optimization numbers in my last post
   matplotlib figimage  : 5.23s    
   matplotlib imshow    : 5.18s    

   # as above but using nxmin and nxmax
   matplotlib figimage  : 2.21s    
   matplotlib imshow    : 2.24s    

So out of the box the next matplotlib will be more than 4x faster than
the last release for images.  A long way from MIDAS and IRAF, but
still satisfying for a day's work.

JDH

On Dec 21, 2004, at 1:02 PM, Xavier Gnata wrote:

>
> Hi,
>
> Please consider this code in colors.py  (from 546) :
>    if vmin is None or vmax is None:
>           rval = ravel(val)
>           if vmin is None:
>               vmin = min(rval)
>           if vmax is None:
>               vmax = max(rval)
>
> On my computer  to be much faster without using rval :
>
>       if vmin is None or vmax is None:
>           #rval = ravel(val)
>           if vmin is None:
>               vmin = val.min()
>           if vmax is None:
>               vmax = val.max()
>
But note that .min and .max are only supported under numarray (I 
believe). There are a number of things we would like to take advantage 
of (especially with regard to array indexing) that we can't in 
matplotlib since the code must work with both Numeric and numarray.

Perry

>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:
  
    Perry> As John later alluded to, the time for the window to come
    Perry> up is a one time cost if you are running from an
    Perry> interactive prompt. It shouldn't be paid for subsequent
    Perry> display updates.

I made some small changes which helped here - eg, deferring the
initialization of the LUTs until they are actually requested.  This
shaved 0.3 s off startup time on my system.  With Todd's help, I also
made some changes in the core "fromarray" in extension code which
delivered some speedups, and removed some extra checks in the
colormapping code which are not needed for data that are properly
normalized.  I also think I found and fixed redundant calls to draw in
some backends due to improper event handling and hold handling that
crept into 0.65.

Here are my current numbers for a 1600x1600 image

# GTKAgg default normalization and colormapping
   matplotlib 0.65 figimage : 9.97s    
   matplotlib 0.65 imshow   : 9.91s    
   matplotlib CVS figimage  : 5.23s    
   matplotlib CVS imshow    : 5.18s    

# GTKAgg prenormalized data and default ("hot") colormapping
   matplotlib 0.65 figimage : 3.46s  
   matplotlib 0.65 imshow   : 3.37s
   matplotlib CVS figimage  : 1.95s
   matplotlib CVS imshow    : 2.01s

# GTKAgg prenormalized data and custom grayscale colormapping
   matplotlib 0.65 figimage : 2.05s    
   matplotlib 0.65 imshow   : 1.95s    
   matplotlib CVS figimage  : 1.15s
   matplotlib CVS imshow    : 1.21s

So the situation is improving.  As I noted before, interaction with
plots via the toolbar should also be notably faster.

This would make a good FAQ....

JDH

Hi,

Please consider this code in colors.py  (from 546) :
 
    if vmin is None or vmax is None:
           rval = ravel(val)
           if vmin is None:
               vmin = min(rval)
           if vmax is None:
               vmax = max(rval)

On my computer  to be much faster without using rval :

       if vmin is None or vmax is None:
           #rval = ravel(val)
           if vmin is None:
               vmin = val.min()
           if vmax is None:
               vmax = val.max()

I am now able to do a "A=rand(2000,2000) figimage(A)" in a almost 
acceptable duration.

Hope this helps

Xavier.
ps : may be in double. sorry.

Hi,  

after spending a nice afternoon profiling the dynamic 
examples and looking a bit through the code, we can make 
a few comments on the performace of the wx backends.
We have used kcachegrind to display the results of
hotshot - all files can be found under
http://www.physik.tu-dresden.de/~baecker/tmp/profiling/


WXAgg (http://www.physik.tu-dresden.de/~baecker/tmp/wxagg.png):
WxAgg leaves the composition of the plot frame to the agg backend 
and blits the calculated picture to the screen. The agg backend seems 
to be quite efficient since we could not find any predominant 
bottleneck. 
But, agg and wx use different image formats, therefore the picture 
has to be transformed using wx.Image.ConvertToBitmap which is 
a deprecated wx method.
On the other hand, gtk-agg uses the C++ routine
agg_to_gtk_drawable  which seems to be faster. 
Optimising the agg to wxPython conversion 
could lead to a similar speedup.

Because wxAgg leaves the drawing to agg it is faster than the 
wx backend.


WX (http://www.physik.tu-dresden.de/~baecker/tmp/profiling/wx.png):
As Chris Barker correctly presumed the wx backend doesn't do any 
caching. Therefore about 15% of program time is spent in the 
freetype2 library (ft2font.FT2Font) and another 7% for changing 
wxPens wxBrushes and sizes. So storing all this information on the
python level (and only altering it if there is a change) would 
lead to a first performance increase. 
(draw_text and set_foreground branch in the picture) 

The wx device context (wxDC) is not passed as local variable or 
instanced as global but selected for every drawing operation and 
unselected afterwards. 
(new_gc branch) 

The drawXXXs (note the plural) commands in wx expect the 
input to be an array of points. The following line of code 
is from the wx-backend. 
> gc.DrawLines([wxPoint(int(x[i]), self.height - int(y[i])) for i in
range(len(x))])
Avoiding the explicit loop and 
vectorizing the expression should give another speed increase
(_draw_solid branch).

Best,

Nikolai and Arnd 

-- 
+++ Sparen Sie mit GMX DSL +++ http://www.gmx.net/de/go/dsl
AKTION für Wechsler: DSL-Tarife ab 3,99 EUR/Monat + Startguthaben

On Dec 20, 2004, at 3:55 AM, Eric Emsellem wrote:

>

Sorry to get into this discussion so late.

> sterday I tried with a 1600x1600 pixels image. NOTE: this is a very 
> reasonable
> size (and typical) in my work and I expect much much bigger ones in a 
> near future
> (up to 20 000 x 20 000).
>
Wow, I want your 20Kx20K image display! Seriously, it sounds like you 
do not intend to display the whole thing at once. In that case I would 
consider doing the slicing and reduction outside of matplotlib and then 
displaying the subsampled or sliced region using the size of the 
window. Applying color transformations on the whole image is going to 
result in a lot of wasted cpu time. I also recognize that 1.6K^2 
displays are not that unreasonable so reasonable performance with this 
size is something one wants to achieve.

> ==> Matplotlib takes ~20 seconds to display it !!!
>  (after 12 seconds the window opens, and then it takes
>  another 8 seconds to be displayed)
>
As John later alluded to, the time for the window to come up is a one 
time cost if you are running from an interactive prompt. It shouldn't 
be paid for subsequent display updates.

> (as compared to less than .2 sec for Midas, Iraf and others so a 
> factor of 100 at least!!!
> and less than a second using the ppgplot routines).
>
For better understanding the comparison, when you use Midas, what are 
you displaying to? It's been a long, long, time since I've used Midas 
so I forget (or likely, it's changed) how image display is done. Are 
you using DS9, ximtool or SAOIMAGE? Or a different display mechanism?

By the way, we do have a module that allows displaying numarray images 
to DS9 and related image display programs (google numdisplay). But if 
you are hoping to combine matplotlib features with image display, this 
isn't going to help much. But if you are looking mainly to use DS9 or 
ximtool features, you can just use them directly and save yourself the 
trouble of trying to emulate them (not that wouldn't be a nice thing to 
have for matplotlib).


Perry

just to answer your question:

>You still haven't answered my question if you typically want grayscale
>or colormapped images ....
>  
>
I am using colourmaps.
Usually a 256 levels is usually more than enough! Here is what I did when
I transformed the midas lut and itt files (3 columns RGB in ascii files 
see below)
into local python lut (pglut for ppgplot lut).

I am now thinking of using another suggestion (from Arnd) on top of what 
you said.
If the image is very large then I only load the central region in imshow
(so I extract the central part, ''central'' meaning center with respect to
some predefined center, by default the centre of the image). Then if I want
to see the full image in the window I scale it (factor N)
by only plotting 1 out of N pixels.... This should reproduce what Midas 
did I think
(not so sure, but Midas has a timed display of < 0.2 sec there in fact 
whatever
the scale is / size of the window)

One note from a local neird: it seems that the normalisation (colour.py)
in the matplotlib 0.65 version
is doing a divide by (vmax - vmin) and this was timed to be quite long
(see previous mails...). But a numarray division should be instantaneous 
even
for such a big image so there is something wrong there. Maytbe first
do : dv = vmax - vmin and divide by dv? (maybe this is not the reason,
using ''divide" and '/' may be different too?)

Eric

"""######################################
   # MIDAS-LIKE LUT and ITT
   ######################################"""
class MidasLut:
    def __init__(self):
        self.LUT_PATH = '/soft/python/pcral/plotutils/midaslut/'
        self.R = arange(256)/255.
        self.G = arange(256)/255.
        self.B = arange(256)/255.
        self.L = arange(256)/255.
    def lut(self,name):   
        sname = str(name)
        sname = self.LUT_PATH+sname+".lasc"
	if os.path.isfile(sname):
           f = open(sname)
           list = f.readlines()
           for i in range(256):
             self.R[i] = float(list[i].split()[0])
             self.G[i] = float(list[i].split()[1])
             self.B[i] = float(list[i].split()[2])
           f.close
	else :
	   print 'ERROR: Lut file', name,'does not exist'
    def itt(name):  
        sname = str(name)
        sname = LUT_PATH+sname+".iasc"
	if os.path.isfile(sname):
           f = open(sname)
           list = f.readlines()
           for i in range(256):
             L[i] = float(list[i].split()[0])
           f.close
	else :
	   print 'ERROR: ITT file', name,'does not exist'

pglut = MidasLut()


-- 
===============================================================
Observatoire de Lyon                 ems...@ob...
9 av. Charles-Andre                      tel: +33 4 78 86 83 84
69561 Saint-Genis Laval Cedex            fax: +33 4 78 86 83 86
France               http://www-obs.univ-lyon1.fr/eric.emsellem
===============================================================

Hi Eric,

On Tue, 21 Dec 2004, Eric Emsellem wrote:

> Hi again,
> thanks a lot for the fixing and comments on figimage. This looks much
> much better now. Indeed a local matplotlib user had also pointed
> out to me the time it spent on cmap.
>
> By the way, the problem I have now is that I would like to have a quick
> way of looking at my images (the 1600x1600) but being able to
> size it INTO the actual opened window (so basically this calls for using
> imshow
> and not figimage).

Just a few thoughts:
- you could gain a speed-up by displaying a reduced image,
  e.g. by passing   image_mat[::2,::2]
  which would correspond to a 800x800 image
  (but latest when zooming one would need the full image ;-(.
- maybe you have to code the zooming youself in terms of figimage
  (In particular if you are still aiming at the 20000x20000 case)
- Is it right that from imshow you would only need
  the  """ extent is a data xmin, xmax, ymin, ymax for making image plots
  registered with data plots.  Default is the image dimensions in pixels"""?
  Somehow I would guess that this should not be a part
  which is too slow, so maybe one could do
  something like this (without any interpolation) also for
  figimage.

Not-more-than-0.5-cents-this-time-ly yours,

Arnd

Hi again,
thanks a lot for the fixing and comments on figimage. This looks much
much better now. Indeed a local matplotlib user had also pointed
out to me the time it spent on cmap.

By the way, the problem I have now is that I would like to have a quick
way of looking at my images (the 1600x1600) but being able to
size it INTO the actual opened window (so basically this calls for using 
imshow
and not figimage). Just to be clear here is what I usually do (e.g. with 
Midas):

1/ I create a display window with the size (and location on my desktop) 
I wish
2/ I ''load'' my big image in there. Depending on the size it will take 
the full window or not.
3/ then depending on the region I wish to look at I scale and recenter 
the image
   to e.g., have it all in the window or just zoom in some regions.
4/ then I can interact with the cursor to e.g. make a cut of the image 
(and display it
in another window) or get some coordinates

so this would look like this in Midas:

1/ create/disp   1 600,600  # create the display window number 1 with 
600x600 px
2/ load image.fits      # load the image: from a fits file
3/ load image.fits scale=-2 center=0,0 # center the image at coord 0,0 and
                                               # scale it down by a 
factor of 2, (- = down, + = up)
4/ get/curs                  #activate the cursor on the image so that a 
click
                                       # prints out the corresponding 
coordinates

By using imshow in matplotlib I think I can solve all these things
(and as soon as the .key fields are activated so I can use the keyboard
to interact with the display), but then it will show again its very slow 
behaviour
(interpolation, etc...) ? Would there be a way to have such a command
but with a much quicker loading / interacting? (note that the pan and zoom
things would probably do it - although it would need to keep the 
coordinates right
so there is no confusion there - , but at the moment this is not usable 
in imshow
with large images as it is much too slow)

Cheers,
Eric

-- 
===============================================================
Observatoire de Lyon                 ems...@ob...
9 av. Charles-Andre                      tel: +33 4 78 86 83 84
69561 Saint-Genis Laval Cedex            fax: +33 4 78 86 83 86
France               http://www-obs.univ-lyon1.fr/eric.emsellem
===============================================================

On Saturday 18 December 2004 08:25 pm, John Hunter wrote:

Thanks John, I got that working and spent some time fiddling
with the classic toolbar as well.  It seems to fit my current 
needs better thant the new toolbar.  However, it appears to 
be a bit more tied to the pylab interface.  My trials at extraplating
from the changes you made for the new interface foundered.  I need
to spend more time getting up to speed on other items, such as adding
menus for user file selection, etc.   

My application involves potentially long time series, perhaps months
or even years long, of flow or water-surface elevation.  It would 
be nice to be able to quickly pan through the series at a large scale
looking for  details that might need correction or where the modeling
does an especially poor job. 

I'm continuing to poke and prode on the source and the draft guide has
helped out on getting a better overview on what is going on.  Clearly 
OOP can be obscure at first.  It is sometimes unclear where the actual 
work gets done.  


> >>>>> "Delbert" == Delbert D Franz <iq...@so...> writes:
> 
>     Delbert> Quite impressed with matplotlib but the learning curve is
>     Delbert> steep and I am feeling my way along a tortuous cave in
>     Delbert> dim light!!
> 
> Well it appears you are doing a good job of it.  Fortunately, there's
> often someone around here to strike a match if you lose your way.
> Documentation is scant, especially on the OO / application embedding
> side.  Make sure you have at least read through
> examples/pythonic_matplotlib.py in the examples subdirectory of the
> source distribution, and here is a link to a draft version of the
> users guide
> http://cvs.sourceforge.net/viewcvs.py/*checkout*/matplotlib/users_guide/users_guide.pdf?rev=1.3
> 
> But these will be no substitute for opening up
> matplotlib/backends/backend_tkagg.py and learning by example.  Ie, if
> you want to figure out how to embed a toolbar in tkagg, the best
> reference is often the source code, which does just that.
> 
> Well, often, but not always.  This list is another good resource to
> turn to.  I myself did not know how to do it for tkagg since I didn't
> write that backend.  So I read through the source and found that the
> tkagg toolbar in the current implementation is actually weakly tied to
> the pylab interface in that is uses a variable called figman, which is
> a reference to the "FigureManager" which is a class pylab uses to
> manage figures.
> 
> No worries, it was trivial to factor out this dependence, and at the
> end of this email, I'll attach a modified backend_tkagg.py which you
> can drop into site-packages/matplotlib/backends that enables the
> toolbar to work with matplotlib embedded in tk apps.  It was a trivial
> change.  I'll also attach some example code showing you how to use
> it.  
> 
> The example code includes the exact toolbar that matplotlib uses.  Of
> course, you'll probably want to customize the toolbar, maybe to add
> some widgets of your own.  To do that, you'll want to subclass
> NavigationToolbar2TkAgg, override the default _init_toolbar method to
> create the buttons and widgets you need, and define functions that
> will be called when your custom buttons are clicked, eg myfunction
> below.  Since you're an avowed OO newbie, I'll give a sketch of that
> approach below, which is untested code meant merely to shine a dim
> light.  But the example I'm including below, embedding_in_tk2.py, does
> work, and adds the default matplotlib toolbar to a tk/matplotlib app.
> Surprisingly, you appear to be the first person to ever attempt to
> embed the tkagg toolbar2 into your own app.
> 
> So here is how you might go about customizing a toolbar
> 
> from backend_tkagg import NavigationToolbar2TkAgg
> 
> class MyToolbar(NavigationToolbar2TkAgg)
>     def _init_toolbar(self):
>         # this was all copied verbatim from backend_tkagg.py
>         xmin, xmax = self.canvas.figure.bbox.intervalx().get_bounds()
>         height, width = 50, xmax-xmin
>         Tk.Frame.__init__(self, master=self.window,
>                           width=width, height=height,
>                           borderwidth=2)
>         
>         self.update()  # Make axes menu
> 
>         self.bHome = self._Button( text="Home", file="home.ppm",
>                                    command=self.home)
> 
>         self.bBack = self._Button( text="Back", file="back.ppm",
>                                    command = self.back)
>         
>         self.bForward = self._Button(text="Forward", file="forward.ppm",
>                                      command = self.forward)
> 
>         self.bPan = self._Button( text="Pan", file="move.ppm",
>                                   command = self.pan)
> 
>         self.bZoom = self._Button( text="Zoom",
>                                    file="zoom_to_rect.ppm",
>                                    command = self.zoom)
> 
>         self.bsave = self._Button( text="Save", file="filesave.ppm",
>                                    command = self.save_figure)
> 
>         ### now I'm going to add a custom button that calls myfunction
>         self.mybutton = self._Button( text="Save", file="myicon.ppm",
>                                       command = self.myfunction)
>         self.message = Tk.StringVar(master=self)
>         self._message_label = Tk.Label(master=self, textvariable=self.message)
>         self._message_label.pack(side=Tk.RIGHT)
>         self.pack(side=Tk.BOTTOM, fill=Tk.X)
> 
>     def myfunction(self, *args):
>         # this function is called when "mybutton" is clicked
>         print "You clicked me!"
> 
> 
> Now, all you need to do is create a MyToolbar instance rather than a
> NavigationToolbar2Tkagg instance in the example code
> embedding_in_tk2.py.
> 
>     Delbert> After 41 years in software development, I get to OOP and
>     Delbert> Python!!  Most of my usage has and will be Fortran but I
>     Delbert> have a smattering of C (a few months on an 8-bit
>     Delbert> machine), a log of PL/1 a long time ago, some PDP 8-I
>     Delbert> assembly, and Basic.  I have not used C++ and do not plan
>     Delbert> to do so.
> 
> It's a long hard road I plowed myself.  I cut my teeth on an A/D
> controller and 8 channel digital oscilloscope that I wrote entirely
> from scratch in quick basic -- not a single external library to help
> me out -- with which I did all my experiments for my dissertation.  I
> also did a lot of numerical analysis purely in FORTRAN in those days.
> Unlike you, I did willingly learn C++ after all that and it was
> sunlight to me after years in the dark -- it appeared designed to
> solve all the problems I had experienced firsthand in my years of
> coding BASIC and FORTRAN.  But you do predate me - I've never touched
> a PL/1 or PDP 8.
> 
> Anyway, you may find yourself backing away from the "and will be" part
> of your statement above.  Time will tell.
> 
> Hope this helps - feel free to post again when and if you get stuck.
> 
> JDH
> 
> 

>>>>> "Dominique" == Dominique Orban <Dom...@po...> writes:

    Dominique> I have also tried to resize the image but the same is
    Dominique> happening. Should i reset the origin of the image
    Dominique> manually somewhere?

This bug was mentioned and fixed in the figimage code I submitted in
the earlier post.  Sorry if it wasn't clear.  Here is the modified
function again; replace the pytlab.py function by the same name

# the modified figimage function
def figimage(*args, **kwargs):    
    # allow callers to override the hold state by passing hold=True|False
    try: ret =  gcf().figimage(*args, **kwargs)
    except ValueError, msg:
        msg = raise_msg_to_str(msg)
        error_msg(msg)
        raise RuntimeError(msg)
    except RuntimeError, msg:
        msg = raise_msg_to_str(msg)
        error_msg(msg)
        raise RuntimeError(msg)
    draw_if_interactive()
    gci._current = ret
    return ret
figimage.__doc__ = Figure.figimage.__doc__ + """
Addition kwargs: hold = [True|False] overrides default hold state"""


-------------------------------------------------------

Hi John,

I looked over the diffs and they look fine;  your analysis sounds
plausible to me.  I'll try to reproduce the problem using the test
script and see if your fix removes it.  If so,  I'll go ahead and commit
the changes.

Todd

On Mon, 2004-12-20 at 13:12 -0600, John Hunter wrote:
> >>>>> "Axel" == Axel Kowald <A.K...@gm...> writes:
>     Axel> This is not really appropriate for me, since I read some
>     Axel> user input and decide then (after the script is running) if
>     Axel> I produce screen output or only a PS file :-(
> 
>     Axel> If you or anyone else finds a solution, please let me know.
> 
> So this has nothing to do with the ps save.  The core problem is
> exposed by
> 
>   from pylab import *
>   plot([1,2,3])
> 
> with interactive false and no call to show.  I traced the source of
> the error message to binding the destroy event to the window.
> Apparently the destroy is being called but the window is never shown.
> By moving the destroy binding into the figure manager show method, all
> appears to be fixed.  Todd, you may want to look over this but I think
> it's sound
> 
> Axel, try replacing the FigureManagerTkAgg code in
> site-packages/matplotlib/backends/backend_tkagg.py with the following
> 
> class FigureManagerTkAgg(FigureManagerBase):
>     """
>     Public attributes
> 
>     canvas      : The FigureCanvas instance
>     num         : The Figure number
>     toolbar     : The tk.Toolbar
>     window      : The tk.Window
>     """
>     def __init__(self, canvas, num, window):
>         FigureManagerBase.__init__(self, canvas, num)        
>         self.window = window
>         self.window.withdraw()
>         self.window.wm_title("Figure %d" % num)
>         self.canvas = canvas
>         self._num =  num
>         t1,t2,w,h = canvas.figure.bbox.get_bounds()
>         w, h = int(w), int(h)
>         self.window.minsize(int(w*3/4),int(h*3/4))
>         if matplotlib.rcParams['toolbar']=='classic':
>             self.toolbar = NavigationToolbar( canvas, self )
>         elif matplotlib.rcParams['toolbar']=='toolbar2':
>             self.toolbar = NavigationToolbar2TkAgg( canvas, self.window )
>         else:
>             self.toolbar = None
>         if self.toolbar is not None:
>             self.toolbar.update()
>         self.canvas._tkcanvas.pack(side=Tk.TOP, fill=Tk.BOTH, expand=1)
>         self._shown = False
>         
>     def resize(self, event):
>         width, height = event.width, event.height
>         self.toolbar.configure(width=width) # , height=height)
> 
>     def show(self):
>         def destroy(*args):
>             self.window = None
>             Gcf.destroy(self._num)
>         if not self._shown: self.window.bind("<Destroy>", destroy)
> 
>         _focus = windowing.FocusManager()
>         self.window.deiconify()
>         self.canvas.draw()
>         self._shown = True
>         
>     def add_subplot(self, *args, **kwargs):
>         a = FigureManagerBase.add_subplot(self, *args, **kwargs)
>         if self.toolbar is not None:
>             self.toolbar.update()
>         return a
>     
>     def add_axes(self, rect, **kwargs):
>         a = FigureManagerBase.add_axes(self, rect, **kwargs)
>         if self.toolbar is not None:
>             self.toolbar.update()
>         return a
>     
>     def set_current_axes(self, a):
>         if a not in self.axes.values():
>             error_msg_tkpaint('Axes is not in current figure')
>         FigureManagerBase.set_current_axes(self, a)
> 
>     def destroy(self, *args):
>         if Gcf.get_num_fig_managers()==0 and not matplotlib.is_interactive():
>             if self.window is not None:
>                 self.window.quit()
>         if self.window is not None:
>             self.window.destroy()
>         self.window = None
> 
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now. 
> http://productguide.itmanagersjournal.com/
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

>>>>> "Axel" == Axel Kowald <A.K...@gm...> writes:
    Axel> This is not really appropriate for me, since I read some
    Axel> user input and decide then (after the script is running) if
    Axel> I produce screen output or only a PS file :-(

    Axel> If you or anyone else finds a solution, please let me know.

So this has nothing to do with the ps save.  The core problem is
exposed by

  from pylab import *
  plot([1,2,3])

with interactive false and no call to show.  I traced the source of
the error message to binding the destroy event to the window.
Apparently the destroy is being called but the window is never shown.
By moving the destroy binding into the figure manager show method, all
appears to be fixed.  Todd, you may want to look over this but I think
it's sound

Axel, try replacing the FigureManagerTkAgg code in
site-packages/matplotlib/backends/backend_tkagg.py with the following

class FigureManagerTkAgg(FigureManagerBase):
    """
    Public attributes

    canvas      : The FigureCanvas instance
    num         : The Figure number
    toolbar     : The tk.Toolbar
    window      : The tk.Window
    """
    def __init__(self, canvas, num, window):
        FigureManagerBase.__init__(self, canvas, num)        
        self.window = window
        self.window.withdraw()
        self.window.wm_title("Figure %d" % num)
        self.canvas = canvas
        self._num =  num
        t1,t2,w,h = canvas.figure.bbox.get_bounds()
        w, h = int(w), int(h)
        self.window.minsize(int(w*3/4),int(h*3/4))
        if matplotlib.rcParams['toolbar']=='classic':
            self.toolbar = NavigationToolbar( canvas, self )
        elif matplotlib.rcParams['toolbar']=='toolbar2':
            self.toolbar = NavigationToolbar2TkAgg( canvas, self.window )
        else:
            self.toolbar = None
        if self.toolbar is not None:
            self.toolbar.update()
        self.canvas._tkcanvas.pack(side=Tk.TOP, fill=Tk.BOTH, expand=1)
        self._shown = False
        
    def resize(self, event):
        width, height = event.width, event.height
        self.toolbar.configure(width=width) # , height=height)

    def show(self):
        def destroy(*args):
            self.window = None
            Gcf.destroy(self._num)
        if not self._shown: self.window.bind("<Destroy>", destroy)

        _focus = windowing.FocusManager()
        self.window.deiconify()
        self.canvas.draw()
        self._shown = True
        
    def add_subplot(self, *args, **kwargs):
        a = FigureManagerBase.add_subplot(self, *args, **kwargs)
        if self.toolbar is not None:
            self.toolbar.update()
        return a
    
    def add_axes(self, rect, **kwargs):
        a = FigureManagerBase.add_axes(self, rect, **kwargs)
        if self.toolbar is not None:
            self.toolbar.update()
        return a
    
    def set_current_axes(self, a):
        if a not in self.axes.values():
            error_msg_tkpaint('Axes is not in current figure')
        FigureManagerBase.set_current_axes(self, a)

    def destroy(self, *args):
        if Gcf.get_num_fig_managers()==0 and not matplotlib.is_interactive():
            if self.window is not None:
                self.window.quit()
        if self.window is not None:
            self.window.destroy()
        self.window = None

>>>>> "John" == John Hunter <jdh...@ac...> writes:


    John> This change alone gives me more than a 2x speedup.  So with
    John> GTKAgg + a custom normalizer (in this case a do nothing
    John> normalizer) you'll be running 4-5 times faster than you were
    John> before, me thinks.

Of the remaining time, about half of it, on my system, is spent in the
colormapping.  Almost all of this is in
LinearSegmentedColormap.__call__, and is split between these numerix
methods

  0.12s  zeros : create the RGBA colormapped output array 
  0.23s  where : make sure the data are in the [0,1] interval
  0.49s  take + makeMappingArray - actually doing the mapping

You may not want or need some of the overhead and extra checking that
matplotlib does in the colormapping.  Do you want colormapping at all
by the way?

If not, you can special case colormapping by simply converting to RGB
in the 0-255 interval.  Note, as I said it is a bit inelegant that
everything has to go through RGB even if you don't need it.  I have
some ideas here, but that will have to wait a bit.


from pylab import *

def mynorm(X):
    return X

class mycmap:
    name = "my gray"

    def __call__(self, X, alpha=None):
        # what is the fastest way to make an MxNx3 array simply
        # duplicating MxN on the last dimension?
        m,n = X.shape
        Z =  zeros((m,n,3), typecode=X.typecode())

        Z[...,0] = X
        Z[...,1] = X
        Z[...,2] = X                

        return Z

#norm = None     # default
norm = mynorm

#cmap = None      # default
cmap = mycmap()

ion()

rc('figure', figsize=(13,12))
X = rand(1600,1600)
figimage(X, cmap=cmap, norm=norm)


With the following numbers

# default cmap
peds-pc311:~/python/projects/matplotlib> time python ~/test.py --numarray -dGTKAgg
1.630u 0.430s 0:02.12 97.1%     0+0k 0+0io 4746pf+0w

# custom cmap
peds-pc311:~/python/projects/matplotlib> time python ~/test.py --numarray -dGTKAgg
1.080u 0.290s 0:01.42 96.4%     0+0k 0+0io 4745pf+0w

Of this 1.42 seconds, the big chunks are

  0.660 The time it takes to do from pylab import *; half of this is
        initializing the colormaps and dicts in cm.  We should do some
        work to get this number down.  But it is a fixed cost that
        doesn't scale with array size

  0.320 RandomArray2.py:97(random)  # creating the dummy data.  We can
        ignore this

  0.540 backend_gtkagg.py:33(draw)  # this creates the entire figure,
        of this, 0.420s is in the function FigureImage.make_image,
        most of which is in the extension code _image.fromarray

So ignoring for now the fixed cost of loading modules and creating
your arrays (Eg by running in pylab using the "run" function you only
pay the module loading costs once), the next place where a big win is
going to come from is in optimizing fromarray.

Note, the matplotlib figure canvas is a drawable widget.  If you want
bare metal speed and don't want any of the features matplotlib offers
(interpolation, normalization, colormapping, etc), you can always dump
your image directly to the gtk canvas, especially if you compile gtk
with image support.  This may be useful enough for us to consider a
backend figimage pipeline which would bypass a number of extra
copies.  Currently we have for gtkagg

 numarray -> agg image buffer -> figure canvas agg buffer -> gtk canvas

With some care, we might be able to simplify this, ideally doing for
the canvas

 numarray -> gtk canvas  # for screen

 numarray -> agg buffer  # for dumping to PNG


There are some subtleties to think through as to how this would work.
It would probably need to be a canvas method and not a renderer
method, for one thing, which would take a bit of redesign.

And then Midas would have to watch its back!

JDH

Hello,

I just downloaded Matplotlib 0.65 in Windows XP, Python 2.3. I just 
wanted to give 2 comments. First, when i try to use spy2() to plot a 
sparsity pattern, i get an error saying that the global name 'where' 
cannot be found. I can't remember if this has been mentioned in the past 
week. I simply solved this by adding 'where' on line 8 of axes.py:

from numerix import MLab, absolute, arange, array, asarray, ones, 
transpose, \
      log, log10, Float, ravel, zeros, Int32, Float64, ceil, min, indices, \
      shape, which, where

Sorry if this was a duplicate.

The second comment is about figimage(). Out of curiosity while reading 
the discussion entitles "Re: [Matplotlib-users] Is really matplotlib 
this slooow for displaying images ? (sorry again)" with Eric Emsellem, i 
have tried the sample script

from matplotlib.pylab import *
X = rand( 100, 100 )
figimage( X )
show()

The window that pops up as an axes box but not plot inside. Instead, a 
small image (representing X i assume) is located in the top left corner 
of the window. This image 'overlaps' with the axes and its lower right 
chunk is masked by the axes. If i rerun the same script using instead

X = rand( 1600, 1600 )

as in the discussion, the all the area _outside_ of the axes is filled 
with the image, and there is nothing inside the axes box. Presumably, 
the whole image area is filled and the axes box is added next, masking 
part of the image.

I have also tried to resize the image but the same is happening. Should 
i reset the origin of the image manually somewhere?

Thanks for a great package,
Dominique

Test done and this is correct, here are the sets of timing :
first line is without ''mynorm'', second is with ''mynorm''.
time python test.py --Numeric -dTkAgg
10.432u 1.663s 0:12.37 97.7%    0+0k 0+0io 0pf+0w
7.258u 1.302s 0:08.64 98.9%     0+0k 0+0io 0pf+0w

time python test.py --Numeric -dGTKAgg
5.209u 0.845s 0:06.10 99.0%     0+0k 0+0io 0pf+0w
4.226u 0.700s 0:04.98 98.7%     0+0k 0+0io 0pf+0w

time python test.py --numarray -dTkAgg
16.391u 1.036s 0:17.96 96.9%    0+0k 0+0io 0pf+0w
5.690u 0.829s 0:06.85 95.0%     0+0k 0+0io 0pf+0w

time python test.py --numarray -dGTKAgg
8.225u 0.546s 0:08.96 97.7%     0+0k 0+0io 0pf+0w
3.363u 0.445s 0:03.86 98.4%     0+0k 0+0io 0pf+0w

Another factor of 10 and you are faster than Midas.. :-)

Eric

John Hunter wrote:

>>>>>>"John" == John Hunter <jdh...@ac...> writes:
>>>>>>            
>>>>>>
>
>    John> I'll spend some time with the profiler looking for some low
>    John> hanging fruit.
>
>God bless the profiler.  It turns out over half of the time to display
>this image is spent in the normalizer, which takes image data in an
>arbitrary scale and maps into the unit interval
>
>  http://matplotlib.sourceforge.net/matplotlib.colors.html#normalize
>
>The normalizer handles a lot of special cases that you may not need.
>In fact, your data may already be normalized.  So you can write a
>custom normalizer
>
>from pylab import *
>
>def mynorm(X):  # do nothing, it's already normalized
>    return X
>
>ion()
>
>rc('figure', figsize=(13,12))
>X = rand(1600,1600)
>figimage(X, cmap=cm.hot, norm=mynorm)
>
>
>This change alone gives me more than a 2x speedup.  So with GTKAgg + a
>custom normalizer (in this case a do nothing normalizer) you'll be
>running 4-5 times faster than you were before, me thinks.
>
>peds-pc311:~/python/projects/matplotlib> time python ~/test.py --numarray
>1.650u 0.450s 0:02.13 98.5%     0+0k 0+0io 4746pf+0w
>
>I'll keep digging through the profiler...
>
>JDH
>
>  
>

-- 
===============================================================
Observatoire de Lyon                 ems...@ob...
9 av. Charles-Andre                      tel: +33 4 78 86 83 84
69561 Saint-Genis Laval Cedex            fax: +33 4 78 86 83 86
France               http://www-obs.univ-lyon1.fr/eric.emsellem
===============================================================

Hi,
thanks for the feedback.

To answer your questions:

- I have both Numeric and numarry but I am using
numarray in principle.

- Running in verbose mode, here is the output:

matplotlib data path /usr/share/matplotlib
loaded rc file /home/emsellem/.matplotlibrc
matplotlib version 0.65
verbose.level helpful
interactive is False
platform is linux2
numerix numarray 1.0
font search path ['/usr/share/matplotlib']
loaded ttfcache file /home/emsellem/.ttffont.cache
Could not load matplotlib icon: 'module' object has no attribute 
'window_set_default_icon_from_file'
backend GTKAgg version 2.0.0

- Then a VERY important note: yes indeed I am an astronomer
and I am used to have a fixed window (I first define its size
or use some default) and THEN ONLY load the image itself.
This is how I use ppgplot in python or Iraf, or Midas.
I indeed to not then use any interpolation scheme there.

- ALSO: there seems to be a bug in the figimage routine as
  it shows the image OUTSIDE an axis filled with white
(this may be because I am not doing the right thing though...)

- And finally here are the timing you asked for:
(there seem to be reasonable considering the difference
in CPU/RAM)

Hope this helps

Eric
==================================================
Before correcting figima
=========================
time python test.py --Numeric -dTkAgg
14.146u 2.363s 0:17.79 92.7%    0+0k 0+0io 10pf+0w

time python test.py --Numeric -dGTKAgg
9.795u 1.697s 0:13.63 84.2%     0+0k 0+0io 12pf+0w

time python test.py --numarray -dTkAgg
22.640u 1.443s 0:25.31 95.1%    0+0k 0+0io 13pf+0w

time python test.py --numarray -dGTKAgg
15.125u 0.925s 0:16.26 98.6%    0+0k 0+0io 0pf+0w

After correcting figima
=========================
time python test.py --Numeric -dTkAgg
10.432u 1.663s 0:12.37 97.7%    0+0k 0+0io 0pf+0w

time python test.py --Numeric -dGTKAgg
5.209u 0.845s 0:06.10 99.0%     0+0k 0+0io 0pf+0w

time python test.py --numarray -dTkAgg
16.391u 1.036s 0:17.96 96.9%    0+0k 0+0io 0pf+0w

time python test.py --numarray -dGTKAgg
8.225u 0.546s 0:08.96 97.7%     0+0k 0+0io 0pf+0w

-- 
===============================================================
Observatoire de Lyon                 ems...@ob...
9 av. Charles-Andre                      tel: +33 4 78 86 83 84
69561 Saint-Genis Laval Cedex            fax: +33 4 78 86 83 86
France               http://www-obs.univ-lyon1.fr/eric.emsellem
===============================================================

>>>>> "John" == John Hunter <jdh...@ac...> writes:

    John> I'll spend some time with the profiler looking for some low
    John> hanging fruit.

God bless the profiler.  It turns out over half of the time to display
this image is spent in the normalizer, which takes image data in an
arbitrary scale and maps into the unit interval

  http://matplotlib.sourceforge.net/matplotlib.colors.html#normalize

The normalizer handles a lot of special cases that you may not need.
In fact, your data may already be normalized.  So you can write a
custom normalizer

from pylab import *

def mynorm(X):  # do nothing, it's already normalized
    return X

ion()

rc('figure', figsize=(13,12))
X = rand(1600,1600)
figimage(X, cmap=cm.hot, norm=mynorm)


This change alone gives me more than a 2x speedup.  So with GTKAgg + a
custom normalizer (in this case a do nothing normalizer) you'll be
running 4-5 times faster than you were before, me thinks.

peds-pc311:~/python/projects/matplotlib> time python ~/test.py --numarray
1.650u 0.450s 0:02.13 98.5%     0+0k 0+0io 4746pf+0w

I'll keep digging through the profiler...

JDH

>>>>> "Eric" == Eric Emsellem <ems...@ob...> writes:

    Eric> Hi, I am using:

    Eric> IPython 0.6.6 with Python 2.3.3 and matplotlib-0.65

OK, too bad, one of the most common causes of slow behavior on earlier
versions of matplotlib was a numeric/numarray incompatibility setting
as discussed here - http://matplotlib.sourceforge.net/faq.html#SLOW.
But with 0.65, this shouldn't be a problem.  But just to be certain

  - do you have Numeric and numarray on your system?

  - what is the output of any test script when run with
    --verbose-helpful.  

It is important that you are not mixing numeric and numarray in your
code, because then matplotlib falls back on the python sequence
protocol.  So I just want to insure that you are using numarray
consistently and that matplotlib knows it.

    Eric> (checked that this is indeed the case)

    Eric> Eric P.S.: by the way, just to let you know (and I will pass
    Eric> the message on the forum) I am sincerely very impressed by
    Eric> matplotlib in general (in fact 5 people just switched to it
    Eric> in the last 2 weeks and our group only amounts to 10 people
    Eric> so 1/2 still to convince!). So this kind of ''negative''
    Eric> feedback/question should not undermine the rest of the soft
    Eric> for sure!!!

I passed it on already.  I suspect there are a number of others who
are interested in and can contribute to this discussion so I'll keep
this on list for a bit.  Thanks for the moral support :-)

A bit of background: as Arnd pointed out, imshow is probably doing a
lot more under the hood than you need, so I suggest we stick with
figimage for a while.  I assume you're an astronomer, since you
mentioned IRAF, no?  Perry Greenfield, also an astronomer, suggested
figimage because he mainly wanted a pixel dump of his data, with no
interpolation that imshow provides.

Do you need color mapping, or mainly grayscale? The reason I ask is
that for simplicity of coding I convert everything to an RGBA under
the hood, which is obviously inefficient in memory and CPU time.  At
some point (maybe now), we'll have to special case grayscale for
people who can't pay those extra costs.  I don't know if this is where
your bottleneck is.

So assuming figimage for now, I wrote a test script to get some
numbers for comparison.  Then I noticed I had introduced a bug in 0.65
in figimage when I wrongly added "hold" control to figimage, where it
doesn't below.  So replace figimage in pylab.py with the function I'm
including below before running any tests.

Since most people can't fit a 1600x1600 image on screen (I can't),
we'll need a big figure window at least to get most of it..  That's
what the figsize command is for.  FYI, my suspicion is we should be
able to get acceptable performance for 1600x1600 or 2000x2000 and
images of this magnitude, with some basic refactoring and
optimization.  I doubt we'll get 20k x 20k in the forseeable future,
in large part because the underlying image library from antigrain only
does 4096x4096.

With this script, I'm running in interactive mode ("ion") so the
figure will be created when the figimage call is made, and then will
immediately terminate rather than go into the tk mainloop since show
is called.


    from pylab import *
    ion()
    rc('figure', figsize=(13,12))
    X = rand(1600,1600)
    figimage(X, cmap=cm.hot)
    #show()


Here are my numbers.  Note I can get almost a 2x performance boost
switching to GTKAgg - there are known performance issues blitting a
large image to tkinter.  Is GTKAgg as possibility for you?

    peds-pc311:~> time python test.py --Numeric -dTkAgg
    5.750u 1.730s 0:08.20 91.2%     0+0k 0+0io 3286pf+0w

    peds-pc311:~> time python test.py --Numeric -dGTKAgg
    3.280u 0.840s 0:04.16 99.0%     0+0k 0+0io 4579pf+0w

    peds-pc311:~> time python test.py --numarray -dTkAgg
    8.830u 1.100s 0:09.96 99.6%     0+0k 0+0io 3455pf+0w

    peds-pc311:~> time python test.py --numarray -dGTKAgg
    4.730u 0.560s 0:05.36 98.6%     0+0k 0+0io 4747pf+0w

with a 3GHz P4 with 1GB of RAM.  How do the numbers for your system
compare?

I'll spend some time with the profiler looking for some low hanging
fruit.

JDH


# the modified figimage function
def figimage(*args, **kwargs):    
    # allow callers to override the hold state by passing hold=True|False

    try: ret =  gcf().figimage(*args, **kwargs)
    except ValueError, msg:
        msg = raise_msg_to_str(msg)
        error_msg(msg)
        hold(b)
        raise RuntimeError(msg)
    except RuntimeError, msg:
        msg = raise_msg_to_str(msg)
        error_msg(msg)
        hold(b)
        raise RuntimeError(msg)
    draw_if_interactive()
    gci._current = ret
    return ret
figimage.__doc__ = Figure.figimage.__doc__ + """
Addition kwargs: hold = [True|False] overrides default hold state"""