matplotlib-devel Mailing List for matplotlib (Page 506)

>>>>> "Nicholas" == Nicholas Young <N.P...@wa...> writes:

    Nicholas> On Wed, 2005-09-07 at 16:15 +0100, Nicholas Young wrote:
    >> I've attached a patch to CVS with the necessary changes below.
    >> There are some issues here:

    Nicholas> My patch contained memory leaks which I've fixed in the
    Nicholas> attachment - but I'm not that experienced in c/c++ so
    Nicholas> there could be more I haven't noticed.

    Nicholas> Nick

You might want to test with the following script

import os

def report_memory(i):
    pid = os.getpid()
    a2 = os.popen('ps -p %d -o rss,sz' % pid).readlines()
    print i, '  ', a2[1],
    return int(a2[1].split()[1])

for i in range(100):
    your_code_here()
    report_memory(i)


You should see little or no leak if everything checks out.

JDH

On Wed, 2005-09-07 at 16:15 +0100, Nicholas Young wrote:
> I've attached a patch to CVS with the necessary changes below.  There
> are some issues here:

My patch contained memory leaks which I've fixed in the attachment - but
I'm not that experienced in c/c++ so there could be more I haven't
noticed.

Nick

Hi,

I've recently come across a need to plot images for which I have
irregular sample points.  As far as I can see the way to do this in
current mpl CVS is either pcolor or contourf (which is sometimes much
faster).

I've implemented a third way with a subclass of AxisImage called
NonUniformImage which creates an axes image using a custom extension to
the PyCXX _image module.  The NonUniformImage class first turns all data
to a MxNx4 UInt8 on initialisation as a cache.  The make_image function
is replaced to call the extension code on each call with the x and y
axes, the RGBA image data, the size of the image to output and the view
limits. This code uses nearest neighbour interpolation to determine the
closest colour and create the output.  By putting the heavy calculations
into C++, by avoiding dealing with sample points that aren't rendered
and by only calculating the sample point to pixel map once per call this
code allows easy viewing and scrolling on fairly high resolution data.
On my laptop (1GB memory) 2.56 million points are handled fairly easily
(test script below:

I've attached a patch to CVS with the necessary changes below.  There
are some issues here:

- I'm not sure what the axes.Axes function to access this should be
called so I haven't made one.

- I'm not sure how to handle image boundaries; I currently have no
boundaries and just choose the nearest sample point - however far away
that is.

- To cope with large images the original array data is not stored and
thus cmap and norm cannot be changed once set_data has been called.

test code:

---

from pylab import *
from Numeric import NewAxis
from matplotlib.image import NonUniformImage

x = arange(-4, 4, 0.005)
y = arange(-4, 4, 0.005)
print 'Size %d points' % (len(x) * len(y))
z = sqrt(x[NewAxis,:]**2 + y[:,NewAxis]**2)

im = NonUniformImage(gca())
im.set_data(x, y, z)
gca().images.append(im)
show()

x2 = x**3

im = NonUniformImage(gca())
im.set_data(x2, y, z)
gca().images.append(im)
show()

---

Hope this is useful,
Nick

On Saturday 03 September 2005 8:47 pm, Eric Firing wrote:
> John,
>
> In the course of looking at the colorbar patch, I came up with two small
> changes for you to consider:
>
> 1) I added an rc option "tick.direction" which can be "in" (default) or
> "out".  With "in", there is no change from present behavior.  With
> "out", all ticks are outside the box, and tick labels are shifted
> accordingly.  I think outward ticks are particularly appropriate for
> things like filled contours and colorbars; inward ticks tend to be
> obscured.

Thank you for doing this, I had been making the same changes to my colorbars.

Darren

John,

In the course of looking at the colorbar patch, I came up with two small 
changes for you to consider:

1) I added an rc option "tick.direction" which can be "in" (default) or 
"out".  With "in", there is no change from present behavior.  With 
"out", all ticks are outside the box, and tick labels are shifted 
accordingly.  I think outward ticks are particularly appropriate for 
things like filled contours and colorbars; inward ticks tend to be obscured.

2) I shifted the axis drawing to a later stage, so that the axes are 
drawn on top, instead of getting overpainted by contourf.

There are also some cleanups:

1) "class ContourfMappable(ScalarMappable)" was not being used, so I 
deleted it.

2) I deleted a redundant initialization of self.images[] from axes.py.

3) Optionally, stripped out trailing blanks.  I will provide two diffs, 
one that ignores blanks, and one that does not, but otherwise the same. 
  I will send you the diffs offline so as not to clutter the list.

Now, regarding the colorbar change committed by Jeff: at present it has 
the minor disadvantage of breaking contourf if the latter is called with 
explicit colors instead of with a colormap.  I think I can fix this and 
improve readability by making some changes in contour.py and in the 
colorbar code.  That's next.

Eric

>>>>> "Robert" == Robert Kern <rk...@uc...> writes:

    Sascha> I am writing a web server app that creates charts among
    Sascha> other things. I am trying to get rid of the temporary file
    Sascha> that I use to transmit the figures created with matplotlib
    Sascha> to the actual web server. Although print_figure says "If
    Sascha> filename is a fileobject, write png to file object (thus
    Sascha> you can, for example, write the png to stdout)" I can't
    Sascha> successfully write anything to stdout. Anyone knows an
    Sascha> example or can give me some hint what I can do to get rid
    Sascha> of the tempfile?
    >>  Short answer: no known way to do this currently, though we'd
    >> like to figure it out.  As far as I know (and could very well
    >> be wrong) libpng requires a FILE*, which StringIO and cStringIO
    >> do not provide.

    Robert> StringIO isn't the issue here; being able to write to
    Robert> sys.stdout, which ought to have a valid FILE* underneath,
    Robert> is the issue.

Thanks for reminding me about this.  I thought it was possible to do
this, but had managed to forget

    import sys
    from pylab import plot, savefig, show
    plot([1,2,3])
    savefig(sys.stdout)
    show()

which I run with 

  > python test.py -dAgg > test.png

produces the expected figure.

JDH

Hi,

On Tue, 30 Aug 2005, Ken McIvor wrote:

> On Aug 30, 2005, at 10:45 AM, Arnd Baecker wrote:
> > I would have thought so as well. However I get:
> <snip>
> > A search for wxPython.h on
> > http://www.debian.org/distrib/packages#search_packages
> > gives no hits.
>
> Sorry, due to a lack of coffee this morning I misread "wxPython.h" as
> "wxWidgets.h" or something daft, yielding the kneejerk "install the
> -dev package" response.
>
> I wasn't aware of this situation, but it has the potential to be a big
> problem for me at work, where we primarily run Debian.  I have emailed
> Ron Lee, the wxgtk2.4 package maintainer, about the situation.  We'll
> see what he has to say on the matter.

Excellent - I was thinking about doing the same.

> > I agree. I wonder if anything improved with wx2.6? (for our
> > PlottingCanvas
> > we even dared to keep drawing DCs around, and it works without
> > problems...)
>
> What PlottingCanvas is this?  I'd be interested in seeing what
> optimizations you guys performed, if the source is available.

http://www.physik.tu-dresden.de/~baecker/python/plot.html

One of our main goals was to plot many points quickly,
in such a way that one appears
after another to get a "dynamic" appearance.
See http://www.physik.tu-dresden.de/~baecker/python/StandardMap.py
as an example.
(note that it still uses the old wx style...)

[...]

> > However, we are currently investigating to use
> > matplotlib for
> > a computational physics course (which will be next summer) and many of
> > the
> > students have *much* slower machines. So we need maximum speed but
> > with a
> > minimum of coding hassle (around 30% of the students have never
> > programmed
> > before ...).
>
> Just leveling the playing field between WXAgg and GtkAgg is exciting
> for me, because that means that future efforts at general optimization
> will net a bigger speed improvement for WXAgg.
>
> I'd imagine the plotting speed will be good enough for something along
> the lines of interactive plotting with iPython or visualizing results
> with pylab.

We have just finished the conversion of all exercices
of our course from scipy.xplt (aka pygist) to matplotlib.
Unfortunately, quite a few are prohibitively slow, even
on our fast machines. But we have some ideas
on possible improvements (both on the side of our code
and on the side of matplotlib) - this is going to be
separate thread though ;-)

> Speaking as a recent survivor of a computational physics
> class, I expect you to see a huge benefit from using Python as the
> language and matplotlib as the visualization, especially if you have
> students who have never programmed before.

I absolutely agree - we have been running our course now
for the third year, so far with scipy and scipy.xplt as plotting
programm. We had very positive feedback (of course,
those who never programmed before, had to work harder ;-).
Just in case: http://www.comp-phys.tu-dresden.de/cp2005/,
however the material (apart from the FAQ) is in German.

Best,

Arnd

On Tue, 2005-08-30 at 21:16 -0700, Robert Kern wrote:
> Why not use PIL where it's available? backend_agg2.py has a start at
> this. Here's a slightly more fleshed-out (but untested) implementation
> for backend_agg.py :
> 
>   if not is_string_like(filename):
>       try:
>           import Image
>           have_pil = True
>       except ImportError:
>           have_pil = False
>       if have_pil:
>           img = Image.frombuffer('RGBA', (self.width, self.height),
>                                  self.buffer_rgba(0, 0))
>           img.write(filename)
>       else:
>           self.renderer._renderer.write_png(filename)

I've previously had a problem with the origin when doing something
similar with buffer_rgba (resulting in a vertical flip).  Doing:

            im = Image.frombuffer('RGBA', (self.width, self.height),
                                  self.buffer_rgba(0, 0),
                                  'raw', 'RGBA', 0, 1)

fixes that problem.

If a method utilising PIL images in this manner is added I'd suggest
adding a to_pil (or similar) method for those who want to process the
resulting image in some manner requiring PIL.

Nick

On Tue, 30 Aug 2005, Ken McIvor wrote:

[...]

> 	# assuming z0 is the initial MxN array that is being decayed...
> 	z_min = min(nx.minimum.reduce(z0))
> 	z_max = max(nx.maximum.reduce(z0))
> 	image = axes.imshow(z0, vmin=z_min, vmax=z_max)
>
> As an aside: I'd love to hear if anyone knows of a nicer way to get
> Numeric to give you the minimum value of a matrix.

What about:
   z_min=min(ravel(z0))

In [3]:Numeric.ravel?
    ravel(m) returns a 1d array corresponding to all the elements of it's
    argument.

OTOH, I am not sure, if the usage of min (which is a python builtin,
operating on sequences) does not cost some performance.
So maybe

  z_min = nx.minimum.reduce(nx.ravel(z0))

is more efficient?

Best,

Arnd

Thanks! Great catch. Sadly, I have been bitten by this before, but had 
completely forgotten about it. I've switched to just doing vmin=0, 
vmax=1, as I know the range before hand.

Abe

Ken McIvor wrote:

> On Aug 30, 2005, at 10:29 PM, John Hunter wrote:
>
>> I suggest you follow pick through set_data and make_image in the image
>> module to see if you can sort out what is going wrong.  Alternatively,
>> if you post an example then I can take a look.
>
>
> I've done Abraham's work for him this time, and attached an example 
> which demonstrates behavior that he *may* be seeing.
>
> The problem is that the minimum and maximum values of the image are 
> determined every time the image is asked to draw itself (in 
> matplotlib.image.AxesImage.__draw()).  I assume that what's happening 
> is that the colors aren't fading down the color ramp as the values 
> decay, but are rather staying constant or blurring a little.
>
> This happens because the color ramp is being applied across the 
> current minimum and maximum of the data, rather than across some 
> absolute scale.  The solution to this (see the attached script for the 
> example) is to specify a vmin and vmax, which will pin the top and 
> bottom of the color ramp to those values:
>
>     # assuming z0 is the initial MxN array that is being decayed...
>     z_min = min(nx.minimum.reduce(z0))
>     z_max = max(nx.maximum.reduce(z0))
>     image = axes.imshow(z0, vmin=z_min, vmax=z_max)
>
> As an aside: I'd love to hear if anyone knows of a nicer way to get 
> Numeric to give you the minimum value of a matrix.
>
> Ken

John Hunter wrote:
>>>>>>"Sascha" == Sascha  <sas...@gm...> writes:
> 
> 
>     Sascha> I am writing a web server app that creates charts among
>     Sascha> other things. I am trying to get rid of the temporary file
>     Sascha> that I use to transmit the figures created with matplotlib
>     Sascha> to the actual web server. Although print_figure says "If
>     Sascha> filename is a fileobject, write png to file object (thus
>     Sascha> you can, for example, write the png to stdout)" I can't
>     Sascha> successfully write anything to stdout. Anyone knows an
>     Sascha> example or can give me some hint what I can do to get rid
>     Sascha> of the tempfile?
> 
> Short answer: no known way to do this currently, though we'd like to
> figure it out.  As far as I know (and could very well be wrong)
> libpng requires a FILE*, which StringIO and cStringIO do not provide.

StringIO isn't the issue here; being able to write to sys.stdout, which
ought to have a valid FILE* underneath, is the issue.

Why not use PIL where it's available? backend_agg2.py has a start at
this. Here's a slightly more fleshed-out (but untested) implementation
for backend_agg.py :

  if not is_string_like(filename):
      try:
          import Image
          have_pil = True
      except ImportError:
          have_pil = False
      if have_pil:
          img = Image.frombuffer('RGBA', (self.width, self.height),
                                 self.buffer_rgba(0, 0))
          img.write(filename)
      else:
          self.renderer._renderer.write_png(filename)


-- 
Robert Kern
rk...@uc...

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

On Aug 30, 2005, at 10:29 PM, John Hunter wrote:
> I suggest you follow pick through set_data and make_image in the image
> module to see if you can sort out what is going wrong.  Alternatively,
> if you post an example then I can take a look.

I've done Abraham's work for him this time, and attached an example 
which demonstrates behavior that he *may* be seeing.

The problem is that the minimum and maximum values of the image are 
determined every time the image is asked to draw itself (in 
matplotlib.image.AxesImage.__draw()).  I assume that what's happening 
is that the colors aren't fading down the color ramp as the values 
decay, but are rather staying constant or blurring a little.

This happens because the color ramp is being applied across the current 
minimum and maximum of the data, rather than across some absolute 
scale.  The solution to this (see the attached script for the example) 
is to specify a vmin and vmax, which will pin the top and bottom of the 
color ramp to those values:

	# assuming z0 is the initial MxN array that is being decayed...
	z_min = min(nx.minimum.reduce(z0))
	z_max = max(nx.maximum.reduce(z0))
	image = axes.imshow(z0, vmin=z_min, vmax=z_max)

As an aside: I'd love to hear if anyone knows of a nicer way to get 
Numeric to give you the minimum value of a matrix.

Ken

>>>>> "Abraham" == Abraham Schneider <ab...@cn...> writes:

    Abraham> I haven't had time to do a full search yet, but was
    Abraham> hoping someone might at least have an idea of where to
    Abraham> begin to look. My current suspicions is that there might
    Abraham> be a caching issue.

It could be -- the image module does do some caching.  But im.set_data
*does* clear the cache with 

        self._imcache =None

I suggest you follow pick through set_data and make_image in the image
module to see if you can sort out what is going wrong.  Alternatively,
if you post an example then I can take a look.

JDH

Sorry, realized I was a little vague. When I say, 'choppy', I mean that 
some cells won't get updated. If there is a lot of activity everything 
looks like, but once activity dies down, some elements stay on in 
various stages (somewhat like bad pixels).

Abraham Schneider wrote:

> Hi. Sorry if this was covered already, but I ran into a recent problem 
> with animating an image using GTKAgg (and matplotlib 0.83.2). In my 
> update_fig() function, I do:
>
> im.set_data(g)
> manager.canvas.draw()
>
> where g is some arbritrary 20x20 array. Over time it's elements will 
> be set to 1, and then decay exponentially. For the majority of the 
> time the array contains all 0s. Under these conditions, it gets very 
> choppy, and skips a large number of frames.
>
> If I do something like: g[0, 0] = 1, then the animation runs smoothly, 
> and looks like it's supposed to (except I now have one cell in the 
> grid set the an incorrect value.
>
> I haven't had time to do a full search yet, but was hoping someone 
> might at least have an idea of where to begin to look. My current 
> suspicions is that there might be a caching issue.
>
> Thanks,
>
> Abe
>
>
> -------------------------------------------------------
> SF.Net email is Sponsored by the Better Software Conference & EXPO
> September 19-22, 2005 * San Francisco, CA * Development Lifecycle 
> Practices
> Agile & Plan-Driven Development * Managing Projects & Teams * Testing 
> & QA
> Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Hi. Sorry if this was covered already, but I ran into a recent problem 
with animating an image using GTKAgg (and matplotlib 0.83.2). In my 
update_fig() function, I do:

im.set_data(g)
manager.canvas.draw()

where g is some arbritrary 20x20 array. Over time it's elements will be 
set to 1, and then decay exponentially. For the majority of the time the 
array contains all 0s. Under these conditions, it gets very choppy, and 
skips a large number of frames.

If I do something like: g[0, 0] = 1, then the animation runs smoothly, 
and looks like it's supposed to (except I now have one cell in the grid 
set the an incorrect value.

I haven't had time to do a full search yet, but was hoping someone might 
at least have an idea of where to begin to look. My current suspicions 
is that there might be a caching issue.

Thanks,

Abe

On Aug 30, 2005, at 10:45 AM, Arnd Baecker wrote:
> I would have thought so as well. However I get:
<snip>
> A search for wxPython.h on
> http://www.debian.org/distrib/packages#search_packages
> gives no hits.

Sorry, due to a lack of coffee this morning I misread "wxPython.h" as 
"wxWidgets.h" or something daft, yielding the kneejerk "install the 
-dev package" response.

I wasn't aware of this situation, but it has the potential to be a big 
problem for me at work, where we primarily run Debian.  I have emailed 
Ron Lee, the wxgtk2.4 package maintainer, about the situation.  We'll 
see what he has to say on the matter.

> I agree. I wonder if anything improved with wx2.6? (for our 
> PlottingCanvas
> we even dared to keep drawing DCs around, and it works without 
> problems...)

What PlottingCanvas is this?  I'd be interested in seeing what 
optimizations you guys performed, if the source is available.

>> However, I think there are a couple pieces of lower-handing fruit, 
>> like not
>> making an extra source copy when blitting (_wxagg.cpp:159-190), that 
>> will
>> offer speedups.
>
> That would be cool (I have to admit, that I am not too worried when  
> FPS>100
> on my machine. However, we are currently investigating to use 
> matplotlib for
> a computational physics course (which will be next summer) and many of 
> the
> students have *much* slower machines. So we need maximum speed but 
> with a
> minimum of coding hassle (around 30% of the students have never 
> programmed
> before ...).

Just leveling the playing field between WXAgg and GtkAgg is exciting 
for me, because that means that future efforts at general optimization 
will net a bigger speed improvement for WXAgg.

I'd imagine the plotting speed will be good enough for something along 
the lines of interactive plotting with iPython or visualizing results 
with pylab.  Speaking as a recent survivor of a computational physics 
class, I expect you to see a huge benefit from using Python as the 
language and matplotlib as the visualization, especially if you have 
students who have never programmed before.

Ken

Ken, all,

I'd tried this patch out on Mac OS X, with standard Mac-based python
and wxPython 2.6. I had to explicitly set the environmental variable
WX_CONFIG, but after that it built fine and runs fine.

For my scripts on a slow G4 Powerbook, I see a small improvement in
speed compared with mpl 0.81.  I'll be eager to try this on Windows.

Thanks Ken,

--Matt Newville <newville at cars.uchicago.edu>

>>>>> "Arnd" == Arnd Baecker <arn...@we...> writes:

    >> It does have wxPython.h, it's just not in the libwxgtk2.4
    >> package.  You'll need to install the libwxgtk2.4-dev package as
    >> well to get the headers.

    Arnd> I would have thought so as well. However I get:
    ....
    Arnd> A search for wxPython.h on
    Arnd> http://www.debian.org/distrib/packages#search_packages gives
    Arnd> no hits.

I found this too -- it may be rightly considered a bug in the dev
package.  I checked out the src package and then copied over the
headers.  The following worked for me


   apt-get source libwxgtk2.5.3
   sudo cp -a wxwidgets2.5-2.5.3.2ubuntu4/include/wx-2.5/wx/wxPython /usr/include/wx-2.5/wx/

JDH

Hi,

On Tue, 30 Aug 2005, Ken McIvor wrote:
> On Aug 30, 2005, at 9:31 AM, Arnd Baecker wrote:
[...]

> > a) debian problem:
> >    Very interestingly debian stable (sarge)
> >    does not have wxPython.h
>
> It does have wxPython.h, it's just not in the libwxgtk2.4 package.
> You'll need to install the libwxgtk2.4-dev package as well to get the
> headers.

I would have thought so as well. However I get:

dpkg -L libwxgtk2.4-dev
/.
/usr
/usr/bin
/usr/bin/wxgtk-2.4-config
/usr/lib
/usr/lib/wx
/usr/lib/wx/include
/usr/lib/wx/include/univ
/usr/lib/wx/include/gtk-2.4
/usr/lib/wx/include/gtk-2.4/wx
/usr/lib/wx/include/gtk-2.4/wx/setup.h
/usr/lib/libwx_gtk-2.4.a
/usr/lib/libwx_gtk_gl-2.4.a
/usr/share
/usr/share/doc
/usr/share/doc/libwxgtk2.4-dev
/usr/share/doc/libwxgtk2.4-dev/copyright
/usr/share/doc/libwxgtk2.4-dev/changelog.gz
/usr/share/man
/usr/share/man/man1
/usr/lib/libwx_gtk-2.4.so
/usr/lib/libwx_gtk_gl-2.4.so
/usr/share/man/man1/wxgtk-2.4-config.1.gz

A search for wxPython.h on
http://www.debian.org/distrib/packages#search_packages
gives no hits.


> >   GTKAgg               FPS: 131.897223221
> >   TkAgg                FPS:  65.5153012595
> >   WXAgg                FPS: 100.55
> >      (200 frames: 1.99 seconds,   200 blits:  0.95 seconds)
> >      (FPS: 100.55                 BPS: 211.00)
> >   WXAgg --no-accel     FPS: 53.21
> >      (200 frames: 3.76 seconds,   200 blits:  2.55 seconds
> >      (FPS: 53.21                  BPS: 78.40)
> >
> >
> > I am running debian sarge, on a PIV, 2.8 GHz and an nVdia Quadro4 NVS
> > graphics card (@1600x1200).
> > So WXAgg is almost on par with GTGAgg - great!!
>
> That's very exciting news.  I hadn't had a chance to test and profile
> the accelerator under Debian, nor had I bothered profiling it against
> GtkAgg and TkAgg under OSX.  I should probably get those backends up
> and running (darn macs) and do so.
>
> > Ken, in _wxagg.cpp you mention that you planned to write a
> > agg_to_wx_bitmap()
> > which draws directly to a bitmap, but that this might not speed up
> > things
> > much because of: AGG->wx.Image->wx.Bitmap before you can blit using a
> > MemoryDC.
>
> Yes, I am concerned that re-implementing that part of the blit into C++
> won't speed things up enough.  I can understand doing the whole
> Image/Bitmap thing for portability (especially when wxWindows got
> started in 1992), but it's rather obnoxious when all it does is make
> extra copies of RGB data.

I agree. I wonder if anything improved with wx2.6?
(for our PlottingCanvas we even dared to keep
drawing DCs around, and it works without problems...)

> However, I think there are a couple pieces
> of lower-handing fruit, like not making an extra source copy when
> blitting (_wxagg.cpp:159-190), that will offer speedups.

That would be cool
(I have to admit, that I am not too worried when  FPS>100
on my machine. However, we are currently investigating
to use matplotlib for a computational physics course
(which will be next summer) and many of the students
have *much* slower machines. So we need maximum speed
but with a minimum of coding hassle (around 30% of the students
have never programmed before ...).

> Unfortunately, they'd be portable across Agg backends, so GtkAgg would
> get faster too.  ;-)

I would not mind too much ;-).

> > Maybe you can contact Chris Barker if he has an idea on this (or are
> > there any
> > solutions available in the context of wxart2d or chaco? - quick
> > googling did not
> > reveal anything ...)
>
> I'll do so, if he doesn't see this part of the message than chime in.
> I have come across some old wxWindows mailing list exchanges indicating
> that there's no way to avoid the Image->Bitmap conversion as a
> requirement for drawing RGB images, so I'm not planning to loose too
> much sleep over it.  I hadn't heard about wxart2d before, but will look
> into it to see how their agg canvas is implemented.  I'll see if I can
> find the chaco repository and do the same there.
>
> Thanks for sharing your anim.py profiling results.  It's heartening to
> see WXAgg's full-redraw speed being limited by the rest of matplotlib,
> rather than the backend.

Indeed this is good news!! Many thanks,

Arnd

On Aug 30, 2005, at 9:31 AM, Arnd Baecker wrote:
> Many thanks for the update of the patch.
> It turned out that I could not apply it to this mornings CVS checkout
> because John's time machine was already active  and he has put your
> patch into CVS ;-)

Really?  Yay for me!  ;-)

> During installation I encountered two glitches:
>
> a) debian problem:
>    Very interestingly debian stable (sarge)
>    does not have wxPython.h

It does have wxPython.h, it's just not in the libwxgtk2.4 package.  
You'll need to install the libwxgtk2.4-dev package as well to get the 
headers.

>   GTKAgg               FPS: 131.897223221
>   TkAgg                FPS:  65.5153012595
>   WXAgg                FPS: 100.55
>      (200 frames: 1.99 seconds,   200 blits:  0.95 seconds)
>      (FPS: 100.55                 BPS: 211.00)
>   WXAgg --no-accel     FPS: 53.21
>      (200 frames: 3.76 seconds,   200 blits:  2.55 seconds
>      (FPS: 53.21                  BPS: 78.40)
>
>
> I am running debian sarge, on a PIV, 2.8 GHz and an nVdia Quadro4 NVS
> graphics card (@1600x1200).
> So WXAgg is almost on par with GTGAgg - great!!

That's very exciting news.  I hadn't had a chance to test and profile 
the accelerator under Debian, nor had I bothered profiling it against 
GtkAgg and TkAgg under OSX.  I should probably get those backends up 
and running (darn macs) and do so.

> Ken, in _wxagg.cpp you mention that you planned to write a 
> agg_to_wx_bitmap()
> which draws directly to a bitmap, but that this might not speed up 
> things
> much because of: AGG->wx.Image->wx.Bitmap before you can blit using a 
> MemoryDC.

Yes, I am concerned that re-implementing that part of the blit into C++ 
won't speed things up enough.  I can understand doing the whole 
Image/Bitmap thing for portability (especially when wxWindows got 
started in 1992), but it's rather obnoxious when all it does is make 
extra copies of RGB data.  However, I think there are a couple pieces 
of lower-handing fruit, like not making an extra source copy when 
blitting (_wxagg.cpp:159-190), that will offer speedups.  
Unfortunately, they'd be portable across Agg backends, so GtkAgg would 
get faster too.  ;-)

> Maybe you can contact Chris Barker if he has an idea on this (or are 
> there any
> solutions available in the context of wxart2d or chaco? - quick 
> googling did not
> reveal anything ...)

I'll do so, if he doesn't see this part of the message than chime in.  
I have come across some old wxWindows mailing list exchanges indicating 
that there's no way to avoid the Image->Bitmap conversion as a 
requirement for drawing RGB images, so I'm not planning to loose too 
much sleep over it.  I hadn't heard about wxart2d before, but will look 
into it to see how their agg canvas is implemented.  I'll see if I can 
find the chaco repository and do the same there.

Thanks for sharing your anim.py profiling results.  It's heartening to 
see WXAgg's full-redraw speed being limited by the rest of matplotlib, 
rather than the backend.

Ken

Hi Ken,

On Mon, 29 Aug 2005, Ken McIvor wrote:

[...]

> I will resubmit the wx-anim.tar.gz patch to sourceforge with this
> correction.

Many thanks for the update of the patch.
It turned out that I could not apply it to this mornings CVS checkout
because John's time machine was already active  and he has put your
patch into CVS ;-)

During installation I encountered two glitches:

a) debian problem:
   Very interestingly debian stable (sarge)
   does not have wxPython.h
   As a work-around I downloaded the corresponding source from debian
    wget
http://ftp.debian.org/debian/pool/main/w/wxwindows2.4/wxwindows2.4_2.4.3.1.tar.gz
   and copied all header files wxwindows2.4-2.4.3.1/wxPython/src/*.h
   to a (newly created) /usr/include/wx/wxPython.

   Then the installation went fine.

b) CVS problem:
   Calling python animation_blit_wx.py from the examples directory
   gives the error

  lib/python2.3/site-packages/matplotlib/__init__.py:788: UserWarning:
could not find rc file; returning defaults
    warnings.warn(message)
  Traceback (most recent call last):
    File "animation_blit_wx.py", line 10, in ?
      import pylab as p
    File "/scratch/abaecker/SOFT//lib/python2.3/site-packages/pylab.py",
line 1, in ?
      from matplotlib.pylab import *
    File
"/scratch/abaecker/SOFT//lib/python2.3/site-packages/matplotlib/pylab.py",
line 194, in ?
      import cm
    File
"/scratch/abaecker/SOFT//lib/python2.3/site-packages/matplotlib/cm.py",
line 372, in ?
      atad['Blues']=_Blues_data
  NameError: name '_Blues_data' is not defined


  Commenting out the corresponding block in matplotlib/cm.py
  everything works fine again
  (John: I think this is a leftover from rev. 1.19 for cm.py "removed
extra cmaps until license is resolved")

OK, now to the new WXAgg (and animation in general):


Running:
  python animation_blit.py
  python animation_blit_tk.py
  python animation_blit_wx.py
  python animation_blit_wx.py --no-accel

gives:

  GTKAgg               FPS: 131.897223221
  TkAgg                FPS:  65.5153012595
  WXAgg                FPS: 100.55
     (200 frames: 1.99 seconds,   200 blits:  0.95 seconds)
     (FPS: 100.55                 BPS: 211.00)
  WXAgg --no-accel     FPS: 53.21
     (200 frames: 3.76 seconds,   200 blits:  2.55 seconds
     (FPS: 53.21                  BPS: 78.40)


I am running debian sarge, on a PIV, 2.8 GHz and an nVdia Quadro4 NVS
graphics card (@1600x1200).
So WXAgg is almost on par with GTGAgg - great!!

Ken, in _wxagg.cpp you mention that
you planned to write a agg_to_wx_bitmap() which
draws directly to a bitmap, but that
this might not speed up things much because
of: AGG->wx.Image->wx.Bitmap before you can blit using a MemoryDC.
Maybe you can contact Chris Barker if he has an idea on this
(or are there any solutions available in the context of wxart2d or
chaco? - quick googling did not reveal anything ...)

Anyway, the speed provided by the animate option
is a fantastic improvement - many thanks John and Ken!!

Just for comparison:

for backend in `echo "Tk TkAgg GTK GTKAgg WX WXAgg"`
do
   echo $backend

   python anim.py -d${backend}
done

Tk     FPS: 35.5331434162
TkAgg  FPS: 20.3487964649
GTK    FPS: 35.528699289
GTKAgg FPS: 26.9074415174
WX     FPS: 13.7550465745
WXAgg  FPS: 26.0163513384

Many thanks,

Arnd

>>>>> "Vinj" == Vinj Vinj <vin...@ya...> writes:

    Vinj> The following code (is a simple modification of
    Vinj> finance_demo.py) produces a stock chart. SOme of the lines
    Vinj> are thicker than the others. All the lines should have the
    Vinj> same width.

This is a little bug that crops up frequently because of how agg
handles subpixel rendering.  I added a little hack to the line
collection code to "snap to pixel center" for len(2) lines.  Should
fix your problem.

Use CVS revision

Checking in src/_backend_agg.cpp;
/cvsroot/matplotlib/matplotlib/src/_backend_agg.cpp,v  <--  _backend_agg.cpp
new revision: 1.89; previous revision: 1.88

or later.

JDH

I may be coming in late to this, but I would suggest:

1. Make sure you are up to date with cvs
2. cvs diff -c -3 > my-mpl-changes.diff

I did the tkagg animation and I would be willing to test your wx changes and 
commit barring any problems.

- Charlie

Ken McIvor wrote:
> On Aug 29, 2005, at 12:03 PM, Arnd Baecker wrote:
> 
>> Also I think according to xvsdiff it
>> should not be __init__.py in matplotlib but the one
>> matplotlib/lib/matplotlib/ which should be patched.
>> So I must be doing something wrong here.
> 
> 
> I doubt you're doing anything wrong here.  I probably don't know how to 
> generate correct CVS diffs for patching, and didn't think to validate my 
> patch by applying it myself.
> 
> Any advice on submitting diffs against CVS, mailing list?
> 
> Ken
> 
> 
> 
> -------------------------------------------------------
> SF.Net email is Sponsored by the Better Software Conference & EXPO
> September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
> Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
> Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

On Aug 29, 2005, at 12:03 PM, Arnd Baecker wrote:
> Also I think according to xvsdiff it
> should not be __init__.py in matplotlib but the one
> matplotlib/lib/matplotlib/ which should be patched.
> So I must be doing something wrong here.

I doubt you're doing anything wrong here.  I probably don't know how to 
generate correct CVS diffs for patching, and didn't think to validate 
my patch by applying it myself.

Any advice on submitting diffs against CVS, mailing list?

Ken

The following code (is a simple modification of
finance_demo.py) produces a stock chart. SOme of the
lines are thicker than the others. All the lines
should have the same width.

VJ



import datetime
from pylab import *
from matplotlib.finance import
quotes_historical_yahoo, plot_day_summary2

import time


axesBG = '#f6f6f6'
figBG = 'w'
date1 = datetime.date( 2002, 2, 1)
date2 = datetime.date( 2004, 12, 12 )

quotes = quotes_historical_yahoo(
    'INTC', date1, date2)

quotetime, open, close, high, low, volume =
zip(*quotes)
numPriceBars = 261


ax = subplot(111)
plot_day_summary2(ax,
open[0:260],close[0:260],high[0:260],low[0:260])
set(ax.get_xticklabels(), 'rotation', 45,
'horizontalalignment', 'right', fontsize=7)

show()