matplotlib-devel Mailing List for matplotlib (Page 528)

Steve Chaplin wrote:

> I created a module that allows interactive matplotlib access from a
> python shell, its a modified version of
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65109
> 
> This method seems to work for me (on Linux) and is not very intrusive to
> matplotlib code - it just requires that 'gui_cmd()' is used to execute
> all gui commands when in interactive mode.
> 
> Any comments?

In case anyone is interested, here's a lightweight, standalone (non-ipython) 
GTK threaded interactive interpreter with matplotlib support.  This code is 
also based on the same ASPN recipe, it's basically the prototype I used to 
understand that recipe when I was adding the matplotlib support to IPython.

The threading tricks are the gold in that recipe, but the rest of the code is 
rather messy.  I pretty much rewrote it when studying it (while keeping the 
threading core).  Perhaps this may be of use to others.  The ASPN recipe 
duplicates a lot of the functionality in the code.py module from the standard 
library, which is completely unnecessary (it's guaranteed to be there in any 
python installation).  By using code.py, the whole thing can be rewritten much 
more cleanly (and you benefit from a good implementation in the stdlib for all 
the interpreter emulation parts).

I think I sent this to John a while back so he could replace the 
interactive.py shipped with matplotlib.  I fully acknowledge in the header the 
ASPN recipe as the origin of the threading solution.

The pylab support in current ipython is basically this same idea, but 
generalized to work with either GTK or WX, and blended into the IPython code 
structure (that wasn't too bad, since ipython itself is based on code.py).

Cheers,

f

Regarding Baptiste's patch
- I'm not too keen on the idea of wrapping gtk.gdk.threads_enter() /
gtk.gdk.threads_leave() around every GDK function call.
- I share Fernando's concern with possible problems when adding new
widgets or embedding a matplotlib window into something else.

My understanding is that wrapping gtk.gdk.threads_enter() /
gtk.gdk.threads_leave() around GDK functions is not necessary if all the
GDK calls occur in the same thread. So I think a simpler solution is to
arrange for all gui calls to go to a single thread and do away with
enter()/leave().

I created a module that allows interactive matplotlib access from a
python shell, its a modified version of
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65109

This method seems to work for me (on Linux) and is not very intrusive to
matplotlib code - it just requires that 'gui_cmd()' is used to execute
all gui commands when in interactive mode.

Any comments?

Regards
Steve

Hi John and Fernando,

John, here is the quick answer to your questions:

* the patch does almost nothing (a few calls to a dummy function) except 
when the gui_protect rcParam is enabled

* when enabled the overhead is acceptable (on my Pentium III 500MHz), 
but import time of pylab is a little longer, because most of the work 
happens there. Also, only the functions and methods defined by 
matplotlib are protected. The idea is that if you call functions or 
methods from the gtk module, you ought to know what you're doing anyway 
(for that matter, I am not as ambitious as gui_thread).

* the patch currently doesn't work with wx because I've not finished the 
work, but it is doable. The GTK or WX mainloop is launched at the time 
the backend is imported, and only the one for the backend you are using 
will be launched.

* if you use interactive.py, or the code in IPython, you'd better not 
enable gui_protect. If you do, 2 GTK mainloops will be launched. This is 
allowed in GTK, but doesn't make sense.

* the GTK mainloop (probably same for WX) can run concurrently with tk. 
This means no problem plotting GTK from IDLE. You can even switch 
backends at runtime by reloading matplotlib.backends and matplotlib.pylab

* the GTK and WX mainloops cannot run concurrently. This is because, at 
least on Linux, WX itself launches a GTK mainloop (it's called WxGTK for 
a reason :). It might be possible to use this very mainloop to also 
display GTK figures, but I've had no luck with this until now.

Fernando, I've been looking again at gui_thread. I now understand the 
beauty and power of it, but it is also more brittle, which means a lot 
of work and testing. It might prove the best solution in the end, though.
I took no offense at all about IPython, so neither should you :-)
I just mentioned it because it is the advised solution in matplotlib doc.

Have a nice day,
BC

PS: You don't have to CC me, I'm subscribed to the list.

John Hunter wrote:
>>>>>>"Baptiste" == Baptiste Carvello <bap...@al...> writes:
> 
> 
>     Baptiste> I know the patch is intrusive, though maybe not as much
>     Baptiste> as it looks (everything except protect_mutex.py is
>     Baptiste> pretty trivial). I didn't expected it to be accepted
>     Baptiste> right away, I see it more as a contribution to the whole
>     Baptiste> gui thread discution.
> 
>     Baptiste> As for gui_thread, I looked at it last year, I just
>     Baptiste> remember I didn't like that code that much. Maybe I
>     Baptiste> should get another look...
> 
> Most of what I've heard of gui thread (caveat, the situation may be
> different now but this was the last I heard) was that the code was
> hairy, required scipy distutils to build, was wx specific and had
> problems that required significant work to fix.  In that context, a
> patch, even an intrusive one, that allowed gtk* and wx* to work from
> an arbitrary python shell would be most welcome.  While I think
> ipython is the one true shell, there are many other shells in
> existence and that won't change.  If for example, this patch allowed
> gtk* to be used from within IDLE or pycrust in addition to the
> standard python shell, that would be a big win.

I certainly have never been a fan of digging too deep into gui_thread myself, 
I admit that much :)  Last time I looked at the code, I got a bit scared.  On 
the other hand, Prabhu recently did a bunch of work on it, so it may be 
cleaner/better, I just don't know.

And as I said, I really was not speaking from the ipython side of things at 
all, I don't know why Baptiste took it that way (I didn't even mention ipython 
in my post).  My opinion was strictly based on the generic approach offered by 
gui_thread, and the potential benefit of doing things that way.

> The problem we need to solve is substantially easier than the one
> gui_thread wants to solve, if I understand the issues correctly.  gui
> thread enables you to interact with general wx apps from a general
> python shell.  We want gtk* and wx* backends to work from a general
> python shell.  If this patch addresses that, it's worth a close look.

One thing that I like about gui_thread is that it works in a generic manner. 
What happens with this patch if someone uses matplotlib but adds new widgets? 
Or if they embed a matplotlib window into something else? Are such usage cases 
covered automatically, or do they require special handling by third-party code?

There certainly are benefits to a matplotlib-based self-contained solution, 
and gui_thread comes with its own baggage, so I'll let you guys judge where 
the balance rests.

Cheers,

f

>>>>> "Fernando" == Fernando Perez <Fer...@co...> writes:

    Fernando> Just a minor comment: insert(0) is an expensive (O(N))
    Fernando> operation on lists. This kind of problem might be better
    Fernando> addressed with a deque
    Fernando> (http://www.python.org/dev/doc/devel/lib/module-collections.html,
    Fernando> could be backported to py2.3).

If I read the patch correctly, usage_stack is being used as a simple
counter.  

I think the data structure we want is a class derived from dict that
pops elements off (preferably intelligently, ie pops the ones which
haven't been accessed in a while) when the dict length gets too long. 
In other words, we don't need to structures (cached and usage_stack)
to do the work of one

There must be such a beast in the cookbook or elsewhere, a cache
object which works like a dict but has a max size...

JDH 

>>>>> "Baptiste" == Baptiste Carvello <bap...@al...> writes:

    Baptiste> I know the patch is intrusive, though maybe not as much
    Baptiste> as it looks (everything except protect_mutex.py is
    Baptiste> pretty trivial). I didn't expected it to be accepted
    Baptiste> right away, I see it more as a contribution to the whole
    Baptiste> gui thread discution.

    Baptiste> As for gui_thread, I looked at it last year, I just
    Baptiste> remember I didn't like that code that much. Maybe I
    Baptiste> should get another look...

Most of what I've heard of gui thread (caveat, the situation may be
different now but this was the last I heard) was that the code was
hairy, required scipy distutils to build, was wx specific and had
problems that required significant work to fix.  In that context, a
patch, even an intrusive one, that allowed gtk* and wx* to work from
an arbitrary python shell would be most welcome.  While I think
ipython is the one true shell, there are many other shells in
existence and that won't change.  If for example, this patch allowed
gtk* to be used from within IDLE or pycrust in addition to the
standard python shell, that would be a big win.

The problem we need to solve is substantially easier than the one
gui_thread wants to solve, if I understand the issues correctly.  gui
thread enables you to interact with general wx apps from a general
python shell.  We want gtk* and wx* backends to work from a general
python shell.  If this patch addresses that, it's worth a close look.
Baptiste, have you have a chance to test with other shells?  Would
your patch break gtk* or wx* under ipython?  Does it work in IDLE
and/or pycrust?  Do you have any ideas on how this would affect
performance?  Obviously it adds some function call overhead -- with
collections the cases where there are large numbers of calls to the
backend renderers is reduced.

Another thought spoken as a threading newbie: could this be done more
simply by, eg subclassing RendererGTK and automating the protect
wrapping.  This latter approach would satisfy a couple of points: it
would not be intrusive in the base class, could be made optional via
interactive setting or rc, and would not adversely impact performance
when it was not needed (on my quick read of the patch, it looks like
the last point is already handled by the call)

  if is_interactive():
      import_protect()


I'll give your patches a test-drive.  Perhaps Steve could also comment
on the gtk changes.

JDH

Baptiste Carvello wrote:
> Hello,
> 
> I'm just making a proposal. As long as the documentation points to 
> IPython for interactive use, you can't say we have a perfect solution on 
> our own :-) (this is not meant to criticize your work on IPython, it's 
> just that I feel more confortable with the usual python interface)
> 
> I know the patch is intrusive, though maybe not as much as it looks 
> (everything except protect_mutex.py is pretty trivial). I didn't 
> expected it to be accepted right away, I see it more as a contribution 
> to the whole gui thread discution.
> 
> As for gui_thread, I looked at it last year, I just remember I didn't 
> like that code that much. Maybe I should get another look...

I actually didn't have ipython in mind at all in my comment.  I agree that a 
generic, shell-independent solution is a good thing to have.  I just think 
that gui_thread provides a cleaner approach (if not clean code :) to the 
overall problem.  gui_thread is messy, but it allows general (meaning, beyond 
matplotlib) access to gui elements from a command line.  I quickly tested it 
with matplotlib and it crashed, so there's work to be done...

I was just trying to point out an alternate direction, ultimately if John is 
happy with this approach, it's his call (he gets to maintain it :)

Cheers,

f

Hello,

I'm just making a proposal. As long as the documentation points to=20
IPython for interactive use, you can't say we have a perfect solution on=20
our own :-) (this is not meant to criticize your work on IPython, it's=20
just that I feel more confortable with the usual python interface)

I know the patch is intrusive, though maybe not as much as it looks=20
(everything except protect_mutex.py is pretty trivial). I didn't=20
expected it to be accepted right away, I see it more as a contribution=20
to the whole gui thread discution.

As for gui_thread, I looked at it last year, I just remember I didn't=20
like that code that much. Maybe I should get another look...

Cheers,
BC

Fernando Perez a =E9crit :
> Just a note: are you aware of the gui_thread module in scipy?  It was=20
> written exactly to do this (though it's currently WX-specific, I think)=
,=20
> and it has seen a pretty extensive amount of work.  I would much rather=
=20
> see a generic, external solution be tested and improved than a=20
> matplotlib-specific rather intrusive patch.  Don't take it badly, I can=
=20
> tell you've put a lot of work into this, but this kind of problem is=20
> IMHO best addressed by a more generic mechanism.  The kind of attribute=
=20
> access which python allows opens the door for such a solution, as=20
> gui_thread has shown.
>=20
> gui_thread currently needs testing and improvements, but I think in the=
=20
> long run it is a cleaner, better alternative.
>=20
> Just my 2 cents.
>=20
> Cheers,
>=20
> f
>=20

Baptiste Carvello wrote:
> Hello,
> 
> here is an imporved version of the patch (minor bug corrected + diffed 
> against latest CVS).

Just a note: are you aware of the gui_thread module in scipy?  It was written 
exactly to do this (though it's currently WX-specific, I think), and it has 
seen a pretty extensive amount of work.  I would much rather see a generic, 
external solution be tested and improved than a matplotlib-specific rather 
intrusive patch.  Don't take it badly, I can tell you've put a lot of work 
into this, but this kind of problem is IMHO best addressed by a more generic 
mechanism.  The kind of attribute access which python allows opens the door 
for such a solution, as gui_thread has shown.

gui_thread currently needs testing and improvements, but I think in the long 
run it is a cleaner, better alternative.

Just my 2 cents.

Cheers,

f

Hello,

here is an imporved version of the patch (minor bug corrected + diffed 
against latest CVS).
By the way, it looks like the web interface is cutting off the patch. 
Probably the right thing given its size. Providing a link to the 
attached file would be nicer, though...

Cheers,
BC

Fernando Perez wrote:
> Just a minor comment: insert(0) is an expensive (O(N)) operation on lists. 
> This kind of problem might be better addressed with a deque 
> (http://www.python.org/dev/doc/devel/lib/module-collections.html, could be 
> backported to py2.3).

Sorry for the extra msg, here's a pure python backport of deque by R. 
Hettinger, the author of the C collections module in py2.4:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/259179

Cheers,

f

Hello,

here is the corrected patch, sorry for the noise.

Cheers,
BC

Joe wrote:
> Hello John and other devels,
> 
> John, you were right on the money in relation to the memory leak that I wrote 
> about in the previous message.  Here's a rough draft of an attempt to patch 
> that up.  This is my first (meaningful) attempt to submit a patch anywhere, 
> so please forgive any errors in the process.  It was made with >patch -p ...  
> I think that is the accepted way to make a patch, but I'm not sure.

> *** text.py	Tue Jan 18 21:03:16 2005
> --- /root/text.py	Tue Jan 18 20:58:56 2005
> *************** class Text(Artist):
> *** 44,49 ****
> --- 44,50 ----
>   
>           Artist.__init__(self)
>           self.cached = {}
> + 	self.usage_stack = []
>           self._x, self._y = x, y
>   
>           if color is None: color = rcParams['text.color']
> *************** class Text(Artist):
> *** 193,198 ****
> --- 194,203 ----
>   
>           ret = bbox, zip(lines, whs, xs, ys)
>           self.cached[key] = ret
> + 	self.usage_stack.insert(0, key)
> + 	if len(self.usage_stack) > 128:
> + 	    self.usage_stack.pop()
> + 	    del self.cached[key]
>           return ret

Just a minor comment: insert(0) is an expensive (O(N)) operation on lists. 
This kind of problem might be better addressed with a deque 
(http://www.python.org/dev/doc/devel/lib/module-collections.html, could be 
backported to py2.3).

Cheers,

f

Hello John and other devels,

John, you were right on the money in relation to the memory leak that I wrote 
about in the previous message.  Here's a rough draft of an attempt to patch 
that up.  This is my first (meaningful) attempt to submit a patch anywhere, 
so please forgive any errors in the process.  It was made with >patch -p ...  
I think that is the accepted way to make a patch, but I'm not sure.

At first I wanted to plug this up by resuing the "cached" variable (by 
removing the last key from the .keys() method if the number of entries got 
too big), but I didn't like how that turned out so I decided to add a list 
which was sorted by last use.  Removal when there became too many items was 
performed on the least recently used entry.  I think that this way turned out 
for the better.

What might improve things is if there was a way to check how big the data 
structures which we are caching are in memory.  The first thought that came 
to my mind was to dump the structures to a Python string via cPickle and 
count the bytes used, but this would be too slow I think.  At the very least, 
the value could be supplied by the .matplotlibrc file, but I'm not very sure 
where all those values are read in.

Anyways too much talk for such a small fix,

Take care and happy coding,
Joe

Hello,

I got confused with my two patches. The pylab.py chunk from the other 
patch actually belongs here.
Sorry for that, I'll send a corrected version tomorrow, I need some sleep :)

Cheers,
BC

Hello,

here is a proposition of a new strategy for interactive plotting in GTK 
(probably also WX), that doesn't need IPython, and runs the mainloop in 
a separate thread from the calculations.
It has no impact on the user unless he chooses to make use of it, and a 
small impact on the backend developper.

It works in two parts:

matplotlib/__init__.py imports a gtk_protect function that returns a 
wrapper around "dangerous" functions or methods.
The one proposed here (protect_mutex.py) encloses the actual function 
call in a gtk.threads_enter() / gtk.threads_leave() pair in order to 
acquire the GTK mutex.
The same strategy seems possible in WX, though I encountered a lot of 
annoying little problems with WX 2.4. If someone is interested in taking 
over that part, please ask me (I usually don't use WX).
For the non-interactive case, matplotlib defines a dummy gtk_protect 
function which returns its own input, i.e. nothing is changed in that case.

Then all the backend as to do is call gtk_protect on dangerous functions 
or methods in order to replace them with a wrapper. The wrapper has 
little overhead, so I may have protected more than needed.

How to use:
after applying the patch, you have to change your rc params to actually 
use it. Change your .matplotlibrc to contain the following:
backend      : GTKAgg # or GTK or GTKCairo
interactive  : True
gui_protect  : Mutex

Afterwards, you can run most of the examples with python -i, and modify 
the plots to your liking :-)

Cheers,
BC

Hello,

I tried to improve the way of plotting two scales (cf 
examples/two_scales.py). The attached patch contains the following changes:

in axis.py: allow right side label, so both y axes can get one. axis 
instances get a new property: label_position, which is left or right for 
yaxis, top or bottom for xaxis.

in axes.py: a new Axes subclass (TwinAxes) that shares the x axis with 
another. Transforms are set accordingly (lazy values for x lims are 
shared). This avoids having to change the xlim or xlabel (and so on) 2 
times.

in pylab.py: a new command (twin) that returns the twin of an axes 
instance (the current one by default), thus making it into a two scales 
plot.

How to use: see the new examples/two_scales.py for the typical use case.

Cheers,
BC

This matplotlib release will be included in enthought's next release
of enthought python, which is widely used by windows users for
scientific computing.  I'd like to get an stable-as-possible release
in, since enthought python is released very infrequently.  Joe Cooper,
who is handling the release, says we can get fixes in until sometime
on Monday, so I'd be much obliged if you all could stress test this
release in case I need to get a bug-fix in.  There have been some
potentially script breaking substantial changes to the numerix module
described below, so these tests are doubly important.

What's new in matplotlib 0.71

numerix refactor

  The organization of the numerix module was refactored to be mindful
  of namespaces. See http://matplotlib.sf.net/API_CHANGES. pylab no
  longer overrides the built-ins min, max, and sum, and provides amin,
  amax and asum as the numerix/mlab versions of these.  pylab defines
  __all__ to prevent surprises when doing from pylab import *.  To see
  the complete list of symbols provided

    >>> import matplotlib.pylab
    >>> matplotlib.pylab.__all__

contour zigzag bug fixed

  Thanks Nadia for the blood, sweat and tears, and Dominique for the
  report.

contour colormaps

  Contour now uses the current colormap if colors is not provided, and
  works with colorbars.  See examples/contour_demo2.py

colorbar enhancements

  Horizontal colorbars supported with keyword arg
  orientation='horizontal' and colorbars can be placed in an arbitrary
  axes with keyword arg cax.

accents in mathtext

  Added accents to mathtext: \hat, reve, \grave, ar, cute, ilde, ec,
  \dot, \ddot. All of them have the same syntax, eg to make an overbar
  you do ar{o} or to make an o umlaut you do \ddot{o}. The shortcuts
  are also provided, eg: "o 'e \`e \~n \.x \^y . See
  examples/accent_demo.py

fixed super/subscript parsing in mathtext

  Widowed superscripts now work, eg r'$^12 m{CO}$'

little bugs and enhancements

  Plugged some memory leaks in wx and image module, fixed x,y args in
  contour, added latex symbol kappa, fixed a yticklabel problem under
  change in clim, fixed colorbar number of color bug, fixed
  set_clip_on bug, reverted pythoninspect in tkagg, fixed event
  handling bugs, fixed matlab-compatible load function, exposed vbox
  attr in FigureManagerGTK.


I did not get a chance to get the aspect=preserve imshow bugs fixed on
this iteration.  Something for next time!

  http://matplotlib.sourceforge.net

JDH

Following the extended discussion of namespace clashes on the dev and
users list, and some discussions with Todd offlist, I took a new
approach to fix these problems.  The central sticking points were

 1) pylab should not override built-ins

 2) numerix should be faithful to the underlying packages while providing
    a consistent interface to them.  Hence numerix would not change
    the names of the functions in mlab

 3) pylab imports the numerix symbols, but didn't want to have a
    different policy than numerix.

The resolution to this dilemma was to refactor numerix to preserve the
namespaces.  numerix now attempts to mirrors the numarray directory
structure for namespaces (though this is currently incomplete as I
just added enough to satisfy the needs of matplotlib).  So you now do

  from matplotlib.numerix.mlab import min, max, mean
  from matplotlib.numerix import arange, array
  from matplotlib.fft import fft

and you'll get the proper symbols from Numeric or numarray depending
on your numerix setting.  For convenience I added amin, amax to mlab
and asum to numerix, though the original python incompatible symbols
min, max, sum still reside in their correct numerix locations
unchanged.  At some point Todd will pull numerix out as a freestanding
module, I think, which will make for less typing since the
matplotlib prefix can be removed.

The decision for numerix package structure to mirror numarray's seems
like a good one to me, since everyone is still hoping to see Numeric
disappear, and this provides the most natural path.  When the great
merging occurs, one can simply globally replace numerix with numarray
and everything will follow.

pylab still imports (almost) all the symbols from numerix,
numerix.mlab, etc, except the ones that clash with python names.  In
their stead, pylab imports amin, amax and asym.  pylab also tries to
be a good citizen and defines __all__, to avoid problems like the time
module problem described by danny on the user's list.

This is explained in a little more detail in API_CHANGES.

I've tested this a good bit and everything seems to be working.  I am
trying to get a release out the door for Joe Cooper to fold into his
enthought build which is going out Monday.  I would like to put a
release up this afternoon on the user's list.

Would you kind folks here mind taking this for a test drive and submit
comments, feedback, problems, back to me as soon as possible?

  http://jdh.uchicago.edu/share/matplotlib-0.71rc1.tar.gz

It would also be helpful if you would inspect the __all__ var of pylab
to see if you think its correct.  Currently:

  date2num, num2date, drange, epoch2num, num2epoch, mx2num,
  DateFormatter, IndexDateFormatter, DateLocator, RRuleLocator,
  YearLocator, MonthLocator, WeekdayLocator, DayLocator, HourLocator,
  MinuteLocator, SecondLocator, rrule, MO, TU, WE, TH, FR, SA, SU,
  YEARLY, MONTHLY, WEEKLY, DAILY, HOURLY, MINUTELY, SECONDLY,
  relativedelta, array, zeros, shape, rank, size, fromstring, take,
  put, putmask, reshape, repeat, choose, searchsorted, asum, cumsum,
  product, cumproduct, alltrue, sometrue, allclose, arrayrange,
  arange, asarray, convolve, swapaxes, concatenate, transpose, sort,
  argsort, argmax, argmin, innerproduct, dot, outerproduct, resize,
  indices, fromfunction, diagonal, trace, ravel, nonzero, shape,
  where, compress, clip, zeros, ones, identity, add, logical_or, exp,
  subtract, logical_xor, log, multiply, logical_not, log10, divide,
  maximum, sin, minimum, sinh, conjugate, bitwise_and, sqrt, power,
  bitwise_or, tan, absolute, bitwise_xor, tanh, negative, ceil,
  greater, fabs, greater_equal, floor, less, arccos, arctan2,
  less_equal, arcsin, fmod, equal, arctan, hypot, not_equal, cos,
  around, logical_and, cosh, arccosh, arcsinh, arctanh,
  cross_correlate, pi, ArrayType, matrixmultiply, rand, randn, eye,
  tri, diag, fliplr, flipud, rot90, tril, triu, ptp, mean, msort,
  median, std, cumsum, prod, cumprod, trapz, diff, cov, corrcoef,
  squeeze, kaiser, blackman, bartlett, hanning, hamming, sinc, eig,
  svd, angle, roots, amin, amax, linspace, window_hanning,
  window_none, conv, detrend, detrend_mean, detrend_none,
  detrend_linear, corrcoef, polyfit, polyval, vander, entropy,
  normpdf, levypdf, find, trapz, prepca, fix, rem, norm, orth, rank,
  sqrtm, prctile, center_matrix, meshgrid, rk4, exp_safe, amap,
  sum_flat, mean_flat, rms_flat, l1norm, l2norm, norm, frange,
  diagonal_matrix, base_repr, binary_repr, log2, ispower2,
  bivariate_normal, inverse, eigenvectors, fft, Int8, UInt8, Int16,
  UInt16, Int32, UInt32, Float32, Float64, Complex32, Complex64,
  Float, Int, Complex, TickHelper, Formatter, FixedFormatter,
  NullFormatter, FuncFormatter, FormatStrFormatter, ScalarFormatter,
  LogFormatter, LogFormatterExponent, LogFormatterMathtext, Locator,
  IndexLocator, FixedLocator, NullLocator, LinearLocator, LogLocator,
  AutoLocator, MultipleLocator, plotting, colormaps,
  get_current_fig_manager, connect, disconnect, get_plot_commands,
  raise_msg_to_str, axis, axes, delaxes, clim, close, clf, colorbar,
  draw, figtext, figimage, figlegend, figure, gca, gcf, gci, get,
  hold, ishold, isinteractive, imread, load, rc, rcdefaults, save,
  savefig, set, subplot, title, xlabel, ylabel, xlim, ylim, xticks,
  rgrids, thetagrids, yticks, polar, over, ioff, ion, axhline,
  axhspan, axvline, axvspan, bar, barh, cohere, contour, csd,
  errorbar, fill, hist, hlines, imshow, loglog, pcolor,
  pcolor_classic, pie, plot, plot_date, psd, scatter, scatter_classic,
  semilogx, semilogy, specgram, spy, spy2, stem, vlines, cla, grid,
  legend, table, text, autumn, bone, cool, copper, flag, gray, hot,
  hsv, jet, pink, prism, spring, summer, winter, Line2D, Polygon,
  Rectangle, Circle, Text, cm, get_cmap, normalize, show




Thanks!
JDH 

Thanks John,

Sheesh, 95 degrees, I can't imagine how I'd take that.   Some Christmases my 
family would go visit relatives in Chicago.  It seemed just a little bit more 
warmer (but not that much) when we returned to Oklahoma.  Never really gone 
anywhere to warm for Christmas.

More below...

On Tuesday 18 January 2005 20:25, John Hunter wrote:
> >>>>> "Joe" == Joe  <gar...@us...> writes:
>
>     Joe> Hello devels, I chose to use matplotlib for a restaurant
>     Joe> simulation.  It supplied all the features I needed and I was
>     Joe> pleasantly surprised.  However the people I was doing this
>     Joe> project for mentioned that the simulation slowed down to a
>     Joe> crawl 10 or so minutes into the simulation.  It sounded like
>     Joe> a memory leak to me so I investigated and found a 100 KB/sec
>     Joe> leak in Windows (via the interpreter as well as frozen with
>     Joe> py2exe) as well as in Linux.  I narrowed it down to a part of
>     Joe> the code where I updated xlim and ylim via Axes::set_xlim,
>     Joe> Axes::set_ylim.  Commenting that part out yielded no leak,
>
> So you're doing animation right?  Are you using images, eg imshow?  I
> just fixed a memory leak in image resizing that amounts to about
> 200K/resize -- the changes are in CVS.  Of course on windows it's hard
> to test because the changes are in extension code, and require
> building on win32.  The instructions for building on win32 are in the
> file setupext.py.

I wish so, but no, I'm not using any images.

> If you are not using images, another place to look is in text caching.
> text layout is expensive (to support arbitrary rotations, etc).  So I
> cache the layout information in a dictionary "cached" in the text
> module.  When you change the xlim/ylim, that would trigger a new
> layout and an addition to the cache.  We need to add some
> auto-clearing mechanisms to prevent the cache from growing w/o bound
> but its not done yet.  I find it hard to believe, though, that this
> would leak 100kb/sec.  Assuming a 20Hz frame rate, the leak is about
> 5kb/sec, and I don't think I'm caching that much info.

I am using 5 GTKAgg FIgure widgets with 2-3 subplots a piece (a maximum of 3 
subplots get drawn at a time).  The simulation runs at 1 Hz.  Therefore 6.67 
kB/sec at max.  Which sounds about right.  I am updating each Axes point set 
when I do the update.  

> You can estimate the memory effect of the cached simply by commenting
> out the line
>
>         self.cached[key] = ret
>
> in the matplotlib.text module.

I'll definitely check that out.  With the above math that just might prove to 
be the culprit.

> What backend are you using?  Are you refreshing the same figure data,
> or creating new figures?  More information would certainly help.....
>
>     Joe> however I lost some critical functionality in my charts
>     Joe> (which behave like the Windows Task Manager resource monitor
>     Joe> graphs).  So I really need this fixed.  I will send shortly a
>     Joe> test case which replicates the problem.
>
> That would be great.
>
>     Joe> Since the maintainer is out on vacation, I was also wondering
>     Joe> if anyone knew enough about the internals of matplotlib that
>     Joe> they could give me an idea of which modules (Python or
>     Joe> otherwise) are affected by Axes::set_xlim and Axes::set_ylim.
>
> I'm back.  I was in Rio for a couple of weeks and have suffered a 95
> degree temperature drop returning to Chicago (sigh)...
>
> JDH

I'll get the test case up on the web ASAP, I just wanted to reply now to let 
everyone know that I'm still working on the problem.

Joe

Hi there,

I just submitted a patch changing the errorhandling of Line2D on non-existant 
linestyle/marker strings. Now I realize that it obviously clashes with the 
current policy for reporting errors.

What happened before was that
 plot(...,linestyle='garbage',...)
would verbose.report_error and set linestyle to a default, while
 set_linestyle(...,linestyle='garbage')
would be silent and still set linestyle to a default.

Clearly, this is inconsistent, so there will be little argument that the 
behavior should be unified. Question is, to what?

What I would expect is, that set_linestyle raises an exception if it is called 
with the wrong arguments. This is, what I expect of any python library, no 
matter whether it offers a GUI or not. When I work in ipython at the 
commandline, I expect the errors to be displayed right there, and not be 
written to a file or displayed in some window.

Obviously, this clashes with the very existance of the 'Verbose' class, used 
to report errors which just writes errors to some file and goes on with work 
afterwards.

The verbose.report_error approach certainly is useful within the code that is 
run in the GUI-thread. Those routines that are called from outside, though, 
should rather raise exception.

Furthermore, the outside functions should do as much checking on the input as 
reasonably possible, so that errors are simpler to debug. Once some erraneous 
input has reached the internals of matplotlib, it really is hard for the user 
to guess what actually caused the error.

Ciao,
Norbert

-- 
_________________________________________Norbert Nemec
         Bernhardstr. 2 ... D-93053 Regensburg
     Tel: 0941 - 2009638 ... Mobil: 0179 - 7475199
           eMail: <No...@Ne...>

>>>>> "Joe" == Joe  <gar...@us...> writes:

    Joe> Hello devels, I chose to use matplotlib for a restaurant
    Joe> simulation.  It supplied all the features I needed and I was
    Joe> pleasantly surprised.  However the people I was doing this
    Joe> project for mentioned that the simulation slowed down to a
    Joe> crawl 10 or so minutes into the simulation.  It sounded like
    Joe> a memory leak to me so I investigated and found a 100 KB/sec
    Joe> leak in Windows (via the interpreter as well as frozen with
    Joe> py2exe) as well as in Linux.  I narrowed it down to a part of
    Joe> the code where I updated xlim and ylim via Axes::set_xlim,
    Joe> Axes::set_ylim.  Commenting that part out yielded no leak,

So you're doing animation right?  Are you using images, eg imshow?  I
just fixed a memory leak in image resizing that amounts to about
200K/resize -- the changes are in CVS.  Of course on windows it's hard
to test because the changes are in extension code, and require
building on win32.  The instructions for building on win32 are in the
file setupext.py.

If you are not using images, another place to look is in text caching.
text layout is expensive (to support arbitrary rotations, etc).  So I
cache the layout information in a dictionary "cached" in the text
module.  When you change the xlim/ylim, that would trigger a new
layout and an addition to the cache.  We need to add some
auto-clearing mechanisms to prevent the cache from growing w/o bound
but its not done yet.  I find it hard to believe, though, that this
would leak 100kb/sec.  Assuming a 20Hz frame rate, the leak is about
5kb/sec, and I don't think I'm caching that much info.

You can estimate the memory effect of the cached simply by commenting
out the line

        self.cached[key] = ret

in the matplotlib.text module.

What backend are you using?  Are you refreshing the same figure data,
or creating new figures?  More information would certainly help.....

    Joe> however I lost some critical functionality in my charts
    Joe> (which behave like the Windows Task Manager resource monitor
    Joe> graphs).  So I really need this fixed.  I will send shortly a
    Joe> test case which replicates the problem.

That would be great.

    Joe> Since the maintainer is out on vacation, I was also wondering
    Joe> if anyone knew enough about the internals of matplotlib that
    Joe> they could give me an idea of which modules (Python or
    Joe> otherwise) are affected by Axes::set_xlim and Axes::set_ylim.

I'm back.  I was in Rio for a couple of weeks and have suffered a 95
degree temperature drop returning to Chicago (sigh)...

JDH

Hello devels,

I chose to use matplotlib for a restaurant simulation.  It supplied all the 
features I needed and I was pleasantly surprised.  However the people I was 
doing this project for mentioned that the simulation slowed down to a crawl 
10 or so minutes into the simulation.  It sounded like a memory leak to me so 
I investigated and found a 100 KB/sec leak in Windows (via the interpreter as 
well as frozen with py2exe) as well as in Linux.  I narrowed it down to a 
part of the code where I updated xlim and ylim via Axes::set_xlim, 
Axes::set_ylim.  Commenting that part out yielded no leak, however I lost 
some critical functionality in my charts (which behave like the Windows Task 
Manager resource monitor graphs).  So I really need this fixed.  I will send 
shortly a test case which replicates the problem.  

Since the maintainer is out on vacation, I was also wondering if anyone knew 
enough about the internals of matplotlib that they could give me an idea of 
which modules (Python or otherwise) are affected by Axes::set_xlim and 
Axes::set_ylim.

Joe

Thanks!

 --- Todd Miller <jm...@st...> wrote: 
> The setup scripts copy _transforms.cpp to
> _na_transforms.cpp and
> _nc_transforms.cpp at build time.  The good news is
> there's only one
> source file for both:  src/_transforms.cpp.
> 
> Regards,
> Todd
> 
> On Wed, 2005-01-12 at 17:30, Brendan Simons wrote:
> > Hi.  I'm attempting to extend the polar_axes class
> to
> > better handle scrolling and zooming, and to allow
> > setting  a lower limit on the r-axis.   There
> isn't
> > much in the way of documentation 'round these
> parts,
> > so I'm kind of stretching.  I have a hunch though
> I
> > may have to add a transform to the extension
> module
> > listed in the subject line.  Can someone point me
> to
> > the source?
> > 
> > Brendan
> > 
> >
>
______________________________________________________________________
> 
> > Post your free ad now! http://personals.yahoo.ca
> > 
> > 
> >
>
-------------------------------------------------------
> > The SF.Net email is sponsored by: Beat the
> post-holiday blues
> > Get a FREE limited edition SourceForge.net t-shirt
> from ThinkGeek.
> > It's fun and FREE -- well,
> almost....http://www.thinkgeek.com/sfshirt
> > _______________________________________________
> > Matplotlib-devel mailing list
> > Mat...@li...
> >
>
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> -- 
> 
>  

______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca

The setup scripts copy _transforms.cpp to _na_transforms.cpp and
_nc_transforms.cpp at build time.  The good news is there's only one
source file for both:  src/_transforms.cpp.

Regards,
Todd

On Wed, 2005-01-12 at 17:30, Brendan Simons wrote:
> Hi.  I'm attempting to extend the polar_axes class to
> better handle scrolling and zooming, and to allow
> setting  a lower limit on the r-axis.   There isn't
> much in the way of documentation 'round these parts,
> so I'm kind of stretching.  I have a hunch though I
> may have to add a transform to the extension module
> listed in the subject line.  Can someone point me to
> the source?
> 
> Brendan
> 
> ______________________________________________________________________ 
> Post your free ad now! http://personals.yahoo.ca
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by: Beat the post-holiday blues
> Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
> It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
--