matplotlib-devel Mailing List for matplotlib

On Dec 14, 2010, at 2:38 PM, Benjamin Root wrote:

> On Tue, Dec 14, 2010 at 4:22 PM, Benjamin Root <ben...@ou...>  
> wrote:
>
>
> On Tue, Dec 14, 2010 at 4:06 PM, Russell Owen <ro...@uw...> wrote:
> On Dec 14, 2010, at 1:50 PM, Benjamin Root wrote:
>
>> On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <ro...@uw...>  
>> wrote:
>> I tried the test script on linux using matplotlib svn trunk rev8840
>> (which appears to include your patch) and found a leak that starts  
>> out
>> small but gets systematically larger. This is with Python 2.6.5 and
>> Tkinter built against Tcl/Tk 8.4.13.
>>
>> Is anyone else seeing this?
>>
>> time   rss memory    mem/sec
>> (sec)     (kb)      (kb/sec)
>>   0.0    36816       nan
>>   5.0    36860       nan
>>  10.0    36860       0.0
>>  15.1    36860       0.0
>>  20.1    36860       0.0
>>  25.1    36896       1.8
>> ...
>>  326.5    36896       0.1
>>  331.6    36972       0.3
>> ...
>>  401.9    36972       0.3
>>  406.9    36980       0.3
>>   ...
>>
>>  602.8    37684       1.4
>>  607.8    37700       1.4
>>
>> This is different behavior than on Mac OS X, but still alarming.
>>
>> -- Russell
>>
>>
>> Russell,
>>
>> I am curious, I recently ran into problems with mixing builds of  
>> numpy and matplotlib at various levels of revisions.  I eventually  
>> had to do a complete clean rebuild.  Note, what I mean by a  
>> complete clean rebuild is that I removed the numpy and matplotlib  
>> source directories and re-checkout the codes from source and then  
>> rebuild.  I would be curious if the problem persists after that.
>
> An interesting question. I can say that this was a clean build of  
> matplotlib since I ran it "in place" (in build/lib.linux- 
> x86_64-2.6/) after building it rather than installing it in site- 
> packages to avoid messing up other users (on the linux system this  
> was a shared python). I modified the script to print  
> matplotlib.__file__ to make sure I was running the right version. I  
> doubt it was a clean build of numpy. But considering no numerics are  
> occurring in this example (it is literally just creating an Axes on  
> a Canvas and calling canvas.draw() repeatedly) do you think numpy  
> could possibly come into this?
>
>
> It is quite possible.  Numpy is used extensively throughout the  
> matplotlib system, regardless of whether you are using it or not.   
> Also, the fact that you are on a shared system is interesting.  For  
> those situations, try doing
>
> "python setupegg.py develop --user"
>
> for the build command.  What this does is builds everything in-place  
> (the build directory symbolically links to those files), and then  
> uses your ~/.local directory to install an egg.lnk file to point  
> back to the build directory.  This should have higher search  
> precedence than the system install of matplotlib and numpy.
>
> Note, I had mixed success with this approach on a RHEL (5?) system  
> recently.  I don't know how recently ~/.local became widely accepted  
> among various distros.
>
> Might you run the script on your system with the clean build?
>
> -- Russell
>
>
> I will give it a shot (once my other analysis programs are done for  
> the day).
>
> Ben Root
>
> I ran your script from the url you posted earlier.  My build does  
> not show any leak for about a minute.  I am fairly certain that what  
> we have here is a build mixing issue or an issue with an unknown  
> combination of libraries interacting.  Could you post your build logs?
>
> Ben Root
>

I tried your suggestion -- installing numpy from scratch (deleting the  
old numpy first) and then building matplotlib fresh. Here are my build  
logs:
<http://www.astro.washington.edu/users/rowen/python/numpy_build_log.txt>
<http://www.astro.washington.edu/users/rowen/python/matplotlib_build_log.txt 
 >

Much like last time (with a fresh matplotlib) the test program shows  
no leak at first, then it starts growing to alarming levels (though it  
took longer to start leaking this time than it did last time):

matplotlib.__file__= /astro/users/rowen/build/matplotlib-trunk/build/ 
lib.linux-x86_64-2.6/matplotlib/__init__.py
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
    0.0    36860       nan
    5.1    36904       nan
   10.1    36904       0.0
   15.1    36904       0.0
   20.2    36904       0.0
   25.2    36904       0.0
   30.2    36904       0.0
   35.2    36904       0.0
   40.2    36904       0.0
   45.2    36904       0.0
   50.3    36904       0.0
   55.3    36904       0.0
   60.3    36904       0.0
   65.4    36904       0.0
   70.4    36904       0.0
   75.4    36904       0.0
   80.4    36904       0.0
   85.4    36904       0.0
   90.5    36904       0.0
   95.5    36904       0.0
  100.5    36976       0.8
  105.5    36976       0.7
  110.6    36976       0.7
  115.6    36976       0.7
  120.6    36976       0.6
  125.6    36976       0.6
  130.7    36976       0.6
  135.7    36976       0.6
  140.7    36976       0.5
  145.7    36976       0.5
  150.8    36976       0.5
  155.8    36976       0.5
  160.8    36976       0.5
  165.8    36976       0.4
  170.9    36976       0.4
  175.9    36976       0.4
  180.9    36976       0.4
  185.9    36976       0.4
  190.9    36976       0.4
  196.0    36976       0.4
  201.0    36976       0.4
  206.0    36976       0.4
  211.0    36976       0.3
  216.1    36976       0.3
  221.1    36976       0.3
  226.1    36976       0.3
  231.1    36976       0.3
  236.1    36976       0.3
  241.2    36976       0.3
  246.2    36976       0.3
  251.2    36976       0.3
  256.3    36976       0.3
  261.3    36976       0.3
  266.3    36976       0.3
  271.4    36976       0.3
  276.4    36976       0.3
  281.4    36976       0.3
  286.5    36976       0.3
  291.5    36976       0.3
  296.6    36976       0.2
  301.6    36976       0.2
  306.6    36976       0.2
  311.6    36976       0.2
  316.6    36976       0.2
  321.6    36976       0.2
  326.7    36976       0.2
  331.7    37048       0.4
  336.7    37048       0.4
  341.7    37048       0.4
  346.8    37048       0.4
  351.8    37048       0.4
  356.8    37048       0.4
  361.8    37048       0.4
  366.8    37048       0.4
  371.8    37048       0.4
  376.9    37048       0.4
  381.9    37048       0.4
  386.9    37048       0.4
  391.9    37048       0.4
  397.0    37048       0.4
  402.0    37048       0.4
  407.0    37072       0.4
  412.0    37084       0.4
  417.1    37108       0.5
  422.1    37124       0.5
  427.1    37144       0.6
  432.1    37156       0.6
  437.2    37180       0.6
  442.2    37196       0.7
  447.2    37216       0.7
  452.2    37228       0.7
  457.2    37248       0.8
  462.3    37264       0.8
  467.3    37284       0.8
  472.3    37300       0.8
  477.3    37320       0.9
  482.3    37336       0.9
  487.4    37356       0.9
  492.4    37372       1.0
  497.4    37392       1.0
  502.4    37404       1.0
  507.4    37424       1.0
  512.5    37444       1.1
  517.5    37460       1.1
  522.5    37476       1.1
  527.6    37496       1.1
  532.6    37512       1.2
  537.6    37532       1.2
  542.6    37548       1.2
  547.6    37568       1.2
  552.6    37584       1.2
  557.7    37604       1.3
  562.7    37620       1.3
  567.7    37636       1.3
  572.8    37652       1.3
  577.8    37676       1.3
  582.9    37688       1.4
  587.9    37708       1.4
  593.0    37724       1.4
  598.0    37744       1.4
  603.0    37760       1.4
  608.0    37780       1.5
  613.0    37796       1.5
  618.1    37816       1.5
  623.1    37828       1.5
  628.1    37848       1.5
  633.2    37864       1.5
  638.2    37884       1.5
  643.3    37904       1.6
  648.3    37920       1.6
  653.4    37936       1.6
  658.4    37956       1.6
  663.4    37972       1.6
  668.4    37992       1.6
(control-C to stop it)

Also like last time (though I did not report this before since I had  
not yet discovered it) once this has occurred the honeymoon is over.  
Leakage is high right from the beginning. It appears that something is  
being cached that causes trouble, but I have no idea what. Here is a  
log from a run immediately after the previous run:

-bash-3.2$ python matplotlibMemoryLeak.py
matplotlib.__file__= /astro/users/rowen/build/matplotlib-trunk/build/ 
lib.linux-x86_64-2.6/matplotlib/__init__.pyc
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
    0.0    33892       nan
    5.0    34168       nan
   10.1    34188       4.0
   15.1    34228       6.0
   20.1    34248       5.3
   25.1    34284       5.8
   30.2    34300       5.2
   35.2    34320       5.0
   40.2    34400       6.6

Might you try running your test for much longer to see if the program  
starts leaking for you?

-- Russell

On Tue, Dec 14, 2010 at 4:22 PM, Benjamin Root <ben...@ou...> wrote:

>
>
> On Tue, Dec 14, 2010 at 4:06 PM, Russell Owen <ro...@uw...> wrote:
>
>> On Dec 14, 2010, at 1:50 PM, Benjamin Root wrote:
>>
>> On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <ro...@uw...> wrote:
>>
>>> I tried the test script on linux using matplotlib svn trunk rev8840
>>> (which appears to include your patch) and found a leak that starts out
>>> small but gets systematically larger. This is with Python 2.6.5 and
>>> Tkinter built against Tcl/Tk 8.4.13.
>>>
>>> Is anyone else seeing this?
>>>
>>> time   rss memory    mem/sec
>>> (sec)     (kb)      (kb/sec)
>>>    0.0    36816       nan
>>>   5.0    36860       nan
>>>  10.0    36860       0.0
>>>  15.1    36860       0.0
>>>  20.1    36860       0.0
>>>  25.1    36896       1.8
>>> ...
>>>  326.5    36896       0.1
>>>  331.6    36972       0.3
>>> ...
>>>  401.9    36972       0.3
>>>  406.9    36980       0.3
>>>   ...
>>>
>>>  602.8    37684       1.4
>>>  607.8    37700       1.4
>>>
>>> This is different behavior than on Mac OS X, but still alarming.
>>>
>>> -- Russell
>>>
>>>
>> Russell,
>>
>> I am curious, I recently ran into problems with mixing builds of numpy and
>> matplotlib at various levels of revisions.  I eventually had to do a
>> complete clean rebuild.  Note, what I mean by a complete clean rebuild is
>> that I removed the numpy and matplotlib source directories and re-checkout
>> the codes from source and then rebuild.  I would be curious if the problem
>> persists after that.
>>
>>
>> An interesting question. I can say that this was a clean build of
>> matplotlib since I ran it "in place" (in build/lib.linux-x86_64-2.6/) after
>> building it rather than installing it in site-packages to avoid messing up
>> other users (on the linux system this was a shared python). I modified the
>> script to print matplotlib.__file__ to make sure I was running the right
>> version. I doubt it was a clean build of numpy. But considering no numerics
>> are occurring in this example (it is literally just creating an Axes on a
>> Canvas and calling canvas.draw() repeatedly) do you think numpy could
>> possibly come into this?
>>
>>
> It is quite possible.  Numpy is used extensively throughout the matplotlib
> system, regardless of whether you are using it or not.  Also, the fact that
> you are on a shared system is interesting.  For those situations, try doing
>
> "python setupegg.py develop --user"
>
> for the build command.  What this does is builds everything in-place (the
> build directory symbolically links to those files), and then uses your
> ~/.local directory to install an egg.lnk file to point back to the build
> directory.  This should have higher search precedence than the system
> install of matplotlib and numpy.
>
> Note, I had mixed success with this approach on a RHEL (5?) system
> recently.  I don't know how recently ~/.local became widely accepted among
> various distros.
>
>
>> Might you run the script on your system with the clean build?
>>
>> -- Russell
>>
>>
> I will give it a shot (once my other analysis programs are done for the
> day).
>
> Ben Root
>

I ran your script from the url you posted earlier.  My build does not show
any leak for about a minute.  I am fairly certain that what we have here is
a build mixing issue or an issue with an unknown combination of libraries
interacting.  Could you post your build logs?

Ben Root

On Tue, Dec 14, 2010 at 4:06 PM, Russell Owen <ro...@uw...> wrote:

> On Dec 14, 2010, at 1:50 PM, Benjamin Root wrote:
>
> On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <ro...@uw...> wrote:
>
>> I tried the test script on linux using matplotlib svn trunk rev8840
>> (which appears to include your patch) and found a leak that starts out
>> small but gets systematically larger. This is with Python 2.6.5 and
>> Tkinter built against Tcl/Tk 8.4.13.
>>
>> Is anyone else seeing this?
>>
>> time   rss memory    mem/sec
>> (sec)     (kb)      (kb/sec)
>>    0.0    36816       nan
>>   5.0    36860       nan
>>  10.0    36860       0.0
>>  15.1    36860       0.0
>>  20.1    36860       0.0
>>  25.1    36896       1.8
>> ...
>>  326.5    36896       0.1
>>  331.6    36972       0.3
>> ...
>>  401.9    36972       0.3
>>  406.9    36980       0.3
>>   ...
>>
>>  602.8    37684       1.4
>>  607.8    37700       1.4
>>
>> This is different behavior than on Mac OS X, but still alarming.
>>
>> -- Russell
>>
>>
> Russell,
>
> I am curious, I recently ran into problems with mixing builds of numpy and
> matplotlib at various levels of revisions.  I eventually had to do a
> complete clean rebuild.  Note, what I mean by a complete clean rebuild is
> that I removed the numpy and matplotlib source directories and re-checkout
> the codes from source and then rebuild.  I would be curious if the problem
> persists after that.
>
>
> An interesting question. I can say that this was a clean build of
> matplotlib since I ran it "in place" (in build/lib.linux-x86_64-2.6/) after
> building it rather than installing it in site-packages to avoid messing up
> other users (on the linux system this was a shared python). I modified the
> script to print matplotlib.__file__ to make sure I was running the right
> version. I doubt it was a clean build of numpy. But considering no numerics
> are occurring in this example (it is literally just creating an Axes on a
> Canvas and calling canvas.draw() repeatedly) do you think numpy could
> possibly come into this?
>
>
It is quite possible.  Numpy is used extensively throughout the matplotlib
system, regardless of whether you are using it or not.  Also, the fact that
you are on a shared system is interesting.  For those situations, try doing

"python setupegg.py develop --user"

for the build command.  What this does is builds everything in-place (the
build directory symbolically links to those files), and then uses your
~/.local directory to install an egg.lnk file to point back to the build
directory.  This should have higher search precedence than the system
install of matplotlib and numpy.

Note, I had mixed success with this approach on a RHEL (5?) system
recently.  I don't know how recently ~/.local became widely accepted among
various distros.


> Might you run the script on your system with the clean build?
>
> -- Russell
>
>
I will give it a shot (once my other analysis programs are done for the
day).

Ben Root

On Dec 14, 2010, at 1:50 PM, Benjamin Root wrote:

> On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <ro...@uw...> wrote:
> I tried the test script on linux using matplotlib svn trunk rev8840
> (which appears to include your patch) and found a leak that starts out
> small but gets systematically larger. This is with Python 2.6.5 and
> Tkinter built against Tcl/Tk 8.4.13.
>
> Is anyone else seeing this?
>
> time   rss memory    mem/sec
> (sec)     (kb)      (kb/sec)
>   0.0    36816       nan
>   5.0    36860       nan
>  10.0    36860       0.0
>  15.1    36860       0.0
>  20.1    36860       0.0
>  25.1    36896       1.8
> ...
>  326.5    36896       0.1
>  331.6    36972       0.3
> ...
>  401.9    36972       0.3
>  406.9    36980       0.3
>   ...
>  602.8    37684       1.4
>  607.8    37700       1.4
>
> This is different behavior than on Mac OS X, but still alarming.
>
> -- Russell
>
>
> Russell,
>
> I am curious, I recently ran into problems with mixing builds of  
> numpy and matplotlib at various levels of revisions.  I eventually  
> had to do a complete clean rebuild.  Note, what I mean by a complete  
> clean rebuild is that I removed the numpy and matplotlib source  
> directories and re-checkout the codes from source and then rebuild.   
> I would be curious if the problem persists after that.

An interesting question. I can say that this was a clean build of  
matplotlib since I ran it "in place" (in build/lib.linux-x86_64-2.6/)  
after building it rather than installing it in site-packages to avoid  
messing up other users (on the linux system this was a shared python).  
I modified the script to print matplotlib.__file__ to make sure I was  
running the right version. I doubt it was a clean build of numpy. But  
considering no numerics are occurring in this example (it is literally  
just creating an Axes on a Canvas and calling canvas.draw()  
repeatedly) do you think numpy could possibly come into this?

Might you run the script on your system with the clean build?

-- Russell

On Tue, Dec 14, 2010 at 3:18 PM, Russell E. Owen <ro...@uw...> wrote:

> I tried the test script on linux using matplotlib svn trunk rev8840
> (which appears to include your patch) and found a leak that starts out
> small but gets systematically larger. This is with Python 2.6.5 and
> Tkinter built against Tcl/Tk 8.4.13.
>
> Is anyone else seeing this?
>
> time   rss memory    mem/sec
> (sec)     (kb)      (kb/sec)
>    0.0    36816       nan
>   5.0    36860       nan
>  10.0    36860       0.0
>  15.1    36860       0.0
>  20.1    36860       0.0
>  25.1    36896       1.8
> ...
>  326.5    36896       0.1
>  331.6    36972       0.3
> ...
>  401.9    36972       0.3
>  406.9    36980       0.3
>  411.9    37004       0.4
>  417.0    37016       0.4
>  422.0    37040       0.4
>  427.0    37052       0.5
>  432.0    37076       0.5
>  437.1    37088       0.5
>  442.1    37112       0.6
>  447.1    37124       0.6
>  452.1    37148       0.6
>  457.2    37160       0.7
>  462.2    37184       0.7
>  467.2    37196       0.7
>  472.2    37220       0.8
>  477.3    37232       0.8
>  482.3    37256       0.8
>  487.3    37268       0.8
>  492.3    37292       0.9
>  497.3    37304       0.9
>  502.3    37328       0.9
>  507.4    37340       1.0
>  512.4    37360       1.0
>  517.4    37376       1.0
>  522.4    37396       1.0
>  527.5    37412       1.1
>  532.5    37432       1.1
>  537.5    37448       1.1
>  542.5    37472       1.1
>  547.6    37488       1.2
>  552.6    37504       1.2
>  557.6    37516       1.2
>  562.6    37540       1.2
>  567.6    37556       1.2
>  572.7    37576       1.3
>  577.7    37592       1.3
>  582.7    37612       1.3
>  587.7    37628       1.3
>  592.7    37648       1.3
>  597.8    37664       1.4
>  602.8    37684       1.4
>  607.8    37700       1.4
>
> This is different behavior than on Mac OS X, but still alarming.
>
> -- Russell
>
>
Russell,

I am curious, I recently ran into problems with mixing builds of numpy and
matplotlib at various levels of revisions.  I eventually had to do a
complete clean rebuild.  Note, what I mean by a complete clean rebuild is
that I removed the numpy and matplotlib source directories and re-checkout
the codes from source and then rebuild.  I would be curious if the problem
persists after that.

Ben Root

I tried the test script on linux using matplotlib svn trunk rev8840 
(which appears to include your patch) and found a leak that starts out 
small but gets systematically larger. This is with Python 2.6.5 and 
Tkinter built against Tcl/Tk 8.4.13.

Is anyone else seeing this?

time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    36816       nan
   5.0    36860       nan
  10.0    36860       0.0
  15.1    36860       0.0
  20.1    36860       0.0
  25.1    36896       1.8
...
 326.5    36896       0.1
 331.6    36972       0.3
...
 401.9    36972       0.3
 406.9    36980       0.3
 411.9    37004       0.4
 417.0    37016       0.4
 422.0    37040       0.4
 427.0    37052       0.5
 432.0    37076       0.5
 437.1    37088       0.5
 442.1    37112       0.6
 447.1    37124       0.6
 452.1    37148       0.6
 457.2    37160       0.7
 462.2    37184       0.7
 467.2    37196       0.7
 472.2    37220       0.8
 477.3    37232       0.8
 482.3    37256       0.8
 487.3    37268       0.8
 492.3    37292       0.9
 497.3    37304       0.9
 502.3    37328       0.9
 507.4    37340       1.0
 512.4    37360       1.0
 517.4    37376       1.0
 522.4    37396       1.0
 527.5    37412       1.1
 532.5    37432       1.1
 537.5    37448       1.1
 542.5    37472       1.1
 547.6    37488       1.2
 552.6    37504       1.2
 557.6    37516       1.2
 562.6    37540       1.2
 567.6    37556       1.2
 572.7    37576       1.3
 577.7    37592       1.3
 582.7    37612       1.3
 587.7    37628       1.3
 592.7    37648       1.3
 597.8    37664       1.4
 602.8    37684       1.4
 607.8    37700       1.4

This is different behavior than on Mac OS X, but still alarming.

-- Russell

In article <4D0...@st...>,
 Michael Droettboom <md...@st...> 
 wrote:

> If you're able to try this with the 1.0.x SVN branch or the SVN trunk 
> (plus my text patch) let me know.  I am not able to reproduce any sort 
> of memory leak with these newer versions, but I am able to with 1.0.0 as 
> released (with or without my text patch).  This is with or without the x 
> axis limits updating you suggested.  I believe there have been other 
> memory leak fixes since the 1.0.0 release.
> 
> Cheers,
> Mike
> 
> On 12/13/2010 08:35 PM, Russell E. Owen wrote:
> > I also wanted to say:
> > - Thank you for the patch. I appreciate the chance to try a fix.
> > - I doubt the issue is related to text because my scripts shows a memory
> > leak even if the displayed text is never updated (comment out the line
> > that modifies the x axis limits to see what I mean; though I confess I
> > did not try that with the trunk matplotlib, only with 1.0.0).
> >
> > -- Russell
> >
> > In article<4D0...@st...>,
> >   Michael Droettboom<md...@st...>
> >   wrote:
> >
> >    
> >> I think I'm on to something -- it seems that text layout information has
> >> a cyclical reference that prevents the Text object from being freed.
> >>
> >> Can you apply the attached patch and let me know if it solves your issue?

I already posted results using the svn trunk plus the svn trunk with 
your patch (as well as the 1.0.0 release). I updated them today with a 
few more options (such as your patch without setting the x limit -- it 
didn't make any difference) and letting them run longer. Here they are. 
All are from running 
<http://www.astro.washington.edu/users/rowen/python/matplotlibMemoryLeak.
py> with various lines commented out (listed before the output). All of 
these are on Mac OS X using python.org Python 2.6.6 and TkAgg with 
Tcl/Tk 8.4.19.

I have confirmed the leak on linux using matplotlib 1.0.0 (indeed it is 
even worse on linux than on Mac OS X -- closer to 28kb/sec than 20) but 
have not tried building svn versions on linux.

-- Russell

matplotlib 1.0.0

        self.subplot.set_xlim(tMin, tMax)
#         self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27676       nan
   5.0    28980       nan
  10.1    29084      20.6
  15.1    29172      19.0
  20.1    29268      19.1
  25.2    29368      19.3
  30.2    29464      19.2
  35.2    29564      19.4
  40.2    29660      19.3
  45.2    29764      19.5
...
1521.6    58248      19.3
1526.6    58348      19.3

        self.subplot.set_xlim(tMin, tMax)
        self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27696       nan
   5.1    29012       nan
  10.2    29152      27.5
  15.2    29292      27.7
  20.2    29436      28.0
  25.3    29580      28.1
  30.4    29728      28.3
  35.4    29876      28.5
  40.5    30024      28.6
  45.5    30160      28.4
  50.6    30308      28.5


matplotlib svn trunk rev8836

        self.subplot.set_xlim(tMin, tMax)
#        self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27664       nan
   5.0    28864       nan
  10.0    28880       3.2
  15.1    28884       2.0
  20.1    28896       2.1
  25.1    28908       2.2
  30.1    28916       2.1
  35.1    28928       2.1
  40.2    28940       2.2
  45.2    28948       2.1

        self.subplot.set_xlim(tMin, tMax)
        self.subplot.clear()
        self.canvas.draw()
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27684       nan
   5.1    28856       nan
  10.1    28856       0.0
  15.1    28856       0.0
  20.1    28864       0.5
  25.2    28872       0.8
  30.2    28880       1.0
  35.2    28884       0.9
  40.2    28892       1.0
  45.2    28900       1.1
  50.2    28920       1.4
  55.3    28924       1.4
  60.3    28932       1.4
  65.3    28940       1.4

After applying Michael Droettboom's patch to svn trunk rev8836:
--- lib/matplotlib/text.py (revision 8819)
+++ lib/matplotlib/text.py (working copy)
@@ -143,6 +143,9 @@
    Handle storing and drawing of text in window or data coordinates.
    """
    zorder = 3
+
+    cached = maxdict(50)
+
    def __str__(self):
        return "Text(%g,%g,%s)"%(self._y,self._y,repr(self._text))

@@ -168,7 +171,6 @@
        """

        Artist.__init__(self)
-        self.cached = maxdict(5)
        self._x, self._y = x, y

        if color is None: color = rcParams['text.color']


        # the preferred code
        self.subplot.set_xlim(tMin, tMax)
#        self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27656       nan
   5.0    28852       nan
  10.1    28864       2.4
  15.1    28876       2.4
  20.1    28888       2.4
  25.1    28908       2.8
  30.1    28916       2.6
  35.1    28924       2.4
  40.2    28936       2.4
...
 803.1    30296       1.8
 808.1    30304       1.8
...
2625.1    33628       1.8
2630.1    33640       1.8
...
4186.1    36476       1.8
4191.2    36488       1.8
...
6228.9    40228       1.8
6233.9    40236       1.8

        self.subplot.set_xlim(tMin, tMax)
        self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27652       nan
   5.0    28836       nan
  10.0    28852       3.2
  15.1    28852       1.6
  20.1    28872       2.4
  25.1    28880       2.2
  30.1    28900       2.6
  35.1    28904       2.3
  40.2    28912       2.2
  45.2    28920       2.1
  50.3    28928       2.0

#         self.subplot.set_xlim(tMin, tMax)
#         self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27636       nan
   5.0    28832       nan
  10.0    28840       1.6
  15.1    28840       0.8
  20.1    28852       1.3
  25.1    28860       1.4
  30.1    28868       1.4
  35.1    28876       1.5
  40.1    28888       1.6
  45.1    28908       1.9
...
 447.0    29620       1.8
 452.1    29628       1.8
...
1180.3    30940       1.8
1185.3    30948       1.8


In article <4D0...@st...>,
 Michael Droettboom <md...@st...> 
 wrote:

> If you're able to try this with the 1.0.x SVN branch or the SVN trunk 
> (plus my text patch) let me know.  I am not able to reproduce any sort 
> of memory leak with these newer versions, but I am able to with 1.0.0 as 
> released (with or without my text patch).  This is with or without the x 
> axis limits updating you suggested.  I believe there have been other 
> memory leak fixes since the 1.0.0 release.
> 
> Cheers,
> Mike
> 
> On 12/13/2010 08:35 PM, Russell E. Owen wrote:
> > I also wanted to say:
> > - Thank you for the patch. I appreciate the chance to try a fix.
> > - I doubt the issue is related to text because my scripts shows a memory
> > leak even if the displayed text is never updated (comment out the line
> > that modifies the x axis limits to see what I mean; though I confess I
> > did not try that with the trunk matplotlib, only with 1.0.0).
> >
> > -- Russell
> >
> > In article<4D0...@st...>,
> >   Michael Droettboom<md...@st...>
> >   wrote:
> >
> >    
> >> I think I'm on to something -- it seems that text layout information has
> >> a cyclical reference that prevents the Text object from being freed.
> >>
> >> Can you apply the attached patch and let me know if it solves your issue?
> >>      
> > ...
> >
> >
> > ----------------------------------------------------------------------------
> > --
> > Lotusphere 2011
> > Register now for Lotusphere 2011 and learn how
> > to connect the dots, take your collaborative environment
> > to the next level, and enter the era of Social Business.
> > http://p.sf.net/sfu/lotusphere-d2d
> > _______________________________________________
> > Matplotlib-devel mailing list
> > Mat...@li...
> > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> >

I

If you're able to try this with the 1.0.x SVN branch or the SVN trunk 
(plus my text patch) let me know.  I am not able to reproduce any sort 
of memory leak with these newer versions, but I am able to with 1.0.0 as 
released (with or without my text patch).  This is with or without the x 
axis limits updating you suggested.  I believe there have been other 
memory leak fixes since the 1.0.0 release.

Cheers,
Mike

On 12/13/2010 08:35 PM, Russell E. Owen wrote:
> I also wanted to say:
> - Thank you for the patch. I appreciate the chance to try a fix.
> - I doubt the issue is related to text because my scripts shows a memory
> leak even if the displayed text is never updated (comment out the line
> that modifies the x axis limits to see what I mean; though I confess I
> did not try that with the trunk matplotlib, only with 1.0.0).
>
> -- Russell
>
> In article<4D0...@st...>,
>   Michael Droettboom<md...@st...>
>   wrote:
>
>    
>> I think I'm on to something -- it seems that text layout information has
>> a cyclical reference that prevents the Text object from being freed.
>>
>> Can you apply the attached patch and let me know if it solves your issue?
>>      
> ...
>
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

Thanks!

Ben

On Tue, Dec 14, 2010 at 9:44 AM, Michael Droettboom <md...@st...> wrote:

>  You need to be granted those permissions in the SourceForge admin
> interface.  I've just gone ahead and done so for you.
>
> Mike
>
>
> On 12/13/2010 04:23 PM, Benjamin Root wrote:
>
> Does some other setting needs to be made to allow me to change the status
> of bug reports in sourceforge?  I can't close reports or assign them to
> myself.
>
> Thanks,
> Ben Root
>
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.http://p.sf.net/sfu/lotusphere-d2d
>
>
> _______________________________________________
> Matplotlib-devel mailing lis...@li...://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
>
>
> --
> Michael Droettboom
> Science Software Branch
> Space Telescope Science Institute
> Baltimore, Maryland, USA
>
>
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>
>

You need to be granted those permissions in the SourceForge admin 
interface.  I've just gone ahead and done so for you.

Mike

On 12/13/2010 04:23 PM, Benjamin Root wrote:
> Does some other setting needs to be made to allow me to change the 
> status of bug reports in sourceforge?  I can't close reports or assign 
> them to myself.
>
> Thanks,
> Ben Root
>
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>    


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA

I also wanted to say:
- Thank you for the patch. I appreciate the chance to try a fix.
- I doubt the issue is related to text because my scripts shows a memory 
leak even if the displayed text is never updated (comment out the line 
that modifies the x axis limits to see what I mean; though I confess I 
did not try that with the trunk matplotlib, only with 1.0.0).

-- Russell

In article <4D0...@st...>,
 Michael Droettboom <md...@st...> 
 wrote:

> I think I'm on to something -- it seems that text layout information has 
> a cyclical reference that prevents the Text object from being freed.
> 
> Can you apply the attached patch and let me know if it solves your issue?
...

I'm afraid your change didn't help (I only applied the patch to 
lib/matplotlib/text.py; the cbook.py change appeared to only affect 
whitespace).

However, the memory leak is smaller using the svn trunk than in 
matplotlib 1.0.0. Also, calling subplot.clear() makes the leak worse in 
matplotlib 1.0.0 but not on the trunk.

Here are my results in detail. Before each run I show a few lines of 
code from:
<http://www.astro.washington.edu/users/rowen/python/matplotlibMemoryLeak.
py>
MemoryLeaker's _updateTimeAxis method. Note that the third column is a 
measure of the leak rate and I hold off measuring the leak rate for few 
seconds to give the application a chance to fully load.

-- Russell

matplotlib 1.0.0

        self.subplot.set_xlim(tMin, tMax)
#         self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27676       nan
   5.0    28980       nan
  10.1    29084      20.6
  15.1    29172      19.0
  20.1    29268      19.1
  25.2    29368      19.3
  30.2    29464      19.2
  35.2    29564      19.4
  40.2    29660      19.3
  45.2    29764      19.5

        self.subplot.set_xlim(tMin, tMax)
        self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27696       nan
   5.1    29012       nan
  10.2    29152      27.5
  15.2    29292      27.7
  20.2    29436      28.0
  25.3    29580      28.1
  30.4    29728      28.3
  35.4    29876      28.5
  40.5    30024      28.6
  45.5    30160      28.4
  50.6    30308      28.5


matplotlib svn trunk rev8836

        self.subplot.set_xlim(tMin, tMax)
#        self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27664       nan
   5.0    28864       nan
  10.0    28880       3.2
  15.1    28884       2.0
  20.1    28896       2.1
  25.1    28908       2.2
  30.1    28916       2.1
  35.1    28928       2.1
  40.2    28940       2.2
  45.2    28948       2.1

        self.subplot.set_xlim(tMin, tMax)
        self.subplot.clear()
        self.canvas.draw()
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27684       nan
   5.1    28856       nan
  10.1    28856       0.0
  15.1    28856       0.0
  20.1    28864       0.5
  25.2    28872       0.8
  30.2    28880       1.0
  35.2    28884       0.9
  40.2    28892       1.0
  45.2    28900       1.1
  50.2    28920       1.4
  55.3    28924       1.4
  60.3    28932       1.4
  65.3    28940       1.4

After applying your patch to svn trunk rev8836:
--- lib/matplotlib/text.py (revision 8819)
+++ lib/matplotlib/text.py (working copy)
@@ -143,6 +143,9 @@
    Handle storing and drawing of text in window or data coordinates.
    """
    zorder = 3
+
+    cached = maxdict(50)
+
    def __str__(self):
        return "Text(%g,%g,%s)"%(self._y,self._y,repr(self._text))

@@ -168,7 +171,6 @@
        """

        Artist.__init__(self)
-        self.cached = maxdict(5)
        self._x, self._y = x, y

        if color is None: color = rcParams['text.color']



        self.subplot.set_xlim(tMin, tMax)
#        self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27656       nan
   5.0    28852       nan
  10.1    28864       2.4
  15.1    28876       2.4
  20.1    28888       2.4
  25.1    28908       2.8
  30.1    28916       2.6
  35.1    28924       2.4
  40.2    28936       2.4

        self.subplot.set_xlim(tMin, tMax)
        self.subplot.clear()
        self.canvas.draw()
rowen:Python bugs rowen$ python matplotlibMemoryLeak.py 
time   rss memory    mem/sec
(sec)     (kb)      (kb/sec)
   0.0    27652       nan
   5.0    28836       nan
  10.0    28852       3.2
  15.1    28852       1.6
  20.1    28872       2.4
  25.1    28880       2.2
  30.1    28900       2.6
  35.1    28904       2.3
  40.2    28912       2.2
  45.2    28920       2.1
  50.3    28928       2.0


In article <4D0...@st...>,
 Michael Droettboom <md...@st...> 
 wrote:

> I think I'm on to something -- it seems that text layout information has 
> a cyclical reference that prevents the Text object from being freed.
> 
> Can you apply the attached patch and let me know if it solves your issue?
> 
> Mike
> 
> On 12/10/2010 02:19 PM, Russell E. Owen wrote:
> > You may have already seen this in the general mailing list, but I've
> > found what I think is a serious memory leak in matplotlib 1.0.0: it
> > leaks memory every time canvas.draw() is called, at least when using
> > TkAgg on unix and Mac.
> >
> > Admittedly many graphs do not need canvas.draw() to be called repeatedly
> > (which I suspect is how it has survived this long). This came up in the
> > context of a strip chart widget, where I am changing the x/time axis
> > limits regularly and calling canvas.draw() so that the change is visible.
> >
> > I submitted ticket 3124990 with a very simple demo script:
> > <https://sourceforge.net/tracker/?func=detail&atid=560720&aid=3124990&gro
> > up_id=80706>
> > (you can disable the setting the x limits if you want to see the leak in
> > its purest form, but then nothing changes visually on the graph).
> >
> > I just wanted to be sure folks know about it in hopes somebody might
> > have an idea how to fix it. I have not tried any other back ends.
> >
> > -- Russell
> >
> >
> > ----------------------------------------------------------------------------
> > --
> > Oracle to DB2 Conversion Guide: Learn learn about native support for 
> > PL/SQL,
> > new data types, scalar functions, improved concurrency, built-in packages,
> > OCI, SQL*Plus, data movement tools, best practices and more.
> > http://p.sf.net/sfu/oracle-sfdev2dev
> > _______________________________________________
> > Matplotlib-devel mailing list
> > Mat...@li...
> > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
> >    
> 
> 
> -- 
> Michael Droettboom
> Science Software Branch
> Space Telescope Science Institute
> Baltimore, Maryland, USA
> 
> ---------------------------------------------------------------------
> Index: lib/matplotlib/cbook.py
> ===================================================================
> --- lib/matplotlib/cbook.py	(revision 8819)
> +++ lib/matplotlib/cbook.py	(working copy)
> @@ -1109,9 +1109,10 @@
>          dict.__init__(self)
>          self.maxsize = maxsize
>          self._killkeys = []
> +
>      def __setitem__(self, k, v):
>          if k not in self:
> -            if len(self)>=self.maxsize:
> +            if len(self) >= self.maxsize:
>                  del self[self._killkeys[0]]
>                  del self._killkeys[0]
>              self._killkeys.append(k)
> Index: lib/matplotlib/text.py
> ===================================================================
> --- lib/matplotlib/text.py	(revision 8819)
> +++ lib/matplotlib/text.py	(working copy)
> @@ -143,6 +143,9 @@
>      Handle storing and drawing of text in window or data coordinates.
>      """
>      zorder = 3
> +
> +    cached = maxdict(50)
> +
>      def __str__(self):
>          return "Text(%g,%g,%s)"%(self._y,self._y,repr(self._text))
>  
> @@ -168,7 +171,6 @@
>          """
>  
>          Artist.__init__(self)
> -        self.cached = maxdict(5)
>          self._x, self._y = x, y
>  
>          if color is None: color = rcParams['text.color']
> ---------------------------------------------------------------------
> ------------------------------------------------------------------------------
> Oracle to DB2 Conversion Guide: Learn learn about native support for PL/SQL,
> new data types, scalar functions, improved concurrency, built-in packages, 
> OCI, SQL*Plus, data movement tools, best practices and more.
> http://p.sf.net/sfu/oracle-sfdev2dev 
> ---------------------------------------------------------------------
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel