matplotlib-devel Mailing List for matplotlib

On 6/30/2012 1:24 PM, Christoph Gohlke wrote:
> On 6/30/2012 12:55 PM, John Hunter wrote:
>>
>>
>> On Sat, Jun 30, 2012 at 2:40 PM, Fernando Perez <fpe...@gm...
>> <mailto:fpe...@gm...>> wrote:
>>
>>      On Sat, Jun 30, 2012 at 11:46 AM, John Hunter <jd...@gm...
>>      <mailto:jd...@gm...>> wrote:
>>      > Well, looks like we better get moving then ;-)
>>
>>      Go MPL!  It would be great to have matching releases of IPython and
>>      MPL, just in time for the Debian freeze and SciPy 2012 :)
>>
>>
>> OK, the v1.1.1 tarball is up at
>> https://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.1.1
>> and this is now the download folder the main site points to.  I'm
>> leaving up the rc2 binaries til Russell and Christoph can build v1.1.1
>> binaries and we get them uploaded.  Sandro, if you're around, you are
>> good to go for including this in debian, hopefully squeaking in under
>> the freeze (sorry for the last minute push).
>>
>> I will hold off on the users and announce list emails til the updated
>> binaries are up.
>>
>> Tagged: git tag v1.1.1 7e47149a7b05f8e5cf1cc899a7e4e7c90dd4244f
>>
>> Thanks to all!
>> JDH
>
> Here are the Windows installers, built against numpy 1.6.2. Sorry, I can
> not upload them to SF. There seems to be some permission problems that
> the SF admins would need to fix manually.
>
> <http://www.lfd.uci.edu/~gohlke/pythonlibs/#matplotlib>
>
> Christoph
>

Sourceforge has fixed the permissions. I uploaded the Windows installers.

Christoph

On Tue, Jul 3, 2012 at 10:03 AM, Tony Yu <ts...@gm...> wrote:
>
>
> On Tue, Jul 3, 2012 at 10:22 AM, Ryan May <rm...@gm...> wrote:
>>
>> On Tue, Jul 3, 2012 at 9:14 AM, Tony Yu <ts...@gm...> wrote:
>> > On Mon, Jul 2, 2012 at 11:42 PM, Tony Yu <ts...@gm...> wrote:
>> >> On Sun, Jun 24, 2012 at 10:02 AM, Tony Yu <ts...@gm...> wrote:
>> >>>
>> >>> I'm having a tough time figuring this out: Saving an animation seems
>> >>> to
>> >>> hang when using `streamplot`. The exact same animation runs without
>> >>> issue
>> >>> when calling show.
>> >>>
>> >>> On my system, the example below hangs consistently at frame 173.
>> >>> However,
>> >>> the number of saved frames (before hanging) varies with density of
>> >>> stream
>> >>> lines (i.e. number of line segments in streamplot): More frames saved
>> >>> for
>> >>> lower density.
>> >>>
>> >>> Seems to be a memory or file-size issue, but
>> >>>
>> >>>    * Memory use doesn't seem to grow. (this suggests that ffmpeg adds
>> >>> frames to disk as opposed to memory.)
>> >>>    * Saving the animation up to the frame just before it hangs gives
>> >>> modest (~1MB) videos.
>> >>> Maybe this is a limitation of `ffmpeg`?
>> >>>
>> >>> Can someone confirm that the example hangs on their system? (Somehow,
>> >>> I
>> >>> often run into bugs that are not reproducible.)
>> >>>
>> >>> In case this is system dependent:
>> >>> * OSX 10.6
>> >>> * Python 2.6.1 (system install)
>> >>> * matplotlib master
>> >>> * ffmpeg 0.10
>> >>>
>> >>> Simple plots seem to work fine, but I should probably try to reproduce
>> >>> using something other than streamplot (maybe create the equivalent
>> >>> number of
>> >>> line and arrow collections).
>> >>>
>> >>> -Tony
>> >>>
>> >>> #~~~ Example
>> >>> import numpy as np
>> >>> import matplotlib.pyplot as plt
>> >>> import matplotlib.animation as animation
>> >>>
>> >>> fig, ax = plt.subplots()
>> >>>
>> >>> # do nothing function that prints frame
>> >>> def update(frame_num):
>> >>>     print frame_num
>> >>>     return []
>> >>>
>> >>> Y, X = np.mgrid[:1:100j, :1:100j]
>> >>> U = np.ones_like(X)
>> >>> V = np.ones_like(X)
>> >>> ax.streamplot(X, Y, U, V, density=1)
>> >>>
>> >>> ani = animation.FuncAnimation(fig, update, save_count=500)
>> >>>
>> >>> # save hangs at frame 173 (this probably varies by system, if it's
>> >>> reproducible).
>> >>> ani.save('animation.avi')
>> >>> # show animates all frames w/o issue.
>> >>> #plt.show()
>> >>>
>> >>>
>> >>
>> >> I finally had some time to revisit this issue. It seems that subprocess
>> >> PIPEs have fairly limited buffers [1] (also see docs for
>> >> `subprocess.Popen.communicate` [2]) and ffmpeg tends to generate a
>> >> decent
>> >> amount of output to stderr. A simple solution is to redirect stderr to
>> >> a
>> >> temporary file. This change fixes the issue on my system.
>> >>
>> >> If this bug is reproducible, I can submit a PR. The temporary file here
>> >> could be created using tempfile so it gets cleaned up automatically, or
>> >> maybe a more permanent log file for debugging. Thoughts?
>> >>
>> >> -Tony
>> >>
>> >> [1]
>> >>
>> >> http://thraxil.org/users/anders/posts/2008/03/13/Subprocess-Hanging-PIPE-is-your-enemy/
>> >>
>> >> [2]
>> >>
>> >> http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
>> >>
>> >
>> > And just to clarify: My original email mentioned that saving would hang
>> > only
>> > when `streamplot` was used to create the figure. It turns out that
>> > ffmpeg
>> > prints more output (keyframes?) for more complex images, so when I ran
>> > simpler plots, they didn't produce enough output to fill the subprocess
>> > `PIPE` (although they would eventually).
>> >
>> > Also, theres's a simpler fix than piping ffmpeg output to a file: Just
>> > set
>> > `-loglevel quiet` in the ffmpeg command.
>> >
>> > -Tony
>>
>> It's not a bad idea to have it logged to a file in a temp directory.
>> We could potentially just do this if debug is set to verbose, however,
>> and just use -loglevel quiet otherwise. Can you manage PR for this or
>> do I need to?
>>
>> Ryan
>>
>
> Hey Ryan,
>
> If you have time, that'd be great. Otherwise, I should have some time at the
> end of the week to submit a PR.
>
> -Tony
>

I might be able to squeeze it in. If you don't see anything by the end
of the week, hit me up.

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

On Tue, Jul 3, 2012 at 10:22 AM, Ryan May <rm...@gm...> wrote:

> On Tue, Jul 3, 2012 at 9:14 AM, Tony Yu <ts...@gm...> wrote:
> > On Mon, Jul 2, 2012 at 11:42 PM, Tony Yu <ts...@gm...> wrote:
> >> On Sun, Jun 24, 2012 at 10:02 AM, Tony Yu <ts...@gm...> wrote:
> >>>
> >>> I'm having a tough time figuring this out: Saving an animation seems to
> >>> hang when using `streamplot`. The exact same animation runs without
> issue
> >>> when calling show.
> >>>
> >>> On my system, the example below hangs consistently at frame 173.
> However,
> >>> the number of saved frames (before hanging) varies with density of
> stream
> >>> lines (i.e. number of line segments in streamplot): More frames saved
> for
> >>> lower density.
> >>>
> >>> Seems to be a memory or file-size issue, but
> >>>
> >>>    * Memory use doesn't seem to grow. (this suggests that ffmpeg adds
> >>> frames to disk as opposed to memory.)
> >>>    * Saving the animation up to the frame just before it hangs gives
> >>> modest (~1MB) videos.
> >>> Maybe this is a limitation of `ffmpeg`?
> >>>
> >>> Can someone confirm that the example hangs on their system? (Somehow, I
> >>> often run into bugs that are not reproducible.)
> >>>
> >>> In case this is system dependent:
> >>> * OSX 10.6
> >>> * Python 2.6.1 (system install)
> >>> * matplotlib master
> >>> * ffmpeg 0.10
> >>>
> >>> Simple plots seem to work fine, but I should probably try to reproduce
> >>> using something other than streamplot (maybe create the equivalent
> number of
> >>> line and arrow collections).
> >>>
> >>> -Tony
> >>>
> >>> #~~~ Example
> >>> import numpy as np
> >>> import matplotlib.pyplot as plt
> >>> import matplotlib.animation as animation
> >>>
> >>> fig, ax = plt.subplots()
> >>>
> >>> # do nothing function that prints frame
> >>> def update(frame_num):
> >>>     print frame_num
> >>>     return []
> >>>
> >>> Y, X = np.mgrid[:1:100j, :1:100j]
> >>> U = np.ones_like(X)
> >>> V = np.ones_like(X)
> >>> ax.streamplot(X, Y, U, V, density=1)
> >>>
> >>> ani = animation.FuncAnimation(fig, update, save_count=500)
> >>>
> >>> # save hangs at frame 173 (this probably varies by system, if it's
> >>> reproducible).
> >>> ani.save('animation.avi')
> >>> # show animates all frames w/o issue.
> >>> #plt.show()
> >>>
> >>>
> >>
> >> I finally had some time to revisit this issue. It seems that subprocess
> >> PIPEs have fairly limited buffers [1] (also see docs for
> >> `subprocess.Popen.communicate` [2]) and ffmpeg tends to generate a
> decent
> >> amount of output to stderr. A simple solution is to redirect stderr to a
> >> temporary file. This change fixes the issue on my system.
> >>
> >> If this bug is reproducible, I can submit a PR. The temporary file here
> >> could be created using tempfile so it gets cleaned up automatically, or
> >> maybe a more permanent log file for debugging. Thoughts?
> >>
> >> -Tony
> >>
> >> [1]
> >>
> http://thraxil.org/users/anders/posts/2008/03/13/Subprocess-Hanging-PIPE-is-your-enemy/
> >>
> >> [2]
> >>
> http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
> >>
> >
> > And just to clarify: My original email mentioned that saving would hang
> only
> > when `streamplot` was used to create the figure. It turns out that ffmpeg
> > prints more output (keyframes?) for more complex images, so when I ran
> > simpler plots, they didn't produce enough output to fill the subprocess
> > `PIPE` (although they would eventually).
> >
> > Also, theres's a simpler fix than piping ffmpeg output to a file: Just
> set
> > `-loglevel quiet` in the ffmpeg command.
> >
> > -Tony
>
> It's not a bad idea to have it logged to a file in a temp directory.
> We could potentially just do this if debug is set to verbose, however,
> and just use -loglevel quiet otherwise. Can you manage PR for this or
> do I need to?
>
> Ryan
>
>
Hey Ryan,

If you have time, that'd be great. Otherwise, I should have some time at
the end of the week to submit a PR.

-Tony

On Tue, Jul 3, 2012 at 9:14 AM, Tony Yu <ts...@gm...> wrote:
> On Mon, Jul 2, 2012 at 11:42 PM, Tony Yu <ts...@gm...> wrote:
>> On Sun, Jun 24, 2012 at 10:02 AM, Tony Yu <ts...@gm...> wrote:
>>>
>>> I'm having a tough time figuring this out: Saving an animation seems to
>>> hang when using `streamplot`. The exact same animation runs without issue
>>> when calling show.
>>>
>>> On my system, the example below hangs consistently at frame 173. However,
>>> the number of saved frames (before hanging) varies with density of stream
>>> lines (i.e. number of line segments in streamplot): More frames saved for
>>> lower density.
>>>
>>> Seems to be a memory or file-size issue, but
>>>
>>>    * Memory use doesn't seem to grow. (this suggests that ffmpeg adds
>>> frames to disk as opposed to memory.)
>>>    * Saving the animation up to the frame just before it hangs gives
>>> modest (~1MB) videos.
>>> Maybe this is a limitation of `ffmpeg`?
>>>
>>> Can someone confirm that the example hangs on their system? (Somehow, I
>>> often run into bugs that are not reproducible.)
>>>
>>> In case this is system dependent:
>>> * OSX 10.6
>>> * Python 2.6.1 (system install)
>>> * matplotlib master
>>> * ffmpeg 0.10
>>>
>>> Simple plots seem to work fine, but I should probably try to reproduce
>>> using something other than streamplot (maybe create the equivalent number of
>>> line and arrow collections).
>>>
>>> -Tony
>>>
>>> #~~~ Example
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> import matplotlib.animation as animation
>>>
>>> fig, ax = plt.subplots()
>>>
>>> # do nothing function that prints frame
>>> def update(frame_num):
>>>     print frame_num
>>>     return []
>>>
>>> Y, X = np.mgrid[:1:100j, :1:100j]
>>> U = np.ones_like(X)
>>> V = np.ones_like(X)
>>> ax.streamplot(X, Y, U, V, density=1)
>>>
>>> ani = animation.FuncAnimation(fig, update, save_count=500)
>>>
>>> # save hangs at frame 173 (this probably varies by system, if it's
>>> reproducible).
>>> ani.save('animation.avi')
>>> # show animates all frames w/o issue.
>>> #plt.show()
>>>
>>>
>>
>> I finally had some time to revisit this issue. It seems that subprocess
>> PIPEs have fairly limited buffers [1] (also see docs for
>> `subprocess.Popen.communicate` [2]) and ffmpeg tends to generate a decent
>> amount of output to stderr. A simple solution is to redirect stderr to a
>> temporary file. This change fixes the issue on my system.
>>
>> If this bug is reproducible, I can submit a PR. The temporary file here
>> could be created using tempfile so it gets cleaned up automatically, or
>> maybe a more permanent log file for debugging. Thoughts?
>>
>> -Tony
>>
>> [1]
>> http://thraxil.org/users/anders/posts/2008/03/13/Subprocess-Hanging-PIPE-is-your-enemy/
>>
>> [2]
>> http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
>>
>
> And just to clarify: My original email mentioned that saving would hang only
> when `streamplot` was used to create the figure. It turns out that ffmpeg
> prints more output (keyframes?) for more complex images, so when I ran
> simpler plots, they didn't produce enough output to fill the subprocess
> `PIPE` (although they would eventually).
>
> Also, theres's a simpler fix than piping ffmpeg output to a file: Just set
> `-loglevel quiet` in the ffmpeg command.
>
> -Tony

It's not a bad idea to have it logged to a file in a temp directory.
We could potentially just do this if debug is set to verbose, however,
and just use -loglevel quiet otherwise. Can you manage PR for this or
do I need to?

Ryan

-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

On Mon, Jul 2, 2012 at 11:42 PM, Tony Yu <ts...@gm...> wrote:

>
>
> On Sun, Jun 24, 2012 at 10:02 AM, Tony Yu <ts...@gm...> wrote:
>
>> I'm having a tough time figuring this out: Saving an animation seems to
>> hang when using `streamplot`. The exact same animation runs without issue
>> when calling show.
>>
>> On my system, the example below hangs consistently at frame 173. However,
>> the number of saved frames (before hanging) varies with density of stream
>> lines (i.e. number of line segments in streamplot): More frames saved for
>> lower density.
>>
>> Seems to be a memory or file-size issue, but
>>
>>    * Memory use doesn't seem to grow. (this suggests that ffmpeg adds
>> frames to disk as opposed to memory.)
>>    * Saving the animation up to the frame just before it hangs gives
>> modest (~1MB) videos.
>> Maybe this is a limitation of `ffmpeg`?
>>
>> Can someone confirm that the example hangs on their system? (Somehow, I
>> often run into bugs that are not reproducible.)
>>
>> In case this is system dependent:
>> * OSX 10.6
>> * Python 2.6.1 (system install)
>> * matplotlib master
>> * ffmpeg 0.10
>>
>> Simple plots seem to work fine, but I should probably try to reproduce
>> using something other than streamplot (maybe create the equivalent number
>> of line and arrow collections).
>>
>> -Tony
>>
>> #~~~ Example
>> import numpy as np
>> import matplotlib.pyplot as plt
>> import matplotlib.animation as animation
>>
>> fig, ax = plt.subplots()
>>
>> # do nothing function that prints frame
>> def update(frame_num):
>>     print frame_num
>>     return []
>>
>> Y, X = np.mgrid[:1:100j, :1:100j]
>> U = np.ones_like(X)
>> V = np.ones_like(X)
>> ax.streamplot(X, Y, U, V, density=1)
>>
>> ani = animation.FuncAnimation(fig, update, save_count=500)
>>
>> # save hangs at frame 173 (this probably varies by system, if it's
>> reproducible).
>> ani.save('animation.avi')
>> # show animates all frames w/o issue.
>> #plt.show()
>>
>>
>>
> I finally had some time to revisit this issue. It seems that subprocess
> PIPEs have fairly limited buffers [1] (also see docs for
> `subprocess.Popen.communicate` [2]) and ffmpeg tends to generate a decent
> amount of output to stderr. A simple solution is to redirect stderr to a
> temporary file. This change fixes the issue on my system.
>
> If this bug is reproducible, I can submit a PR. The temporary file here
> could be created using tempfile so it gets cleaned up automatically, or
> maybe a more permanent log file for debugging. Thoughts?
>
> -Tony
>
> [1]
> http://thraxil.org/users/anders/posts/2008/03/13/Subprocess-Hanging-PIPE-is-your-enemy/
>
> [2]
> http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
>
>
And just to clarify: My original email mentioned that saving would hang
only when `streamplot` was used to create the figure. It turns out that
ffmpeg prints more output (keyframes?) for more complex images, so when I
ran simpler plots, they didn't produce enough output to fill the subprocess
`PIPE` (although they would eventually).

Also, theres's a simpler fix than piping ffmpeg output to a file: Just
set `-loglevel quiet` in the ffmpeg command.

-Tony

On Sun, Jun 24, 2012 at 10:02 AM, Tony Yu <ts...@gm...> wrote:

> I'm having a tough time figuring this out: Saving an animation seems to
> hang when using `streamplot`. The exact same animation runs without issue
> when calling show.
>
> On my system, the example below hangs consistently at frame 173. However,
> the number of saved frames (before hanging) varies with density of stream
> lines (i.e. number of line segments in streamplot): More frames saved for
> lower density.
>
> Seems to be a memory or file-size issue, but
>
>    * Memory use doesn't seem to grow. (this suggests that ffmpeg adds
> frames to disk as opposed to memory.)
>    * Saving the animation up to the frame just before it hangs gives
> modest (~1MB) videos.
> Maybe this is a limitation of `ffmpeg`?
>
> Can someone confirm that the example hangs on their system? (Somehow, I
> often run into bugs that are not reproducible.)
>
> In case this is system dependent:
> * OSX 10.6
> * Python 2.6.1 (system install)
> * matplotlib master
> * ffmpeg 0.10
>
> Simple plots seem to work fine, but I should probably try to reproduce
> using something other than streamplot (maybe create the equivalent number
> of line and arrow collections).
>
> -Tony
>
> #~~~ Example
> import numpy as np
> import matplotlib.pyplot as plt
> import matplotlib.animation as animation
>
> fig, ax = plt.subplots()
>
> # do nothing function that prints frame
> def update(frame_num):
>     print frame_num
>     return []
>
> Y, X = np.mgrid[:1:100j, :1:100j]
> U = np.ones_like(X)
> V = np.ones_like(X)
> ax.streamplot(X, Y, U, V, density=1)
>
> ani = animation.FuncAnimation(fig, update, save_count=500)
>
> # save hangs at frame 173 (this probably varies by system, if it's
> reproducible).
> ani.save('animation.avi')
> # show animates all frames w/o issue.
> #plt.show()
>
>
>
I finally had some time to revisit this issue. It seems that subprocess
PIPEs have fairly limited buffers [1] (also see docs for
`subprocess.Popen.communicate` [2]) and ffmpeg tends to generate a decent
amount of output to stderr. A simple solution is to redirect stderr to a
temporary file. This change fixes the issue on my system.

If this bug is reproducible, I can submit a PR. The temporary file here
could be created using tempfile so it gets cleaned up automatically, or
maybe a more permanent log file for debugging. Thoughts?

-Tony

[1]
http://thraxil.org/users/anders/posts/2008/03/13/Subprocess-Hanging-PIPE-is-your-enemy/

[2]
http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate