matplotlib-devel Mailing List for matplotlib

False alarm. The Travis logs includes (but hides) the install output, and
test_mlab was running for me, but I was looking at the wrong line
numbers.   Still though, I would be curious as to any differences, but I
can't seem to download the log file.

Ben


On Thu, Jun 26, 2014 at 6:57 PM, Benjamin Root <ben...@ou...> wrote:

> Just noticed an oddity with the tests on Travis versus the tests on my
> machine. The test log on Travis for a single run has over 10,000 lines.
> But, for me, it is over ~4800 lines. At a glance, I can see that test_mlab
> is not executed for me, but they are for Travis. I am very suspicious of
> the test_mlab run on Travis because it seems to be running multiple times,
> but I can't be sure.
>
> Michael, can I get the test log for one of the recent Travis runs?
>
> Thanks,
> Ben Root
>
>
>
> On Wed, Jun 4, 2014 at 1:49 PM, Benjamin Root <ben...@ou...> wrote:
>
>> So, I just tried comparing memory usage for a plot displayed via show()
>> versus savefig() as a PNG. It would seem that saving to pngs uses more
>> memory. Not sure why, though.
>>
>> Ben
>> On Jun 4, 2014 12:57 PM, "Eric Firing" <ef...@ha...> wrote:
>>
>>> On 2014/06/04 6:26 AM, Benjamin Root wrote:
>>>
>>>> A theory...
>>>>
>>>> If I remember correctly, the nosttests was set up to execute in parallel
>>>> using the default Multiprocessing settings, which is to have a process
>>>> worker for each available CPU core. Perhaps this might be the crux of
>>>> the issue with so many simultaneous tests running that the amount of
>>>> memory used at the same time becomes too large. Or, am I thinking of the
>>>> doc build system?
>>>>
>>>> Ben Root
>>>>
>>>
>>> Ben,
>>>
>>> Top shows a single process.  The VM is configured with 2 cores.
>>>
>>> Eric
>>>
>>
>

Just noticed an oddity with the tests on Travis versus the tests on my
machine. The test log on Travis for a single run has over 10,000 lines.
But, for me, it is over ~4800 lines. At a glance, I can see that test_mlab
is not executed for me, but they are for Travis. I am very suspicious of
the test_mlab run on Travis because it seems to be running multiple times,
but I can't be sure.

Michael, can I get the test log for one of the recent Travis runs?

Thanks,
Ben Root



On Wed, Jun 4, 2014 at 1:49 PM, Benjamin Root <ben...@ou...> wrote:

> So, I just tried comparing memory usage for a plot displayed via show()
> versus savefig() as a PNG. It would seem that saving to pngs uses more
> memory. Not sure why, though.
>
> Ben
> On Jun 4, 2014 12:57 PM, "Eric Firing" <ef...@ha...> wrote:
>
>> On 2014/06/04 6:26 AM, Benjamin Root wrote:
>>
>>> A theory...
>>>
>>> If I remember correctly, the nosttests was set up to execute in parallel
>>> using the default Multiprocessing settings, which is to have a process
>>> worker for each available CPU core. Perhaps this might be the crux of
>>> the issue with so many simultaneous tests running that the amount of
>>> memory used at the same time becomes too large. Or, am I thinking of the
>>> doc build system?
>>>
>>> Ben Root
>>>
>>
>> Ben,
>>
>> Top shows a single process.  The VM is configured with 2 cores.
>>
>> Eric
>>
>

This is very interesting. I have also noticed that ticking has been a
bottleneck in rendering, but always suspected the computation of the ticks
rather than the actual rendering of the marks. Perhaps this might spur some
renewed interest in identifying the specific reason for the bottleneck and
solving that issue now that we know that there are significant performance
gain potential here?

Looking forward to reviewing this at some point in the next few weeks.

Cheers!
Ben Root



On Thu, Jun 26, 2014 at 2:10 PM, Joel B. Mohler <jo...@ki...>
wrote:

> About a week ago I sent a message to the users mailing list with tick mark
> performance problems.  I now have a proof of concept patch which
> (mis-)uses the
> projection keyword in add_subplot to use a custom Axes class.  Import one
> python file, use "projection='fastticks'" -> boring scatter plots render
> about
> 2x as fast and plots with lots of minor ticks render 6x faster.
>
> The code is at https://github.com/jbmohler/mplfastaxes ; the core idea is
> in
> fastaxes.py which uses a single Line2D for all tick marks of a given flavor
> rather than a Line2D for every single tick mark.
>
> There are two reasons this isn't a total win:
> 1)  it's not done yet in all tick/grid configurations.
> 2)  it loses flexibility in tick labeling
>
> The lost flexibility may matter a great deal to other people.  In my
> use-case,
> the labeling flexibility simply seems misguided and the performance is much
> much preferred.
>
> Comments welcome about how this could move forward toward being included in
> MPL.
>
> Joel
>
>
> ------------------------------------------------------------------------------
> Open source business process management suite built on Java and Eclipse
> Turn processes into business applications with Bonita BPM Community Edition
> Quickly connect people, data, and systems into organized workflows
> Winner of BOSSIE, CODIE, OW2 and Gartner awards
> http://p.sf.net/sfu/Bonitasoft
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

About a week ago I sent a message to the users mailing list with tick mark
performance problems.  I now have a proof of concept patch which (mis-)uses the
projection keyword in add_subplot to use a custom Axes class.  Import one
python file, use "projection='fastticks'" -> boring scatter plots render about
2x as fast and plots with lots of minor ticks render 6x faster.

The code is at https://github.com/jbmohler/mplfastaxes ; the core idea is in
fastaxes.py which uses a single Line2D for all tick marks of a given flavor
rather than a Line2D for every single tick mark.

There are two reasons this isn't a total win:
1)  it's not done yet in all tick/grid configurations.
2)  it loses flexibility in tick labeling

The lost flexibility may matter a great deal to other people.  In my use-case,
the labeling flexibility simply seems misguided and the performance is much
much preferred.

Comments welcome about how this could move forward toward being included in
MPL.

Joel

On 25 June 2014 17:55, Benjamin Root <ben...@ou...> wrote:

> The mplot3d tests are not run automatically, so I have no idea when this
> change happened. But I would suspect it is related to the changes in
> triangulation rather than with mplot3d itself. I have zero expertise to say
> if one result is more correct than the other. See attached.
>
> I should note that this particular difference was within the default
> tolerance. It was the rendering to PDF that seemed to have enough
> difference to trigger a failure.
>

Ben,

You are right, this change occurred when the underlying triangulation code
was switched to using qhull.  Both before and after pictures are equally
correct.

A rectangle in the x-y plane is split into two triangles by adding a
diagonal.  The shortest of the two diagonals should added, so for the
analytical solution either diagonal is equally valid.  In practice the
choice of diagonal depends on the order of operations in the underlying
algorithm and how these interact with finite machine precision.

Rather than just changing the test image, a better solution would be to
modify the relevant example/test code to avoid such degenerate cases that
can give problems in the future.  The source code for trisurf3d_demo (
http://matplotlib.org/examples/mplot3d/trisurf3d_demo.html) is clearly
derived from tripcolor_demo (
http://matplotlib.org/examples/pylab_examples/tripcolor_demo.html), but
omits the key line near the beginning of

angles[:,1::2] += math.pi/n_angles

Putting this line back in will avoid similar problems in the future, and
improve the images too.

Ian