matplotlib-devel Mailing List for matplotlib

>>>>> "Eric" == Eric Firing <ef...@ha...> writes:

    Eric> Here is a simple illustration of the problem and
    Eric> inconsistency, using just the first few points in
    Eric> Tennessee's plot and a few lines from his script.  With a
    Eric> log scale for y, the Agg backends are leaving gaps (breaking
    Eric> the line) at the points where y==0; the PS and SVG backends
    Eric> are removing those points, but *not* breaking the line.  I
    Eric> think the Agg behavior is much better.  But maybe it would
    Eric> be better yet to raise an exception, forcing the user to
    Eric> deal with the error explicitly.  For example, the user might
    Eric> want to change the zero values to a very small number, and
    Eric> then explicitly set the y range to exclude the small
    Eric> numbers.  This is illustrated in the second subplot.

This issue has a long history, which I'll summarize here.  In the old
days, we raised an exception when non-positive numbers were passed to
log plotting.  Most people found this objectionable, and prefer to see
the valid points plotted rather than nothing (though a warning would
be nice).  This is what matlab does.  I needed to make some
architectural changes to support this, because the transformations
happened in the lines class which passed the transformed data to the
backend. 

I looked into filtering the data in the lines class (basically what
you did for ma data) but did not think it would be fast enough,
because we potentially would have had to create a lot of line
segments.

So I changed the backend API and modified draw_lines to take the
transformation as an argument.  Then the backend can do the following
(eg _backend_agg)

    if (needNonlinear)
      try {
	mpltransform->nonlinear_only_api(&thisx, &thisy);
      }
      catch (...) {
	moveto = true;
	continue;
      }

Ie if there is a nonlinear transformation that raises an exception,
the path state is changed from lineto to moveto.  This is very fast,
at least an order of magnitude faster than trying to compute the
segments at the python level I suspect.

At the same time I introduced some optimizations for marker drawing
with a new backend method called draw_markers that also does the
transformations in the backend.  This made marker drawing up to 25
times faster (we used to draw every marker as a separate function call
in lines.py)

We decided to support both old and new style APIs in the "transition
period" which made it easy for other backends to do ..... nothing.  So
that is why the other backends haven't implemented this feature yet.

Steve worked on it a bit for Cairo and Darren and I both did for PS
but never got a working product.  You can see the attempt in
backend_ps _draw_markers method (underscore hidden) and in
backend_cairo's draw_markers_OLD method.  There was extensive
discussion on this list with the API and implementation details so
just search for draw_markers if you are interested.

So the backend is now in what I call a schizophrenic state, in two
ways.  

  1) some backends use the old and some use the "newstyle" API

  2) Even in the newstyle API, some methods do the transformation in
     the front end (draw_rectangle) and some in the backend
     (draw_markers).  

There is a need to simplify, unify and rationalize this API, but since
it mostly works, there has not been a lot of pressure to do so.  And
it would break a lot of backends that may not be actively maintained.

A modest short term goal should be to get the newstyle draw_lines and
draw_markers working for the PS backend, both to fix this log problem
and because it is faster and potentially much smaller in terms of PS
file sizes.  Something for a rainy day.

JDH

John Hunter wrote:
>>>>>>"Tennessee" == Tennessee Leeuwenburg <ten...@te...> writes:
> 
>     Tennessee> That could certainly explain it.
> 
>     Tennessee> I don't know that I agree that it's correct -- I'm not
>     Tennessee> graphing log(X), I'm graphing X, and 0 is a valid value
>     Tennessee> for it. Only the scale is logarithmic, but it still
>     Tennessee> contains a 0. I'll bow to a more expert opinion if
>     Tennessee> there's disagreement.
> 
> If you posst your script, we might be able to give a more informed
> opinion of what the desired output should be :-)  I understand that
> there are some largish data files that you don't want to post, but if
> we could just read the script alone that might be enough.
> 
> JDH


John,

Here is a simple illustration of the problem and inconsistency, using 
just the first few points in Tennessee's plot and a few lines from his 
script.  With a log scale for y, the Agg  backends are leaving gaps 
(breaking the line) at the points where y==0; the PS and SVG backends 
are removing those points, but *not* breaking the line.  I think the Agg 
behavior is much better.  But maybe it would be better yet to raise an 
exception, forcing the user to deal with the error explicitly.  For 
example, the user might want to change the zero values to a very small 
number, and then explicitly set the y range to exclude the small 
numbers.  This is illustrated in the second subplot.

I haven't looked into it, but I suspect the log(0) values are becoming 
nans, so what we are seeing is differences in nan-handling among the 
backends.

Eric

>>>>> "Tennessee" == Tennessee Leeuwenburg <ten...@te...> writes:
    Tennessee> That could certainly explain it.

    Tennessee> I don't know that I agree that it's correct -- I'm not
    Tennessee> graphing log(X), I'm graphing X, and 0 is a valid value
    Tennessee> for it. Only the scale is logarithmic, but it still
    Tennessee> contains a 0. I'll bow to a more expert opinion if
    Tennessee> there's disagreement.

If you posst your script, we might be able to give a more informed
opinion of what the desired output should be :-)  I understand that
there are some largish data files that you don't want to post, but if
we could just read the script alone that might be enough.

JDH

On Mon, 2006-03-13 at 20:08 -1000, Eric Firing wrote:
> Tennessee Leeuwenburg wrote:
> > I've sent Eric the files off-list. Anyone else who wants them, let me
> > know. The bundle is about 400Kb -- not huge but not tiny. The script
> > itself is small, it's just the data file and the output images which
> > take up the space.
> > 
> > The .png and onscreen renderer are both missing line segments, and
> > behave in the same way. The .svg renderer behaves as expected.
> > 
> > I'm running a very up-to-date linux, and I installed matplotlib newly
> > about 4 days ago.
> > 
> > Cheers,
> > -T
> 
> Tennessee,
> 
> The first thing I notice is that you are trying to plot with log-log 
> scales, but some of your y values are zero.  Log(0) is undefined. The 
> Agg renderers are doing something sensible: leaving out the 0 values. 
> It is actually the ps and svg renderers that are incorrect, in that they 
> seem to be substituting a value of 1 for each zero; they are bridging 
> what *should* be gaps.
> 
> Eric

That could certainly explain it. 

I don't know that I agree that it's correct -- I'm not graphing log(X),
I'm graphing X, and 0 is a valid value for it. Only the scale is
logarithmic, but it still contains a 0. I'll bow to a more expert
opinion if there's disagreement.

Cheers,
-T

Tennessee Leeuwenburg wrote:
> I've sent Eric the files off-list. Anyone else who wants them, let me
> know. The bundle is about 400Kb -- not huge but not tiny. The script
> itself is small, it's just the data file and the output images which
> take up the space.
> 
> The .png and onscreen renderer are both missing line segments, and
> behave in the same way. The .svg renderer behaves as expected.
> 
> I'm running a very up-to-date linux, and I installed matplotlib newly
> about 4 days ago.
> 
> Cheers,
> -T

Tennessee,

The first thing I notice is that you are trying to plot with log-log 
scales, but some of your y values are zero.  Log(0) is undefined. The 
Agg renderers are doing something sensible: leaving out the 0 values. 
It is actually the ps and svg renderers that are incorrect, in that they 
seem to be substituting a value of 1 for each zero; they are bridging 
what *should* be gaps.

Eric

I've sent Eric the files off-list. Anyone else who wants them, let me
know. The bundle is about 400Kb -- not huge but not tiny. The script
itself is small, it's just the data file and the output images which
take up the space.

The .png and onscreen renderer are both missing line segments, and
behave in the same way. The .svg renderer behaves as expected.

I'm running a very up-to-date linux, and I installed matplotlib newly
about 4 days ago.

Cheers,
-T