matplotlib-devel Mailing List for matplotlib (Page 2)

>>>>> "Darren" == Darren Dale <dd...@co...> writes:

    Darren> The coordinates (80.640 31.680) are rendered twice; I can
    Darren> comment one of these lines out of the PS file and the tick
    Darren> still renders. Its not a bug in draw_markers, the square
    Darren> data markers are only rendered once, it seems to be
    Darren> specific to tickmarks.

Strange....  I'll look into this later.

    Darren> I think we could get a performance boost if all similar
    Darren> ticks were passed together to draw_markers, right now they
    Darren> are passed independently.

We could, but it would require some redesign.  Tick is a class, and
the axis contains a list of ticks.  Thus it would take some top-level
redesign.  


    Darren> Yeah, I realized I had made a boneheaded observation just
    Darren> after I hit the send button.

It's always that way :-) That is what the send button is for: self
enlightenment.

    Darren> OK. Would you add the signature to backend_bases?

Not yet.  I was just suggesting you use this internally.

    def draw_markers(self, gc, path, rgbFace, x, y, transform):

        self.push_gc(gc)

        while 1: 
          .... snip...

and later when it becomes part of the api, you'll already have done
the hard part.  You can also call this function from draw_ps.
Basically, all you need to do is rip the gc setting part of out of
draw_ps.

JDH

Darren Dale wrote:

>% draw_markers 
>/marker { gsave
>newpath
>translate
>0.000 0.000 m
>0.000 4.000 l
>closepath
>stroke
>grestore } bind def
>0.500 setlinewidth
>0 setlinecap
>80.640 31.680 marker
>80.640 31.680 marker
>stroke
>
>The coordinates (80.640 31.680) are rendered twice; I can comment one of these 
>lines out of the PS file and the tick still renders. Its not a bug in 
>draw_markers, the square data markers are only rendered once, it seems to be 
>specific to tickmarks. 
>
>I think we could get a performance boost if all similar ticks were passed 
>together to draw_markers, right now they are passed independently.
>  
>
Yes, this would be good, since the same marker could be save and then 
just translated from position to position.

 -- Paul

-- 
Paul Barrett, PhD      Space Telescope Science Institute
Phone: 410-338-4475    ESS/Science Software Branch
FAX:   410-338-4767    Baltimore, MD 21218

Hi John,

>
>     Darren> 2) I think each tickmark is listed in agg.path_storage
>     Darren> twice, and therefore gets rendered twice in PS.
>
> Why do you think this?  Which ticks?

I was checking the output of the files I was generating, here is a clip 
responsible for rendering a single xtickmark:

% draw_markers 
/marker { gsave
newpath
translate
0.000 0.000 m
0.000 4.000 l
closepath
stroke
grestore } bind def
0.500 setlinewidth
0 setlinecap
80.640 31.680 marker
80.640 31.680 marker
stroke

The coordinates (80.640 31.680) are rendered twice; I can comment one of these 
lines out of the PS file and the tick still renders. Its not a bug in 
draw_markers, the square data markers are only rendered once, it seems to be 
specific to tickmarks. 

I think we could get a performance boost if all similar ticks were passed 
together to draw_markers, right now they are passed independently.

>     Darren> 5) Im not doing anything with vec6 =
>     Darren> transform.as_vec6_val(). I'm not sure what it is used for.
>
> This is in case you want to do the affine transformation yourself.
> The transform is a nonlinear part plus an affine.  Note that
> backend_ps is currently doing
>
>         if transform.need_nonlinear():
>             x,y = transform.nonlinear_only_numerix(x, y)
>         x, y = transform.numerix_x_y(x, y)
>
> which is wrong -- it will fail for nonlinear transforms like log
> because the numerix_x_y call does the nonlinear and the affine part
> and so you will be doing the nonlinear part twice.  

I'll get up to speed on this eventually. I just copied those three lines from 
backend_cairo.draw_markers.

>     Darren> 6) draw_lines is getting a long pathlist from agg.
>
> That is not surprising.  matplotlib plots what you give it.  

Yeah, I realized I had made a boneheaded observation just after I hit the send 
button.

> Now, onto the subject of how you might be able to make this faster.
[...]
> It might be worth implementing a push_gc method
> that sets the current gc state, and then calling this at the top of
> draw_markers and not inside the loop.  We'll probably want to
> implement this as a default gc method across backends anyway in the
> near term, so it would be a worthwhile change.

OK. Would you add the signature to backend_bases?

-- 

Darren

>>>>> "Darren" == Darren Dale <dd...@co...> writes:


    Darren> I made a first (and second) attempt at implementing
    Darren> draw_markers and draw_lines in the postscript backend. The
    Darren> changes are in CVS, although I left draw_markers masked as
    Darren> _draw_markers, it needs to be unmasked if you want to try
    Darren> it out.

Hey Darren, thanks for working on this.  

    Darren> I found some places for speed/memory/ps-filesize
    Darren> improvements. With draw_markers masked, the script below
    Darren> took 2.43 seconds to generate and write the 1.5MB eps
    Darren> file. With draw_markers unmasked, it took 0.69 seconds to
    Darren> make a 350KB eps file.

A good start.  You'll might be able to get this number down a bit
more, which I discuss below.

    Darren> 1) Circles are being drawn with draw_markers, but
    Darren> agg.path_storage has no curve information in it? Circles
    Darren> are faithfully reproduced in ps output, but it takes 50
    Darren> line segments to draw each circle in
    Darren> plot(arange(10000),'-o').

This is a wart slated for destruction.  We plan to replace circles and
ellipses with splines rather than vertices.  Just hasn't been done
yet.  

    Darren> 2) I think each tickmark is listed in agg.path_storage
    Darren> twice, and therefore gets rendered twice in PS.

Why do you think this?  Which ticks?

    Darren> 3) I expected marker paths to be terminated with the
    Darren> agg.path_cmd_end_poly code. This is not the case. What is
    Darren> the purpose of path_cmd_end_poly?

Only marker paths that are polygons have end poly (eg draw_circle).  A
lot of the paths (eg tick marks) are not polygons and so don't have an
end_poly code.

    Darren> 4) I am getting an unrecognized agg.path_commands_e
    Darren> code. They should be one of 0,1,2,3,4,6,0x0F, and I am
    Darren> getting a value of 70. ?? I just ignore it and PS seems to
    Darren> render fine.

I had to track this one down myself.  lines.py calls

      path.end_poly()

agg_path_storage::end_poly calls

    add_vertex(0.0, 0.0, path_cmd_end_poly | flags);

where flags is agg_basics path_flags_e::path_flags_close = 0x40.  You
can test for end poly using the agg module with

  >>> 0x40 | 6
  70
  >>> from matplotlib.agg import path_storage, is_end_poly
  >>> is_end_poly(71)
  False
  >>> is_end_poly(70)
  True

    Darren> 5) Im not doing anything with vec6 =
    Darren> transform.as_vec6_val(). I'm not sure what it is used for.

This is in case you want to do the affine transformation yourself.
The transform is a nonlinear part plus an affine.  Note that
backend_ps is currently doing

        if transform.need_nonlinear():
            x,y = transform.nonlinear_only_numerix(x, y)
        x, y = transform.numerix_x_y(x, y)

which is wrong -- it will fail for nonlinear transforms like log
because the numerix_x_y call does the nonlinear and the affine part
and so you will be doing the nonlinear part twice.  The motivation for
separating out the nonlinear and affine parts was to let the backend
machinery do the affine part (in the great majority of cases, the
transforms are pure affine anyway).  So you might want to do

        if transform.need_nonlinear():
            x,y = transform.nonlinear_only_numerix(x, y)

        vec6 = transform.as_vec6_val()

and then set the current ps affine to vec6.

    Darren> 6) draw_lines is getting a long pathlist from agg. Rather
    Darren> than draw a straight line between two points, it is doing
    Darren> something like

    Darren> 50.106 249.850 moveto 53.826 249.850 lineto 57.546 249.850
    Darren> lineto 61.266 249.850 lineto

    Darren> and thats just for the line in the legend! The straight
    Darren> line in the actual plot has many, many intermediate
    Darren> points.

That is not surprising.  matplotlib plots what you give it.  If you
specify a straight line of 10000 points as you did in your example

  plot(arange(10000),'-s')

matplotlib will plot all 10000 vertices of the line.  It's incumbent
on the user not to pass in redundant data.

Now, onto the subject of how you might be able to make this faster.
One of the primary motivations of draw_markers is that you should only
have to set the graphics context state once.  In the current
implementation, we have

        while start < len(x):
            to_draw = izip(x[start:end],y[start:end])
            ps = ['%1.3f %1.3f marker' % point for point in to_draw] 
            self._draw_ps("\n".join(ps), gc, None)
            start = end
            end   += 1000
        
and _draw_ps sets the gc state.  Now this isn't really a huge deal,
since you are chunking the data in 1000 length buckets.  But for very
large data sets (500k markers) it will result in 500 superfluous calls
to set the gc state.  It might be worth implementing a push_gc method
that sets the current gc state, and then calling this at the top of
draw_markers and not inside the loop.  We'll probably want to
implement this as a default gc method across backends anyway in the
near term, so it would be a worthwhile change.

Hope this helps, thanks again.

JDH

On Wednesday 30 March 2005 10:39 pm, John Hunter wrote:
>     JDH> - implement draw_markers and draw_lines with the new API
>     JDH> (transform is done in backend).  

[..snip..]

I made a first (and second) attempt at implementing draw_markers and 
draw_lines in the postscript backend. The changes are in CVS, although I left 
draw_markers masked as _draw_markers, it needs to be unmasked if you want to 
try it out.

I found some places for speed/memory/ps-filesize improvements. With 
draw_markers masked, the script below took 2.43 seconds to generate and write 
the 1.5MB eps file. With draw_markers unmasked, it took 0.69 seconds to make 
a 350KB eps file. 

Some comments:

1) Circles are being drawn with draw_markers, but agg.path_storage has no 
curve information in it? Circles are faithfully reproduced in ps output, but 
it takes 50 line segments to draw each circle in plot(arange(10000),'-o').

2) I think each tickmark is listed in agg.path_storage twice, and therefore 
gets rendered twice in PS.

3) I expected marker paths to be terminated with the agg.path_cmd_end_poly 
code. This is not the case. What is the purpose of path_cmd_end_poly?

4) I am getting an unrecognized agg.path_commands_e code. They should be one 
of 0,1,2,3,4,6,0x0F, and I am getting a value of 70. ?? I just ignore it and 
PS seems to render fine.

5) Im not doing anything with vec6 = transform.as_vec6_val(). I'm not sure 
what it is used for.

6) draw_lines is getting a long pathlist from agg. Rather than draw a straight 
line between two points, it  is doing something like 

50.106 249.850 moveto
53.826 249.850 lineto
57.546 249.850 lineto
61.266 249.850 lineto

and thats just for the line in the legend! The straight line in the actual 
plot has many, many intermediate points.


Feedback appreciated!


from pylab import *
from time import clock

figure(1)
plot(arange(10000),'-s')
l=legend(('1e4 markers',))
d = clock()
savefig('temp.eps')
print clock()-d

-- 

Darren

Hi,

Replying to myself to make things clear here.

John forwarded my message to Enthought and David C. Morrill answered
that the problem was probably in pydoc and its way of inspecting
modules, rather than in ctraits.c, since when importing traits.py
(without pydoc) and examining the traits.CTrait.__bases__, one gets
(<type 'cTrait'>,), which looks correct.

Thanks.

-- 
Florent