matplotlib-users Mailing List for matplotlib

That definitely helps.  Here's what I did.

First.

Yeah, the results are totally acceptable if I do '-' as my line/marker. 
The pdf renders and loads just fine.

If I do 'o' or even ','  as my marker, then the pdf is horrendously
slow.  I'm talking minutes to render a page.

So, I tried your idea of altering the backend

If I change that line the "if True:" then I get MUCH better results. 
But I also get enormous file sizes.

I've taken a subset of 10 of my 750 graphs.

Those 10, before changing the backend, would make file sizes about about
290KiB.  After changing the backend, if I use plot(x, y, '-') I still
get a file size about 290KiB.

But after changing the backend, if I use plot(x, y, '.') for my markers,
my file size is no 21+ MB.  Just for 10 of my graphs.  I'm afraid making
all 750 in the same pdf may be impossible at those size.

BUT, at least now I can render those 10 in vector format.  Before it
took the pdf minutes to load a page.  Now it only takes maybe 15-20
seconds to load a page of 10 graphs.

So that definitely helped.  Thanks!

Is there anyway to do this even better?  At this rate I'd have to split
my pdf file into multiple chunks, which really isn't ideal to have to
send people 70 pdf files. 

Is there anyway to have reasonable pdf sizes as well as this improved
performance for keeping them in vector format?

Thanks again.

On 05/01/2014 01:19 PM, Jouni K. Seppänen [via matplotlib] wrote:
> nertskull <[hidden email]
> </user/SendEmail.jtp?type=node&node=43348&i=0>> writes:
>
> > The problem, is the pdf is unbearably slow when plotting as a
> scatter plot
> > or as a line with markers.
> >
> > If I make a regular line plot, with no markers, just a single line,
> it is
> > plotted and the pdf is fine.  But then it connects my points which I
> don't
> > want.
>
> Others have commented on the volume of data, but that paragraph makes
> me curious: are you saying that the results are acceptable if you do
> something like
>
>   plot(x, y, '-')
>
> but not if you do
>
>   plot(x, y, 'o')  or  plot(x, y, '-o')?
>
> The amount of data in the pdf file should be within a constant factor in
> all cases, but the '-' case there are only moveto and lineto commands,
> while the two other cases render markers as something called an XObject,
> which is repeated a lot of times on the page. I wonder if the overhead
> from using an XObject is making the rendering application slow.
>
> Does it help at all to use a simpler marker, e.g. plot(x, y, ',')? One
> change you could try if you're feeling adventurous is the following
> function in lib/matplotlib/backends/backend_pdf.py:
>
>     def draw_markers(self, gc, marker_path, marker_trans, path, trans,
>                      rgbFace=None):
>         # For simple paths or small numbers of markers, don't bother
>         # making an XObject
>         if len(path) * len(marker_path) <= 10:
>             RendererBase.draw_markers(self, gc, marker_path,
> marker_trans,
>                                       path, trans, rgbFace)
>             return
>         # ...
>
> The comment is not quite right: only if the path is short *and* the
> number of markers is small does the XObject code get skipped. You could
> just change the if statemt to "if True:" and rerun your code (possibly
> with the ',' marker style). If that helps, it's evidence that we need to
> revisit the condition for using XObjects for markers.
>
> -- 
> Jouni K. Seppänen
> http://www.iki.fi/jks
>
>
> ------------------------------------------------------------------------------
>
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform
> available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Matplotlib-users mailing list
> [hidden email] </user/SendEmail.jtp?type=node&node=43348&i=1>
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
> ------------------------------------------------------------------------
> If you reply to this email, your message will be added to the
> discussion below:
> http://matplotlib.1069221.n5.nabble.com/Millions-of-data-points-saved-to-pdf-tp43338p43348.html
>
> To unsubscribe from Millions of data points saved to pdf, click here
> <http://matplotlib.1069221.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=43338&code=bmVydHNrdWxsQGdtYWlsLmNvbXw0MzMzOHwtMTQ3Nzk2OTQ5Nw==>.
> NAML
> <http://matplotlib.1069221.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>





--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Millions-of-data-points-saved-to-pdf-tp43338p43349.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

nertskull <ner...@gm...> writes:

> The problem, is the pdf is unbearably slow when plotting as a scatter plot
> or as a line with markers.
>
> If I make a regular line plot, with no markers, just a single line, it is
> plotted and the pdf is fine.  But then it connects my points which I don't
> want.

Others have commented on the volume of data, but that paragraph makes
me curious: are you saying that the results are acceptable if you do
something like

  plot(x, y, '-')

but not if you do

  plot(x, y, 'o')  or  plot(x, y, '-o')?

The amount of data in the pdf file should be within a constant factor in
all cases, but the '-' case there are only moveto and lineto commands,
while the two other cases render markers as something called an XObject,
which is repeated a lot of times on the page. I wonder if the overhead
from using an XObject is making the rendering application slow.

Does it help at all to use a simpler marker, e.g. plot(x, y, ',')? One
change you could try if you're feeling adventurous is the following
function in lib/matplotlib/backends/backend_pdf.py:

    def draw_markers(self, gc, marker_path, marker_trans, path, trans,
                     rgbFace=None):
        # For simple paths or small numbers of markers, don't bother
        # making an XObject
        if len(path) * len(marker_path) <= 10:
            RendererBase.draw_markers(self, gc, marker_path, marker_trans,
                                      path, trans, rgbFace)
            return
        # ...

The comment is not quite right: only if the path is short *and* the
number of markers is small does the XObject code get skipped. You could
just change the if statemt to "if True:" and rerun your code (possibly
with the ',' marker style). If that helps, it's evidence that we need to
revisit the condition for using XObjects for markers.

-- 
Jouni K. Seppänen
http://www.iki.fi/jks

"""
The pyplot interface is generally preferred for non-interactive
plotting (i.e., scripting). The pylab interface is convenient for
interactive calculations and plotting, as it minimizes typing. Note
that this is what you get if you use the ipython shell with the -pylab
option, which imports everything from pylab and makes plotting fully
interactive.
"""

Gotta remember to update this paragraph... the -pylab option has been long
deprecated, and is supposedly about to be removed in an upcoming release of
ipython.

Ben Root



On Thu, May 1, 2014 at 10:38 AM, Yuxiang Wang <yw...@vi...> wrote:

> Hi Neal,
>
> I always followed what has been written here:
>
>
> http://matplotlib.org/faq/usage_faq.html#matplotlib-pylab-and-pyplot-how-are-they-related
>
> And they said,
>
> ------------------
> Matplotlib, pylab, and pyplot: how are they related?
>
> Matplotlib is the whole package; pylab is a module in matplotlib that
> gets installed alongside matplotlib; andmatplotlib.pyplot is a module
> in matplotlib.
>
> Pyplot provides the state-machine interface to the underlying plotting
> library in matplotlib. This means that figures and axes are implicitly
> and automatically created to achieve the desired plot. For example,
> calling plot from pyplot will automatically create the necessary
> figure and axes to achieve the desired plot. Setting a title will then
> automatically set that title to the current axes object:
>
> import matplotlib.pyplot as plt
>
> plt.plot(range(10), range(10))
> plt.title("Simple Plot")
> plt.show()
>
> Pylab combines the pyplot functionality (for plotting) with the numpy
> functionality (for mathematics and for working with arrays) in a
> single namespace, making that namespace (or environment) even more
> MATLAB-like. For example, one can call the sin and cos functions just
> like you could in MATLAB, as well as having all the features of
> pyplot.
>
> The pyplot interface is generally preferred for non-interactive
> plotting (i.e., scripting). The pylab interface is convenient for
> interactive calculations and plotting, as it minimizes typing. Note
> that this is what you get if you use the ipython shell with the -pylab
> option, which imports everything from pylab and makes plotting fully
> interactive.
> ------------------
>
>
> -Shawn
>
> On Thu, May 1, 2014 at 2:16 AM, Michiel de Hoon <mjl...@ya...>
> wrote:
> > 99.9% of the time I am using pyplot, as it usually does what I want
> without me having to understand an api.
> > I don't care so much if pyplot agrees with matlab or not, but it should
> be something easy that new users can pick up quickly.
> >
> > Best,
> > -Michiel
> >
> > --------------------------------------------
> > On Wed, 4/30/14, Neal Becker <ndb...@gm...> wrote:
> >
> >  Subject: [Matplotlib-users] Which api to learn?
> >  To: mat...@li...
> >  Date: Wednesday, April 30, 2014, 7:49 AM
> >
> >  I've never used matlab (and hope
> >  never to have to).  But I've been using pyplot
> >  api for mpl for quite a while.
> >
> >  Is there any good reason to move to the "native" mpl api and
> >  drop pyplot?  I ask
> >  because as I understand, pyplot is intended as a matlab
> >  workalike, and since I
> >  never learned matlab I have no need for that crutch.
> >  OTOH, I'm quite used to
> >  the pyplot api at this point.
> >
> >
> >
>  ------------------------------------------------------------------------------
> >  "Accelerate Dev Cycles with Automated Cross-Browser Testing
> >  - For FREE
> >  Instantly run your Selenium tests across 300+ browser/OS
> >  combos.  Get
> >  unparalleled scalability from the best Selenium testing
> >  platform available.
> >  Simple to use. Nothing to install. Get started now for
> >  free."
> >  http://p.sf.net/sfu/SauceLabs
> >  _______________________________________________
> >  Matplotlib-users mailing list
> >  Mat...@li...
> >  https://lists.sourceforge.net/lists/listinfo/matplotlib-users
> >
> >
> >
> ------------------------------------------------------------------------------
> > "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> > Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> > unparalleled scalability from the best Selenium testing platform
> available.
> > Simple to use. Nothing to install. Get started now for free."
> > http://p.sf.net/sfu/SauceLabs
> > _______________________________________________
> > Matplotlib-users mailing list
> > Mat...@li...
> > https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
>
> --
> Yuxiang "Shawn" Wang
> Gerling Research Lab
> University of Virginia
> yw...@vi...
> +1 (434) 284-0836
> https://sites.google.com/a/virginia.edu/yw5aj/
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>

Hi Neal,

I always followed what has been written here:

http://matplotlib.org/faq/usage_faq.html#matplotlib-pylab-and-pyplot-how-are-they-related

And they said,

------------------
Matplotlib, pylab, and pyplot: how are they related?

Matplotlib is the whole package; pylab is a module in matplotlib that
gets installed alongside matplotlib; andmatplotlib.pyplot is a module
in matplotlib.

Pyplot provides the state-machine interface to the underlying plotting
library in matplotlib. This means that figures and axes are implicitly
and automatically created to achieve the desired plot. For example,
calling plot from pyplot will automatically create the necessary
figure and axes to achieve the desired plot. Setting a title will then
automatically set that title to the current axes object:

import matplotlib.pyplot as plt

plt.plot(range(10), range(10))
plt.title("Simple Plot")
plt.show()

Pylab combines the pyplot functionality (for plotting) with the numpy
functionality (for mathematics and for working with arrays) in a
single namespace, making that namespace (or environment) even more
MATLAB-like. For example, one can call the sin and cos functions just
like you could in MATLAB, as well as having all the features of
pyplot.

The pyplot interface is generally preferred for non-interactive
plotting (i.e., scripting). The pylab interface is convenient for
interactive calculations and plotting, as it minimizes typing. Note
that this is what you get if you use the ipython shell with the -pylab
option, which imports everything from pylab and makes plotting fully
interactive.
------------------


-Shawn

On Thu, May 1, 2014 at 2:16 AM, Michiel de Hoon <mjl...@ya...> wrote:
> 99.9% of the time I am using pyplot, as it usually does what I want without me having to understand an api.
> I don't care so much if pyplot agrees with matlab or not, but it should be something easy that new users can pick up quickly.
>
> Best,
> -Michiel
>
> --------------------------------------------
> On Wed, 4/30/14, Neal Becker <ndb...@gm...> wrote:
>
>  Subject: [Matplotlib-users] Which api to learn?
>  To: mat...@li...
>  Date: Wednesday, April 30, 2014, 7:49 AM
>
>  I've never used matlab (and hope
>  never to have to).  But I've been using pyplot
>  api for mpl for quite a while.
>
>  Is there any good reason to move to the "native" mpl api and
>  drop pyplot?  I ask
>  because as I understand, pyplot is intended as a matlab
>  workalike, and since I
>  never learned matlab I have no need for that crutch.
>  OTOH, I'm quite used to
>  the pyplot api at this point.
>
>
>  ------------------------------------------------------------------------------
>  "Accelerate Dev Cycles with Automated Cross-Browser Testing
>  - For FREE
>  Instantly run your Selenium tests across 300+ browser/OS
>  combos.  Get
>  unparalleled scalability from the best Selenium testing
>  platform available.
>  Simple to use. Nothing to install. Get started now for
>  free."
>  http://p.sf.net/sfu/SauceLabs
>  _______________________________________________
>  Matplotlib-users mailing list
>  Mat...@li...
>  https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users



-- 
Yuxiang "Shawn" Wang
Gerling Research Lab
University of Virginia
yw...@vi...
+1 (434) 284-0836
https://sites.google.com/a/virginia.edu/yw5aj/

This makes me wonder if you would be better served with something like
bokeh:

http://bokeh.pydata.org/

Cheers!
Ben Root



On Thu, May 1, 2014 at 9:28 AM, nertskull <ner...@gm...> wrote:

> No we definitely aren't really interested in the gaps.  Gaps are just where
> we were unable to collect the data.
>
> I don't know if we can attach pictures to this thread or not, but I'm going
> to try.
>
> The attached is roughly what I want, but with all 750 as vectors.
>
> I want to see the 'movement' of the line, but I need the gaps to remain, so
> I know where they are.
>
> The problem with plotting a reduced data set, is I lose some of the very
> small sections of line.  I'll play around with that idea, but we want to be
> able to zoom in on a vector file, and see the tiny areas of less than
> 10points that would be lost if we plot a reduced data set.
>
> But what it sounds like, is it is unlikely this will work in vector
> graphics
> form.  Its just too much to do without reducing the dataset.
>
> <http://matplotlib.1069221.n5.nabble.com/file/n43344/figure_1.png>
>
>
>
> --
> View this message in context:
> http://matplotlib.1069221.n5.nabble.com/Millions-of-data-points-saved-to-pdf-tp43338p43344.html
> Sent from the matplotlib - users mailing list archive at Nabble.com.
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
>

No we definitely aren't really interested in the gaps.  Gaps are just where
we were unable to collect the data.

I don't know if we can attach pictures to this thread or not, but I'm going
to try.

The attached is roughly what I want, but with all 750 as vectors.

I want to see the 'movement' of the line, but I need the gaps to remain, so
I know where they are.

The problem with plotting a reduced data set, is I lose some of the very
small sections of line.  I'll play around with that idea, but we want to be
able to zoom in on a vector file, and see the tiny areas of less than
10points that would be lost if we plot a reduced data set.

But what it sounds like, is it is unlikely this will work in vector graphics
form.  Its just too much to do without reducing the dataset.

<http://matplotlib.1069221.n5.nabble.com/file/n43344/figure_1.png> 



--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Millions-of-data-points-saved-to-pdf-tp43338p43344.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

when reading the number of points you have in each plot, I have to ask
why you need so many (plotted) data points. If you plot e.g. every
10th or 50th data point, you reduce the number of points by a factor
of 10 (or 50). This should make the PDF smaller and faster and even if
you zoom into each plot, you should be able to see enough details (of
course, if there are one or two outliers you might not see them). And
probably you are not able to distinguish between two data points if
they are too close to each other so you probably don't need every data
point.

Cheers,

Dominik

On 05/01/2014 02:09 PM, nertskull wrote:
> I am trying to create a multipage pdf of about 750 different
> graphs.
> 
> Each graph has around 5,000 - 15,000 data points, giving me roughly
> 7 million points across the pdf.  I make it in a large pdf with a
> page length of about 20 inches, and then plot about 10 graphs to a
> page.  So I end up with basically 75 pages in my pdf.  I'm
> basically trying to graph a line of XY data points.
> 
> The problem, is the pdf is unbearably slow when plotting as a
> scatter plot or as a line with markers.
> 
> If I make a regular line plot, with no markers, just a single line,
> it is plotted and the pdf is fine.  But then it connects my points
> which I don't want.
> 
> I assume this is all because its making the pdf in vector format.
> And when I convert it to single lines, I only have ~750 line
> vectors.  But when I try to scatter plot, or line plot with
> markers, I end up with millions of vectors.
> 
> I've tried the 'rasterized=True' and that definitely works.  But
> the quality is really bad.  I need to be able to zoom in close on
> the pdf and still see rough resolution of the points.
> 
> For clarity, I don't actually need to see each individual points.
> The graphs have two lines on them, and I just need to be able to
> distinguish between the two lines.  The two lines are just made up
> of thousands of points each.
> 
> Is there anyway to keep scalable vectors and do this?  Or will I
> just be forced to go to a rasterized image file in order to load
> the pdf in a reasonable time.
> 
> Thanks.
> 
> 
> 
> -- View this message in context:
> http://matplotlib.1069221.n5.nabble.com/Millions-of-data-points-saved-to-pdf-tp43338.html
>
> 
Sent from the matplotlib - users mailing list archive at Nabble.com.
> 
> ------------------------------------------------------------------------------
>
> 
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.
> Get unparalleled scalability from the best Selenium testing
> platform available. Simple to use. Nothing to install. Get started
> now for free." http://p.sf.net/sfu/SauceLabs 
> _______________________________________________ Matplotlib-users
> mailing list Mat...@li... 
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users
> 

- -- 
Dominik Klaes
Deputy student representative of the AIfA
Argelander-Institut für Astronomie
Room 2.027a
Auf dem Hügel 71
53121 Bonn

Telefon: 0228/73-5773
E-Mail: dk...@as...
Homepage: http://www.astro.uni-bonn.de/~dklaes/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTYjxlAAoJEG4xBfJ3OX2tck4QALZ4raj6ZHW50Ie2uj31dC7n
q7LynLhBCfYr8Hs/m8wad/VfNStNqdoyJta683YF4ev6qskUY6lvh3qzYRNZdYMk
8yzT2CmhrWss+jEQYyrfKjrjSZbWMtCRaNHdrF/Ne6Je7VaPE/y6+4PXmkKNNeKU
bCdRdyUsUb+cjQPXgIn0bN9AFqNDOcVpMkkkzxfHU2n0kGGOymVRpOLRqQbSFR5X
AsyawYmia08RRr312dNja27BJcvXA9JHJ+qk0r7UPCOZow2GZGb6BlCu+eE/xtBt
j/r9Ym5KUgSD0q35veT34BLoMuD/L3Q7RujCxkToKezTdNGbRZh/8EkRKbu5Zc48
SnZAkmcAO8GTXqdBaD32l67iOjSCK1qqXLv8/Sb+0OOhZ3gMdF/6PbxvBM3m7U0g
zthwBYWBkJrZUuBRyi3fzYs5olvlQDW6RWPnB4tA5acrOmrHAhSqp5I4nk+ln/mZ
s41R+uG2sx5+F3aJGMLEL0lNpRtQbWUIPQ7RHJ48TVHhrG3oB/41Li4FOcYvboKr
8B5XIygw8eNULja7Q7Coz2T/uFg42pMyRMMMDz92eLhOxlmk8k0bWqAqUp4dj85l
Hodz94PwYvloWV6tagSHVAYiUtkWhkyHOYBSH1oPSfB4Tpsh8aiGCk5oIZn92Kty
kswsXBVzAe4npHOfl05Y
=9Tfy
-----END PGP SIGNATURE-----

What do you consider a gap?Perhaps if you know that you can find those in your data and if you really want to visualize the gaps, plot those instead of the data.  
—
Sent from Mailbox

On Thu, May 1, 2014 at 2:41 PM, Alan G Isaac <ala...@gm...> wrote:

> Suppose each data point is only 1 point (1/72 ") in diameter.
> A solid line across a 20" page is less than 1500 points.
> You're using a fraction of a page per graph and trying to
> plot 5,000-15,000 points per graph.  This is pointless (pun
> intended) for visual display, especially since you do not
> care about the individual points.  What happens if you
> decimate the points?  Is the result acceptable?
> Perhaps you could do even better than that, given your
> posted description.  Fit a line to the points, and only
> plot the fitted line.  Or use something like `hexbin`.
> Alan Isaac
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get 
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

How about different line styles or colors instead of markers?—
Sent from Mailbox

On Thu, May 1, 2014 at 2:10 PM, nertskull <ner...@gm...> wrote:

> I am trying to create a multipage pdf of about 750 different graphs.
> Each graph has around 5,000 - 15,000 data points, giving me roughly 7
> million points across the pdf.  I make it in a large pdf with a page length
> of about 20 inches, and then plot about 10 graphs to a page.  So I end up
> with basically 75 pages in my pdf.  I'm basically trying to graph a line of
> XY data points.
> The problem, is the pdf is unbearably slow when plotting as a scatter plot
> or as a line with markers.
> If I make a regular line plot, with no markers, just a single line, it is
> plotted and the pdf is fine.  But then it connects my points which I don't
> want.  
> I assume this is all because its making the pdf in vector format.  And when
> I convert it to single lines, I only have ~750 line vectors.  But when I try
> to scatter plot, or line plot with markers, I end up with millions of
> vectors.
> I've tried the 'rasterized=True' and that definitely works.  But the quality
> is really bad.  I need to be able to zoom in close on the pdf and still see
> rough resolution of the points.  
> For clarity, I don't actually need to see each individual points.  The
> graphs have two lines on them, and I just need to be able to distinguish
> between the two lines.  The two lines are just made up of thousands of
> points each.
> Is there anyway to keep scalable vectors and do this?  Or will I just be
> forced to go to a rasterized image file in order to load the pdf in a
> reasonable time.
> Thanks.
> --
> View this message in context: http://matplotlib.1069221.n5.nabble.com/Millions-of-data-points-saved-to-pdf-tp43338.html
> Sent from the matplotlib - users mailing list archive at Nabble.com.
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get 
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Matplotlib-users mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-users

On 5/1/2014 6:47 AM, Björn Opitz wrote:
> How would you create a figure as in
> "fig = plt.figure()", but without pyplot?

http://econpy.googlecode.com/svn/trunk/software4econ.xhtml#mpl-hints

hth,
Alan Isaac

Suppose each data point is only 1 point (1/72 ") in diameter.
A solid line across a 20" page is less than 1500 points.
You're using a fraction of a page per graph and trying to
plot 5,000-15,000 points per graph.  This is pointless (pun
intended) for visual display, especially since you do not
care about the individual points.  What happens if you
decimate the points?  Is the result acceptable?

Perhaps you could do even better than that, given your
posted description.  Fit a line to the points, and only
plot the fitted line.  Or use something like `hexbin`.

Alan Isaac

I am trying to create a multipage pdf of about 750 different graphs.

Each graph has around 5,000 - 15,000 data points, giving me roughly 7
million points across the pdf.  I make it in a large pdf with a page length
of about 20 inches, and then plot about 10 graphs to a page.  So I end up
with basically 75 pages in my pdf.  I'm basically trying to graph a line of
XY data points.

The problem, is the pdf is unbearably slow when plotting as a scatter plot
or as a line with markers.

If I make a regular line plot, with no markers, just a single line, it is
plotted and the pdf is fine.  But then it connects my points which I don't
want.  

I assume this is all because its making the pdf in vector format.  And when
I convert it to single lines, I only have ~750 line vectors.  But when I try
to scatter plot, or line plot with markers, I end up with millions of
vectors.

I've tried the 'rasterized=True' and that definitely works.  But the quality
is really bad.  I need to be able to zoom in close on the pdf and still see
rough resolution of the points.  

For clarity, I don't actually need to see each individual points.  The
graphs have two lines on them, and I just need to be able to distinguish
between the two lines.  The two lines are just made up of thousands of
points each.

Is there anyway to keep scalable vectors and do this?  Or will I just be
forced to go to a rasterized image file in order to load the pdf in a
reasonable time.

Thanks.



--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Millions-of-data-points-saved-to-pdf-tp43338.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Dear all,

> Is there any good reason to move to the "native" mpl api and drop pyplot?

I notice I don't even understand this question, but I'd like to. From
http://matplotlib.org/faq/usage_faq.html#matplotlib-pylab-and-pyplot-how-are-they-related,
I see the difference between *pylab* and the pyplot module. But what
would be "the" native mpl api? How would you create a figure as in
"fig = plt.figure()", but without pyplot? If this is explained
anywhere in the docs, I haven't come across it.


Best,

Björn

99.9% of the time I am using pyplot, as it usually does what I want without me having to understand an api.
I don't care so much if pyplot agrees with matlab or not, but it should be something easy that new users can pick up quickly.

Best,
-Michiel

--------------------------------------------
On Wed, 4/30/14, Neal Becker <ndb...@gm...> wrote:

 Subject: [Matplotlib-users] Which api to learn?
 To: mat...@li...
 Date: Wednesday, April 30, 2014, 7:49 AM
 
 I've never used matlab (and hope
 never to have to).  But I've been using pyplot 
 api for mpl for quite a while.
 
 Is there any good reason to move to the "native" mpl api and
 drop pyplot?  I ask 
 because as I understand, pyplot is intended as a matlab
 workalike, and since I 
 never learned matlab I have no need for that crutch. 
 OTOH, I'm quite used to 
 the pyplot api at this point.
 
 
 ------------------------------------------------------------------------------
 "Accelerate Dev Cycles with Automated Cross-Browser Testing
 - For FREE
 Instantly run your Selenium tests across 300+ browser/OS
 combos.  Get 
 unparalleled scalability from the best Selenium testing
 platform available.
 Simple to use. Nothing to install. Get started now for
 free."
 http://p.sf.net/sfu/SauceLabs
 _______________________________________________
 Matplotlib-users mailing list
 Mat...@li...
 https://lists.sourceforge.net/lists/listinfo/matplotlib-users