matplotlib plot array size limit?

Question

I've created a program that retrieves data from a device on the serial port every half second or so. It then appends that data to the array that sets the data points and then updates the plot. Everything goes fine until it's been running for an hour or so, at which point the program stops responding.

Does anyone know if there is a size limit for this array? If anyone has any ideas on handling a data set that could be millions of points, I would love to hear your thoughts.

You might want to specify your platform, the version of matplotlib and the backend you're using. — NPE
– NPE, Commented Nov 30, 2012 at 20:47
A minimal but complete example that demonstrates the problem would also be helpful. — NPE
– NPE, Commented Nov 30, 2012 at 20:49
I'm writing this in python on a windows system using the QtAgg4 back end. I've written a simplified version, and I've gotten well past 100,000 points, so, I know it's not as simple as just having too many points. I wish I could post my code, but, it's several very large classes. My biggest problem, is that I don't get an error message, it just stops responding. — Stephen
– Stephen, Commented Nov 30, 2012 at 20:56
Another thing that I noticed was that the program will run twice as long if it's running as an executable as opposed to running in my eclipse IDE. — Stephen
– Stephen, Commented Nov 30, 2012 at 21:11

Ohad Eytan · Accepted Answer · 2019-04-16 20:17:41Z

5

Using the code below I was able to get matplotlib to show a simple graph of ten million points. I suspect the problem isn't with the array size.

import numpy as np
import matplotlib.pyplot as plt
import random

nsteps = 10000000
draws = np.random.randint(0,2,size=nsteps)
steps = np.where(draws>0,1,-1)
walk = steps.cumsum()
plt.plot(np.arange(nsteps), np.array(walk), 'r-')
plt.title("Big Set Random Walk with $\pm1$ steps")
plt.show()

edited Apr 16, 2019 at 20:17

Ohad Eytan

8,5341 gold badge24 silver badges32 bronze badges

answered Nov 30, 2012 at 20:38

BKay

1,4771 gold badge16 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Stephen Over a year ago

Fair enough. Does anyone know of any tools I can use to determine what is causing the lock up?

BKay Over a year ago

Given Akavall's comment, this must depend at least in part on the chart type.

Akavall · Accepted Answer · 2012-11-30 20:44:34Z

1

There seems to be a some limit. I just tried

import pylab
import numpy as np

n = 10000000 # my code works fine for n = 1000000

x = np.random.normal(0,1,n)

pylab.plot(x)

pylab.show()

And got the following error:

OverflowError: Agg rendering complexity exceeded. Consider downsampling or decimating your data.

answered Nov 30, 2012 at 20:44

Akavall

86.9k58 gold badges215 silver badges261 bronze badges

1 Comment

Stephen Over a year ago

10 million data points at about 2 data points per second would be roughly 1300 hours of polling, that would probably work for me. :)

Collectives™ on Stack Overflow

matplotlib plot array size limit?

2 Answers 2

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related