Using matplotlib in GAE

Question

My tags and title quite clearly state my problem. I want to use matplotlib to create real-time plots in Google App Engine. I've read the documentation and searched on SO and Google. I found a post, pointing to this working demo. But when I try it on my own, it doesn't work for me.

I created a simple application, consisting only of a handler-script hello_world.py

import numpy as np
import os
import sys
import cStringIO

print "Content-type: image/png\n"

os.environ["MATPLOTLIBDATA"] = os.getcwdu()  # own matplotlib data
os.environ["MPLCONFIGDIR"] = os.getcwdu()    # own matplotlibrc
import matplotlib.pyplot as plt

plt.plot(np.random.random((20))) #imshow(np.random.randint((10,10)))

sio = cStringIO.StringIO()
plt.savefig(sio, format="png")
sys.stdout.write(sio.getvalue())

and a a configuration file app.yaml

application: helloworldtak
version: 1
runtime: python27
api_version: 1
threadsafe: no

handlers:
- url: /.*
  script: hello_world.py

libraries:
- name: numpy
  version: "latest"
- name: matplotlib
  version: "latest"

I want to plot something and then return the content as png-image. This procedure works fine for a normal web-server like Apache or IIS, I did this a million times.

The problem is rather: when I run my script locally within the development server, I get an error that is probably due to my MPL version 1.1.1, which is only "experimental" in GAE. But when I deploy my app to GAE, I get a completely different, uncorrelated error.

Looking at the looks, the traceback is:

Traceback (most recent call last):
  File "/base/data/home/apps/s~helloworldtak/1.364765672279579252/hello_world.py", line 16, in <module>
    import matplotlib.pyplot as plt
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/pyplot.py", line 23, in <module>
    from matplotlib.figure import Figure, figaspect
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/figure.py", line 18, in <module>
    from axes import Axes, SubplotBase, subplot_class_factory
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/axes.py", line 14, in <module>
    import matplotlib.axis as maxis
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/axis.py", line 10, in <module>
    import matplotlib.font_manager as font_manager
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/font_manager.py", line 1324, in <module>
    _rebuild()
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/font_manager.py", line 1278, in _rebuild
    fontManager = FontManager()
  File "/python27_runtime/python27_lib/versions/third_party/matplotlib-1.1.1/matplotlib/font_manager.py", line 995, in __init__
    self.defaultFont['ttf'] = self.ttffiles[0]
IndexError: list index out of range

It seems to have to do something with the fonts-cache of MPL. I read in the docs that caching and file-access is one of the problems with MPL in GAE, but obviously, the import works for others.

What am I doing wrong?

Edit Based on the answer below, I changed my code to be

import numpy as np
import cStringIO
import matplotlib.pyplot as plt

import webapp2

class MainPage(webapp2.RequestHandler):
    def get(self):
        plt.plot(np.random.random((20)),"r-")
        sio = cStringIO.StringIO()
        plt.savefig(sio, format="png")
        self.response.headers['Content-Type'] = 'image/png'

        self.response.out.write(sio.getvalue())

app = webapp2.WSGIApplication([('/', MainPage)],
                              debug=True)

and like this, it's working.

To be clear, it fails under localhost? and where did you get your app.yaml file from? (Python 7 normally calls the main script as hello_world.app) — user784435
– user784435, Commented Jan 22, 2013 at 15:34
I created it according to the "Getting started" guide. I then migrated it to a 2.7-version according to the manual. — Thorsten Kranz
– Thorsten Kranz, Commented Jan 22, 2013 at 20:38
It fails under localhost and when deployed to GAE. But: the exception and traceback differ. I didn't include the local one as I wanted to exclude local incompatibilities as a possible origin of the problem. — Thorsten Kranz
– Thorsten Kranz, Commented Jan 23, 2013 at 6:40

Gianni Di Noia · Accepted Answer · 2013-01-22 16:23:58Z

7

I'm not familiar with sys module. To give an answer to the question I prefer using webapp2. This is a working handler:

import webapp2
import StringIO
import numpy as np
import matplotlib.pyplot as plt


class MainPage(webapp2.RequestHandler):
    def get(self):
        plt.plot(np.random.random((20)))
        sio = StringIO.StringIO()
        plt.savefig(sio, format="png")
        img_b64 = sio.getvalue().encode("base64").strip()
        plt.clf()
        sio.close()
        self.response.write("""<html><body>""")
        self.response.write("<img src='data:image/png;base64,%s'/>" % img_b64)
        self.response.write("""</body> </html>""")

app = webapp2.WSGIApplication([('/', MainPage)], debug=True)

Alternatively, you could write the sio.getvalue() in the blobstore with files api and use the method get_serving_url() of images api for avoid to encode in base64.

answered Jan 22, 2013 at 16:23

Gianni Di Noia

1,57810 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Thorsten Kranz Over a year ago

thanks for your feedback. my problem is that I can't even import matplotlib. maybe my app configuration is the problem? how does yours look?

Gianni Di Noia Over a year ago

your app.yaml looks good except two things: hello_world.APP as commented @peterretief and the webapp2 import in libraries. this script works only in production.

Thorsten Kranz Over a year ago

I succeeded to get it working based on your sample, though I really don't want to create HTML, but PNG-files. I'll attach my code to my question.

mgilson Over a year ago

I found this answer to be quite helpful (+1). One recommendation that I would make is to use cStringIO rather than StringIO. I think there would be some pretty significant performance increases if you're doing this repeatedly.

Gianni Di Noia Over a year ago

@mgilson you're right, in fact the module is not blacklisted developers.google.com/appengine/kb/libraries

mgiuca · Accepted Answer · 2013-01-28 22:58:05Z

2

The problem was that you were setting the MATPLOTLIBDATA and MPLCONFIGDIR environment variables to your app directory before importing matplotlib. Since you didn't have any fonts in your app directory, it couldn't load any fonts.

answered Jan 28, 2013 at 22:58

mgiuca

21.4k7 gold badges56 silver badges71 bronze badges

Collectives™ on Stack Overflow

Using matplotlib in GAE

2 Answers 2

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related