14

I'm writing a web-application in Python, I haven't decided if I want to use Flask, web.py or something else yet, and I want to be able to do profile on the live application.

There seems to be very little information on how you go about implementing the instrumentation to do performance measurement, short of doing a lot of print datetime.now() everywhere.

What is the best way of going about instrumenting your Python application to allow good measurements to be made. I guess I'm looking for something similar to the Stackoverflow teams mvc-mini-profiler.

Improve this question
2
  • 3
    docs.python.org/library/profile.html Commented Jun 21, 2011 at 17:54
  • I don't really like the fact the profiling of individual functions done the way the standard profile suggests. It seems mostly useful if you expect the application to terminate at some point. I do see how it could be done elegantly with decorators, so perhaps it's worth looking into writing some test for. Commented Jun 21, 2011 at 18:08

4 Answers 4

Reset to default
8

You could simply run cProfile tool that comes with Python:

python -m cProfile script.py

Of course, you would have to create the script.py file that would execute the parts of the code that you want to test. If you had some unit tests, you could also use that.

Or you couse use:

import cProfile 
cProfile.run('foo()')

to profile it from foo entry point.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I would really prefer having the profiler code in the function, rather than "wrapping" the function in the profiler. Also doesn't it make it harder to turn profiling on and of for parts of the applications?
Turns out that despite my reluctance to use the standard profiler, this is the correct answer, but with a small modification. Someone wrote this: mg.pov.lt/profilehooks which allows you to add a decorator to the functions you want to profile. It's not perfect, as it fails to collect everything like Stackoverflow mvc profiler does, but I guess that's the price of having something a bit more generic.
Thanks for the comment, I didn't know about profilehooks. Cheers
4

Amir Salihefendic wrote a short (150 LOC) RequestProfiler, which is described in this blog post:

I haven't tried it, but since it is a WSGI middleware, it should be somewhat pluggable.

Improve this answer

3 Comments

Looks intersting, a bit limited perhaps and I don't really understand how it manages to track database and memcache queries, but it looks like a quick way for locating bottlenecks.
I haven't looked at the source myself - I trust the author ;) However, I hope other answers appear to this question, since I have an interest in this topic myself (maybe the question title can be a bit more focused e.g. on WSGI profiling).
It wasn't meant to be WSGI profiling. I actually don't care for solution that is WSGI only, I wan't to be able to use the same tools for web and ordinary applications.
0

You can just use a general purpose web application performance tool, such as httpperf. This works using an external client and works with any framework since it works against a standard interface (HTTP). Therefore it tests the full stack performance.

Improve this answer

3 Comments

But it won't tell me where time is spent, just that a given http query is slow.
But will help you compare framework performace that you seemed to imply that you want to do. Otherwise, you can use the profiling tools available that you say you don't actually want to use for some reason.
May that was my mistake to talk about web frame works, it was meant as "I want something that supports any framework". I don't want to use the standard Python profiler, because it doesn't seem like something you would like to run constantly in a production system, but more something you use to measure performance before deploying.
0

Use New Relic's Free monitoring system. You simply install an agent on the server and point to your flask init.py file. Once you run the application with proper agent setup, you will start seeing application metrics in see New Relic's online dashboard called APM. By default it will show you graphs of your application's throughput (QPS/RPM), app response time, top transactions, error rate, error stack trace if any(eg for 500 error), calls to external services etc. In addition you can monitor your System stats too.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.