4

I have a program that is performing waaaay under par, and I would like to profile it. However, it is multithreaded, so I can't seem to find a good way to profile this thing. Any advice?

I've tried yappi, but it segfaults on OS X :(

EDIT: This is in python, sorry for putting it under profiling...

2 Answers 2

2

Are you multithreading or multiprocessing? If you are just multithreading, then that is the problem. Python currently has problems with multithreading on a multiprocessor system because of the Global Interpreter Lock (GIL). They are working on fixing it for Python 3.2 - at least so that your program will run as fast on a single core as on multiple cores.

If you aren't convinced take a look at the shootout results for the thread-ring program. Running with a single core is faster than running with quad cores.

Now, if you use multiprocessing instead, profiling can be difficult as well because then you have to run CProfiler from each separate process. There are some questions that point you in the right direction though.

Sign up to request clarification or add additional context in comments.

1 Comment

thread-ring isn't a good example because thread-ring is all about task-switching - so Java single core is also faster than Java quad core. shootout.alioth.debian.org/u32/…
2

Depending on how far you've come in your troubleshooting, there are some tools that might point you in the right direction.

  • "top" is a helpful start to show you if your problem is burning CPU time or simply waiting for stuff.

  • "dtruss -c" can show you where you spend time and what system calls takes most of your time.

Both these can give you a hint without knowing anything about python.

If you just want to use yappi, it isn't too much work to set up a virtualbox and install some sort of Linux on your machine. I find myself doing that from time to time when I want to try something.

There might of course be things I don't know about that makes it impossible or not worth the effort. Also, profiling on another OS running virtualized might not give the exact same results, but it might still be helpful.

1 Comment

As it happens, I set up yappi on a virtual machine as well, and sure enough, yappi fails.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.