How do I get meaningful results from gprof on an MPI code?

Question

I am optimising an MPI code and I am working with Gprof. The problem is that the results I obtained are completely unreasonable. My workflow is the following:

compiling the code adding -pg as a compilation flag.
running the code mpirun -np Nproc EXEC.exe arg1 ... argN.
running gprof on the executable gprof EXEC.exe

What's wrong in this?

Bart · Accepted Answer · 2022-12-23 05:43:40Z

10

Instructions for running gprof typically assume the program is serial, or single process but multi-threaded.

To run gprof with a multi-process program like an MPI program, you'll want to

make sure each process outputs its own file
explicitly sum the files across processes
run gprof on the results.

This blog post or these instructions at LLNL are good starting points:

Set the poorly documented GMON_OUT_PREFIX environment variable, e.g. in bash, export GMON_OUT_PREFIX=gmon.out- before running the mpiexec command (then, depending on the environment, you may have to run mpirun -x GMON_OUT_PREFIX -np Nproc EXEC.exe arg1 ... argN to make sure each process has the environment variable)
Use gprof itself to collect and sum the results, gprof -s EXEC.exe gmon.out-*
gprof EXEC.exe gmon.sum (or just examine the individual files, or the files collectively, with gprof EXEC.exe gmon.out-12345 or gprof EXEC.exe gmon.out-*)

edited Dec 23, 2022 at 5:43

Bart

10.4k5 gold badges54 silver badges83 bronze badges

answered Dec 15, 2018 at 18:06

Jonathan Dursi

51.1k10 gold badges131 silver badges160 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How do I get meaningful results from gprof on an MPI code?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related