0

I have a small problem with a program that is run using MPI. I need to run this program on a HPC and I need to run it this way. The program needs to have 1 MPI process per node, use the number of cores allocated and create a certain number of OpenMP threads. The program must have 1 MPI process per node, use the number of cores allocated, and create a certain number of OpenMpi threads. For example, if I have 1 node and I want to have 2 OpenMP threads with 13 cores, with this string

mpirun --bind-to core -hostfile /path/to/hostfile -np 1 -x OMP_NUM_THREADS --map-by ppr:1:node:PE=13 /program/to/run

the program will create 1 MPI process with 2 OpenMP threads, but it will use only 2 cores. How can I fix this. PBS allocate the right number of resouce and othres tool works fine, so the problem is in the way I lunch the program. Thanks in advance.

I have tried to start mpirun with OMP_PROC_BIND OMP_PLACES --bind to none, and other combinations, but the program still uses 1 core per OpenMP thread.

Improve this question
4
  • I think, you are confusing OpenMPI, which is a specific MPI implementation, and OpenMP. OpenMPI by itself will always only use a single core per MPI process. Each process might spawn multiple OpenMP threads to utilize the cores assigned to each MPI process. What value do you set for OMP_NUM_THREADS? Is your code compiled with OpenMP flag (-fopenmp)? Commented Oct 21, 2024 at 19:04
  • @Joachim Sorry for the confusion, you are absolutely right. The code was compiled with the fopenmp flag. OMP_NUM_THREADS is set to the number of OpenMP threads I want. So in this case, OMP_NUM_THREADS is set to 2. From htop, I see that it correctly creates the number of threads requested. Commented Oct 21, 2024 at 21:25
  • Each string of execution (thread) will always only use one core at a time. If you use OMP_PROC_BIND=close OMP_PLACES=cores (or other values for these variables), each thread will stick to it's core. If you want to use all 13 cores, you need to start 13 threads. If you want to utilize them equally, you need to distribute the work to these 13 threads equally. Commented Oct 22, 2024 at 8:24
  • @Joachim Thanks for the answer. So to sum up, an OpenMP thread can't use more than one core, right? Or is it possible to make my OpenMPI process that creates 2 OpenMP threads use 1300% of the resources? (the 13 allocated cores) Commented Oct 22, 2024 at 8:48
Related questions
16
Unable to use all cores with mpirun
0
-cpu-set in openmpi doesn't work as it suppose to be
0
Different behaviour of MPI 2.1 and MPI 3.0 implementaion. (MPI + OpenMP)
Load 4 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.