OpenMP: Working with Anaconda Python / Cython, but not with System (Arch) Python / Cython

Question

I have a Python/Cython application, which is parallelized using OpenMP and which makes several calls to the Intel MKL. Usually, i determine the number of threads via OMP_NUM_THREADS=xx. Both the cython script as well as MKL (Pardiso solver calls) correctly start several threads when i run my script using a Anaconda distribution (Python 3.6). The CPU load and the number of loaded cores can be seen very well in the system monitor.

However, when using the systems Python distribution (Python 3.6 under Arch Linux), only one thread is started, for both the cython module as well as the Intel MKL.

At least for my cython module i can tell that the correct number of threads is requested (via prange() ), but just one thread is obtained.

No compilation errors arise, and of course flag '-fopenmp' is used for compilation. Since the issue affects both my cython module as well as the Intel MKL, i assume it is somehow related to my systems OpenMP. What is the issue here? Thank you!

The system python will also be using system openmp, compiler, toolchain etc. So there may be a global OMP_NUM_THREADS setting for the system that does not apply to anaconda. Try setting that env variable manually when running the python script. — danny
– danny, Commented Apr 6, 2018 at 14:31
True, but that's what i already do: i set the env variable via OMP_NUM_THREADS=... . — mneuner
– mneuner, Commented Apr 7, 2018 at 6:37

Sia · Accepted Answer · 2018-04-11 03:04:17Z

0

Try to specify number of threads inside the code just before the loop in case OMP_NUM_THREADS is overwritten somewhere outside, like

import openmp
openmp.omp_set_num_threads(NumThreads)
# parallel loop here

answered Apr 11, 2018 at 3:04

Sia

2072 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

OpenMP: Working with Anaconda Python / Cython, but not with System (Arch) Python / Cython

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related