Which operations in numpy/scipy are multithreaded?

Question

I'm working on an algorithm, and I've made no attempt to parallelize it other than just by using numpy/scipy. Looking at htop, sometimes the code uses all of my cores and sometimes just one. I'm considering adding parallelism to the single-threaded portions using multiprocessing or something similar.

Assuming that I have all of the parallel BLAS/MKL libraries, is there some rule of thumb that I can follow to guess whether a numpy/scipy ufunc is going to be multithreaded or not? Even better, is there some place where this is documented?

To try to figure this out, I've looked at: https://scipy.github.io/old-wiki/pages/ParallelProgramming, Python: How do you stop numpy from multithreading?, multithreaded blas in python/numpy.

For future readers, the joblib maintainers have a cool package for controlling threads in blas/etc: github.com/joblib/threadpoolctl — mostsquares
– mostsquares, Commented Dec 18, 2019 at 20:45

Gennady.F · Accepted Answer · 2019-02-27 14:10:29Z

2

You may try to take IDP package (Intel® Distribution for Python) which contains versions of NumPy*, SciPy*, and scikit-learn* with integrated Intel® Math Kernel Library.

This would give you threading of all Lapack routines automatically whether this make sense to do. Here you find out the list of threaded mkl’s functions : https://software.intel.com/en-us/mkl-linux-developer-guide-openmp-threaded-functions-and-problems

answered Feb 27, 2019 at 14:10

Gennady.F

5812 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Kalo Over a year ago

Why did you put an asterisk next to the package names in your answer? The same is used in Intel's documentation (and others), and I have never seen it explained.

newkid · Accepted Answer · 2019-02-07 17:12:39Z

The routines intrinsic to numpy and scipy allow single threads by default. You can change that if you so choose.

# encoding: utf-8
# module numpy.core.multiarray
# from /path/to/anaconda/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-darwin.so
# by generator 1.145
# no doc
# no imports

# Variables with simple values

ALLOW_THREADS = 1

When compiling numpy, you can control threading by changing NPY_ALLOW_THREADS:

./core/include/numpy/ufuncobject.h:#if NPY_ALLOW_THREADS
./core/include/numpy/ndarraytypes.h:        #define NPY_ALLOW_THREADS 1

As for the external libraries, I've mostly found numpy and scipy to wrap around legacy Fortran code (QUADPACK, LAPACK, FITPACK ... so on). All the subroutines in these libraries compute on single threads.

As for the MKL dependencies, the SO posts you link to sufficiently answer the question.

Chenxin · Accepted Answer · 2022-03-31 16:26:10Z

0

please try to set the global variable OMP_NUM_THREADS. It works for my scipy and numpy. The functions I use are,

ling.inv() and A.dot(B)

answered Mar 31, 2022 at 16:26

Chenxin

215 bronze badges

Collectives™ on Stack Overflow

Which operations in numpy/scipy are multithreaded?

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related