1

Benchmarks of different languages and related questions are everywhere on the Internet. However, I still cannot figure out an answer of whether I should switch to C in my program.

Basically, The most time consuming part in my program involves a lot of matrix inverse and matrix multiplication. I have several plans:

  1. stick with numpy.
  2. use C with LAPACK/BLAS.
  3. rewrite my python program and change the most time consuming part into C and then use python to call C.

I know numpy is just something wrapped around LAPACK/BLAS. So will 2 or 3 be substantially(500%) faster than 1?

Improve this question
7
  • cython actually can be regarded as another language. That's not what I want. Commented Dec 7, 2015 at 9:04
  • 4
    You're going to have to show us what you're doing and how you're using NumPy. For linear algebra operations, NumPy is just calling down to the optimised LAPACK/BLAS routines (assuming they're on your system) - you're unlikely to get much faster than those. Hand-written C code is likely to much slower than calling these routines. Commented Dec 7, 2015 at 9:37
  • @ajcr, edited accordingly Commented Dec 7, 2015 at 9:53
  • You can have a look into sagemath . Its based on python and has a lot of stuff included. For my part I use it for big matrix multiplication in GF(2^8). Commented Dec 7, 2015 at 9:55
  • Profile your current code - if are spending almost all of your time inside BLAS/LAPACK routines then you will see essentially no performance benefit from options 2 or 3. One rare scenario where it might make sense to re-write some portion of your code in C would be if you are calling BLAS/LAPACK routines a lot of times on many small matrices. In practice you should probably be thinking about whether the efficiency of your algorithm can be improved, whether you are linked against the fastest possible BLAS/LAPACK implementation, and possibly whether you could benefit from using the GPU. Commented Dec 7, 2015 at 19:30

1 Answer 1

Reset to default
1

I just wanted to ask a very similar question when i saw yours. I have tested this question from various directions. From quite some time I am trying to beat numpy.dot function by my code.

I have large complex matrices and their multiplication is the primary bottleneck of my program. I have tested following methods

  1. simple c code.
  2. cython code with various optimizations, using cblas.
  3. python 32 bit and 64 bit versions and found that 64 bit version is 1.5-2 times faster than the 32 bit.
  4. ananconda's MKL implementation but no luck there also.
  5. einsum for the matrix multiplication
  6. python 3 and python 2.7 are same python 3 @ operator is also same
  7. numpy.dot(a,b,c) is marginally faster than c=numpy.dot(a,b)

by far the numpy.dot is the best. It beat every other method, sometimes marginally (einsum) but mostly significantly.

During my research i come across one article namely Ultrafast matrix multiplication which tells that apple's altivec implementation can multiply 2500x2500 matrix in less than a second. On my PC with intel core i3 4th generation 2.3 GHZ 4 gb ram it took 73 seconds using numpy.dot hence I am still searching for faster implementation on PC.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

73 seconds is still quite a long time for dotting two 2500x2500 float matrices together. I suspect you are probably using a slow, single-threaded BLAS implementation. The same dot product takes ~650ms on my laptop when numpy is linked against multithreaded OpenBLAS, but ~16 seconds when linked against the reference CBLAS implementation.
Thanks for your comment, I have just checked my code again and found that the above timings were for integer matrices, for float 64 matrices it reduced to 270 ms (anaconda 3 64 bit with python 3.5). I will be more careful with data types while working with matrices. But still the numpy dot is the best.
BLAS and LAPACK only support floating point matrices. For matrix multiplication with other dtypes, numpy will fall back on its own C implementation, which will be much slower.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.