Efficient way to find mode in a large 2D matrix in python(Scipy)

Question

I am using stats.mode() from scipy to calculate mode in a 800*500 matrix. And the execution time is like:

   `Time taken to execute 0.4359015187888584
    Time taken to execute 0.42199154405135975
    Time taken to execute 0.4250416821138656
    Time taken to execute 0.4100701556064723
    Time taken to execute 0.4371956395342953`

But I need it under:

 Excution time 0.09193338154885265

Is there any method to make it efficient?

Is there any specific reason for your lower bound time requirement? — dennlinger
– dennlinger, Commented Jun 29, 2018 at 6:32
yeah!.Actually matrices are sequence of frames in a video.And I have to find modes of those frames before showing the output video. So when I apply that method,output video lags few secs from actual video speed. — priyanka
– priyanka, Commented Jun 29, 2018 at 6:43
Do you need the row-wise mode or just one mode per frame? Also what data type and range are your pixels ? — Paul Panzer
– Paul Panzer, Commented Jun 29, 2018 at 10:14
@PaulPanzer ,Now I am trying to find row wise mode. (one mode per frame would also be appreciated,but prefer row-wise). Data type I am using is int8 and range is -128 to 127. — priyanka
– priyanka, Commented Jun 29, 2018 at 10:46

Paul Panzer · Accepted Answer · 2018-06-29 11:03:54Z

1

I have no idea why scipy.stats.mode is so slow. Anyway, you can get a much faster result using np.bincount:

# create random frame
>>> a = np.random.randint(0, 256, (800, 500)).astype(np.int8)
>>> 
# add row offsets to make bincount create separate bins for each row
>>> counts = np.bincount((a.view(np.uint8) + 256 * np.arange(800)[:, None]).ravel(), minlength=256*800).reshape(800, 256)
# find the mode
>>> values = np.argmax(counts, axis=1)
# discard the counts for all other values
>>> counts = counts[np.arange(800), values]
# convert modes back to original dtype
>>> values = values.astype(np.uint8).view(np.int8)

answered Jun 29, 2018 at 11:03

Paul Panzer

53.3k3 gold badges60 silver badges103 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

priyanka Over a year ago

Fantastic, Thanks a lot @PaulPanzer

Collectives™ on Stack Overflow

Efficient way to find mode in a large 2D matrix in python(Scipy)

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related