Return common element indices between two numpy arrays

Question

I have two arrays, a1 and a2. Assume len(a2) >> len(a1), and that a1 is a subset of a2.

I would like a quick way to return the a2 indices of all elements in a1. The time-intensive way to do this is obviously:

from operator import indexOf
indices = []
for i in a1:
    indices.append(indexOf(a2,i))

This of course takes a long time where a2 is large. I could also use numpy.where() instead (although each entry in a1 will appear just once in a2), but I'm not convinced it will be quicker. I could also traverse the large array just once:

for i in xrange(len(a2)):
    if a2[i] in a1:
        indices.append(i)

But I'm sure there is a faster, more 'numpy' way - I've looked through the numpy method list, but cannot find anything appropriate.

Many thanks in advance,

D

Alok Singhal · Accepted Answer · 2016-06-19 06:45:03Z

18

How about

numpy.nonzero(numpy.in1d(a2, a1))[0]

This should be fast. From my basic testing, it's about 7 times faster than your second code snippet for len(a2) == 100, len(a1) == 10000, and only one common element at index 45. This assumes that both a1 and a2 have no repeating elements.

edited Jun 19, 2016 at 6:45

answered Feb 25, 2010 at 11:47

Alok Singhal

96.9k21 gold badges131 silver badges158 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Dave Over a year ago

I compared your solution to Dave Kirby's above, with this one being approx 1.35X faster for len(a2) == 12347424, len(a1) == 1338, so this solution get's my vote - thanks!

Alok Singhal Over a year ago

For anyone reading this: it seems like setmember1d has been renamed to in1d since numpy 1.4.

Alok Singhal Over a year ago

@DanielWatkins Since numpy 1.4 is very old now, I have updated my answer to use in1d.

Dave Kirby · Accepted Answer · 2010-02-25 20:13:25Z

2

how about:

wanted = set(a1)
indices =[idx for (idx, value) in enumerate(a2) if value in wanted]

This should be O(len(a1)+len(a2)) instead of O(len(a1)*len(a2))

NB I don't know numpy so there may be a more 'numpythonic' way to do it, but this is how I would do it in pure python.

edited Feb 25, 2010 at 20:13

answered Feb 25, 2010 at 11:38

Dave Kirby

26.7k5 gold badges72 silver badges84 bronze badges

1 Comment

Dave Over a year ago

should that be enumerate(a2)?

chriad · Accepted Answer · 2013-10-26 12:14:42Z

1

index = in1d(a2,a1)
result = a2[index]

answered Oct 26, 2013 at 12:14

chriad

1,41217 silver badges22 bronze badges

Comments

Philippe Miron · Accepted Answer · 2017-08-18 03:49:13Z

1

Very similar to @AlokSinghal, but you get an already flattened version.

numpy.flatnonzero(numpy.in1d(a2, a1))

answered Aug 18, 2017 at 3:49

Philippe Miron

1781 silver badge10 bronze badges

Comments

Eelco Hoogendoorn · Accepted Answer · 2016-06-19 08:33:26Z

0

The numpy_indexed package (disclaimer: I am its author) contains a vectorized equivalent of list.index; performance should be similar to the currently accepted answer, but as a bonus, it gives you explicit control over missing values as well, using the 'missing' kwarg.

import numpy_indexed as npi
indices = npi.indices(a2, a1, missing='raise')

Also, it will also work on multi-dimensional arrays, ie, finding the indices of one set of rows in another.

answered Jun 19, 2016 at 8:33

Eelco Hoogendoorn

10.8k1 gold badge46 silver badges43 bronze badges

Comments

ankit agrawal · Accepted Answer · 2022-02-10 20:34:23Z

0

These all methods are slow for me. Following method is doing quite fast. The index list has the index of the elements from first list which are common in second list.

index=[]
d={}
for j in range(len(first_list)):
    name=first_list[j]
    d[name]=j
    
for i in range(len(second_list)):
    name=second_list[i]
    index.append(d[name])

answered Feb 10, 2022 at 20:34

ankit agrawal

3113 silver badges18 bronze badges

Collectives™ on Stack Overflow

Return common element indices between two numpy arrays

6 Answers 6

3 Comments

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

3 Comments

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related