Efficient grouping in numpy

Question

I have a list of about 10⁶ pairs, where each element of the pair is either -1, 0, or 1:

[
 [ 0,  1],
 [-1, -1],
 [ 0, -1],
 [ 1,  0],
 ...
]

I want to split these pairs into two groups (i.e. lists of pairs) according to whether the first element of the pair is -1 or not¹.

Is there a way to do this efficiently with numpy?

Despite the terminology and notation I used above, I am in fact agnostic about the actual types of the pairs and the "lists" of pairs. Use whatever numpy or python data structure leads to the most efficient solution. (But no pandas, please.)

EDIT:

For example, if the initial list of pairs is

[
 [ 0, -1],
 [ 0, -1],
 [ 1, -1],
 [-1, -1],
 [ 1,  0],
 [-1,  1],
 [-1, -1],
 [ 0,  0],
 [ 0,  1],
 [-1,  0]
]

...an acceptable result would consist of the two lists

[
 [-1, -1],
 [-1,  1],
 [-1, -1],
 [-1,  0]
]

...and

[
 [ 0, -1],
 [ 0, -1],
 [ 1, -1],
 [ 1,  0],
 [ 0,  0],
 [ 0,  1]
]

The last two lists preserve the ordering of elements as they appeared in the original lists. This would be my preference, but it is not essential. For example, a solution consisting of

[
 [-1, -1],
 [-1, -1],
 [-1,  0],
 [-1,  1]
]

...and

[
 [ 0, -1],
 [ 0, -1],
 [ 0,  0],
 [ 0,  1],
 [ 1, -1],
 [ 1,  0],
]

...would also be acceptable.

^{¹ In other words, all the pairs in one group should have -1 at their first position, and all the elements of the other group should have either 0 or 1 at their first position.}

what do you mean by efficient solution? Efficient wrt to what? Memory/storage? Computational complexity? — Nikos M.
– Nikos M., Commented Apr 21, 2019 at 11:04
for example why isnt a simple for loop to separate the list in two, efficient? — Nikos M.
– Nikos M., Commented Apr 21, 2019 at 11:05
Please also add expected output to your question as this is raising answers which seem aren't what you were expecting. — Austin
– Austin, Commented Apr 21, 2019 at 11:05

Sheldore · Accepted Answer · 2019-04-21 11:31:17Z

2

How about just using the condition twice to check for positive and negative as

import numpy as np

a = np.array([ [ 0, -1], [ 0, -1], [ 1, -1], [-1, -1], [ 1,  0], 
                    [-1,  1], [-1, -1], [ 0,  0], [ 0,  1], [-1,  0]])

pos = a[a[:, 0]!=-1]
neg = a[a[:, 0]==-1]

print (pos)
# [[ 0 -1]
#  [ 0 -1]
#  [ 1 -1]
#  [ 1  0]
#  [ 0  0]
#  [ 0  1]]

print (neg)
# [[-1 -1]
#  [-1  1]
#  [-1 -1]
#  [-1  0]]

answered Apr 21, 2019 at 11:31

Sheldore

39.2k9 gold badges63 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

freude Over a year ago

Is conditional indexing faster than my answer below?

Sheldore Over a year ago

@freude: I wouldn't called it same and yes, indeed it is different. The indexing which you and I used are to my mind the straightforward approach. It amused me though why you didn't use the indexing for the positives and resorted to delete. Regarding your first comment, I haven't performed any timing study. Feel free to do so.

freude Over a year ago

I have shown two equal approaches not making preferences to any of them. I am pretty sure the timing is equal. But if you think that you have significantly contributed to the discussion, so it be. That is fine.

hpaulj Over a year ago

In this binary grouping we could construct one mask=a[:,0]==-1, and apply a[mask] and a[~mask]. Speed is some what better.

Sheldore Over a year ago

@hpaulj: Thanks for the nice suggestion

Raphael · Accepted Answer · 2019-04-21 11:21:13Z

0

import numpy as np
a = np.random.randint(-1, 2, size=(10, 2))

print(a)
[[ 0  0]
 [ 1  1]
 [ 1  1]
 [-1 -1]
 [ 0 -1]
 [ 1  1]
 [-1  1]
 [-1  0]
 [ 1 -1]
 [ 1  1]]

minus, zero, one = [np.array([r for r in a if r[0] == c]) for c in [-1, 0, 1]]


print(minus)
[[-1 -1]
 [-1  1]
 [-1  0]]
print(zero)
[[ 0  0]
 [ 0 -1]]
print(one)
[[ 1  1]
 [ 1  1]
 [ 1  1]
 [ 1 -1]
 [ 1  1]]

edited Apr 21, 2019 at 11:21

answered Apr 21, 2019 at 11:06

Raphael

1,8112 gold badges10 silver badges27 bronze badges

Comments

Narcisse Doudieu Siewe · Accepted Answer · 2019-04-21 16:20:32Z

-1

you ca, do it yourself! the only efficiency I see is generator or something like that which will save memory at the cost of computation time

def sanitize(yllp):#yllp: list-like of pair
    y = yield
    yield
    for x in yllp:
        if (x[0] in {0,1} and y != -1) or x[0] == -1 == y:
           yield x

Example:

L = [
     (-1,1), 
     (0,1), 
     (0,1), 
     (-1,1), 
     (-1,0), 
     (-1,-1), 
     (0,0), 
     (1,0)
    ]

#get list starting by 0 or 1
w=sanitize(L)    
w.next()
w.send(0)
for i in w:print(i)

#get list starting by -1
t=sanitize(L)
t.next()
t.send(-1)
for i in t:print(i)

edited Apr 21, 2019 at 16:20

answered Apr 21, 2019 at 11:52

Narcisse Doudieu Siewe

1,0941 gold badge7 silver badges9 bronze badges

2 Comments

hpaulj Over a year ago

Your list(w) produces the not -1 group, but where's the other?

Narcisse Doudieu Siewe Over a year ago

for the other side it is simple: w=sanitize(L) w.next() w.send(-1) for i in w:print(i) I have just edited teh code

Collectives™ on Stack Overflow

Efficient grouping in numpy

3 Answers 3

5 Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related