Merging lists based on elementwise condition

Question

Problem 1

I have four lists of equal length: A_old, A_new, B_old, B_new and want to have the following behaviour: If A_new[i] > A_old[i] then set B_old[i] = B_new[i].

Tried a straightforward loop implementation:

for i, (a_new, a_old) in enumerate(izip(A_new, A_old)):
    if a_new > a_old:
        B_old[i] = B_new[i]

And an implementation using a map (as suggested here):

B_old = map(lambda (a_new, a_old, b_new, b_old): b_new if a_new > a_old else b_old,
            izip(A_new, A_old, B_new, B_old))

But none of these approaches are sufficiently fast. Note that I cannot use numpy arrays and boolean arrays because the elements of B_new, B_old are not numeric.

To put things on a more formal footing, a sample implementation is provided here: https://gist.github.com/1986588 and the runtimes are as shown below:

Naive: 1.13343191147
Mapping: 1.45240283012
kev: 1.09499907494
Numpy: 0.0525879859924
Where: 0.0651860237122

Problem 2

Having evaluated different options numpy slicing with boolean arrays seems to be the way to go. However, I have now encountered a new problem:

Let B_new be not an array but simply a value val. Then the best variant reads

B_old[A_new > A_old] = val

This works fine as long as val is not iterable but fails when it is. In particular, val is a tuple here. I would like to assign the tuple val to the elements of B_old where A_new > A_old.

A workaround seems to be to let

val2 = np.empty(1, dtype=object)
val2[0]=val
B_old[A_new > A_old] = val

but this seems convoluted. The problem is that I do not know what the type of val is. It might be iterable and it might not.

What exactly are you looking for the performance? What do you mean by "not sufficiently fast"? — ronakg
– ronakg, Commented Mar 6, 2012 at 14:02
Sorry, should have been clearer: The algorithm I am working on computes many convolutions which are rather expensive. However, the above consumes ~50% of the computational resources. On the other hand, a simple np.maximum(A_new, A_old) consumes hardly any resources but is of the same basic form as the above problem. — Till Hoffmann
– Till Hoffmann, Commented Mar 6, 2012 at 14:09
FWIW, you can use numpy arrays for non-numeric objects. But I'm not sure this would speed up anything here. — wim
– wim, Commented Mar 6, 2012 at 14:28
Have you considered writing a C extension for that? You shouldn't be able to get a lot faster than the straightforward for loop. — Niklas B.
– Niklas B., Commented Mar 6, 2012 at 14:44
Thanks for the info! I was having trouble when creating a numpy array and subsequently adding elements of a different type. E.g. the following fails a=np.empty(10); a.fill('test') with the following error message ValueError: could not convert string to float: test. Having tried it for numerical values it gives bad results. — Till Hoffmann
– Till Hoffmann, Commented Mar 6, 2012 at 14:46

kev · Accepted Answer · 2012-03-06 14:18:04Z

3

B_old = [B_new[i] if A_new[i]>A_old[i] else B_old[i] for i in range(len(A_old))]

answered Mar 6, 2012 at 14:18

kev

163k49 gold badges286 silver badges282 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Till Hoffmann Over a year ago

The above suggestion is unfortunately slower than the two implementations I already tried.

Collectives™ on Stack Overflow

Merging lists based on elementwise condition

Problem 1

Problem 2

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Problem 1

Problem 2

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related