Removing NaNs in numpy arrays

Question

I have two numpy arrays that contains NaNs:

A = np.array([np.nan,   2,   np.nan,   3,   4])
B = np.array([   1  ,   2,     3   ,   4,  np.nan])

are there any smart way using numpy to remove the NaNs in both arrays, and also remove whats on the corresponding index in the other list? Making it look like this:

A = array([  2,   3, ])
B = array([  2,   4, ])

are you also using pandas? (there's dropna that does what you want with DataFrame objects) — Kos
– Kos, Commented Mar 18, 2015 at 11:18

EdChum · Accepted Answer · 2015-03-18 12:04:13Z

10

What you could do is add the 2 arrays together this will overwrite with NaN values where they are none, then use this to generate a boolean mask index and then use the index to index into your original numpy arrays:

In [193]:

A = np.array([np.nan,   2,   np.nan,   3,   4])
B = np.array([   1  ,   2,     3   ,   4,  np.nan])
idx = np.where(~np.isnan(A+B))
idx
print(A[idx])
print(B[idx])
[ 2.  3.]
[ 2.  4.]

output from A+B:

In [194]:

A+B
Out[194]:
array([ nan,   4.,  nan,   7.,  nan])

EDIT

As @Oliver W. has correctly pointed out, the np.where is unnecessary as np.isnan will produce a boolean index that you can use to index into the arrays:

In [199]:

A = np.array([np.nan,   2,   np.nan,   3,   4])
B = np.array([   1  ,   2,     3   ,   4,  np.nan])
idx = (~np.isnan(A+B))
print(A[idx])
print(B[idx])
[ 2.  3.]
[ 2.  4.]

edited Mar 18, 2015 at 12:04

answered Mar 18, 2015 at 11:21

EdChum

397k204 gold badges836 silver badges583 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Oliver W. Over a year ago

Using np.where is entirely superfluous though.

EdChum Over a year ago

@OliverW. Hmm. yes you are correct, I was originally thinking that I needed the integer indices but yes it's unnecessary here, I'll update my answer

FuzzyDuck · Accepted Answer · 2015-03-18 11:22:23Z

8

A[~(np.isnan(A) | np.isnan(B))]

B[~(np.isnan(A) | np.isnan(B))]

answered Mar 18, 2015 at 11:22

FuzzyDuck

1,53112 silver badges14 bronze badges

Collectives™ on Stack Overflow

Removing NaNs in numpy arrays

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related