Python - add variables including NaN

Question

Example Dataset:

var1  var2  result
1     4     5
2     NaN   2
3     5     8
NaN   6     6
NaN   NaN   NaN

I would like to create a new variable by summing var1 & var2, as seen above. However - I don't want to just replace NaN in var2 with 0s.

This question was closed because someone pointed me towards a question that would fill var2 with zeros. I need the sum to work if either are NaN, and if var1 & var2 are both NaN, I need the resulting new variable to by NaN. I feel like this wasn't answered by the question duplicate I was directed to.

Thanks in Advance

Try: v3 = v1 + (v2 if not math.isnan(v2) else 0)

jarmod
– jarmod

2020-11-10 16:14:49 +00:00
Commented Nov 10, 2020 at 16:14 — jarmod
– jarmod, Commented Nov 10, 2020 at 16:14

anatolyg · Accepted Answer · 2020-11-30 18:29:07Z

3

A straightforward solution:

def mysum(var1, var2):
    if math.isnan(var1):
        return var2
    else:
        if math.isnan(var2):
            return var1
        else:
            return var1 + var2

If you want it to support a list of variables:

def mysum(vars):
    vars = [x for x in vars if not math.isnan(x)]  # delete all nans
    if vars:
        return sum(vars)
    else:
        return float('nan')

Note: numpy happens to have a function nansum which does exactly that, but its behavior was later changed for the case of all nans, so better define your own one.

edited Nov 30, 2020 at 18:29

answered Nov 30, 2020 at 18:21

anatolyg

28.5k9 gold badges66 silver badges149 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Cainã Max Couto da Silva · Accepted Answer · 2020-11-30 18:28:03Z

2

You can specify min_count=1 in pandas sum:

df['result'] = df[['var1','var2']].sum(axis=1, min_count=1)

Result:

   var1  var2  result
0   1.0   4.0     5.0
1   2.0   NaN     2.0
2   3.0   5.0     8.0
3   NaN   6.0     6.0
4   NaN   NaN     NaN

From docs:

min_countint, default 0
The required number of valid values to perform the operation. If fewer than min_count non-NA values are present the result will be NA.

edited Nov 30, 2020 at 18:28

answered Nov 30, 2020 at 18:22

Cainã Max Couto da Silva

4,9691 gold badge15 silver badges39 bronze badges

Collectives™ on Stack Overflow

Python - add variables including NaN

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related