Propagate/forward-fill nan values in numpy array [duplicate]

Question

I need to forward-fill nan values in a numpy array along the columns (axis=0). I am looking for a pure numpy solution that yields the same result as pd.fillna(method="ffill").

import numpy as np
import pandas as pd

arr = np.array(
    [
        [5, np.nan, 3],
        [4, np.nan, np.nan],
        [6, 2, np.nan],
        [2, np.nan, 6],
    ]
)

expected = pd.DataFrame(arr).fillna(method="ffill", axis=0)  # I need this line in pure numpy

print(f"Original array:\n {arr}\n")
print(f"Expected array:\n {expected.values}\n")

Original array:
 [[ 5. nan  3.]
 [ 4. nan nan]
 [ 6.  2. nan]
 [ 2. nan  6.]]

Expected array:
 [[ 5. nan  3.]
 [ 4. nan  3.]
 [ 6.  2.  3.]
 [ 2.  2.  6.]]

I had this same issue. Frustrated the hell out of me. Ended up using a chained pandas instruction. [pd.Series(array).bfill().to_numpy()]. — Paul
– Paul, Commented Jul 30, 2023 at 7:27

Vikrant Gupta · Accepted Answer · 2022-01-11 11:56:47Z

2

No inbuilt function in numpy to do this. Below simple code will generate desired result using numpy array only.

row,col = arr.shape
mask = np.isnan(arr)
for i in range(1,row):
    for j in range(col):
        if mask[i][j]:
            arr[i][j] =arr[i-1][j]

answered Jan 11, 2022 at 11:56

Vikrant Gupta

2472 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ashwiniku918 · Accepted Answer · 2022-01-11 11:30:51Z

0

Bottleneck push function is a good option to forward fill. It's normally used internally in packages like Xarray.

from bottleneck import push
push(arr, axis=0)

answered Jan 11, 2022 at 11:30

Ashwiniku918

3312 silver badges7 bronze badges

2 Comments

Andi Over a year ago

Usually this answer would be sufficient, however, I am not able to install the package in my environment. Hence, I am looking for a pure numpy (or maybe numba) solution.

Ashwiniku918 Over a year ago

This link might help with various : stackoverflow.com/questions/41190852/…

Collectives™ on Stack Overflow

Propagate/forward-fill nan values in numpy array [duplicate]

2 Answers 2

Comments

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Linked

Related