Python: Fast way to remove horizontal black line in image

Question

I would like to remove horizontal black lines on an image:

To do this, I interpolate the RGB values of each column of pixels.

The black line disappear but I think it is possible to optimize this function:

def fillMissingValue(img_in):
    img_out = np.copy(img_in)

    #Proceed column by column
    for i in tqdm(range(img_in.shape[1])):
        col = img_in[:,i]

        col_r = col[:,0]
        col_g = col[:,1]
        col_b = col[:,2]
        r = interpolate(col_r)
        g = interpolate(col_g)
        b = interpolate(col_b)
        img_out[:,i,0] = r
        img_out[:,i,1] = g
        img_out[:,i,2] = b

    return img_out

def interpolate(y):
    x = np.arange(len(y))
    idx = np.nonzero(y)
    interp = interp1d(x[idx],y[idx],fill_value="extrapolate" )

    return interp(x)

if __name__ == "__main__":
    img = cv2.imread("lena.png")
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

    img = cv2.resize(img, (1024,1024))


    start = time.time()
    img2 = fillMissingValue(img)
    end = time.time()

    print("Process time: {}".format(np.round(end-start,3)))

Do you have any ideas ? I thought of doing a prepocessing step by identifying the position of the black lines. And thus only interpolated neighboring pixels. But I don't think it's faster

Current result:

Can you post your original image without the outer tick marks and labels? — fmw42
– fmw42, Commented Feb 15, 2022 at 19:38
Minor, but seems to me that in interpolate, for given size of an image, x will be the same in all calls, no need to recalculate it (and allocate a new array) each time. — Dan Mašek
– Dan Mašek, Commented Feb 15, 2022 at 19:50
Can we assume that the lines are truly horizontal, and run from edge to edge? If so, we can also calculate idx once for the whole image... and then with a little bit of reshaping do the whole thing in a single call to interp1d. A quick prototype: pastebin.com/zHxYmdjx -- variant 3 takes 1/6 of the time yours did (without the progress bar). — Dan Mašek
– Dan Mašek, Commented Feb 15, 2022 at 20:36
your code is somewhat obfuscated. you use np.nonzero to differentiate between picture and BLACK line (zero).... — Christoph Rackwitz
– Christoph Rackwitz, Commented Feb 16, 2022 at 0:04
as for different solutions... inpainting should work nicely. or detect each black line, and then take the adjacent two non-black lines (check they are non-black!) and calculate the average and assign that. — Christoph Rackwitz
– Christoph Rackwitz, Commented Feb 16, 2022 at 12:10

Jérôme Richard · Accepted Answer · 2022-02-16 23:38:43Z

interp1d is not very efficient.

As proposed by @ChristophRackwitz in the comments, you can detect the location of the lines and use the inpainting method provided by OpenCV:

img = cv2.imread('lena.jpg')

# Locate the relatively black lines
threshold = 25
lineIdx = np.where(np.mean(img, axis=(1,2)) < threshold)

# Perform inpainting on the located lines
mask = np.zeros(img.shape[:2], dtype=np.uint8)
mask[lineIdx] = 255

# Actual inpainting.
# Note: using 2 or 1 instead of 3 make the computation 
# respectively ~2 and ~4 time faster on my machine but 
# the result is not as beautiful with 3.
img2 = cv2.inpaint(img, mask, 3, cv2.INPAINT_NS)

The computational part takes 87 ms on my machine while your code takes 342 ms. Note that because of JPEG compression, the result is not so great. You can inpaint the neighbour lines (eg. lineIdx-1 and lineIdx+1) so to get a much better result at the expense of the slower computation (about 2.5 slower on my machine).

An alternative solution is to perform the interpolation yourself in Numpy:

%%time
# Locate the relatively black lines
threshold = 25
lineIdx = np.where(np.mean(img, axis=(1,2)) < threshold)[0]
lineIdxSet = set(lineIdx)

img2 = img.copy()
start, end = None, None
interpolate = False
for i in range(img.shape[0]+1):
    if i in lineIdxSet:
        if start is None:
            start = i
        end = i
    else:
        if not (start is None):
            assert not (end is None)
            # The first lines are black
            if start <= 0:
                i0, i1 = end+1, end+1
            # The last lines are black
            elif end >= img.shape[0]-1:
                i0, i1 = start-1, start-1
            # usual case
            else:
                i0, i1 = start-1, end+1
            # The full image is black
            if i0 < 0 or i1 >= img.shape[0]:
                continue
            end = min(end, img.shape[0]-1)
            # Actual linear interpolation (of a bloc of lines)
            coef = np.linspace(0, 1, end-start+3)[1:-1].reshape(-1, 1)
            l0 = img[i0].reshape(-1)
            l1 = img[i1].reshape(-1)
            img2[start:end+1] = (coef * l0 + (1.0-coef) * l1).reshape(-1, img.shape[1], 3)
        start, end = None, None

This code takes only 5 ms on my machine. The result should be similar to the one of your original code except that it works line-by-line and not column by column and that the detection is not independent for each colour channel. Note that the inpainting method give more beautiful results when large blocs of lines are black.

Collectives™ on Stack Overflow

Python: Fast way to remove horizontal black line in image

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related