Enhancing OpenCV Masking

Question

I made a program that applies a mask over an object as described in this StackOverflow question. I did so using colour thresholding and making the mask select only the colour range of human skin (I don't know if it works for white people as I am not white and it works well for me). the problem is when I run it, some greys (grey area on the wall or a shadow) are also picked up on the mask and it is applied there. I wanted to know whether there was a way to remove the unnecessary bits in the background, and/or if there was a way using object detection I could solve this. PS I tried using createBackgroundSubtractorGMG/MOG/etc but that came out very weird and way worse. Here is my code:

import cv2
from cv2 import bitwise_and
from cv2 import COLOR_HSV2BGR
import numpy as np
from matplotlib import pyplot as plt




cap = cv2.VideoCapture(0)

image = cv2.imread('yesh1.jpg')
bg = cv2.imread('kruger.jpg')
bg = cv2.cvtColor(bg, cv2.COLOR_BGR2RGB)

kernel1 = np.ones((1,1),np.uint8)
kernel2 = np.ones((10,10),np.uint8)

while (1):
     ret, frame = cap.read()
     hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
     lowerBound = np.array([1, 1, 1])
     upperBound = np.array([140, 255 ,140])

     mask = cv2.inRange(hsv, lowerBound, upperBound)
     blur = cv2.GaussianBlur(mask,(5,5),0)
     ret1,mask = cv2.threshold(blur,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
     mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel1)
     contourthickness = cv2.cvtColor(mask, cv2.IMREAD_COLOR)
     res = bitwise_and(frame, frame, mask = mask)
     crop_bg = bg[0:480, 0:640]
     final = frame + res
     final = np.where(contourthickness != 0, crop_bg, final)


     cv2.imshow('frame', frame)
     cv2.imshow('Final', final) # TIS WORKED BBYY



     key = cv2.waitKey(1) & 0xFF
     if key == 27:
         break

cv2.destroyAllWindows()

EDIT: Following @fmw42 's comment, I am adding the original image as well as a screenshot of how the different frames look. The masked image also changes colour. Something to fix that will also be helpful.

Post your original image so others can test your code and see where it might be improved rather than trying to remove the bad noise. Perhaps something earlier in your process will help. Possibly it is the otsu thresholding that is not adequate. — fmw42
– fmw42, Commented Apr 14, 2022 at 19:39
You are never going to get a very robust solution using OpenCV basics. I found this backgroundremover package that works great (there are other deep learning based solutions that are specialized in human body). Here is the result. — Rotem
– Rotem, Commented Apr 15, 2022 at 9:16
As per @Rotem suggestion, see also remove.bg. I found that to be the best online bg removal — fmw42
– fmw42, Commented Apr 15, 2022 at 15:15

score 1 · Accepted Answer · 2022-04-15 13:53:14Z

1

@Jeremi. Your code working 100%. Using White wall for background. Avoid door(it is not white, it is cream), shadow around edge, to prevent noising. If you have white bed sheet or white walls. I am using Raspberry pi 4b/8gb, 4k monitor. I can't get actual size of window.

Here is output:

What you see on my output. I placed my hand behind white sheet closer to camera. I do not have white wall on my room. My room is greener. That why you see logo on background. Btw, I can move my hand no problem.

Collectives™ on Stack Overflow

Enhancing OpenCV Masking

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related