1

I am working on a project and I ran into a situation. I want to detect a rectangle object (a black keyboard) in an IR image. The background is pretty clean so it's not really a hard problem, I used simple threshold and minAreaRect in OpenCV to solve it. Easy case of the problem

But I also want the program to track this object when I use my hand to move it (yes, in real time). And my hand will cover a small part of the object like this case. Tricky case of the problem

My initial thought is to learn the object size in the easy case, and for the hard case, try to match my "learned rectangle" to cover as many white pixels as possible.

Anyone has a better solution, maybe a feature-based approach? I don't know if using features can improve the situation because object is mostly black in these IR images.

Thank you in advance.

Improve this question

2 Answers 2

Reset to default
2

How about using morphological operations like dilation and erosion (Opencv has implementations for these) on the thresholded image. Once you get that, you could try some corner detection/contour detection or line detectors(in opencv contrib module) to understand the shape of the object.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thank you. I tried using morphological operations and it seemed to work.
@duy_sau_rom If you found the answer useful, and it solved your problem, please consider upvoting and marking the answer as 'accepted'
I am new on stackoverflow and need a little bit more reputation to upvote. I will come back and upvote for you later. :')
1

Your "tricky" case is still fairly simple, can be solved with dilate/erode (as mentioned by Shawn Mathew) and then the same minAreaRect. Here, on the right is your thresholded image after erosion and dilation with a 5x5 kernel, minAreaRect finds a rotated rectangle for it, drawn over the original thresholded image on the left:

enter image description here

Are you interested in more complicated cases, for example, where you hand covers one of the short edges of the keyboard entirely?

Improve this answer

4 Comments

Thank you for this answer. I guess that case is pretty tough and can't be solved easily without knowing the original size of the keyboard right?
I don't think it can be solved at all without knowing the original size, but if you know not even the size, but the long/short side ratio, it becomes somewhat simple: minAreaRect gives you a shortened version of rectangle, you estimate, which one of the short side has less white pixels next to it, then extend the rectangle in that direction. Something like that.
What do you think about the case with a white (or another color) keyboard. I think we can still look at it and know it's a keyboard, but it's not that easy for a program to do so.
It may be not too difficult to cook something up for any particular setup (e. g. "a white keyboard of known size under known viewing angle, lighting conditions, background etc.") What would probably be really difficult (if at all possible) is to do it for a more general case (e. g. "a keyboard of any color, any size, under any angle, etc.", the more of those "any" you want, the more difficult it gets).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.