Text line segmentation using OpenCV Python

Question

I am trying to extract the text lines from the below figure and I tried the below code but I am getting only a single image without any data. While masking the images all the lines are masked perfectly. Below i attached the masked image, final output image and desired output image.

img = cv.imread('Handwritten_data/Ostu_images/H_1.jpg')

lower = (0, 0, 0)
upper = (0, 120, 150)

# threshold on border color
mask = cv.inRange(img, lower, upper)

# dilate threshold
kernel = cv.getStructuringElement(cv.MORPH_RECT, (250,10))
mask = cv.morphologyEx(mask, cv.MORPH_DILATE, kernel)

# recolor border to white
img[mask==255] = (255,255,255)


# Inverting the mask by
# performing bitwise-not operation
mask_black = cv.bitwise_not(mask)
Mask = cv.bitwise_and(img, img, mask = mask_black)

gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)

# otsu threshold
thresh = cv.threshold(gray, 0, 255, cv.THRESH_OTSU )[1] 

# apply morphology open
kernel = cv.getStructuringElement(cv.MORPH_RECT, (250,10)) 
morph = cv.morphologyEx(thresh, cv.MORPH_OPEN, kernel)


# creating a folder
try: 
    # creating a folder named data
    if not os.path.exists('Image_0'):
        os.makedirs('Image_0')
  
# if not created then raise error
except OSError:
    print ('Error: Creating directory of data')

# find contours and bounding boxes
bboxes = []
bboxes_img = img.copy()
contours = cv.findContours(morph, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
    x,y,w,h = cv.boundingRect(cntr)
    cv.rectangle(bboxes_img, (x, y), (x+w, y+h), (0,0,255), 1)
    bboxes.append((x,y,w,h))

for j in range(len(bboxes)):
    (x,y,w,h) = bboxes[j]
    crop = img[y-10:y+h+10, x-10:x+w+10]
    cv.imwrite(f'Image_0/S_{j}.jpg', crop)

Any suggestions or help to solve this problem.

Below is the masking image

final image ouput

desired image output be like

Thanks in advance

maybe first use print() to see what you have in variables and which part of code is executed. It is called "print debuging" and it helps to see what code is doing. Maybe you create list with only one item, or make other mistake. — furas
– furas, Commented Apr 5, 2022 at 15:16
you can use os.makedirs('Image_0', exist_ok=True) without checking if not os.path.exists('Image_0'): — furas
– furas, Commented Apr 5, 2022 at 15:17
maybe you should display image after every operation to see what you have on image. it seems you removed all text from image using img[mask==255] = (255,255,255) — furas
– furas, Commented Apr 5, 2022 at 15:23
it seems all lines are recognized but as single contour, not separated elements. — furas
– furas, Commented Apr 5, 2022 at 15:26
One issue you have is that most of the masked lines connect so that when you extract the contour bounding boxes, you will not get separated lines. You need to modify your morphology so that your masked regions are not touching. Make your morphology kernel only 1 or 2 pixels tall, not 10. — fmw42
– fmw42, Commented Apr 5, 2022 at 16:32

Jeru Luke · Accepted Answer · 2022-04-05 18:45:25Z

3

The idea you presented for text line segmentation is correct. But the approach needs some tweaking.

Note:

Whenever you want to mask a portion of the image, make sure the masked region is in white. Because when finding contours the algorithm looks for white regions. Since you were looking for contours that were in black the algorithm missed it.

The following is modified with the above idea.

Solution:

img = cv2.imread(image_file)
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh2 = cv2.threshold(img_gray, 150, 255, cv2.THRESH_BINARY_INV)
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (150,2))
mask = cv2.morphologyEx(thresh2, cv2.MORPH_DILATE, kernel)

Below is the mask image:

bboxes = []
bboxes_img = img.copy()
contours = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
    x,y,w,h = cv2.boundingRect(cntr)
    cv2.rectangle(bboxes_img, (x, y), (x+w, y+h), (0,0,255), 1)
    bboxes.append((x,y,w,h))

The following is the final result with bounding boxes:

You can now add your portion of the code to save each region as a separate image file. You can try modifying the kernel parameters and exploring different morphological operations to get a better mask region.

Hope this puts you on the right track!

answered Apr 5, 2022 at 18:45

Jeru Luke

21.4k13 gold badges85 silver badges91 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Nikhil Over a year ago

thank you for your time and suggestions for improvement of the code

Collectives™ on Stack Overflow

Text line segmentation using OpenCV Python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related