4

I am having some issues with detecting specific "blobs" in a set of images. Not all images are the same, but I suppose the same parameters would be used to detect anyways. Image1

If you zoom in, you will see small, yellow aphids on the leaf. My goal is to single these out and count them. I don't really need to do much to the image, just obtain a count of them.

Right now, I have this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Emgu.CV;
using Emgu.CV.Features2D;
using Emgu.CV.Structure;
using Emgu.CV.Util;

namespace AphidCounter
{
    class Program
    {
        static void Main(string[] args)
        {
            // Read image
            Mat im_in = CvInvoke.Imread("myimage1.jpg", Emgu.CV.CvEnum.LoadImageType.Grayscale);
            //Mat im_in = CvInvoke.Imread("myimage2.png", Emgu.CV.CvEnum.LoadImageType.Color);
            Mat im = im_in;
            CvInvoke.Threshold(im_in, im, 40, 255, Emgu.CV.CvEnum.ThresholdType.BinaryInv);  // 60, 255, 1

            //CvInvoke.NamedWindow("Blob Detector", Emgu.CV.CvEnum.NamedWindowType.AutoSize);

            DetectBlobs(im, 0);

            CvInvoke.WaitKey(0);
        }

        static void DetectBlobs(Mat im, int c)
        {

            int maxT = 50;
            int minA = 125; // Minimum area in pixels
            int maxA = 550; // Maximum area in pixels

            SimpleBlobDetectorParams EMparams = new SimpleBlobDetectorParams();
            SimpleBlobDetector detector;

            EMparams.MinThreshold = 0;
            EMparams.MaxThreshold = 100;

            if (minA < 1) minA = 1;
            EMparams.FilterByArea = true;
            EMparams.MinArea = minA;
            EMparams.MaxArea = maxA;

            if (maxT < 1) maxT = 1;
            EMparams.MinConvexity = (float)maxT / 1000.0F; // 0.67

            EMparams.FilterByInertia = true;
            EMparams.MinInertiaRatio = 0.01F;

            EMparams.FilterByColor = true;
            EMparams.blobColor = 0;

            VectorOfKeyPoint keyPoints = new VectorOfKeyPoint();

            detector = new SimpleBlobDetector(EMparams);
            detector.DetectRaw(im, keyPoints);

            Mat im_with_keypoints = new Mat();
            Bgr color = new Bgr(0, 0, 255);
            Features2DToolbox.DrawKeypoints(im, keyPoints, im_with_keypoints, color, Features2DToolbox.KeypointDrawType.DrawRichKeypoints);

            // Show blobs
            CvInvoke.Imwrite("keypoints1.jpg", im_with_keypoints);
            CvInvoke.Imshow("Blob Detector " + keyPoints.Size, im_with_keypoints);

            System.Console.WriteLine("Number of keypoints: " + keyPoints.Size);

        }
    }
}

However, this is the result: Image2

Am I not getting the parameters right? Or is there something else that I'm missing?

Improve this question
1
  • Please provide intermediate images like your binary image. Commented Feb 24, 2016 at 11:52

1 Answer 1

Reset to default
4

It is not because of some wrong parameters. The image segmentation part itself has its limitation.

Grayscale based thresholding may not work when the contrast between the blob and the background is very low. Yet a threshold value around 160 is quite tolerable in this example but not any accurate.

I would suggest to go for colour based thresholding since there is a decent colour gap.

Here is a C++ implementation of colour based thresholding. Blobs are filtered using the same SimpleBlobDetector.

I have converted the image from RGB to ‘Lab’ for better segmentation.

As the image provided is too huge, it took more time to process. So I cropped a key part of the image and tuned the blob params for the same. So I provide the cropped image too (755 x 494px).

leaf_cropped

Colour based thresholding and blob filtering:

#include "opencv2\imgproc\imgproc.hpp";
#include "opencv2\highgui\highgui.hpp";
#include "opencv2\features2d\features2d.hpp";

using namespace cv;
using namespace std;

void main()
{
    char image_path[] = "E:/Coding/media/images/leaf_small.jpg";
    Mat img_color, img_lab, img_thresh, img_open, img_close, img_keypoints;

    img_color = imread(image_path, IMREAD_ANYCOLOR);

    //Convert image to CIE Lab colorspace for better colour based segmentation
    cvtColor(img_color, img_lab, CV_BGR2Lab);

    //create window before creating trackbar
    namedWindow("win_thresh", WINDOW_NORMAL);
    namedWindow("win_blob", WINDOW_NORMAL);

    //Using trackbar calculate the range of L,a,b values to seperate blobs
    int low_L = 150, low_A = 0, low_B = 155,
        high_L = 255, high_A = 255, high_B = 255;

    //*Use trackbars to caliberate colour thresholding
    createTrackbar("low_L", "win_thresh", &low_L, 255);
    createTrackbar("low_A", "win_thresh", &low_A, 255);
    createTrackbar("low_B", "win_thresh", &low_B, 255);
    createTrackbar("high_L", "win_thresh", &high_L, 255);
    createTrackbar("high_A", "win_thresh", &high_A, 255);
    createTrackbar("high_B", "win_thresh", &high_B, 255);

    int minArea = 35, maxArea = 172, minCircularity = 58, minConvexity = 87, minInertiaRatio = 21;

    //Use trackbar and set Blob detector parameters
    createTrackbar("minArea", "win_blob", &minArea, 200);
    createTrackbar("maxArea", "win_blob", &maxArea, 200);
    createTrackbar("minCircular", "win_blob", &minCircularity, 99);
    createTrackbar("minConvex", "win_blob", &minConvexity, 99);
    createTrackbar("minInertia", "win_blob", &minInertiaRatio, 99);

    SimpleBlobDetector::Params params;
    vector<KeyPoint> keypoints;

    while (waitKey(1) != 27) //press 'esc' to quit
    {
        //inRange thresholds basedon the Scalar boundaries provided
        inRange(img_lab, Scalar(low_L, low_A, low_B), Scalar(high_L, high_A, high_B), img_thresh);

        //Morphological filling
        Mat strucElement = getStructuringElement(CV_SHAPE_ELLIPSE, Size(5, 5), Point(2, 2));
        morphologyEx(img_thresh, img_close, MORPH_CLOSE, strucElement);

        imshow("win_thresh", img_close);

        //**SimpleBlobDetector works only in inverted binary images
        //i.e.blobs should be in black and background in white.
        bitwise_not(img_close, img_close); // inverts matrix

        //Code crashes if minArea or any miin value is set to zero
        //since trackbar starts from 0, it is adjusted here by adding 1
        params.filterByArea = true;
        params.minArea = minArea + 1;
        params.maxArea = maxArea + 1;

        params.filterByCircularity = true;
        params.filterByConvexity = true;
        params.filterByInertia = true;

        params.minCircularity = (minCircularity + 1) / 100.0;
        params.minConvexity = (minConvexity + 1) / 100.0;
        params.minInertiaRatio = (minInertiaRatio + 1) / 100.0;

        SimpleBlobDetector detector(params);
        detector.detect(img_close, keypoints);
        drawKeypoints(img_color, keypoints, img_keypoints, Scalar(0, 0, 255), DrawMatchesFlags::DEFAULT);

        stringstream displayText;
        displayText = stringstream();
        displayText << "Blob_count: " << keypoints.size();
        putText(img_keypoints, displayText.str(), Point(0, 50), CV_FONT_HERSHEY_PLAIN, 2, Scalar(0, 0, 255), 2);

        imshow("win_blob", img_keypoints);
    }
    return;
}

Output Screenshot

output screenshot

Tune the blob parameters according to the actual HD image.

Since the veins of the leaf are almost of the same colour and intensity of the aphid, this method also may utterly fail when an aphid sits close to or exactly on top of a vein.

This can be an ad-hoc fix but not robust enough. There got to be a simple and robust method to achieve the result, using some filters, transformation or edge detection. Please share any other optimal solution if available.

EDIT: Opting Grayscale thresholding as previous approach failed

Colour thresholding approach failed for this_image

Colour based thresholding has a very narrow bandwidth, if the image falls within the bandwidth the accuracy will be really good, on the other hand colour shifts totally ruin the accuracy. Since you will be processing 100s of images, colour thresholding may not be suitable.

I tried normal Grayscale thresholding with some morphological erosion and filling, and got a decent accuracy. Also Grayscale thresholding has better immunity to colour shifts.

Additionally we have auto thrsholding option using OTSU Thresholding which selects the threshold value based on the image.

Code snippet:

threshold(img_gray, img_thresh, 0, 255, THRESH_OTSU);

Mat strucElement = getStructuringElement(CV_SHAPE_ELLIPSE, Size(3, 3), Point(1, 1));
morphologyEx(img_thresh, img_open, MORPH_OPEN, strucElement);

Rest of the code remains the same.

Parameter values:

minArea = 75, maxArea = 1000, minCircularity = 50, minConvexity = 20, minInertiaRatio = 15

The white ants are hard to differentiate from aphids as we are not using colour information. So the min_area has to be carefully tuned in order to exclude them.

Processed images can be found here img_1, img_2.

Tweak the morphology methods and blob parameters to obtain an optimal average count.

Improve this answer
Sign up to request clarification or add additional context in comments.

12 Comments

what sort of processing times did you get with and without the cropped image? My goal is to do this sort of stuff on the fly (20-45secs per image). There is an acceptable level of error that I assume will happen. I just want on average, the level to be no more than 5%-10% over a set of 100 images.
Here are a few pictures that I got from this. As you can see, the more aphids, the more it misses. Is this perhaps a color variance problem? i.imgur.com/zPizCnL.jpg i.imgur.com/uD8XK29.jpg
I have edited the answer to modify the thresholding method. It takes around 15-30 secs per HD image. My image processing knowledge is limited or else I would have tried advanced methods like templateMatching or foreground masking etc.
Also look for pre-processing functions to enhance the image before thresholding, toobtain better results.
Awesome. Sorry for the long delay - had some issues come up. One thing I've noticed is that it will double count some of them. I've tweaked the parameters and haven't found a good "middle ground" to reduce them without losing accuracy. Is it practical to exclude locations that were already marked, based on size? So if it detects an aphid of a certain size, we save the inner area of the circle and check to make sure subsequent hits are not inside those areas? I wonder how much that will slow it down...
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.