5

Purpose of the Code: to recognize a button on the screen/get its coordinates automatically without human intervention. (The code is supposed to find a 2D array inside a bigger 2D array).

How i tried to solve it: I stored each RGB pixel value in a 2D array (a[][] also called the big array). I Stored the button pixels/smaller 2Darray in the optionArrayButton[][]. Then coded these steps: (Look at the arrays below while reading this).

  1. Get the number for SmallerArray[0][0] = firstSmallerArray Number
  2. Check Biger Array for FirstSmallerArray number by going through [0][0] to [0][end] then [1][0] to [1][end] and so on.
  3. If firstSmallerArray number is not found return -1 or not Found.
  4. Else Get position of where it is found in the bigger Array.
  5. Get the height of Smaller Array (smallerArray.length) and width (smallerArray[0].length).
  6. Using firstSmallerArray number, smallerArray.length, and smallerArray[0].length store in temp array.
  7. check if the temp == smallerArray and get coordinates.

What i need help with: for some reason even though the smaller array is inside the larger array it says button is not found (foundButton returns false). I have spent two days on it and couldn't find whats wrong.

since the arrays I am using has 2 million+ RGB values I am just going to give these arrays instead for example. Bigger array:

[3 3 1 0 9]
[4 1 5 4 5]
[7 5 6 2 8]
[8 2 7 3 5]
[1 8 7 6 4]

Smaller array:

[5 6 2]
[2 7 3]
[8 7 6]

I am bit of a noob at coding so I likely won't understand java/coding terms. Again thanks for anyone that can help.

DataStorage DataStorageObject = new DataStorage();
int[][] optionArrayButton = DataStorageObject.optionArrayButton();

int firstSmallerArrayNumber = optionArrayButton[0][0]; //Step 1
int heightOfSmallerArray = optionArrayButton.length; //Step 5
int widthOfSmallerArray = optionArrayButton[0].length; //Step 5

boolean foundButton = false;

//a[][] has the screens rgb values
for(int yaxisCounter = 0; yaxisCounter < 300; yaxisCounter++) //Step 2
{
   for(int xaxisCounter = 0; xaxisCounter < 300; xaxisCounter++) //Step 2
   {
       if(a[yaxisCounter][xaxisCounter] == firstSmallerArrayNumber) //Step 4
       {
          int[][] tempArray = new int[heightOfSmallerArray][widthOfSmallerArray];  //Step 6
        //  System.out.println(" " + yaxisCounter + ", " + xaxisCounter);
          for(int ycounterForTemp = 0; ycounterForTemp < heightOfSmallerArray; ycounterForTemp++)  //Step 6
          {
             for(int xcounterForTemp = 0; xcounterForTemp < widthOfSmallerArray; xcounterForTemp++)  //Step 6
             {
                tempArray[ycounterForTemp][xcounterForTemp] = a[yaxisCounter][xaxisCounter];  //Step 6
         //       System.out.println("Storing in temp");
             }
          }

          foundButton = isArrayEqual(tempArray, optionArrayButton); //Step 7
        //  System.out.println("Button found is a " + foundButton + " statement");

          if(foundButton)
          {
              basePointy = yaxisCounter;
              basePointx = xaxisCounter;

         //    System.out.println("Base Point y is: " + basePointy);
         //    System.out.println("Base Point x is: " + basePointx);

           }

           //If there are any problems this is where it would happen
           else
           {
             //   System.out.println("Button Found is a : " + "false"  + " statement");
              //  System.out.println("In the nested Else");
                continue;
            }


        }
        else
        {
         //   System.out.println("In the else");
            continue;
        }
    }
}

//    System.out.println("Button Found is a : " + foundButton + " statement");
2
  • How exactly are you recognizing a button in your algorithm? Commented Dec 29, 2013 at 17:31
  • 1
    Please don't forget to add a '?' to questions! Some people do a search in the page for '?' and if none exists in the 'question' go directly to the next (actual) question in line. Commented Dec 29, 2013 at 17:38

1 Answer 1

3

Use this approach:

int[][] matrix = ...;
int[][] submatrix = ...;

loopX: for (int x = 0; x < matrix.length - submatrix.length + 1; ++x)
loopY: for (int y = 0; y < matrix[x].length - submatrix[0].length + 1; ++y)
{
    for (int xx = 0; xx < submatrix.length; ++xx)
    for (int yy = 0; yy < submatrix[0].length; ++yy)
    {
        if (matrix[x + xx][y + yy] != submatrix[xx][yy])
        {
            continue loopY;
        }
    }

    // Found the submatrix!
    System.out.println("Found at: " + x + " " + y);
    break loopX;
}
System.out.println("Done");

Your magic number 300 is suspicious. Maybe your button is further than 300 pixels from the left or top? Also be sure that you use lossless images. Only 1 bit wrong and this fails.

Sign up to request clarification or add additional context in comments.

1 Comment

i was just trying something with the 300 didn't know it was there

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.