I am trying to use the python selenium web-driver to locate the tile/button elements for googles minesweeper game, but they do not appear as individual HTML elements that can be referenced when inspected. I can reference the element containing the tiles to click any tile with selenium methods, but I have no way of accessing the state of each tile with python, as the element.text method returns None when called on the larger element. Is there any way to access the state of the tiles without screenshotting/text image recognition?
I took a screenshot of the element with selenium, and used Pillow to get a specific single pixel color from each tile that I hoped I would be able to consistently match with an expected tile state. The code looked like this:
def get_tile_color(self, x, y):
#updated screenshot
self.get_screenshot()
# converted number of tiles to pixels
x = (x * 30) - 15
y = (x * 30) - 15
with Image.open(r"board_screenshot.png") as im:
# tuple contained four values, so the first three are returned
return tuple(im.getpixel((x,y))[num] for num in range(3))
This approach was not consistent in the RGB values it returned.
