Python - Take screenshot including mouse cursor

Question

I am currently trying to take a screenshot who include mouse cursor using PIL.ImageGrab.

This is my code:

import ctypes, win32gui, win32ui
from PIL import Image, ImageGrab

size = round(ctypes.windll.shcore.GetScaleFactorForDevice(0) / 100 * 32)

cursor = get_cursor()

pixdata = cursor.load()
minsize = [size, None]

width, height = cursor.size
for y in range(height):
    for x in range(width):

        if pixdata[x, y] == (0, 0, 0, 255):
            pixdata[x, y] = (0, 0, 0, 0)

        else:
            if minsize[1] == None:
                minsize[1] = y

            if x < minsize[0]:
                minsize[0] = x

ratio = ctypes.windll.shcore.GetScaleFactorForDevice(0) / 100

img = ImageGrab.grab(bbox=None, include_layered_windows=True)

pos_win = win32gui.GetCursorPos()
pos = (round(pos_win[0]*ratio), round(pos_win[1]*ratio))


img.paste(cursor, pos, cursor)

img.save("screenshot.png")

And this is my get_cursor() function:

def get_cursor():

    hcursor = win32gui.GetCursorInfo()[1]
    hdc = win32ui.CreateDCFromHandle(win32gui.GetDC(0))
    hbmp = win32ui.CreateBitmap()
    hbmp.CreateCompatibleBitmap(hdc, 36, 36)
    hdc = hdc.CreateCompatibleDC()
    hdc.SelectObject(hbmp)
    hdc.DrawIcon((0,0), hcursor)
    
    bmpinfo = hbmp.GetInfo()
    bmpbytes = hbmp.GetBitmapBits()
    bmpstr = hbmp.GetBitmapBits(True)
    cursor = Image.frombuffer('RGB', (bmpinfo['bmWidth'], bmpinfo['bmHeight']), bmpstr, 'raw', 'BGRX', 0, 1).convert("RGBA")
    
    win32gui.DestroyIcon(hcursor)    
    win32gui.DeleteObject(hbmp.GetHandle())
    hdc.DeleteDC()


    pixdata = cursor.load()
    minsize = [32, None]

    width, height = cursor.size
    for y in range(height):
        for x in range(width):

            if pixdata[x, y] == (0, 0, 0, 255):
                pixdata[x, y] = (0, 0, 0, 0)

            else:
                if minsize[1] == None:
                    minsize[1] = y

                if x < minsize[0]:
                    minsize[0] = x


    return cursor

The problem is that some cursors are not pasted at the right position because they have pixels to the left of their position like this (do not pay attention to the quality).

How can I place the cursor image correctly (or otherwise solve the problem)?

Add a delay before the image grab so that you can position the cursor in the correct place. This is similar to the delay feature in the windows snipping tool — cup
– cup, Commented May 30, 2022 at 7:43
@cup The problem is not the actual position of the cursor but how the image returned by my get_cursor function is placed. On both screenshots, the real cursor is exactly at the same position (and is not moving) — crazycat256
– crazycat256, Commented May 30, 2022 at 7:51
github.com/BoboTiG/python-mss/issues/55#issuecomment-580481146 there is a part of code you want — Kal-1
– Kal-1, Commented May 31, 2022 at 9:33
@Kal-1 I tried this solution but I can't save or display this image format — crazycat256
– crazycat256, Commented May 31, 2022 at 12:31
"The problem is not the actual position of the cursor but how the image returned by my get_cursor function is placed." Yes; cursors have a "hot spot" which indicates where the OS will position the cursor image relative to the actual mouse position. It's effectively hard-coded data that depends on the image. It might be represented in the win32gui.GetCursorInfo() result somewhere; try reading the documentation. In the worst case, you can just hard-code the offset yourself. — Karl Knechtel
– Karl Knechtel, Commented Jun 1, 2022 at 20:17

crazycat256 · Accepted Answer · 2022-06-02 06:39:03Z

Mouse cursors have a "hot spot" which is the position where the image is placed relative to the cursor position This "hot spot" can be obtained like this:

import win32gui

hcursor = win32gui.GetCursorInfo()[1]
hotspot = win32gui.GetIconInfo(hcursor)[1:3]

So the full code is:

import ctypes, win32gui, win32ui
from PIL import Image, ImageGrab


def get_cursor():
    hcursor = win32gui.GetCursorInfo()[1]
    hdc = win32ui.CreateDCFromHandle(win32gui.GetDC(0))
    hbmp = win32ui.CreateBitmap()
    hbmp.CreateCompatibleBitmap(hdc, 36, 36)
    hdc = hdc.CreateCompatibleDC()
    hdc.SelectObject(hbmp)
    hdc.DrawIcon((0,0), hcursor)
    
    bmpinfo = hbmp.GetInfo()
    bmpstr = hbmp.GetBitmapBits(True)
    cursor = Image.frombuffer('RGB', (bmpinfo['bmWidth'], bmpinfo['bmHeight']), bmpstr, 'raw', 'BGRX', 0, 1).convert("RGBA")
    
    win32gui.DestroyIcon(hcursor)    
    win32gui.DeleteObject(hbmp.GetHandle())
    hdc.DeleteDC()


    pixdata = cursor.load()


    width, height = cursor.size
    for y in range(height):
        for x in range(width):

            if pixdata[x, y] == (0, 0, 0, 255):
                pixdata[x, y] = (0, 0, 0, 0)


    hotspot = win32gui.GetIconInfo(hcursor)[1:3]

    return (cursor, hotspot)

cursor, (hotspotx, hotspoty) = get_cursor()
cursor.save("cursor.png")


ratio = ctypes.windll.shcore.GetScaleFactorForDevice(0) / 100

img = ImageGrab.grab(bbox=None, include_layered_windows=True)

pos_win = win32gui.GetCursorPos()
pos = (round(pos_win[0]*ratio - hotspotx), round(pos_win[1]*ratio - hotspoty))


img.paste(cursor, pos, cursor)

img.save("screenshot.png")

Ahmed Anwar · Accepted Answer · 2024-05-16 05:19:55Z

1

You can try using a pointer and pyautogui for the purpose. It's pretty simple and might serve your purpose.

import pyautogui
from PIL import ImageGrab, ImageDraw

# Take the screenshot
img = ImageGrab.grab()

# Get the cursor position
x, y = pyautogui.position()

# Create an ImageDraw object
draw = ImageDraw.Draw(img)

# Draw a circle at the cursor position
draw.ellipse((x, y, x+15, y+15), fill='white')

# Save the screenshot
img.save('screenshot_with_cursor.png')

answered May 16, 2024 at 5:19

Ahmed Anwar

112 bronze badges

1 Comment

Community Over a year ago

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

Collectives™ on Stack Overflow

Python - Take screenshot including mouse cursor

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related