How to record mouse and keyboard movement simultaneously with Python?

Question

I want to create a function that records both mouse and keyboard events until a specific key is pressed and then replays them together.

I think that this can be achieved with the keyboard and mouse modules. In an earlier question, I asked how to record the mouse movement until a key is pressed, and I got the following code:

import mouse
import keyboard

events = []                 #This is the list where all the events will be stored
mouse.hook(events.append)   #starting the mouse recording
keyboard.wait("a")          #Waiting for 'a' to be pressed
mouse.unhook(events.append) #Stopping the mouse recording
mouse.play(events)          #Playing the recorded events

That works fine. As both modules were made by the same people, I assumed that the same would work with the keyboard module. But it doesn't.

mouse_events = []
keyboard_events = []

mouse.hook(mouse_events.append)
keyboard.hook(keyboard_events.append)

keyboard.wait("a")

mouse.unhook(events.append)
keyboard.unhook(events.append)
keyboard.play(events)

The keyboard.hook(events.append) line in the code above throws an error: TypeError: unhashable type: 'list'.

I tried to check the module files but I fail to understand most of it.

So, to summarize: How can I start the mouse and keyboard recording at the same moment, stop them both at the same time and run both simultaneously? Are the mouse and keyboard modules the best option to achieve this?

Nouman · Accepted Answer · 2019-08-27 17:45:14Z

10

There is a problem with your code.
Your lists are:

mouse_events = []
keyboard_events = []

But you are using events.append rather than the list name. Looks like you forgot to modify the code.

The error is thrown because the module keyboard uses dict for hook unlike module mouse and you can't use list as keys.

You can solve this by using lambda:

keyboard.hook(lambda _: keyboard_events.append(_))

There is more simpler way to do this without using hook but its only for module keyboard

Use start_recording() and stop_recording()

1) start_recording() to enable the recording of keyboard events. It doesn't take a callback and you can record once at a time.
2) stop_recording() to stop the started recording. It returns the list of recorded events.

_{mouse module doesn't have stop/start_recording}
So your final code will look like this:

import mouse
import keyboard

mouse_events = []


mouse.hook(mouse_events.append)
keyboard.start_recording()       #Starting the recording

keyboard.wait("a")

mouse.unhook(mouse_events.append)
keyboard_events = keyboard.stop_recording()  #Stopping the recording. Returns list of events

Playing the events together:

The only way to play both events at the same time is to use threading

Here is an example for your code:

import threading
import mouse
import keyboard

mouse_events = []


mouse.hook(mouse_events.append)
keyboard.start_recording()

keyboard.wait("a")

mouse.unhook(mouse_events.append)
keyboard_events = keyboard.stop_recording()

#Keyboard threadings:

k_thread = threading.Thread(target = lambda :keyboard.play(keyboard_events))
k_thread.start()

#Mouse threadings:

m_thread = threading.Thread(target = lambda :mouse.play(mouse_events))
m_thread.start()

#waiting for both threadings to be completed

k_thread.join() 
m_thread.join()

edited Aug 27, 2019 at 17:45

answered Aug 27, 2019 at 8:21

Nouman

7,3257 gold badges38 silver badges66 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Paulo Schreiner Over a year ago

Thanks, but can you play them together, at the same time? Using mouse.play and then keyboard.play is not what I expected to do. Any ideas?

Nouman Over a year ago

@PauloSchreiner Its possible through threading. Check the edit

Paulo Schreiner Over a year ago

Perfect. Thanks!

Wboy Over a year ago

The events are out of sync tho, the mouse plays faster than the keyboard. Is there a fix for this?

Nouman Over a year ago

@Wboy I am not sure but the possible reason is that modules are glitching

|

RMPR · Accepted Answer · 2020-06-28 22:04:28Z

2

You can also use a software developed for this purpose, plus the events will be in sync, here is a GIF showcasing a basic use:

In the background it creates a simple Python file with the modules Pyautogui and the recording is done with Pynput

Disclaimer: I am the author.

edited Jun 28, 2020 at 22:04

answered Jun 28, 2020 at 18:46

RMPR

3,5314 gold badges22 silver badges33 bronze badges

Collectives™ on Stack Overflow

How to record mouse and keyboard movement simultaneously with Python?

2 Answers 2

6 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related