Quite apart from the whether the user exists in the docker container, I think you should also understand when $USER is resolved. When you run this command in your shell:
docker exec -it --user my-user my-container echo $USER
The string $USER is replaced with root before even calling upon the docker executable. (I'm assuming you're running it as as root, otherwise I've no idea where it's from.)
To avoid this, you'd have to escape the $, like this:
docker exec -it --user my-user my-container echo \$USER
When I tested this, the value of $USER was not known to the shell inside the container. If you want to see that the user running inside the container is indeed your user, try this:
docker exec -it --rm --user my-user my-container /bin/id -u
That will print the userid of my-user. Unfortunately, that user won't exist until you create it in the container, so it doesn't have a name. Another person has already posted a link in this question to address that.