aws-sam-cli Python requirements.txt missing from deployed Lambda when custom template.yaml

I have several AWS SAM templates for the same project, where the lambdas use Python 3.8, partly for different environments, and partly to try out new things, and I use -t my-template.yaml to specify which template to use.

I build using the regular sam build -t my-template.yaml && sam deploy -t my-template.yaml.

Problem: the requirements.txt file seems to be respected in the build step appears to be ignored by the deploy step.

.aws-sam/build/MyFunction contains the requirements:

$ ls
app.py   
certifi-2020.12.5.dist-info  
chardet-4.0.0.dist-info  
idna-2.10.dist-info  
pymysql                  
rds_config.py  
requests-2.25.1.dist-info  
urllib3
certifi  
chardet                      
idna                     
__init__.py          
PyMySQL-1.0.2.dist-info  
requests       
requirements.txt           
urllib3-1.26.3.dist-info

But when running in the cloud, the lambda fails with an exception saying that pymysql is not found, and when I check the function in the AWS UI, there are no requirements included (eg. no PyMySQL-1.0.2.dist-info).

I noticed that if I don't use the -t flag on sam deploy and just have the default template.yaml file there, then the requirements appear to be included correctly, and the lambda runs successfully.

What may I be doing wrong?

requirements.txt contents:

$ cat requirements.txt
requests
pymysql

SAM version:

$ sam --version
SAM CLI, version 1.18.1

OS:

$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
....