I am creating a Python AWS Lambda function that connects to db to extract data as CSV then sftp that CSV into an SFTP server (abc.example.com). I am using pysftp and Paramiko. Looks like pysftp needs a private key file for password less connection to SFTP host. How do I get this private key file?
Do we need to create a public/private key pair (ssh-keygen) at destination SFTP host? And then use the public part of that key within Lambda function?
Thanks
Yes, if you don't have it already then you have to create keys using ssh-keygen on sftp host and use it.
import pysftp
with pysftp.Connection('hostname', username='me', private_key='/path/to/keyfile') as sftp:
#
# ... do sftp operations
#
Reference: https://pysftp.readthedocs.io/en/release_0.2.8/cookbook.html
with pysftp.Connection(abc.example.com, username=u1009697, private_key=id_rsa.pub, cnopts=cnopts) as sftp: logger.debug("SFTP object created") but it says SSH Error:not a valid DSA private key file
Just setup a public key authentication the same way you would do it for a normal (GUI/commandline) SFTP or SSH client. There's nothing pysftp/Python/Lambda-specific about that.
There are zillions of guide on the Internet showing how to do that.
For example my article Set up SSH public key authentication.
And then use the private key in your Python/pysftp code:
Connect to SFTP with key file using Python pysftp
As pysftp requires the key in a physical file, what can be complicated to do in AWS Lambda, you can also hard-code the key in the Python code, if you switch to Paramiko:
SSH/SCP through Paramiko with key in string
(see pysftp vs. Paramiko)
with pysftp.Connection(abc.example.com, username=u1009697, private_key=id_rsa.pub, cnopts=cnopts) as sftp: logger.debug("SFTP object created") but it says SSH Error:not a valid DSA private key file
private_key parameter obviously takes the private key (id_rsa), not the public key (id_rsa.pub). 3) The public key has to go to the authorized_keys file. – Please follow the instructions I have linked in my answer. First try to setup the authentication with some GUI/commandline SFTP/SSH client. And only then try to code the client.