Sftp in python by invoking unix-shell commands

You should pipe your commands into sftp. Try something like this:

import os
import subprocess

dstfilename="/var/tmp/hi.txt"
samplefilename="/var/tmp/sample.txt"
target="[email protected]"

sp = subprocess.Popen(['sftp', target], shell=False, stdin=subprocess.PIPE)

sp.stdin.write("cd /tmp\n")
sp.stdin.write("put %s\n" % dstfilename)
sp.stdin.write("bye\n")

[ do other stuff ]

sp.stdin.write("put %s\n" % otherfilename)

[ and finally ]

sp.stdin.write("bye\n")
sp.stdin.close()

But, in order to answer your question:

os.system("echo put %(dstfilename)s %locals()) >>sample.txt" // Seems this is syntatically not correct.

Of course it isn't. You want to pass a stringto os.system. So it has to look like

os.system(<string expression>)

with a ) at the end.

The string expression consists of a string literal with an applied % formatting:

"string literal" % locals()

And the string literal contains the redirection for the shell:

"echo put %(dstfilename)s >>sample.txt"

And together:

os.system("echo put %(dstfilename)s >>sample.txt" % locals())

. But as said, this is the worst solution I can imagine - better write directly to a temp file or even better pipe directly into the sub process.

Well, I think the literal solution to your question would look something like this:

import os
dstfilename="/var/tmp/hi.txt"
samplefilename="/var/tmp/sample.txt"
host="abc.com"
user="sa"

with open(samplefilename, "w") as fd:
    fd.write("cd /tmp\n")
    fd.write("put %s\n" % dstfilename)
    fd.write("bye\n")

os.system("sftp -B %s %s@%s" % (samplefilename, user, host))

As @larsks says, use a proper filehandler to make the tmp file for you, and my personal preference is to not to do string formatting using locals().

However depending on the use case, I don't think this is a particularly suitable approach - how does the password the sftp site get entered for example?

I think you'd get a more robust solution if you took a look at the SFTPClient in Paramiko, or failing that, you might need something like pexpect to help with ongoing automation.

If you want a non-zero return code if any of the sftp commands fail, you should write the commands to a file, then run an sftp batch on them. In this fashion, you can then retrieve the return code to check if the sftp commands had any failure.

Here's a quick example:

import subprocess

host="abc.com"
user="sa"

user_host="%s@%s" % (user, host)

execute_sftp_commands(['put hi.txt', 'put myfile.txt'])

def execute_sftp_commands(sftp_command_list):
    with open('batch.txt', 'w') as sftp_file:
        for sftp_command in sftp_command_list:
            sftp_file.write("%s\n" % sftp_command)
        sftp_file.write('quit\n')
    sftp_process = subprocess.Popen(['sftp', '-b', 'batch.txt', user_host], shell=False)
    sftp_process.communicate()
    if sftp_process.returncode != 0:
        print("sftp failed on one or more commands: {0}".format(sftp_command_list))

Quick disclaimer: I did not run this in a shell so a typo might be present. If so, send me a comment and I will correct.