How to Transfer Pandas DataFrame to .csv on SFTP using Paramiko Library in Python?

Question

I want to transfer a Python dataframe directly as a .csv file to a remote server using Paramiko module. Currently, I save the dataframe as a .csv then I push that .csv file to the server. I stumbled by this similar question How to write pandas dataframe to csv/xls on FTP directly, but is it possible using Paramiko module? Thanks in advance!

This is the simple script I use to transport a .csv file from my directory to the remote server:

import pandas as pd
import paramiko

# Save DataFrame as CSV
file_name = 'file.csv'
df.to_csv(file_name,index=False)

# Connect to Server Via FTP
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(hostname='host',username='user_name',password='password')
ftp_client= ssh_client.open_sftp()

# Upload 'file.csv' to Remote Server
ftp_client.put('path_to_file.csv','path_to_remote_file')

Martin Prikryl · Accepted Answer · 2019-03-20 07:31:56Z

15

Just use SFTPClient.open

with sftp.open('path_to_remote_file', "w") as f:
    f.write(df.to_csv(index=False))

edited Mar 20, 2019 at 7:31

Martin Prikryl

205k64 gold badges560 silver badges1.1k bronze badges

answered Mar 19, 2019 at 19:52

fn.

2,9663 gold badges30 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Albert Sosa Over a year ago

worked like a charm. thanks man! I just changed it for my script to "with ftp_client.open" for future references

Martin Prikryl Over a year ago

You should enable buffering, otherwise the code can be very slow. See Writing to a file on SFTP server opened using pysftp “open” method is slow.

Martin Prikryl Over a year ago

It's probably more efficient to use df.to_csv(f, index=False), as with that whole contests does not have be to serialized to memory first. Instead, it can be written line by line. See Read CSV/Excel files from SFTP file, make some changes in those files using Pandas, and save back. That also negates my previous comment, as actually with your approach the slow problem with writing line by line does not happen, as you write whole file at once.

Collectives™ on Stack Overflow

How to Transfer Pandas DataFrame to .csv on SFTP using Paramiko Library in Python?

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related