Paramiko uploading file successfully but it's empty

Question

Trying to export a dataframe to csv using StringIO and transfer it via SFTP using paramiko. The file comes through successfully but it's empty. Any idea why?

import pandas as pd
from StringIO import StringIO
import paramiko

names = ['Bob','Jessica','Mary','John','Mel']
births = [968, 155, 77, 578, 973]

BabyDataSet = list(zip(names,births))
df = pd.DataFrame(data = BabyDataSet, columns=['Names', 'Births'])

buf = StringIO()
df.to_csv(buf, sep = ',', header = True, index = False)
#buf.getvalue() # Correct output

transport = paramiko.Transport(('localhost', 22))
transport.connect(username='user', password='pass')
sftp = paramiko.SFTPClient.from_transport(transport)

upload_path = '/Users/user/Desktop/test.csv'
sftp.putfo(buf, upload_path)
sftp.close()

Simon Fraser · Accepted Answer · 2017-06-06 16:08:29Z

7

With this line, you have written to buf as though it's a file.

df.to_csv(buf, sep = ',', header = True, index = False)

It's a file-like object, so it has a reference to how far through that 'file' you are. When putfo tries to read from buf it will read from the last place used, and since you've written to the buffer, that location is at the end of your data. buf.getvalue() returns the entire contents of the buffer, regardless of the current position in the file. Seek to the start to get everything when reading normally:

df.to_csv(buf, sep = ',', header = True, index = False)
buf.seek(0) # rewind to the beginning of the 'file'
sftp.putfo(buf, upload_path)

answered Jun 6, 2017 at 16:08

Simon Fraser

2,81821 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Paramiko uploading file successfully but it's empty

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related