Skip first rows when writing csv (pandas.DataFrame.to_csv)

Question

In my python script I am reading a csv file via

df = pd.read_csv('input.csv', sep=';', encoding = "ISO-8859-1", skiprows=2, skipfooter=1, engine='python')

I am the skipping the first two rows in the csv file because they are just discriptions I don't need.

After importing, I am filtering and separating the data. I want to write the data back to csv files while having the same format as before (first two rows either empty or the description as before the import). How can I do that?

Currently I am using

df.to_csv('output.csv'), sep=';', encoding = "ISO-8859-1")

Is there something like a parameter "skiprows" for exporting? I can't find one in the api documentation for .to_csv.

Could you not write your description first and then append to the file? Or just write 2 new lines and then append? There isn't a built in api in pandas for this — EdChum
– EdChum, Commented Jul 24, 2018 at 13:34
Maybe you are right. I see you can use mode='a' for appending when writing csv. So I just need to create a csv file with two empty rows before exporting. — Marco
– Marco, Commented Jul 24, 2018 at 13:37
Appending to a pre-existing file is trivial, there isn't a method built in pandas to do this using to_csv — EdChum
– EdChum, Commented Jul 24, 2018 at 13:38

jezrael · Accepted Answer · 2018-07-24 13:43:27Z

7

One possible solution is write DataFrame with NaNs first and then append original DataFrame:

df1 = pd.DataFrame({'a':[np.nan] * 2})
df1.to_csv('output.csv', index=False, header=None)
df.to_csv('output.csv', sep=';', encoding = "ISO-8859-1", mode='a')

Or same original header to df1 and this write first, only necessary no value | in header data:

df1 = pd.read_csv('input.csv', sep='|', encoding = "ISO-8859-1", nrows=2, names=['tmp'])

df1.to_csv('output.csv', index=False, header=None)
df.to_csv('output.csv', sep=';', encoding = "ISO-8859-1", mode='a')

edited Jul 24, 2018 at 13:43

answered Jul 24, 2018 at 13:38

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Marco Over a year ago

Perfect, first solution works for me. I will switch to the second one if i really need the description later. Thanks!

pc_pyr Over a year ago

Thanks for this answer @jezrael, just wanted to know if we have a way to skip first row alone in .to_csv() ?

rigo · Accepted Answer · 2020-05-14 23:01:49Z

0

You want to copy the header from your original file and write to your new file. Then you append your dataframe by setting mode to 'a'

with open("my_csv.csv") as f:
    header =''
    for line in range(0,header_length_in_lines):
        header +=f.readline()

with open('my_new_csv.csv','w+') as f:
    f.write(header)
df.to_csv('my_new_csv.csv', mode='a', index=False)

answered May 14, 2020 at 23:01

rigo

3252 silver badges10 bronze badges

1 Comment

Ranjan Pal Over a year ago

I want to export a dataframe (500 rows,2 coulmns) from python to a csv file. However, I need to ensure that 1st 20 rows has some text/strings written and then the dataframe should should start from 21st row onwards. Can somebody please let me know how do we do this?

Collectives™ on Stack Overflow

Skip first rows when writing csv (pandas.DataFrame.to_csv)

2 Answers 2

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related