Unable to convert csv file to text tab delimited file in Python

Question

Instead of manually convert csv file to text tab delimited file using excel software

I would like to automate this process using Python.

However, using the following code

with open('endnote_csv.csv', 'r') as fin:
       with open('endnote_deliminated.txt', 'w', newline='') as fout:
           reader = csv.DictReader(fin, delimiter=',')
           writer = csv.DictWriter(fout, reader.fieldnames, delimiter='|')
           writer.writeheader()
           writer.writerows(reader)

Return an error of

ValueError: dict contains fields not in fieldnames: None

May I know where did I do wrong,

The csv file is accessible via the following link

Thanks in advance for any insight.

your source file is not a proper .csv file. It has tons of commas in the middle of the entries. — AirSquid
– AirSquid, Commented Jul 6, 2020 at 16:57
this may help out if you want to avoid pandas: stackoverflow.com/questions/21527057/… — AirSquid
– AirSquid, Commented Jul 6, 2020 at 17:04

d-man · Accepted Answer · 2020-07-06 16:59:34Z

2

You can use the Python package called pandas to do this:

import pandas as pd
fname = 'endnote_csv'
pd.read_csv(f'{fname}.csv').to_csv(f'{fname}.tsv', sep='\t', index=False)

Here's how it works:

pd.read_csv(fname) - reads a CSV file and stores it as a pd.DataFrame object (not important for this example)
.to_csv(fname) - writes a pd.DataFrame to a CSV file given by fname
sep='\t' - replaces the ',' used in CSVs with a tab character
index=False - use this to remove the row numbers

If you want to be a bit more advanced and use the command line only, you can do this:

# csv-to-tsv.py
import sys

import pandas as pd

fnames = sys.argv[1:]

for fname in fnames:
    main_name = '.'.join(fname.split('.')[:-1])
    pd.read_csv(f'{main_name}.csv').to_csv(f'{main_name}.tsv', sep='\t', index=False)

This will allow you to run a command like this from the command line and change all .csv files to .tsv files in one go:

python csv-to-tsv.py *.csv

answered Jul 6, 2020 at 16:59

d-man

5065 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

rpb Over a year ago

Thanks for the suggestion, I will try this and come back to you.

rpb Over a year ago

Hi, apparently, this save the doc as tsv instead of txt format

d-man Over a year ago

just change everywhere it says '.tsv' to '.txt' - for example: pd.read_csv(f'{main_name}.csv').to_csv(f'{main_name}.txt', sep='\t', index=False)

Ravikant Khond · Accepted Answer · 2020-07-06 17:22:13Z

0

It is erroring out on comma seperated author names. It appears that columns in the underline rows exceeds number of headers.

answered Jul 6, 2020 at 17:22

Ravikant Khond

515 bronze badges

Collectives™ on Stack Overflow

Unable to convert csv file to text tab delimited file in Python

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related