write CSV columns out in a different order in Python

Question

I realize this is very similar to this question. However, I have a CSV file that always comes in the same format that I need to write out with columns in a different order to move it down the data processing pipeline. If my csv file contains headers and data like this:

Date,Individual,Plate,Sample,test,QC
03312011,Indiv098,P342,A1,deep,passed
03312011,Indiv113,P352,C3,deep,passed

How would I write out a csv file with the same columns as the original input csv but in the following order:

test,QC,Plate,Sample
deep,passed,P342,A1
deep,passed,P352,C3

My initial thought was to do something like this:

f = open('test.csv')
lines = f.readlines()
for l in lines:
    h = l.split(",")
    a, b, c, d, e, f  = h
    for line in h:
        print e, f, c, d,

If you just need a tool rather than code to do this, take a look at my FOSS tool at code.google.com/p/csvfix which makes re-ordering of CSV fields (among other things) trivial. — user2100815
– user2100815, Commented May 24, 2011 at 23:05
Famous Last Words: "file that always comes in the same format" — John Machin
– John Machin, Commented May 24, 2011 at 23:08
Famous Last Thoughts: "My CSV file can be parsed successfully with str.split(',')" — John Machin
– John Machin, Commented May 24, 2011 at 23:35
new_row = [row[4], row[5], row[2], row[3]] See stackoverflow.com/questions/8905165/… — Sudo1234
– Sudo1234, Commented Dec 22, 2015 at 21:36

John Machin · Accepted Answer · 2011-05-24 23:33:40Z

5

If there's the slightest chance that the input file or the output file won't have the same layout each time, here's a more general way to get your "reorderfunc":

writenames = "test,QC,Plate,Sample".split(",") # example
reader = csv.reader(input_file_handle)
writer = csv.writer(output_file_handle)
# don't forget to open both files in binary mode (2.x)
# or with `newline=''` (3.x)
readnames = reader.next()
name2index = dict((name, index) for index, name in enumerate(readnames))
writeindices = [name2index[name] for name in writenames]
reorderfunc = operator.itemgetter(*writeindices)
writer.writerow(writenames)
for row in reader:
    writer.writerow(reorderfunc(row))

answered May 24, 2011 at 23:33

John Machin

83.2k12 gold badges147 silver badges193 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ignacio Vazquez-Abrams · Accepted Answer · 2011-05-24 23:03:03Z

5

reorderfunc = operator.itemgetter(4, 5, 2, 3)

 ...

newrow = reorderfunc(oldrow)
 ...

answered May 24, 2011 at 23:03

Ignacio Vazquez-Abrams

804k160 gold badges1.4k silver badges1.4k bronze badges

Comments

Mark Tolonen · Accepted Answer · 2011-05-25 00:39:51Z

4

Given your input as src.csv:

import csv

with open('x.csv','rb') as i:
    with open('y.csv','wb') as o:
        r = csv.DictReader(i)
        w = csv.DictWriter(o,'test QC Plate Sample'.split(),extrasaction='ignore')
        w.writeheader()
        for a in r:
            w.writerow(a)

Output

test,QC,Plate,Sample
deep,passed,P342,A1
deep,passed,P352,C3

answered May 25, 2011 at 0:39

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

1 Comment

stackit Over a year ago

the best python working python code, who cares for speed nowadays thanks to moore laws

user3392191 · Accepted Answer · 2014-03-08 00:15:27Z

0

#Use CSV library
import csv
media = {}
files=['Online.txt']
directory = "C:/directory/"
rowCnt=0

for file in files:

    file=directory+file

    with open(file, 'rb') as f:
        reader = csv.reader(f, delimiter='|') #use pipe delimiter

        for row in reader:
            rowCnt+=1
            if (rowCnt % 1000) == 0:
                print ('"%s","%s","%s","%s","%s","%s","%s","%s","%s"')% (row[1],row[4],row[14],row[17],row[18],row[24],row[25],row[28],row[30])

answered Mar 8, 2014 at 0:15

user3392191

Collectives™ on Stack Overflow

write CSV columns out in a different order in Python

4 Answers 4

Comments

Comments

Output

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Output

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related