Unix Script to add header in awk script resulting in header on every other line

Question

I am trying to add a header to a split file but with this code the header is appearing every other line:

awk -F, '{print "eid,devicetype,meterid,lat,lng" > $7"-"$6".csv"}{print $1",", $2",", $3",", $4",", $5"," >> $7"-"$6".csv"}' path/filename

The awk code by itself works but I need to apply a header in the file. The script splits the file based on the values in columns 6 & 7 as well as names the end file with those values. Then it removes columns 6 & 7 it only puts columns 1 - 5 in the output file. This is on Unix in a shell script run from PowerCenter. I am sure it is probably simple fix for others more experienced.

Ed Morton · Accepted Answer · 2014-08-07 21:20:42Z

1

awk '
BEGIN { FS=OFS="," }
{ fname = $7 "-" $6 ".csv" }
!seen[fname]++ { print "eid", "devicetype", "meterid", "lat, "lng" > fname}
{ print $1, $2, $3, $4, $5 > fname }
' path/filename

answered Aug 7, 2014 at 21:20

Ed Morton

209k18 gold badges90 silver badges212 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Tom Fenech Over a year ago

So the > only erases the file once, not every time?

Ed Morton Over a year ago

Right, the semantics of > and >> in awk are different from shell.

anubhava · Accepted Answer · 2014-08-08 13:42:52Z

1

You can use:

awk -F, '!a[$7,$6]++{print "eid,devicetype,meterid,lat,lng" > $7 "-" $6 ".csv"}
  {print $1,$2,$3,$4,$5 > $7 "-" $6 ".csv"}' OFS=, /path/filename.csv

NR==1 will make sure that header is printed for 1st record.

edited Aug 8, 2014 at 13:42

answered Aug 7, 2014 at 19:41

anubhava

790k67 gold badges603 silver badges671 bronze badges

7 Comments

Thomas J. Huber Over a year ago

It worked for one file but sometimes the export is multiple files. I need it to add a header to all output files. Can you help? Also I have a space aftereach comma in my output files. What do I need to remove those spaces?

anubhava Over a year ago

To use comma instead of space in output use awk -v OFS=, .... Also if you provide your sample input file in your question I can take a look on why its not working.

Etan Reisner Over a year ago

!a[$7,$6]++ instead of NR==1 as the pattern might do what he wants. One header for each unique $7/$6 combination.

Ed Morton Over a year ago

Right and you need to parenthesize the string concatenation on the right of the > and it should be > instead of >> on the 2nd line (though functionally it won't matter in this case).

Thomas J. Huber Over a year ago

The !a[$7,$6]++ worked for adding the Header to each file. Thank you. The -v OFS=, is not working for the comma. The spaces are how I am putting the columns into the new files: $1",",$2",",$3",",$4",",$5"," but the results are showing as: column1, column2, column3, column4, column5, where as I would want them to show as column1,column2,column3,column4,column5

|

Collectives™ on Stack Overflow

Unix Script to add header in awk script resulting in header on every other line

2 Answers 2

2 Comments

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related