Awk: Print to different files using multiple defined variables

Question

I am opening up a file and checking if the items in columns 1 & 8 match certain specs. If yes, write output to a file x. If the items in column 1 match specs but column 8 does not match the specs, write output to file y.

I am defining multiple variables (awk -v v=$var,f1=$file,f2=$output), and I believe how I reference f1 & f2 is the problem. If I remove the quotes:

print $0 >> f2
awk: cmd. line:5: (FILENAME=- FNR=2) fatal: expression for `>>' redirection has null string value

If I put in a $:

print $0 >> $f2

I end up with a bunch of files with odd names that I don't want, and the files I do want are empty (except for the echoed line).

if I put "":

print $0 >> "f2"

The files I want are almost empty, and it creates a file called f2.

#!/bin/bash

output="output.txt"
echo -e "C1\tSeqID\tAminoAcid\tCD1\tCD2\tCD3\tGene\tEnvironment\tFilename" > $output
inputFile="input.txt.gz"

for var in A B C D E F G H I J K L
do
   file=$var".txt"
   echo -e "C1\tSeqID\tAA\tCD1\tCD2\tCD3\tGene\tEnvironment\tFilename" > $file

   #---Wrong, forgot to catch $8 != v
   #zcat $inputFile |  awk -v v=$var '{
#  if ($8 == v && ($1 == "V1" || $1 == "V2" || $1 == "V3" || $1 == "V4" || $1 == "V5" || $1 == "V6" || $1 == "V7" || $1 == "V8" || $1 == "V9" || $1 == "V10"))
#  print $0
#  }' | tee -a $file $output  

   zcat $inputFile |  awk -v v=$var,f1=$file,f2=$output '{
   if ($8 == v && ($1 == "V1" || $1 == "V2" || $1 == "V3" || $1 == "V4" || $1 == "V5" || $1 == "V6" || $1 == "V7" || $1 == "V8" || $1 == "V9" || $1 == "V10"))
   print $0 >> "file"
   else if ($8 != v && ($1 == "V1" || $1 == "V2" || $1 == "V3" || $1 == "V4" || $1 == "V5" || $1 == "V6" || $1 == "V7" || $1 == "V8" || $1 == "V9" || $1 == "V10"))
   print $0 >> "f2"
   }' 

   gzip $file
done

gzip $output

I can run through the loop and have two separate awk commands that write to different files. However, it is a very large file (4G compressed) and it is more efficient to use my current approach (or something similar to it). Any guidance on how to reference the 2nd & 3rd variable are greatly appreciated.

Ed Morton · Accepted Answer · 2014-11-22 21:49:08Z

2

Use separate -vs:

awk -v v="$var" -v f1="$file" -v f2="$output" '...'

answered Nov 22, 2014 at 21:49

Ed Morton

209k18 gold badges90 silver badges212 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

cer Over a year ago

Ahh. Thank you @Ed Morton. It seems to be working sans error & writing to f1, however, f2 is still empty (aside from echoed line). I will see how it finished and if there is still a problem, I will ask a new question. Thank you again.

gboffi · Accepted Answer · 2014-11-22 21:50:39Z

0

% awk -v v=qw,f1=we,f2=as 'BEGIN{print v, "*", f1, "*", f2}'
qw,f1=we,f2=as *  * 
% awk -v v=qw -v f1=we -v f2=as 'BEGIN{print v, "*", f1, "*", f2}'
qw * we * as
%

Do you need anything else to proceed?

answered Nov 22, 2014 at 21:50

gboffi

25.4k10 gold badges62 silver badges98 bronze badges

Collectives™ on Stack Overflow

Awk: Print to different files using multiple defined variables

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related