Error in awk code in shell script

Question

I'm using the ls command to list files to be used as input. For each file found, I need to

Perform a system command (importdb) and write to a log file.
Write to an error log file if the first character of column 2, line 6 of the log file created in step 1 is not "0".
rename the file processed so it won't get re-processed on the next run.

My script:

#!/bin/sh
ls APCVENMAST_[0-9][0-9][0-9][0-9]_[0-9][0-9] |
while read LINE 
 do
       importdb -a test901 APCVENMAST ${LINE} > importdb${LINE}.log
 awk "{if (NR==6 && substr($2,1,1) != "0")      
       print "ERROR processing ", ${LINE} > importdb${LINE}err.log
        }" < importdb${LINE}.log
       mv  ${LINE} ${LINE}.PROCESSED
 done

This is very preliminary code, and I'm new to this, but I can't get passed parsing errors as the one below.

The error context is:

{if (NR==6 && >>>  substr(, <<< awk The statement cannot be correctly parsed.

What is this script intended to actually accomplish? There's almost certainly a better way that doesn't use awk (or ls) at all. — Charles Duffy
– Charles Duffy, Commented Oct 3, 2013 at 21:48

Chris Seymour · Accepted Answer · 2013-10-03 21:52:47Z

6

Issues:

Never double quote an awk script.
Always quote literal strings.
Pass in shell variables correctly either by using -v if you need to access the value in the BEGIN block or after the scripts i.e. awk -v awkvar="$shellvar" 'condition{code}' file or by awk condition{code}' awkvar="$shellvar"
Always quote shell variables.
Conditional should be outside block.
There is ambiguity with redirection and concatenation precedence so use parenthesis.

So the corrected (syntactical) script:

 awk 'NR==6 && substr($2,1,1) != 0 {       
           print "ERROR processing ", line > ("importdb" line "err.log")
      }' line="${LINE}" "importdb${LINE}.log"

You have many more issues but as I don't know what you are trying to achieve it's difficult to suggest the correct approach...

You shouldn't parse the output of ls
Awk reads files you don't need to loop using shell constructs

edited Oct 3, 2013 at 21:52

answered Oct 3, 2013 at 21:08

Chris Seymour

86.4k32 gold badges166 silver badges209 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

CS Pei Over a year ago

Are you sure it works? in my linux, awk can not take shell variable line directly, you have to use "$line".

Chris Seymour Over a year ago

I don't define a shell variable line. I define an awk variable line that has the same value as the shell variable $LINE. In awk if you do $line then you would be accessing the field number contain in line if any. $ and $ have completely different meaning in shell and awk. Awk is not part of the shell!

Imagination Over a year ago

one point I don't understand. Conditional should be outside block why? awk has if - else statement, right? what if a script needs {if(foo){code}else if (foo2){code} else if (bar){code} else{code}} ?

Chris Seymour Over a year ago

@Imagination that is the structure of awk programs. You apply blocks of code to lines that match conditionals. Yes, they are case where using traditional if statements are appropriate but this isn't one. You're example would be foo{code;next}foo2{code;next}bar{code;next}{code}.

Imagination Over a year ago

@sudo_O thank you for the explanation. so you are saying this point is only for this question. btw, the if-else is not as same as your foo{code;next} because before/after the if-else there could be other codes, it is not always stopping processing the current record. But to this specific question, you are right. thanks again.

|

Collectives™ on Stack Overflow

Error in awk code in shell script

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related