Regular expression matching using shell script

Question

I have created a shell script below that checks whether the file out.txt contains any line with the text "Billing_20160210". I am getting the error in the regular expression part which states that "[[: not found". Not sure whether i am missing something. Any help would be appreciated.

#!/bin/bash
bq ls test:Dataset_test > out.txt
cat out.txt | while read LINE
do 
  if [ LINE = *"Billing_20160210"* ]; 
  then 
    echo "Processing $LINE"
  fi
done

Edit not working:

#!/bin/bash
bq ls geotab-bigdata-test:JS_test > out.txt
cat out.txt | while read LINE
do 
 if [[ $LINE == *"DailyBillingTest_20160210"* ]];
 then 
  echo "Processing $LINE"
 fi
done

There is no regex here just glob. Replace[ LINE = *"Billing_20160210"* ] with [[ $LINE == *"Billing_20160210"* ]] — anubhava
– anubhava, Commented Aug 3, 2016 at 14:53
[[: not found also implies that you aren't actually using bash as your interpreter to run this script. Are you invoking it with sh yourscript? If so, that would override the #!/bin/bash shebang. If invoking with an explicit interpreter, that needs to be bash yourscript, not sh yourscript. — Charles Duffy
– Charles Duffy, Commented Aug 3, 2016 at 14:58
What exactly is that "Change:" edit? Is that saying that this is what you've also tried that's having the same effect? That this is your proposed solution? (If so, solutions should be in answers, not edits to questions; it's acceptable to add an answer to your own question, though polite to encourage someone whose comment helped you to put that comment in an answer themselves). — Charles Duffy
– Charles Duffy, Commented Aug 3, 2016 at 14:59
(...btw, filename extensions on scripts are bad form -- a .sh extension falsely implies that sh can be used to run a script, which is simply untrue if it's a bash script [hence your bug here], and using an extension at all means you need to either change your callers to invoke the script differently or have its name be inaccurate if you rewrite it in a different language. You don't run ls.elf; why would you use extensions on other commands you write yourself?) — Charles Duffy
– Charles Duffy, Commented Aug 3, 2016 at 15:04

hek2mgl · Accepted Answer · 2016-08-03 15:01:23Z

1

Don't use a shell loop to filter a file line by line. Use grep as a filter here:

grep 'Billing_20160210' out.txt | while read -r line ; do
    echo "processing ${line}"
done

Or even without a temporary file:

bq ls test:Dataset_test | grep 'Billing_20160210' | while read -r line ; do
    echo "processing ${line}"
done

answered Aug 3, 2016 at 15:01

hek2mgl

159k31 gold badges263 silver badges279 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Charles Duffy Over a year ago

Granted, moving to this (POSIX-compliant) code moots the issue, but it might not hurt to also address interpreter selection, as that was the most immediate fault. Surely the OP will also write bash-specific code on other occasions in their future; understanding why one can't use sh to run a bash script thus has utility beyond this single immediate question.

user3447653 Over a year ago

@CharlesDuffy: Thank you. It works. This is the first time i am trying to run a shell script, so did not get that point in the beginning.

hek2mgl Over a year ago

@CharlesDuffy I see your point. If a file extension is desired it should be .bash.. I'm also used to .sh but I admit that it is not correct as long as the code isn't 100% POSIX compliant.

Charles Duffy Over a year ago

nod. The approach I tend to encourage is to use an extension reflecting the minimum compatible shell (.sh for POSIX, .bash/.ksh/.zsh etc if appropriate) if a script is intended to be sourced, or leave it off if it's only expected to be executed -- if being sourced, the interpreter to be used is information the user needs to have available, and it's consistent with standard behavior in other languages (ie. Python modules end in .py, Python executable entrypoints generated with setuptools etc don't).

Collectives™ on Stack Overflow

Regular expression matching using shell script

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related