How do I avoid the usage of the "for" loop in this bash function?

Question

I am creating this function to make multiple grep's over every line of a file. I run it as following:

cat file.txt | agrep string1 string2 ... stringN 

function agrep () {  
   for a in $@;  do
     cmd+=" | grep '$a'";
   done ;
   while read line ; do
     eval "echo "\'"$line"\'" $cmd";
   done;
}

The idea is to print every line that contains all the strings: string1, string2, ..., stringN. This already works but I want to avoid the usage of the for to construct the expression:

| grep string1 | grep string2 ... | stringN

And if it's possible, also the usage of eval. I tried to make some expansion as follows:

echo "| grep $"{1..3}

And I get:

| grep $1 | grep $2 | grep $3

This is almost what I want but the problem is that when I try:

echo "| grep $"{1..$#}

The expansion doesn't occur because bash cant expand {1..$#} due to the $#. It just works with numbers. I would like to construct some expansion that works in order to avoid the usage of the for in the agrep function.

rob mayoff · Accepted Answer · 2016-04-03 02:28:17Z

5

agrep () {
    if [ $# = 0 ]; then
        cat
    else
        pattern="$1"
        shift
        grep -e "$pattern" | agrep "$@"
    fi
}

answered Apr 3, 2016 at 2:28

rob mayoff

387k69 gold badges842 silver badges887 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

neo33 Over a year ago

thanks for the support this was very useful, but i have one question, the if statement is optional or is necessary, it's unclear for me that if the if is true then execute cat, i am not sure about the usage of that.

rob mayoff Over a year ago

In the case of no arguments given at all, or after all of the arguments have been peeled off, agrep needs to pass its input through to its output unmodified. The cat command does that.

neo33 Over a year ago

ok thanks for the support an explanation, this was very useful.

Community · Accepted Answer · 2017-05-23 10:28:27Z

3

Instead of running each multiple greps on each line, just get all the lines that match string1, then pipe that to grep for string2, etc. One way to do this is make agrep recursive.

agrep () {
    if (( $# == 0 )); then
        cat   # With no arguments, just output everything
    else
        grep "$1" | agrep "${@:2}"
    fi
}

It's not the most efficient solution, but it's simple.

(Be sure to note Rob Mayoff's answer, which is the POSIX-compliant version of this.)

edited May 23, 2017 at 10:28

CommunityBot

11 silver badge

answered Apr 3, 2016 at 2:30

chepner

538k77 gold badges595 silver badges747 bronze badges

Comments

karakfa · Accepted Answer · 2016-04-03 02:22:14Z

2

awk to the rescue!

you can avoid multiple grep calls and constructing the command by switching to awk

awk -v pat='string1 string2 string3' 'BEGIN{n=split(pat,p)} 
               {for(i=1;i<=n;i++) if($0!~p[i]) next}1 ' file

enter your space delimited strings as in the example above.

answered Apr 3, 2016 at 2:22

karakfa

67.8k8 gold badges45 silver badges59 bronze badges

3 Comments

chepner Over a year ago

Depending on the patterns to match, it may be tricky to find a way to pack them all into a single delimited string. (What if one of the strings itself contains whitespace, for example?)

karakfa Over a year ago

it doesn't have to be space delimited (mimicking original example) you can pick any unused char as a delimiter but need to split with that too.

chepner Over a year ago

In general, you can't guarantee there is an unused character, or at least know ahead of time what is unused. It's the same problem arrays were introduced to solve.

Community · Accepted Answer · 2017-05-23 11:45:07Z

Not building a string for the command is definitely better (see chepner's and Rob Mayoff's answers). However, just as an example, you can avoid the for by using printf:

agrep () { 
    cmd=$(printf ' | grep %q' "$@")
    sh -c "cat $cmd"
}

Using printf also helps somewhat with special characters in the patterns. From help printf:

In addition to the standard format specifications described in printf(1),
printf interprets:

  %b    expand backslash escape sequences in the corresponding argument
  %q    quote the argument in a way that can be reused as shell input
  %(fmt)T output the date-time string resulting from using FMT as a format
        string for strftime(3)

Since the aim of %q is providing output suitable for shell input, this should be safe.

Also: You almost always want to use "$@" with the quotes, not just plain $@.

Collectives™ on Stack Overflow

How do I avoid the usage of the "for" loop in this bash function?

4 Answers 4

3 Comments

Comments

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

Comments

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related