How do you loop over files containing A and B in bash?

Question

I'm looking for all the files in a directory that contain "AAA" and "BBB". I then want to modify/do stuff with each of the found files. The file names do contain spaces.

grep piping to grep would seem simple but requires the xargs to function correctly.

The multiple match functionality in grep (from what I can tell) is only for "AAA" or "BBB".

What I've got is:

for FILENAME in "$(grep -ilrZ "AAA" ./files/* | xargs -0 grep -ilr "BBB")"
do
  echo "$FILENAME"
  echo "match"
done

However, this treats the list as files (including the newlines) as a single entity (i.e. "match" will only print once).

Removing the double quotes from the for sub-shell means every space is a new thing to be looped over, which breaks up filenames.

tripleee · Accepted Answer · 2022-05-10 08:08:08Z

Your use of grep -Z in the first invocation is a solution to the problem in the second case, too.

grep -ilrZ "AAA" ./files/* |
xargs -r0 grep -Zil "BBB" |
xargs -r0 -i echo "{} match"

Using -r in the second grep makes no sense because you only want to examine precisely the files from the output of the first grep.

If you just want to list the files, of course, just grep -l already does that; presumably, you will want to do something more complex in the second xargs.

An alternative solution might be

find ./files -type f \
    -exec grep -qi "AAA" {} \; \
    -exec grep -qi "BBB" {} \; \
    -exec sh -c 'for f; do echo "$f matched"; done' _ {} +

The way this works is, if one -exec fails, the file for which it failed will be regarded as a failed predicate by find, and so the remainder of the predicates will be skipped. The files which reach the final predicate will have returned success from all the previous predicates.

The latter should work on non-Linux platforms which lack the GNU extensions grep -Z, xargs -0 etc.

In case it's not obvious, find and grep -r examine subdirectories, too. If you just want to examine the current directory, omitting the -r from the first grep should work. For find, you can add -maxdepth 1 before -type f to only examine the current directory.

Demo: https://ideone.com/ELqKxZ

For much more on this topic, see also https://mywiki.wooledge.org/BashFAQ/020 and perhaps https://mywiki.wooledge.org/DontReadLinesWithFor

glenn jackman · Accepted Answer · 2022-05-07 18:07:39Z

1

@tripleee has an excellent, robust answer. An alternate take. Assuming your shell is bash and your awk is GNU awk

shopt -s globstar    # enable recursive globbing with **
gawk '
    BEGINFILE {a = b = 0}
    /AAA/ {a = 1}
    /BBB/ {b = 1}
    a && b {print FILENAME; nextfile}
' ./files/** 2>/dev/null

answered May 7, 2022 at 18:07

glenn jackman

249k42 gold badges233 silver badges362 bronze badges

Collectives™ on Stack Overflow

How do you loop over files containing A and B in bash?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related