How to grep a file using search parameters from another file

Question

I am trying to use a file containing IP addresses as the basis for searching through a Cisco firewall configuration file. Normally, I would use something like:

for i in $(cat ip.file); do grep $i fw.config; done

But doing that returns absolutely nothing. If I put the above script into a file and execute it with the bash -xv flags, each line returns something like this:

+ for i in '`cat ip.file`'
+ grep $'1.2.3.4\r' fw.config  (each IP address is different)

grep 1.2.3.4 fw.config is exactly what I want to happen, but I get nothing back from this command.

I know of the grep -f option, but that also returns nothing. I am not an experienced coder, so I might be overlooking something obvious.

This won't fix your problem, but you should always quote variables when they're arguments to a command. It's not a bad idea to use -- to stop option processing when one of the arguments is a variable, too. grep -- "$i" fw.config — Dennis Williamson
– Dennis Williamson, Commented Sep 2, 2010 at 2:44

John Kugelman · Accepted Answer · 2010-09-02 00:45:04Z

22

It looks like ip.file is in DOS format and has \r\n line endings. Run dos2unix on it to convert to UNIX format. This will get rid of the errant \r carriage returns that are messing up grep.

By the way, you can use grep -f FILE to pass grep a list of patterns to search for. It will then do a single pass searching for any of those patterns.

# After doing `dos2unix ip.file'...
grep -f ip.file fw.config

# Or...
grep -f <(dos2unix < ip.file) fw.config

edited Sep 2, 2010 at 0:45

answered Sep 2, 2010 at 0:34

John Kugelman

365k70 gold badges555 silver badges600 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Gordon Davisson Over a year ago

the -f approach also has the advantage that it interprets the search pattern as simple strings -- e.g. the dots in "1.2.3.4" will only match dots, not any character.

Eric Over a year ago

This was the answer. The list of IP addresses was grepped from a dns zone file which I downloaded from my dns provider via a web browser. I had no idea that it was in dos format. Good catch. After running dos2unix, grep -f worked as expected. Thanks.

ghostdog74 · Accepted Answer · 2010-09-02 00:40:25Z

5

GNU grep,

grep -f ip.txt config

Its advisable also not to use for loop with cat. (If you do, you should change IFS to $'\n'). Use while read loop instead.

while read -r line
do
  ....
done <"ip.txt"

answered Sep 2, 2010 at 0:40

ghostdog74

346k62 gold badges264 silver badges349 bronze badges

Comments

user304132 · Accepted Answer · 2010-09-02 07:53:12Z

1

for i in $(tr '\r' '\n' < ip.file); do grep $i fw.config; done

answered Sep 2, 2010 at 7:53

user304132

462 bronze badges

Collectives™ on Stack Overflow

How to grep a file using search parameters from another file

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related