Bash parameter expansion in brackets not working as expected

Question

I am writing a script that wraps the find command to search for specific source file types under a given directory. A sample invocation would be :

./find_them.sh --java --flex --xml dir1

The above command would search for .java, .as and .xml files under dir1.

To do this manually I came up with the following find command :

find dir1 -type f -a \( -name "*.java" -o -name "*.as" -o -name "*.xml" \)

As I am doing this in a script where I want to be able specify different file sets to search for you end up with the following structure :

find_cmd_file_sets=$(decode_file_sets) # Assume this creates a string with the file sets e.g. -name "*.java" -o -name "*.as" etc
dirs=$(get_search_dirs) # assume this gives you the list of dirs to search, defaulting to the current directory

for dir in $dirs 
do
    find $dir -type f -a \( $find_cmd_file_sets \)
done

The above script doesn't behave as expected, you execute the script and the find command churns for a while before returning no results. I'm certain the equivalents of decode_file_sets and get_search_dirs I've created are generating the correct results.

A simpler example if to execute the following directly in a bash shell

file_sets=' -name "*.java" -o -name "*.as" '
find dir -type f -a \( $file_sets \) # Returns no result
# Executing result of below command directly in the shell returns correct result
echo find dir -type f -a \\\( $file_sets \\\)

I don't understand why variable expansion in brackets of the find command would change the result. If it makes any difference I am using git-bash under Windows.

This is really frustrating. Any help would be much appreciated. Most importantly I would like to understand why the variable expansion of $file_sets is behaving as it is.

Why do you believe that quotes in a variable are the same as quotes at the command line? — Ignacio Vazquez-Abrams
– Ignacio Vazquez-Abrams, Commented Nov 5, 2014 at 10:26
I don't know what would make you think I believe that. I am using single quotes so the double quotes in the variable should be preserved when used as an argument to find — ahjmorton
– ahjmorton, Commented Nov 5, 2014 at 10:38
I guess there is a mistake in the for loop -- it should be for dir in $dirs — Jdamian
– Jdamian, Commented Nov 5, 2014 at 10:52
@ahjmorton The double quotes stored in a parameter are not used to quote *.java; they are treated literally. You cannot nest syntactic quotes like this. bash arrays were introduced for precisely this reason. — chepner
– chepner, Commented Nov 5, 2014 at 14:08

Amit Kumar · Accepted Answer · 2014-11-05 11:02:17Z

2

Hope this will work, Its tested on bash.

file_sets=' -name "*.java" -o -name "*.as" '
command=`echo "find $dir -type f -a \( $file_sets \)"`

eval $command

answered Nov 5, 2014 at 11:02

Amit Kumar

3131 gold badge5 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

ahjmorton Over a year ago

This approach does work, would like to know why the original version did not behave as expected

Amit Kumar Over a year ago

Don't know exactly but in my opinion or atleast what i concluded is due to find command functionality which take everything as its argument and not replacing variable to its value. considering variable as its argument to -a option.

chepner Over a year ago

The echo in a command subsitution is redundant; command="find $dir -type -f -a $ $file_sets $" works as well.

chepner Over a year ago

When possible, avoid the use of eval--and it's almost always possible.

danadam · Accepted Answer · 2014-11-05 12:08:14Z

TLDR: Don't use quotes in find_cmd_file_sets variable and disable pathname expansion (set -f) before calling find.

When you have "special" character in a variable content and then you try to expand that variable without quotes than bash will surround each word with "special" character with single quotes, e.g.:

#!/usr/bin/env bash
set -x
VAR='abc "def"'
echo $VAR

The output is:

+ VAR='abc "def"'
+ echo abc '"def"'
abc "def"

As you can see, bash surrounded "def" with single quotes. In your case, the call to find command becomes:

find ... -name '"*.java"' ...

So it tries to find files which start with " and end with .java"

To prevent that behavior, the only thing you can do (which I'm aware of) is to use double quotes when expanding the variable, e.g.:

#!/usr/bin/env bash
set -x
VAR='abc "def"'
echo "$VAR"

The output is:

+ VAR='abc "def"'
+ echo 'abc "def"'
abc "def"

The only problem, as you probably noticed already, is that now the whole variable is in quotes and is treated as single argument. So this won't work in your find command.

The only option left is to not use quotes, neither in variable content nor when expanding the variable. But then, of course, you have a problem with pathname expansion:

#!/usr/bin/env bash
set -x
VAR='abc *.java'
echo $VAR

The output is:

+ VAR='abc *.java'
+ echo abc file1.java file2.java
abc file1.java file2.java

Fortunately you can disable pathname expansion using set -f:

#!/usr/bin/env bash
set -x
VAR='abc *.java'
set -f
echo $VAR

The output is:

+ VAR='abc *.java'
+ set -f
+ echo abc '*.java'
abc *.java

To sum up, the following should work:

#!/usr/bin/env bash

pattern='-name *.java'
dir="my_project"
set -f
find "$dir" -type f -a \( $pattern \)

Accepting this as it is the most complete description of the solution.

chepner · Accepted Answer · 2014-11-05 14:09:46Z

0

bash arrays were introduced to allow this kind of nested quoting:

file_sets=( -name "*.java" -o -name "*.as" )
find dir -type f -a \( "${file_sets[@]}" \)

answered Nov 5, 2014 at 14:09

chepner

538k77 gold badges594 silver badges746 bronze badges

Collectives™ on Stack Overflow

Bash parameter expansion in brackets not working as expected

3 Answers 3

4 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related