Using a string which contains quotes as parameter to a command in bash without additional escaping

Question

Background

What I'm trying to do is exclude all submodules of a git repository in a find command. I know that I can exclude a single directory like this:

find . -not -path './$submodule/*'

So I built a command that generates a lot of these statements and stores them:

EXCLUDES=$(for submodule in $(git submodule status | awk '{print $2}'); do
    echo -n "-not -path './$submodule/*' ";
done)

Problem

But when I run find . $EXCLUDES, this does not work. I suspect this is because of a bash quoting strategy that I do not understand. For example, lets assume (# marks output):

tree .
# .
# ├── bar
# │   └── baz.scala
# └── foo.scala

set -x
EXCLUDES="-not -path './bar/*'"
find . -type f $EXCLUDES
# + find . -not -path ''\''./bar/*'\''' <---- weird stuff
# foo.scala
# baz.scala

find . -type f -not -path './bar/*'
# + find . -type f -not -path './bar/*'
# foo.scala

How do I tell bash not to to the weird quoting stuff its doing (see marked line above)?

Edit: @eddiem suggested using git ls-files, which I will do in this concrete case. But I'm still interested in how I'd do this in the general case where I have a variable with quotes and would like to use it as arguments to a command.

Are you just trying to grep for something in source files that are in your primary repo, but not in submodules? If so, you can accomplish that with git grep. — eddiem
– eddiem, Commented Oct 25, 2016 at 17:00
you're protecting your submodule env. variable in simple quotes. So it's not evaluated. Use no quotes or double quotes. — Jean-François Fabre
– Jean-François Fabre ♦, Commented Oct 25, 2016 at 17:00
@eddiem Nope, not grepping. It's to be part of a linting build step. For which I need a certain set of files. — fresskoma
– fresskoma, Commented Oct 25, 2016 at 17:05
@Jean-FrançoisFabre I don't understand. Why would it be evaluated (do you mean globbing?) ? Changing everything to double quotes does not make a difference in the example below. — fresskoma
– fresskoma, Commented Oct 25, 2016 at 17:05
@fresskoma Do you only care about files that are in the repo (as opposed to unstaged new files, for example)? If so, you can just use git ls-files. — eddiem
– eddiem, Commented Oct 25, 2016 at 17:10

cxw · Accepted Answer · 2016-10-25 17:54:15Z

The "weird stuff" you note is because bash only expands $EXCLUDES once, by substituting in the value you stored in EXCLUDES. It does not recursively process the contents of EXCLUDES to remove single-quotes like it does when you specify the quoted string on the command line. Instead, bash escapes special characters in $EXCLUDES, assuming that you want them there:

-not -path './bar/*'

becomes

-not -path ''\''./bar/*'\'''
             ^^         ^^ escaped single quotes
           ^^             ^^ random empty strings I'm actually not sure about
               ^       ^ single quotes around the rest of your text.

So, as @Jean-FrançoisFabre said, if you leave off the single quotes in EXCLUDES=..., you won't get the weird stuff.

So why isn't the first find working as expected? Because bash expands $EXCLUDES into a single word, i.e., a single element of argv that gets passed to find.^* However, find expects its arguments to be separate words. As a result, find does not do what you expect.

The most reliable way I know of to do this sort of thing is to use an array:

declare -a EXCLUDES    #make a new array
EXCLUDES+=("-not" "-path" './bar/*')
    # single-quotes       ^       ^ so we don't glob when creating the array

and you can repeat the += line any number of times for exclusions that you want. Then, to use these:

find . -type f "${EXCLUDES[@]}"

The "${name[@]}" form, with all that punctuation, expands each element of the array to a separate word, but does not further expand those words. So ./bar/* will stay as that and not be globbed. (If you do want globbing, find . -type f ${EXCLUDES[@]} (without the "") will expand each element of the array.)

Edit By the way, to see what's in your array, do set|grep EXCLUDES. You will each each element listed separately. You can also do echo "${EXCLUDES[@]}", but I find that less useful for debugging since it doesn't show the indices.

^* see the "expansion" section of the man page. "Parameter expansion," expanding things that start with $, cannot change the number of words on the command line — except for "$@" and "${name[@]}".

Collectives™ on Stack Overflow

Using a string which contains quotes as parameter to a command in bash without additional escaping

Background

Problem

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Background

Problem

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related