bash Command Substitution with Spaces Issue

Question

I'm trying to populate some exclusions to run a backup script on the command line by reading the values from an external file and prefixing these with -x. There will be a variable number of exclusions, including none at all. The trouble I'm running into is that when bash expands this variable, it frames it with single quotes. This confuses the backup script.

#cat sss2.sh:

#!/bin/bash
IFS=$'\n'
xcl=""
for x in $(cat /root/exfile)
 do
 xcl="$xcl -x $x"
 done
backupbin /sourcepath /destination "$xcl"

# cat exfile
"*.tgz"
"*.zip"

When I run this, bash expands the variable $xcl with single quotes, which prevents backupbin from running. It is due to the spaces, because if I didn't have the -x, the single quotes disappear.

# /bin/bash -x ./sss2
+ IFS=''
+ xcl=
++ cat exfile
+ for x in '$(cat /root/exfile)'
+ xcl=' -x "*.tgz"'
+ for x in '$(cat exfile)'
+ backupbin /sourcepath /destination ' -x "*.tgz" -x "*.zip"'

I have tried this as expanding a populated array, and have tried different combinations of single quotes, double quotes, and back ticks. I've tried using eval without success:

backupbin /sourcepath /destination $(eval echo "$xcl")

Finally, I've tried creating an array and expanding this, but I get the same result:

IFS=$'\n' 
excludefile=($(cat /root/exfile))
if [ ${#excludefile[@]} -eq 0 ]; then
 echo ""
 else
 for i in "${excludefile[@]}"
 do
 printf "%s" " -x $i"
 done

What am I doing wrong?

@basin He doesn't need to use eval if he wants the shell to expand them. With $(cat /root/exfile), glob patterns would still be expanded. Word splitting is not the only thing that happens there. The behaviour should also be the same with $(</root/exfile). — konsolebox
– konsolebox, Commented Jun 4, 2014 at 6:03
That's the actual reason why for x in $<something> is to be avoided unless pathname expansion is really intended and relying from IFS would be safe enough. There's a safer solution to it actually, but I'm not going to allow another pioneer idea to be stolen by posturing FAQ/tutorial authors who actually just gathered ideas from old forums :) — konsolebox
– konsolebox, Commented Jun 4, 2014 at 6:11
Thanks, I didn't know that. Do you know how to expand this? glob='* *' — basin
– basin, Commented Jun 4, 2014 at 6:14

rici · Accepted Answer · 2014-06-04 13:50:18Z

5

As always with this sort of problem, the simplest solution to this kind of problem is to use an array, rather than to try to juggle quotes. (Quote+eval solutions almost always fail in some corner case, sometimes disastrously.)

#!/bin/bash
IFS=$'\n'

xcl=()
for x in $(< /root/exfile); do
  xcl+=(-x "$x")
done

backupbin /sourcepath /destination "${xcl[@]}"

See the bash FAQ.

Edit: as @konsolebox notes, the use of $(</root/exfile) is subject to pathname expansion. So if the file contained a line like "foo*" and one or more files whose names start with foo exist in the current working directory, then the exclusion pattern -- which was probably intended to be a wildcard exclusion -- will instead by replaced by the list of filenames. It would better to use:

#!/bin/bash
xcl=()
while IFS= read -r x; do xcl+=(-x "$x"); done < /root/exfile
backupbin /sourcepath /destination "${xcl[@]}"

edited Jun 4, 2014 at 13:50

answered Jun 4, 2014 at 5:13

rici

243k30 gold badges263 silver badges364 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

konsolebox · Accepted Answer · 2014-06-04 05:51:48Z

2

@rici's suggestion about using an array is already correct. As for reading lines, better use this form:

while read -r x; do
    xcl+=(-x "$x")
done < /root/exfile

Reading values through word splitting is not recommended.

answered Jun 4, 2014 at 5:51

konsolebox

76.3k13 gold badges110 silver badges114 bronze badges

4 Comments

rici Over a year ago

Good point about word splitting, but unfortunately you still need to set IFS to get read to work correctly in the case that a line begins or ends with whitespace: while IFS= read -r x; will work. mapfile is another possibility.

basin Over a year ago

This will definitely not expand globs.

konsolebox Over a year ago

@basin Yep I made the answer before making the comment and seeing the possibility that the OP would actually want them expanded.

konsolebox Over a year ago

@rici It's rare for filenames to be having leading and trailing spaces but yes that could be helpful as well. About mapfile yes I thought about it (readarray) but you have to insert -x before every element anyway that you'd still run a loop so I just decided to skip it.

Collectives™ on Stack Overflow

bash Command Substitution with Spaces Issue

2 Answers 2

Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related