Using a command stored as Bash array inside a heredoc

Question

Background: I'm writing a wrapper script which handles sending our researchers' jobs to our computing clusters (one of which uses SLURM, the other one Torque/PBS). It also has to serve as a drop-in replacement for wrapper scripts included in a few pieces of analysis software (eg. FSL) so they can interact with the cluster directly.

On the user side, the script would be called like this:

submit_job [arguments] <command to run>

The call to the scheduler gets assembled and <command to run> should end up in a small script that gets submitted to the scheduler. Currently, this is implemented like this:

# In case of SLURM
sbatch <sbatch arguments> <<EOF
#!/usr/bin/env bash
srun "${@}"
EOF

However, if I just use ${@}, the quotes are not preserved. The command could contain quotes (eg. paths with spaces), and there could be multiple commands separated with semicolons (then the whole thing would be wrapped in quotes though). I cannot really control what gets put in there by the combination of people and software that use it.

So, how can I get the command (which could contain quotes, eg. paths with spaces, and possibly multiple commands separated with semicolons) from A (script parameters) to B (variable) to C (heredoc), without messing up the quoting?

I assemble the scheduler commands using Bash arrays, as recommended in the oft-cited FAQ entry, and it works fine. However, while I can do this:

#/usr/bin/env bash
run=( mkdir "foo bar" )
set -x
"${run[@]}"
set +x

and the directory foo bar is created, I cannot do this:

#/usr/bin/env bash
run=( mkdir "foo bar" )
/usr/bin/env bash <<EOF
set -x
$run[@]
"${run[@]}"
set +x
EOF

The first one creates directories foo and bar, because expansion happens, and the second one tries, and fails, to run mkdir foo bar.

So, what am I left with?

Thank you in advance

KamilCuk · Accepted Answer · 2018-08-23 09:03:25Z

4

Escape your args using printf "%q". This is a nice technique for passing your args around.

#/usr/bin/env bash
run=(mkdir "foo bar")
/usr/bin/env bash <<EOF
set -x
${run[@]}  # this expands into `mkdir foo bar`
$(printf "%q " "${run[@]}")  # this expands into `mkdir foo\ bar`
set +x
EOF

answered Aug 23, 2018 at 9:03

KamilCuk

146k8 gold badges86 silver badges155 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Julius Juurmaa Over a year ago

This works brilliantly, thank you! I stumbled upon printf "%q " sometime before finally asking -- didn't quite put the pieces together back then.

Collectives™ on Stack Overflow

Using a command stored as Bash array inside a heredoc

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related