Passing arguments to a Python script from a shell variable containing quoted spaces

Question

I am calling a python script through a bash wrapper, but I'm having trouble dealing with arguments that contain quoted spaces.

I assemble the arguments to the python script into a bash variable, such as

opt="-c start.txt"
opt+="--self 'name Na'"

Then call the python script with something like:

python test_args.py $opt

When printing sys.argv in Python, I get

['test-args.py', '-c', 'start.txt', '--self', "'name", "Na'"]

instead of the expected

['test-args.py', '-c', 'start.txt', '--self', 'name Na']

I tried using an array when calling the script, such as

python test_args.py ${opt[@]}

but then I get

['test-args.py', "-c start.txt --self 'name Na'"]

Any other ideas?

Maybe this will help (look at the eval solution).

KBart
– KBart

2013-01-31 14:54:48 +00:00
Commented Jan 31, 2013 at 14:54 — KBart
– KBart, Commented Jan 31, 2013 at 14:54

Gordon Davisson · Accepted Answer · 2013-01-31 15:17:03Z

4

Use an array, but store each argument as a separate element in the array:

opt=(-c start.txt)
opt+=(--self 'name Na')

python test_args.py "${opt[@]}"

See BashFAQ #050.

answered Jan 31, 2013 at 15:17

Gordon Davisson

128k16 gold badges141 silver badges168 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

englebip Over a year ago

This is exactly what I came up with in the end, thanks for writing it up!

bgporter · Accepted Answer · 2013-01-31 14:55:34Z

0

This is what the shlex module is for.

The shlex class makes it easy to write lexical analyzers for simple syntaxes resembling that of the Unix shell. This will often be useful for writing minilanguages, (for example, in run control files for Python applications) or for parsing quoted strings.

answered Jan 31, 2013 at 14:55

bgporter

36.9k8 gold badges65 silver badges67 bronze badges

Comments

William · Accepted Answer · 2013-01-31 22:20:28Z

-1

Your instinct to embed spaces inside the variable's value was good, but when the value is simply expanded during the command line parsing their special meaning is lost as you saw. You need to expand the variable before the command line to your python script is parsed:

set -f
eval python test_args.py $opt
set +f

That will expand to:

python test_args.py -c start.txt --self 'name Na'

Which will then be parsed correctly with the quotes regaining their special meaning.

Edit: I've added set -f/+f (aka -/+o noglob) around the eval to disable file globbing although that wasn't an issue in the OP's example that's not an unheard of issue with eval. (Another, stronger caveat is to never eval user input unless you take extreme care to make sure it won't blow up into something nasty. If you don't control the value being eval-ed, you can't be sure what will happen.)

edited Jan 31, 2013 at 22:20

answered Jan 31, 2013 at 15:24

William

4,9652 gold badges17 silver badges14 bronze badges

2 Comments

Gordon Davisson Over a year ago

Please don't use eval, it has a well-deserved reputation for causing bizarre bugs. For example, try this with opt="-c start.txt --self 'name * Na'" and try to explain the result.

William Over a year ago

I reject your risk assessment in the context of the OP which shows the script controlling the content of the value to be eval-ed, but agree a caveat is called for. Might have been more helpful though if you'd suggested surrounding the eval with set -f ... set +f to disable file globbing. (Which is the problem you're hinting it, of course.)

Collectives™ on Stack Overflow

Passing arguments to a Python script from a shell variable containing quoted spaces

3 Answers 3

1 Comment

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related