Syntax error from the shell on a command being passed to os.system

Question

I have some odd behaviour occurring in my Python program.

perf_join_cmd = "join <(sort -k1,1 {}) <(sort -k1,1 {}) > {}.ScoreProfile"
perf_join_cmd = perf_join_cmd.format(meta, extracted_snps, out)
os.system(perf_join_cmd)

I am creating a string, passing in arguments to the string. The string represents a join/sort piped command to be ran on the shell, hence my call to os.system.

The error is:

sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `join <(sort -k1,1 ../../ADHD_GWAS.tbl) <(sort -k1,1 ../adhd-c/bin/adhd-merged.subset) > adhd-c.ScoreProfile'

I've been hacking away at this for long enough and I don't see anything that looks like a syntax error - so I suspect there's something deeper happening here and I was hoping someone could shed some light.

As an FYI, this is the output of the perf_jon_cmd when printed:

join <(sort -k1,1 ../../ADHD_GWAS.tbl) <(sort -k1,1 ../adhd-c/bin/adhd-merged.subset) > adhd-c.ScoreProfile

Any help appreciated!

Thanks.

Did you try debugging this in bash first before running it through python? — cs95
– cs95, Commented Jul 21, 2017 at 19:13
Yes. Running the exact command in bash (just copying it from the print) runs fine. — TimelordViktorious
– TimelordViktorious, Commented Jul 21, 2017 at 19:14
Have you looked at this: stackoverflow.com/questions/17584124/… — Cory Madden
– Cory Madden, Commented Jul 21, 2017 at 19:15
Ah, so doing os.system(perf_join_cmd, shell=True) should solve it? — cs95
– cs95, Commented Jul 21, 2017 at 19:16
Hm, not sure. The solution here suggests using /bin/bash -c, can you try that? stackoverflow.com/questions/21822054/… — Cory Madden
– Cory Madden, Commented Jul 21, 2017 at 19:19

janos · Accepted Answer · 2017-07-21 19:30:51Z

sh: -c: line 0: syntax error near unexpected token `('

The sh: at the start indicates that the command was executed by /bin/sh, and it is not Bash. The <(...) syntax is not supported in /bin/sh.

You can try to run through Bash, just be careful with the quoting:

perf_join_cmd = "join <(sort -k1,1 {}) <(sort -k1,1 {}) > {}.ScoreProfile"
perf_join_cmd = perf_join_cmd.format(meta, extracted_snps, out)
os.system("bash -c '{}'".format(perf_join_cmd))

Note that the above won't work if perf_join_cmd contains any single quotes. If that's a problem, you could try to enclose in double quotes instead. (If perf_join_cmd doesn't contain double quotes...)

It's very fragile to run complex shell commands like this. Users are very often confused about quoting and shell expansion rules when working in simple shell environments. When doing that through another language, it gets even more confusing and error-prone.

A good middle ground could be to encapsulate the complex command into a script, so that you can run simply:

os.system("script.sh args ...")

The bash -c '...' wrapping won't be necessary here, the script.sh should have a correct #!/bin/bash shebang line, and it will behave as expected.

Ian Kirkpatrick · Accepted Answer · 2017-07-21 19:24:41Z

-1

perf_join_cmd = "join <(sort -k1,1 {}) <(sort -k1,1 {}) > {}.ScoreProfile"

you probably just need to close the first <.

perf_join_cmd = "join <(sort -k1,1 {})> <(sort -k1,1 {}) > {}.ScoreProfile"

answered Jul 21, 2017 at 19:24

Ian Kirkpatrick

2,02018 silver badges35 bronze badges

1 Comment

TimelordViktorious Over a year ago

Does not work unfortunately. It seems like I need to use bash.

Collectives™ on Stack Overflow

Syntax error from the shell on a command being passed to os.system

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related