awk commands within python script

Question

I need to write a python script where I need to call a few awk commands inside of it.

#!/usr/bin/python
import os, sys
input_dir = '/home/abc/data'

os.chdir(input_dir)
#wd=os.getcwd()
#print wd
os.system ("tail -n+2 ./*/*.tsv|cat|awk 'BEGIN{FS="\t"};{split($10,arr,"-")}{print arr[1]}'|sort|uniq -c")

It gives an error in line 8: SyntaxError: unexpected character after line continuation character

Is there a way I can get the awk command get to work within the python script? Thanks

Why? there is nothing sed/awk/sort/uniq can do that you cannot do directly from within python and as a bonus you get a solution that runs on all platform python is ported to! — Fredrik Pihl
– Fredrik Pihl, Commented May 21, 2013 at 19:19

Aya · Accepted Answer · 2013-05-21 16:55:38Z

14

You have both types of quotes in that string, so use triple quotes around the whole thing

>>> x = '''tail -n+2 ./*/*.tsv|cat|awk 'BEGIN{FS="\t"};{split($10,arr,"-")}{print arr[1]}'|sort|uniq -c'''
>>> x
'tail -n+2 ./*/*.tsv|cat|awk \'BEGIN{FS="\t"};{split($10,arr,"-")}{print arr[1]}\'|sort|uniq -c'

edited May 21, 2013 at 16:55

Aya

42.5k6 gold badges60 silver badges56 bronze badges

answered May 21, 2013 at 16:45

TehTris

3,2371 gold badge23 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

TehTris Over a year ago

stack overflow doesn't format it correctly in the code blocks, but if you copy paste the line into the interpreter you'll see what I mean.

Schorsch · Accepted Answer · 2013-05-21 16:55:24Z

9

You should use subprocess instead of os.system:

import subprocess
COMMAND = "tail -n+2 ./*/*.tsv|cat|awk 'BEGIN{FS=\"\t\"};{split($10,arr,\"-\")}{print arr[1]}'|sort|uniq -c"  

subprocess.call(COMMAND, shell=True)

As TehTris has pointed out, the arrangement of quotes in the question breaks the command string into multiple strings. Pre-formatting the command and escaping the double-quotes fixes this.

edited May 21, 2013 at 16:55

answered May 21, 2013 at 16:44

Schorsch

7,9618 gold badges41 silver badges67 bronze badges

2 Comments

tripleee Over a year ago

Adding even more |cats to the pipeline would make it even better, no?

tommy.carstensen Over a year ago

Why should he use subprocess instead of os.system? Thanks.

Collectives™ on Stack Overflow

awk commands within python script

2 Answers 2

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related