Submit results of a bash command to python script as sys.argv[1:]

Question

I have a list of files to be used as input that is very long. I was looking at the file input module, which seems to have some attractive features, [https://docs.python.org/3/library/fileinput.html][1]

But I could not figure out how to use it without entering each file name by hand, which would be prohibitively long.

What I would love to do is submit something like

python myscript.py (ls -d -1 dir/* | grep summary.txt)

Or I could push this same bash command to filelist.txt and submit that way; either way the trick would be to submit without having to type each file.

Maybe this will work: ls -d -1 dir/* | xargs grep summary.txt | xargs python myscript.py — Marcin
– Marcin, Commented Jun 12, 2014 at 4:46
You should probably not be calling on an external utility to generate the file names to begin with. Python is perfectly capable of all of that. Never use ls output in a program either. — ormaaj
– ormaaj, Commented Jun 12, 2014 at 7:07
What are the contents of summary.txt? Under no circumstances should you be processing the output of ls. (Also, grep reads from standard input or a file argument, not both.) — chepner
– chepner, Commented Jun 12, 2014 at 12:30
@chepner - why "under no circumstances" could you help me understand why this is bad practice? — Vincent Laufer
– Vincent Laufer, Commented Jun 12, 2014 at 17:41

GleasonK · Accepted Answer · 2014-06-12 04:42:38Z

1

You could use the os package or subprocess. You would just have to figure out the exact command you would need to run in the terminal then read it and split it.

import os
files = os.popen('ls').read()
# "file1\nfile2\nfile3\nfile4"
files.rsplit()
# ['file1', 'file2', 'file3', 'file4']

Now you have a list of inputs that you can pass to the function

answered Jun 12, 2014 at 4:42

GleasonK

1,0228 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

chepner Over a year ago

Use os.listdir instead of a pipe to ls.

A.J. Uppal · Accepted Answer · 2014-06-12 04:44:22Z

0

I'm pretty sure you cannot use sys.argv for this. Try using subprocess.Popen():

import subprocess
args = subprocess.Popen(['ls', '-d', '-1', 'dir/*', '|', 'grep', 'summary.txt'], stdout=subprocess.PIPE)
out, err = args.communicate()
#Do something with out

answered Jun 12, 2014 at 4:44

A.J. Uppal

19.3k7 gold badges48 silver badges82 bronze badges

1 Comment

ormaaj Over a year ago

This won't work. ls doesn't understand globs and definitely can't create a pipe or execute the grep utility. You should be executing a shell (or better, not relying on external programs for this simple task at all).

Collectives™ on Stack Overflow

Submit results of a bash command to python script as sys.argv[1:]

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related