Send each line as a single argument in bash

Question

I have a variable in bash, which is something like

filenames='file 1
file 2
file 3'

I need to send each line in the above variable's content as a single argument to a program. But I can't get bash to that. Here's what I tried:

python -c 'import sys; print sys.argv' $filenames
['-c', 'file', '1', 'file', '2', 'file', '3']

or

python -c 'import sys; print sys.argv' "$filenames"
['-c', 'file 1\nfile 2\nfile 3']

What I'm expecting is something like this

['-c', 'file 1', 'file 2', 'file 3']

I've tried fiddling with the IFS setting too, but couldn't get it right. Any ideas on this?

Gordon Davisson · Accepted Answer · 2012-02-12 17:42:47Z

2

Double-evaluating a command with either eval or sh -c can lead to weird bugs if you don't get the extra level of quoting just right. I'd recommend using an array instead, either just by storing the file list as an array to begin with:

filearray=("file 1" "file 2" "file 3")
python -c 'import sys; print sys.argv' "${filearray[@]}"

or by converting it from newline-delimited string to an array:

filenames='file 1
file 2
file 3'
oldIFS="$IFS"; IFS=$'\n'
filearray=($filenames)
IFS="$oldIFS"
python -c 'import sys; print sys.argv' "${filearray[@]}"

answered Feb 12, 2012 at 17:42

Gordon Davisson

128k16 gold badges141 silver badges168 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

sharat87 Over a year ago

+1 This is the perfect solution I've been wanting. Thanks a bunch.

perelman · Accepted Answer · 2012-02-12 10:39:30Z

2

$ export filenames="file 1
file 2
file 3"
$ echo "$filenames" | xargs -d\\n python -c 'import sys; print sys.argv'
['-c', 'file 1', 'file 2', 'file 3']

Another way using read instead of xargs:

#!/bin/bash
filenames="file 1
file 2
file 3"
cmd="python -c 'import sys; print sys.argv; print sys.stdin.readlines()'"
while read file
do
    args="$args \"$file\""
done < <(echo "$filenames")
echo $cmd $args
echo `echo hi | sh -c "$cmd $args"`

Output:

$ ./test.sh 
python -c 'import sys; print sys.argv; print sys.stdin.readlines()' "file 1" "file 2" "file 3"
['-c', 'file 1', 'file 2', 'file 3'] ['hi\n']

edited Feb 12, 2012 at 10:39

answered Feb 12, 2012 at 10:06

perelman

1,78514 silver badges25 bronze badges

4 Comments

sharat87 Over a year ago

whoa! I thought xargs runs the command once for each line of input, no?

sharat87 Over a year ago

Also, I need something else to be going in the stdin of the python command, so I don't think this'll do it in its current form.

perelman Over a year ago

Nope, you would need to use the -n1 option to get that behavior. By default it uses as many arguments as will fit in a line for each invocation (you can redefine "fit" with the -s and -n options).

sharat87 Over a year ago

Ah, eval. I had a feeling this'd be the solution. Also, the while loop can be turned to this: args="$(echo "$filenames" | sed 's/^\|$/"/g' | tr '\n' ' ')" :)

tripleee · Accepted Answer · 2012-02-12 20:09:58Z

1

Just to add another option,

OLDIFS=$IFS
IFS='
'
set -- $filenames
python -c 'import sys; print sys.argv; print sys.stdin.readlines()' "$@"
IFS=$OLDIFS

Edit: Add code to save old value in OLDIFS and restore it afterwards. IFS='whatever' set -- $var on a single line does not work.

edited Feb 12, 2012 at 20:09

answered Feb 12, 2012 at 17:29

tripleee

192k37 gold badges318 silver badges367 bronze badges

3 Comments

sharat87 Over a year ago

And I was trying IFS="\n" and the like, #facepalm. Thanks.

tripleee Over a year ago

In bash, $'\n' is a literal newline.

sharat87 Over a year ago

Hm, I remember trying that too. Oh well.

Huang F. Lei · Accepted Answer · 2012-02-12 17:51:05Z

0

Yet another option:

while read line ; do
  python -c 'import sys; print sys.argv' $line;
done <(echo $filenames)

answered Feb 12, 2012 at 17:51

Huang F. Lei

1,86515 silver badges23 bronze badges

1 Comment

tripleee Over a year ago

Wouldn't any newlines be mutilated because of the echo without quoting?

bhdnx · Accepted Answer · 2014-06-19 12:01:10Z

0

What about xargs:

$ seq 10 | xargs  python -c 'print __import__("sys").argv'
['-c', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10']

answered Jun 19, 2014 at 12:01

bhdnx

5407 silver badges6 bronze badges

Collectives™ on Stack Overflow

Send each line as a single argument in bash

5 Answers 5

1 Comment

4 Comments

3 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

4 Comments

3 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related