204

Suppose I have a Unix shell variable as below

variable=abc,def,ghij

I want to extract all the values (abc, def and ghij) using a for loop and pass each value into a procedure.

The script should allow extracting arbitrary number of comma-separated values from $variable.

2
  • What exactly you want to do by iterating over? Commented Dec 30, 2014 at 9:11
  • 1
    I want to pass each of the field (say abc) as an argument to a procedure. Commented Dec 30, 2014 at 9:26

10 Answers 10

271

Not messing with IFS
Not calling external command

variable=abc,def,ghij
for i in ${variable//,/ }
do
    # call your procedure/other scripts here below
    echo "$i"
done

Using bash string manipulation http://www.tldp.org/LDP/abs/html/string-manipulation.html

Sign up to request clarification or add additional context in comments.

13 Comments

The nice thing about this is you can nest multiple loops using different delimiters. Good suggestion!
Nice tip for avoiding mess with IFS!
Small warning - if any of the values in $i contain spaces, they'll be split (and that might be the reason that it's passed as comma-separated rather than space-separated)
If a previous function changed the IFS setting, then this doesn't work... Change it to this: for i in ${variable//,/$IFS} do; echo "$i"; done
@anandhu make sure you are running 'sh' or 'bash'
|
193

You can use the following script to dynamically traverse through your variable, no matter how many fields it has as long as it is only comma separated.

variable=abc,def,ghij
for i in $(echo $variable | sed "s/,/ /g")
do
    # call your procedure/other scripts here below
    echo "$i"
done

Instead of the echo "$i" call above, between the do and done inside the for loop, you can invoke your procedure proc "$i".


Update: The above snippet works if the value of variable does not contain spaces. If you have such a requirement, please use one of the solutions that can change IFS and then parse your variable.

4 Comments

This doesn't work if $variable contains whitespace, e.g. variable=a b,c,d => prints 4 lines (a | b | c | d) rather than 3 (a b | c | d)
As well as it is getting added new quotes like ** "value **. could you please update if there is any regex to remove that ..
i am getting shellcheck warnings with this script
Shellcheck was released in 2020. This answer is from 2014 and was answered to address the original poster's question. If possible, you can edit the answer to improve it. :)
77

If you set a different field separator, you can directly use a for loop:

IFS=","
for v in $variable
do
   # things with "$v" ...
done

You can also store the values in an array and then loop through it as indicated in How do I split a string on a delimiter in Bash?:

IFS=, read -ra values <<< "$variable"
for v in "${values[@]}"
do
   # things with "$v"
done

Test

$ variable="abc,def,ghij"
$ IFS=","
$ for v in $variable
> do
> echo "var is $v"
> done
var is abc
var is def
var is ghij

You can find a broader approach in this solution to How to iterate through a comma-separated list and execute a command for each entry.

Examples on the second approach:

$ IFS=, read -ra vals <<< "abc,def,ghij"
$ printf "%s\n" "${vals[@]}"
abc
def
ghij
$ for v in "${vals[@]}"; do echo "$v --"; done
abc --
def --
ghij --

8 Comments

Makes me happy when someone actually knows how to use the shell, rather than pipe together a bunch of binutils.
do you have to set IFS back to whatever it was before ?
@pstanton no, because you are setting it just for the scope of this specific command. It is different if you would do IFS=, and then read -ra vals <<< "a,b,c" in another line. See Setting IFS for a single statement
@fedorqui It did! That's the only variable I wanted to loop through and it made my yaml file easier to read (instead of one LONG environment variable it has a bunch of rows)
This IFS=","; for i in $var; do echo "$i"; done works in bash, but didn't work for me in zsh. Apparently, zsh doesn't word split by default, but you can tell it to split with an extra =: IFS=","; for i in $=var; do echo "$i"; done.
|
13

I think syntactically this is cleaner and also passes shell-check linting

variable=abc,def,ghij
for i in ${variable//,/ }
do
    # call your procedure/other scripts here below
    echo "$i"
done

Comments

8
#/bin/bash   
TESTSTR="abc,def,ghij"

for i in $(echo $TESTSTR | tr ',' '\n')
do
echo $i
done

I prefer to use tr instead of sed, becouse sed have problems with special chars like \r \n in some cases.

other solution is to set IFS to certain separator

Comments

3

Another solution not using IFS and still preserving the spaces:

$ var="a bc,def,ghij"
$ while read line; do echo line="$line"; done < <(echo "$var" | tr ',' '\n')
line=a bc
line=def
line=ghij

1 Comment

This works correctly in bash, however zsh swallows the spaces.
2

Here is an alternative tr based solution that doesn't use echo, expressed as a one-liner.

for v in $(tr ',' '\n' <<< "$var") ; do something_with "$v" ; done

It feels tidier without echo but that is just my personal preference.

2 Comments

Nice solution, but unfortunately doesn't work. Can be fixed for example by setting IFS to \n.
How doesn't it work? If you copy/paste as is of course it wont work... otherwise, maybe be more specific and useful? Sorrt, comments dont support markdown so this will be poorly formatted: $ v=abc,def,ghij $ for v in $(tr ',' '\n' <<< "$var") ; do printf "Thing=%s\n" "$v" ; done Thing=abc Thing=def Thing=ghij It probably isn't whitespace proof, but the question didn't mention that.
2

The following solution:

  • doesn't need to mess with IFS
  • doesn't need helper variables (like i in a for-loop)
  • should be easily extensible to work for multiple separators (with a bracket expression like [:,] in the patterns)
  • really splits only on the specified separator(s) and not - like some other solutions presented here on e.g. spaces too.
  • is POSIX compatible
  • doesn't suffer from any subtle issues that might arise when bash’s nocasematch is on and a separator that has lower/upper case versions is used in a match like with ${parameter/pattern/string} or case

beware that:

  • it does however work on the variable itself and pop each element from it - if that is not desired, a helper variable is needed
  • it assumes var to be set and would fail if it's not and set -u is in effect
while true; do
        x="${var%%,*}"
        echo $x
        #x is not really needed here, one can of course directly use "${var%%:*}"

        if [ -z "${var##*,*}" ]  &&  [ -n "${var}" ]; then
                var="${var#*,}"
        else
                break
        fi
done

Beware that separators that would be special characters in patterns (e.g. a literal *) would need to be quoted accordingly.

Comments

0

Here's my pure bash solution that doesn't change IFS, and can take in a custom regex delimiter.

loop_custom_delimited() {
    local list=$1
    local delimiter=$2
    local item
    if [[ $delimiter != ' ' ]]; then
        list=$(echo $list | sed 's/ /'`echo -e "\010"`'/g' | sed -E "s/$delimiter/ /g")
    fi
    for item in $list; do
        item=$(echo $item | sed 's/'`echo -e "\010"`'/ /g')
        echo "$item"
    done
}

Comments

-1

Try this one.

#/bin/bash   
testpid="abc,def,ghij" 
count=`echo $testpid | grep -o ',' | wc -l` # this is not a good way
count=`expr $count + 1` 
while [ $count -gt 0 ]  ; do
     echo $testpid | cut -d ',' -f $i
     count=`expr $count - 1 `
done

9 Comments

but what if i am not sure about the number of fields in the variable testpid?
Also, i need to pass each filed of the above variable as an argument to a procedure. And i want to do this until the length of the variable becomes zero. (ie; all the fields should be passed once to the procedure for processing)
I don't know about the procedure. From this link You can get the count of the fields. http://stackoverflow.com/questions/8629330/unix-count-of-columns-in-file. After that Make that for loop into while loop. You can get that one.
This is to count the fields from a file i guess. What if i need to count the fields in a variable (suppose the variable is testpid=abc,def,ghij)
echo the variable and pipe the output to the awk. Or else see my updated answer. May be it is useful.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.