Why is a variable changed within a loop unchanged outside the loop

Question

I use command | awk '{ print $1; }' | while read val ; do to loop through the command output. Recently I wanted to calculate the sum, and I discovered some strange behaviour in bash:

content of test.txt

100
200
300

content of test.sh

sum='0'
cat test.txt | awk '{ print $1; }' | while read val ; do
        sum=`expr $sum + $val`
        echo "sum is now: $sum"
done

echo "FINAL SUM: $sum"

output from executing test.sh

sum is now: 100
sum is now: 300
sum is now: 600
FINAL SUM: 0

the final sum should be 600. What can I do to fix this?

imp25 · Accepted Answer · 2012-10-10 13:00:49Z

2

To expand on what newfurniturey states, but in a way where you can use it with an arbritary input command, not just cat:

sum='0' 
while read val ; do
     sum=`expr $sum + $val`
     echo "sum is now: $sum" 
done < <(cat test.txt | awk '{ print $1 }')
echo "FINAL SUM: $sum"

Replace cat test.txt with whatever command which outputs the input you require.

answered Oct 10, 2012 at 13:00

imp25

2,35716 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ghoti Over a year ago

+1. I'll just add to this, that the documentation for what's going on here is on the bash man page in a section entitled "Process Substitution".

Steve · Accepted Answer · 2012-10-10 12:53:49Z

2

There's no need for the bash pipe. You can do it all with awk:

awk '{sum+= $1; printf "The sum is now: %s\n", sum } END { print "FINAL SUM:", sum }' file.txt

Results:

The sum is now: 100
The sum is now: 300
The sum is now: 600
FINAL SUM: 600

answered Oct 10, 2012 at 12:53

Steve

55.1k13 gold badges94 silver badges105 bronze badges

1 Comment

ghoti Over a year ago

+1. Pipes are a powerful tool, not to be used by those who don't understand them. Awk, however, should be used by everybody. :)

newfurniturey · Accepted Answer · 2012-10-10 13:00:46Z

2

The cause is the use of cat, which spawns another subshell. This means that the sum variable is incremented in the second subshell, and then goes out of scope (and returns to it's previous value of 0) when the loop is finished.

Try updating your loop to not use cat:

sum='0'
while read val ; do
        sum=`expr $sum + $val`
        echo "sum is now: $sum"
done < test.txt

echo "FINAL SUM: $sum"

If you don't actually need the loop (i.e. - if you're not handling any other column/content-processing), you could use awk directly and store it's value into the sum variable:

sum=`awk '{ sum += $1; } END { print sum }' test.txt`;

edited Oct 10, 2012 at 13:00

answered Oct 10, 2012 at 12:50

newfurniturey

38.6k10 gold badges99 silver badges104 bronze badges

3 Comments

imp25 Over a year ago

The OP wished to pre-process the input with awk, to do this you can replace the closing line of the while loop with done < <(awk '{ print $1 }' test.txt)

newfurniturey Over a year ago

@imp25 The contents of the sample file only contain a single column - so awk '{print $1}' is redundant; however, my updated answer allows the full use of that without the need for a loop (considering the OP doesn't need any additional processing inside the loop).

JMW Over a year ago

as i told in my case it is not really a cat command. i just used the cat to illustrate my problem

doubleDown · Accepted Answer · 2012-10-15 13:35:01Z

0

This strange behaviour is actually caused by the bash pipes

To quote the Bash reference manual

Each command in a pipeline is executed in its own subshell

Think of sum as a local variable in your while loop, that's why sum appears not to be set when you go out of the while-loop.

The solutions proposed by the others will work fine.

answered Oct 15, 2012 at 13:35

doubleDown

8,4581 gold badge36 silver badges50 bronze badges

Collectives™ on Stack Overflow

Why is a variable changed within a loop unchanged outside the loop

4 Answers 4

1 Comment

1 Comment

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

1 Comment

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related