Bash build array in while loop (does not persist)? [duplicate]

Question

Consider this example, say, test.sh:

cat > test.txt <<'EOF'
test 1
test 2
test 3
EOF

declare -a myarr
declare -p myarr  # "declare: myarr: not found"
myarr=()
declare -p myarr  # "declare -a myarr='()'"
#for (( i=1; i<=3; i++ )); do # ok
sed -n 's!test!TEST!p' test.txt | while read i; do # not preserved ?!
  myarr=("${myarr[@]}" "pass $i")
  declare -p myarr
done
declare -p myarr  # "declare -a myarr='()'" ?!

If I uncomment the for ((... line, and comment the sed -n ... line, then the output of bash test.sh is as expected:

test.sh: line 8: declare: myarr: not found
declare -a myarr='()'
declare -a myarr='([0]="pass 1")'
declare -a myarr='([0]="pass 1" [1]="pass 2")'
declare -a myarr='([0]="pass 1" [1]="pass 2" [2]="pass 3")'
declare -a myarr='([0]="pass 1" [1]="pass 2" [2]="pass 3")'

However, if I run the script as posted, then the myarr builds in the while loop, but once outside, it's empty:

test.sh: line 8: declare: myarr: not found
declare -a myarr='()'
declare -a myarr='([0]="pass TEST 1")'
declare -a myarr='([0]="pass TEST 1" [1]="pass TEST 2")'
declare -a myarr='([0]="pass TEST 1" [1]="pass TEST 2" [2]="pass TEST 3")'
declare -a myarr='()'

So, why is myarr in this case (in, or rather, after the while loop) empty - and how do I get it to preserve its value?

This is BashFAQ #24.

Charles Duffy
– Charles Duffy

2017-11-10 05:39:51 +00:00
Commented Nov 10, 2017 at 5:39 — Charles Duffy
– Charles Duffy, Commented Nov 10, 2017 at 5:39

anubhava · Accepted Answer · 2017-11-10 05:08:28Z

8

Problem is that due to use of pipeline, you are forking a subshell and populating entries in your array inside the subshell. Once your loop ends, subshell terminates and all the changes get lost there.

You can use a process substitution to avoid this:

myarr=()

while IFS= read  -r line; do
   myarr+=("pass $line")
done < <(sed -n 's!test!TEST!p' test.txt)

# examine the result
declare -p myarr

Output:

declare -a myarr=([0]="pass TEST 1" [1]="pass TEST 2" [2]="pass TEST 3")

answered Nov 10, 2017 at 5:08

anubhava

790k67 gold badges603 silver badges671 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

sdaau Over a year ago

Many thanks @anubhava -> that was indeed the issue! Cheers!

ephemient Over a year ago

I prefer using other forms such as exec {fd}< <(sed ...); while read -r -u $fd line; do ...; done; exec {fd}<&- because it prevents accidentally consuming the same source within the loop, but this is the right way to do it.

Collectives™ on Stack Overflow

Bash build array in while loop (does not persist)? [duplicate]

1 Answer 1

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Linked

Related