1

I have came across weird behavior of Bash variable in redirection after curl command:

curl -s 'https://www.google.com' > "./out_curl_$((++i)).txt"

I created following test script ./curl_test.sh:

#!/bin/env bash
i=0
curl -s 'https://www.google.com' > "./out_curl_$((++i)).txt"
curl -s 'https://www.google.com' > "./out_curl_$((++i)).txt"
echo "This is i: ${i}"
ls -1 ./out_curl_*.txt

j=0
echo 'hello' > "./out_echo_$((++j)).txt"
echo 'hello' > "./out_echo_$((++j)).txt"
echo "This is j: ${j}"
ls -1 ./out_echo_*.txt

exit 0

Output:

$ ./curl_test.sh
This is i: 0
./out_curl_1.txt
This is j: 2
./out_echo_1.txt
./out_echo_2.txt

Expected output:

$ ./curl_test.sh
This is i: 2
./out_curl_1.txt
./out_curl_2.txt
This is j: 2
./out_echo_1.txt
./out_echo_2.txt

Please does anybody know why is that? What already tried out:

wget has the same behavior:

wget -q -O - 'https://www.google.com' > "./out_wget_$((++k)).txt"

Encapsulating curl into sub-shell ( ... ) does not help.

This 2 liner of course works:

((i++))
curl -s 'https://www.google.com' > "./out_curl_${i}.txt"

Thank You for any hints.

0

1 Answer 1

1

Processing redirections is typically done after forking off a subprocess, but necessarily done before using the execve() to hand over control from the shell to the new executable (in your case curl or wget) being run in that process.

Because you've already fork()'d, changes to variables are happening in the new process's context, and aren't propagated back to the parent.

This doesn't apply to the default echo because it's a shell built-in, and is thus in-process (requiring no fork at all), but it does apply to /bin/echo on shells implemented in this manner (which the POSIX standard neither prohibits nor requires).


A simpler reproducer for this issue -- showing that it applies to all non-builtin commands -- follows, as generated with bash 3.2.57 and also reproduced against 4.4.23:

$ i=0; echo >>"test$((++i)).tmp"; echo "$i"
1
$ i=0; /bin/echo >>"test$((++i)).tmp"; echo "$i"
0

If you want to work around this problem, you can do so by performing the redirection for an entire command group:

$ i=0; { /bin/echo; } >>"test$((++i)).tmp"; echo "$i"
1

The { ...; } construct performs redirection for the entire block of commands, and thus executes the redirection before forking off the child process which will be replaced with /bin/echo.

Sign up to request clarification or add additional context in comments.

1 Comment

Thank You very much Charles for explanation. Executing commands curl, wget as a group { ... ; } works as expected and as You described.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.