bash for loop with error when detecting non 200 response from curl

Question

I got a bash script which reads URLs from different .txt files (ven Array defines the txt files) and I curl them and if the Response Code is not 200 it writes an error in error.txt

Sadly it always does it at the end, even if there is no error in any of the links, any idea why?

for i in "${ven[@]}"; do
        while IFS='' read -r line || [[ -n "$line" ]]; do
                        IP=$(curl --write-out '%{http_code}' --silent --output /dev/null $line?upstream=$1)
                                if [ $IP != 200 ]
                                then
                                counter=$((counter + 1))
                                echo $(date +"%d.%m.%y %T") : $line >> error.txt
                                fi
        done < $i
done

ven=("1.txt" "2.txt" "3.txt" "4.txt") - looks fine to me - am I missing smth? — Isengo
– Isengo, Commented Jul 29, 2017 at 7:24
It was a bogus in one of the textfiles :/ one line i oversaw in the 2.txt — Isengo
– Isengo, Commented Jul 29, 2017 at 7:33

l'L'l · Accepted Answer · 2017-07-29 07:22:11Z

1

Your script has some quoting issues, although the main issue is the inconsistent use of test expression brackets (double vs. single) and how you are checking for the number.

if [[ "$IP" -ne 200 ]]

-ne means "not equal", and since you've already used double brackets stay consistent.

The other stuff is more preventative in the way you quote the variables:

for i in "${ven[@]}"; do
    while IFS='' read -r line || [[ -n "$line" ]]; do
        IP=$(curl --write-out '%{http_code}' --silent --output /dev/null "$line?upstream=$1")
        if [[ "$IP" -ne 200 ]]; then
            counter=$((counter + 1))
            echo "$(date +'%d.%m.%y %T')" : "$line" >> error.txt
        fi
    done < "$i"
done

NOTE: If a site is redirecting ( 301 ) then it will show an error — something to maybe consider.

answered Jul 29, 2017 at 7:22

l'L'l

47.5k12 gold badges102 silver badges154 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Isengo Over a year ago

That does not seem to be the issue, but thank you for pointing that out. I haven´t done to much with bash yet. I will get an editor to point out the formatting stuff. 301 redirects should also throw an error, since there URLs should not be redirected - but thanks.

l'L'l Over a year ago

The script worked fine after making the changes; if you are missing code in your question you should add it since it's not reproducible here. (eg. counter=$((counter + 1)) is useless)...

Isengo Over a year ago

Found the mistake, there was an emtpy line in one of the txt files. thanks.I use the counter later to log the number (echo -e 'Curl Requests had' $counter 'Errors' >> error.txt)

Collectives™ on Stack Overflow

bash for loop with error when detecting non 200 response from curl

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related