BASH Recursive Function

Question

I have a test site that performs a chain of 302 redirects. I'm looking to write a BASH script that will print the redirect URL from the response header from each page in the chain.

The following command will strip the URL from an individual response header and print it.

$ curl -Is http://iperformalotofredirects.com/ | sed -ne 's/Location: //' -e6p
http://iperformalotofredirects.com/firststep/

$ curl -Is http://iperformalotofredirects.com/firststep/ | sed -ne 's/Location: //' -e6p
http://iperformalotofredirects.com/nextstep/

I wrote a python script that functions properly, but for the sake of research, I'm also trying to do the same thing in a BASH script.

#!/bin/bash

geturl() {
  if [[ $1 == http* ]]
  then
    echo $1
    geturl $(curl -Is $1 | sed -ne 's/Location: //' -e6p)
  fi
}

geturl http://iperformalotofredirects.com/

The script comes up empty after the first iteration

$ ./curlit.sh
http://iperformalotofredirects.com/
http://iperformalotofredirects.com/firststep/

What I need is for the script to continue until exhaustion. My manual results show that more comes after http://iperformalotofredirects.com/firststep/ whereas my script is failing at achieving this.

Edit: Included @konsolebox's answer as reference

#!/bin/bash

main() {
    local location=$1

    while [[ $location == http* ]]; do
        echo "$location"
        location=$(curl -Is "$location" | awk '/Location: / { print $2 }')
    done
}

main "$@"

Copied and pasted this script identically. Ran it:

$ ./curlit.sh http://iperformalotofredirects.com/
http://iperformalotofredirects.com/
http://iperformalotofredirects.com/firststep/

What I expect to see:

$ ./curlit.sh http://iperformalotofredirects.com/
http://iperformalotofredirects.com/
http://iperformalotofredirects.com/firststep/
http://iperformalotofredirects.com/nextstep/
http://iperformalotofredirects.com/anotherstep/
http://iperformalotofredirects.com/moresteps/
http://iperformalotofredirects.com/finalstep/

Edit: fravadona requested the entire header

$ curl -Is http://iperformalotofredirects.com/
HTTP/1.1 302 Moved Temporarily
Server: nginx/1.18.0
Date: Fri, 07 Jan 2022 09:20:56 GMT
Content-Type: text/html
Content-Length: 145
Location: http://iperformalotofredirects.com/firststep/
Connection: keep-alive

$ curl -Is http://iperformalotofredirects.com/firststep/
HTTP/1.1 302 Moved Temporarily
Server: nginx/1.18.0
Date: Fri, 07 Jan 2022 09:20:56 GMT
Content-Type: text/html
Content-Length: 145
Location: http://iperformalotofredirects.com/nextstep/
Connection: keep-alive

So what do you need? If you're asking for a review, consider posting in Code Review instead. Also, Bash does not support tail call optimization so soon enough you would reach "stack overflow" with that code. — konsolebox
– konsolebox, Commented Jan 7, 2022 at 8:46
@konsolebox I'm not asking for a review. I thought it was apparent. I added a sentence at the bottom of the original post for explanation. — sirEgghead
– sirEgghead, Commented Jan 7, 2022 at 8:50
I appreciate the info about the stack overflow. Definitely not looking to break anything. I'd be happy to have the recursive function changed out for a while loop. I tried something similar, and failed at that attempt. — sirEgghead
– sirEgghead, Commented Jan 7, 2022 at 9:01
How about against https://kernel.org, did it work as you expected or not? — konsolebox
– konsolebox, Commented Jan 7, 2022 at 9:15
What is the entire response header when inquiring http://iperformalotofredirects.com/firststep/? — Fravadona
– Fravadona, Commented Jan 7, 2022 at 9:18

Fravadona · Accepted Answer · 2022-01-09 00:22:06Z

2

Try this code for debugging:

#!/bin/bash

main() {
    local location=$1

    while [[ $location == http* ]]; do 
        printf '%q\n' "$location"
        location=$(curl -Is "$location" | awk '/^Location: / { print $2 }')
    done
}

main "$@"

For me, running main https://kernel.org outputs $'https://www.kernel.org/\r', which means that there might be an unwanted carriage return at the end of the location.

A possible fix would be to strip that carriage return:

#!/bin/bash

main() {
    local location=$1

    while [[ $location == http* ]]; do 
        printf '%q\n' "$location"
        location=$(curl -Is "$location" | awk '/^Location: / { print $2 }')
        location=${location%$'\r'}
    done
}

main "$@"

edited Jan 9, 2022 at 0:22

answered Jan 7, 2022 at 9:54

Fravadona

17.6k1 gold badge29 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

konsolebox Over a year ago

This definitely it. I thought about CRLF but only about the script.

sirEgghead Over a year ago

Good call. Let me have a look.

Collectives™ on Stack Overflow

BASH Recursive Function

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related