6

I get the following error when I try to use bash for HTTP request. Does anybody know how to fix the problem? Thanks.

$ exec 3<>/dev/tcp/httpbin.org/80
$ echo -e 'GET /get HTTP/1.1\r\nUser-Agent: bash\r\nAccept: */*\r\nAccept-Encoding: gzip\r\nhost: http://httpbin.org\r\nConnection: Keep-Alive\r\n\r\n' >&3
$ cat <&3
HTTP/1.1 400 Bad Request
Server: awselb/2.0
Date: Wed, 31 Mar 2021 00:43:01 GMT
Content-Type: text/html
Content-Length: 524
Connection: close

<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
</body>
</html>
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
Improve this question

1 Answer 1

Reset to default
5

Your Host header is incorrect. That should be a hostname, not a URL:

$ echo -e 'GET /get HTTP/1.1\r
User-Agent: bash\r
Accept: */*\r
Accept-Encoding: gzip\r
host: httpbin.org\r
Connection: Keep-Alive\r
\r
' >&3

Which results in:

$ cat <&3
HTTP/1.1 200 OK
Date: Wed, 31 Mar 2021 00:55:23 GMT
Content-Type: application/json
Content-Length: 279
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip",
    "Host": "httpbin.org",
    "User-Agent": "bash",
    "X-Amzn-Trace-Id": "Root=1-6063c87b-09303a470da318290e856d71"
  },
  "origin": "96.237.56.197",
  "url": "http://httpbin.org/get"
}

Regarding your second question about getting your script to terminate properly, one option is to parse the Content-Length header and only read that many bytes. Something like:

#!/bin/bash

exec 3<>/dev/tcp/httpbin.org/80

cat <<EOF | dos2unix >&3
GET /get HTTP/1.1
User-Agent: bash
host: httpbin.org

EOF

while :;  do
    read line <&3
    line=$(echo "$line" | tr -d '\r')
    [[ -z $line ]] && break

    if [[ $line =~ "Content-Length" ]]; then
        set -- $line
        content_length=$2
    fi
done

echo "length: $content_length"
dd bs=1 count=$content_length <&3 2> /dev/null

That works for this one particular test case, but it's awfully fragile (e.g., what if there is no Content-Length header?).

I would just use curl instead of trying to use bash as an http client.

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

I put the three commands in a script, but the script hangs there. How to let the script terminate upon finishing running the three comands?
Are the '\r's necessary? I deleted all of them. It seems still working?
If the Content-Length is not available, there is no way to know all the data is sent so it can be finished? How wget/curl handle this case when the length is not known?
I suspect curl and wget simply have better networking code than bash and are able to respond correctly to a closed socket.
@user15502206 , @larsks One option, change Connection header value to close i.e. Connection: close. ` cat <&3` will then terminate as expected. You will have to run exec 3<>/dev/tcp/httpbin.org/80 again before attempting the http call. Which I think is ok. This is definitely not "efficient", but common, we are using BASH to do HTTP requests, so I think that ship has sailed :)
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.