5

I try to read/fetch this file:

https://blockchain-office.com/file.txt with a bash script over dev/tcp without using curl,wget, etc..

I found this example:

exec 3<>/dev/tcp/www.google.com/80
echo -e "GET / HTTP/1.1\r\nhost: http://www.google.com\r\nConnection: close\r\n\r\n" >&3
cat <&3

I change this to my needs like:

exec 3<>/dev/tcp/www.blockchain-office.com/80
echo -e "GET / HTTP/1.1\r\nhost: http://www.blockchain-office.com\r\nConnection: close\r\n\r\n" >&3
cat <&3

When i try to run i receive:

400 Bad Request
Your browser sent a request that this server could not understand

I think this is because strict ssl/only https connections is on.

So i change it to :

exec 3<>/dev/tcp/www.blockchain-office.com/443
echo -e "GET / HTTP/1.1\r\nhost: https://www.blockchain-office.com\r\nConnection: close\r\n\r\n" >&3
cat <&3

When i try to run i receive:

400 Bad Request
Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.

So i even can't get a normal connection without get the file!

All this post's does not fit, looks like ssl/tls is the problem only http/80 works, if i don't use curl, wget, lynx, openssl, etc...:

how to download a file using just bash and nothing else (no curl, wget, perl, etc.)

Using /dev/tcp instead of wget

How to get a response from any URL?

Read file over HTTP in Shell

I need a solution to get/read/fetch a normal txt file from a domain over https only with /dev/tcp no other tools like curl, and output in my terminal or save in a variable without wget, etc.., is it possible and how, or is it there an other solution over the terminal with the standard terminal utilities?

Improve this question
7
  • i dont think bash's /dev/tcp supports ssl/tls. Commented Jul 12, 2022 at 9:19
  • Any other command for the terminal, with the standard utilities? Commented Jul 12, 2022 at 9:20
  • Than this is the same problem no ssl/tls unix.stackexchange.com/questions/83926/… Commented Jul 12, 2022 at 9:29
  • With telnet (echo 'GET /'; echo ""; sleep 1; ) | telnet www.blockchain-office.com 80 | 503 Service Temporarily Unavailable | same problem no ssl/tls Commented Jul 12, 2022 at 9:35
  • If you use /dev/tcp for a http/https connection you have to manage the complete handshake including ssl/tls, http headers, chunks and more. Or you use curl/wget that manage it for you. Commented Jul 12, 2022 at 9:57

2 Answers 2

Reset to default
5

You can use openssl s_client to perform the equivalent operation but delegate the SSL part:

#!/bin/sh

host='blockchain-office.com'
port=443
path='/file.txt'

crlf="$(printf '\r\n_')"
crlf="${crlf%?}"

{
  printf '%s\r\n' \
    "GET ${path} HTTP/1.1" \
    "host: ${host}" \
    'Connection: close' \
    ''
} |
  openssl s_client -quiet -connect "${host}:${port}" 2 >/dev/null | {

  # Skip headers by reading up until encountering a blank line
  while IFS="${crlf}" read -r line && [ -n "$line" ]; do :; done

  # Output the raw body content
  cat
}

Instead of cat to output the raw body, you may want to check some headers like Content-Type, Content-Transfer-Encoding and even maybe navigate and handle recursive MIME chunks, then decode the raw content to something.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

thanks for the answer but as i wrote i was looking for a solution without installing tools like openssl, wget, curl, etc.. only with /dev/tcp or something from the standard terminal utilities.
Blockhain Office, then shell is the wrong tool because it is not capable of performing any of the SSL handshake without using external resources/commands. Now relieve and use what you want and can from what I show you here as the cleanest and most portable POSIX-shell grammar implementation of a minimal HTTP session through SSL. And then maybe it is time to consider alternative options (not using HTTPS, using languages with built-in or standard library SSL support).
We receive lists/txt from different domains with url's for a ssh network. Some of the domains we can handle the config , like deactivate the strict https but that is not the solution. On other domains we can handle/config nothing. There are a lot of microservices in containers with a minimal bash only with the standard tools that will receive this files and handle the content. After all comments and check the internet, yes we have to use tools like openssl, wget, curl, etc.. Because we can handle only http/80 with the standard shell tools.
1

After all the comments and research, the answer is no, we can't get/fetch files using only the standard tools with the shell like /dev/tcp because we can't handle ssl/tls without handle the complete handshake.

It is only possbile with the http/80.

i dont think bash's /dev/tcp supports ssl/tls

If you use /dev/tcp for a http/https connection you have to manage the complete handshake including ssl/tls, http headers, chunks and more. Or you use curl/wget that manage it for you.

then shell is the wrong tool because it is not capable of performing any of the SSL handshake without using external resources/commands. Now relieve and use what you want and can from what I show you here as the cleanest and most portable POSIX-shell grammar implementation of a minimal HTTP session through SSL. And then maybe it is time to consider alternative options (not using HTTPS, using languages with built-in or standard library SSL support).

We will use curl, wget and openssl on seperate docker containers now.

I think there are still some requirements in the future to see if we keep only one of them or all of them.

We will use the script from @Léa Gris in a docker container too.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.