1

I have a file hosted on an https server and I'd like to be able to transfer it to my client using openssl s_client as follows: openssl s_client -connect <my_ip:my_port>/my_file.

I'm able to currently get the contents of the file by running that command and then typing GET my_file, but I'd like to automate this so that it's not interactive. Using the -quiet switch doesn't help either.

Note: the device I'm trying to use this on doesn't have any other file transfer programs, so I'm stuck with openssl

1
  • Seriously, no wget or curl? That's a shame Commented Oct 26, 2020 at 21:57

2 Answers 2

3

If you really can use HTTP/0.9 then this will work for you

printf '%s\r\n\r\n' 'GET my_file' | openssl s_client -connect <my_ip:my_port>
4
  • Why should they use HTTP/0.9? Is it really that hard to add a host header? printf 'GET %s HTTP/1.1\r\nHost: %s\r\nConnection: close\r\n\r\n' / www.google.com | openssl s_client -quiet -connect www.google.com:443 Commented Oct 26, 2020 at 23:29
  • 1
    @User414777 of course it's not hard to add a host header. Or a protocol indicator. The OP said that a plain unadorned GET was sufficient, which in these days surprised me. Please re-read their question Commented Oct 26, 2020 at 23:35
  • 1
    It is sufficient with some servers, like google's. But to get the data, you should use at least -ign_eof (or -quiet). And probably -verify_return_error and -servername, if you're really about using TLS ;-) Commented Oct 26, 2020 at 23:43
  • With newer version of OpenSSL, -servername takes its value from -connect by default. So -servername may not be necessary. Commented Oct 27, 2020 at 0:25
1

openssl s_client is not a particularly great tool for this, but it can be done. Let's break this down into two parts. First, making the HTTP request, and second, extracting your content from the response.

Making the HTTP request

The hardest part here is that s_client closes the connection when its stdin gets closed. So you need to keep stdin open until the connection is closed. Based on this ServerFault answer, you can do something like:

(printf "GET $PATH HTTP/1.0\r\nHost: $HOST\r\n\r\n"; sleep 10) | \
    openssl s_client -connect $HOST:443 -quiet) > /tmp/output.tmp

There are some big limitations here. First is that you need to set the time for sleep long enough that the transfer can complete. But not so long that you wait forever. Second is that there's no redirect handling, so you need a URL that will serve the data directly on the URL you request rather than 302ing you off somewhere else.

Extracting the data

Now you have a /tmp/output.tmp that has the HTTP response in it, which includes your data and some headers. Stripping the headers is harder than it sounds, because you may have a mixture of \r\n and \n newlines, and you may be transferring binary data.

If your file is Unix style text (e.g., a shell script), then you can get rid of all the \r characters and remove everything up to and including the first blank line:

tr -d '\r' < /tmp/output.tmp | sed '1,/^$/d' > /tmp/output

If your file is binary (e.g., an executable), then you can't just delete the \r characters. Assuming the first few bytes of your file are fairly unique (let's say \xF7ELF in this example), and assuming you have GNU grep available, you can do something like this:

SKIP=$(grep -aboF "$(printf "\xf7ELF")" output | head -1 | awk -F: '{print $1}')
dd ibs=1 if=/tmp/output.tmp of=/tmp/output skip=$SKIP
2
  • I'm trying to use this for dl a file stored in a AWS S3 (because I'm on a docker image I can't modify which doesn't have curl installed). But I don't manage to dl the file Commented Sep 30, 2022 at 11:14
  • Thank you for the answer. Why finding the first CRLFCRLF would fail to separate the file content from the headers if the HTTP standard requires that the header block should be separated from the body by CRLF and even when there are no headers, the first line should also have CRLF at the end? In this case should this work for any file content: LANG=C grep -boazP -m 1 '\x0d\x0a\x0d\x0a' /tmp/my-file | head -1 | awk -F ':' '{print $1 + 4}'? Commented Oct 7, 2022 at 15:44

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.