Is there a status code for ssl is not supported?
I would like to send an error code if someone tries to call a redirected domain with https:// because right now nginx falls back to using the default_server which ends in an certificate error
You can pick what status you want form the list that you can find here.
In this page you can also find the list of custom statuses supported by ngix. I guess your case is falling under this:
495 SSL Certificate Error: An expansion of the 400 Bad Request response code, used when the client has provided an invalid client certificate.
The HTTP Error code response is somewhat moot if there is not a valid certificate as the client (browser) will first process the certificate and warn the user before the user see's any error codes.
I'm not sure what the best practice is for SEO but I would consider configuring the web server to not respond at all if you don't support SSL for the domain / port being requested. This way the user never see's the invalid certificate.