6

I'm trying to get a simple websocket connection working on my server running in a Kubernetes cluster.

Websocket connections are able to establish on my local test machine but I can't connect my client side to the server after I deploy to GKE with nginx-ingress.

ingress yaml file:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.org/websocket-services : "socket-cluster-ip-service"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/websocket-services : "socket-cluster-ip-service"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
spec:
  rules:
    - host: my-socket.com
      http:
        paths:
          - path: /
            backend:
              serviceName: socket-cluster-ip-service
              servicePort: 3000
    - host: www.my-socket.com
      http:
        paths:
          - path: /
            backend:
              serviceName: socket-cluster-ip-service
              servicePort: 3000

service yaml file:

apiVersion: v1
kind: Service
metadata:
  name: socket-cluster-ip-service
spec:
  type: ClusterIP
  selector:
    component: socket
  ports:
  - port: 3000
    targetPort: 3000

deployment yaml file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: socket-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      component: socket
  template:
    metadata:
      labels:
        component: socket
    spec:
      containers:
      - name: socket
        image: gcr.io/my-socket/socket
        ports:
        - containerPort: 3000

express server code:

...

var server = http.createServer(app);
const WebSocket = require('ws');

const wss = new WebSocket.Server({ server: server });

wss.on('connection', function connection(ws) {
  console.log("new connection", ws)
  ws.on('message', function incoming(message) {
    console.log('received: %s', message);
  });

  ws.send('something');
});

...

client side code:

var ws = new WebSocket('ws://www.my-socket.com/');

ws.onopen = () => {
  // connection opened
  ws.send('something'); // send a message
};

ws.onmessage = (e) => {
  // a message was received
  console.log(e.data);
};

ws.onerror = (e) => {
  // an error occurred
  console.log(e.message);
};

ws.onclose = (e) => {
  // connection closed
  console.log(e.code, e.reason);
};

I keep getting the error message "Invalid Sec-WebSocket-Accept response" from the client side. But I think it's just the fact that the client could not establish websocket connection with the server.

The server side library I used is ws

I've added the following annotations to my ingress yaml file as suggested by other post and official ingress-nginx documentation but it's still not working:

nginx.org/websocket-services : "socket-cluster-ip-service"
nginx.ingress.kubernetes.io/websocket-services : "socket-cluster-ip-service"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"

2 Answers 2

1

For me it works like a charm!
I'm using kong-docker-kubernetes-ingress-controller.bintray.io/kong-ingress-controller:0.9.0 on my private kubernetes cluster.
Thus, your issue might be related to GCE.

Please also check your TLS configuration.
In your post you were mentioning ws://... rather than wss://, which indicates you don't even use TLS. This might be rejected by your browser (e.g. Chrome).

Sign up to request clarification or add additional context in comments.

1 Comment

@ Christian Arnheiter, i am facing same issue, in my local, able to establish web socket connection from client (angular ng2-stompjs) to server (spring boot). But when i deploy client and server applications on k8s cluster, i see "java.io.IOException: Unable to unwrap data, invalid status [CLOSED]" error after protocol switch is success (Completed 101 SWITCHING_PROTOCOLS). We are using Kong Ingress Controller v0.9.1, do we need to set any proxy configurations at ingress level?
0

im not on GKE however this might help the above issue: https://medium.com/k8scaleio/running-websocket-app-on-kubernetes-2e13eabb4c4f

it's to do with persistent state. I'm having the same issue works perfectly, when i run the docker image locally(docker run...) but the image deployed inside the cluster (k3d with loadbalacncer exposing its port )...web-socket will not connect. other services exposed (rest->mysql etc all work fine only the nodejs app with web-socket fails)

solution is to add annotations to your ingress controller rules:

annotations section in the yaml file:

certmanager.k8s.io/cluster-issuer: [your issuer]
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/websocket-services: ws-service
nginx.org/websocket-services: ws-service

1 Comment

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.