3

I try to make sockets to communicate with my clients.

A socket would be created after some requests to my API. It means, a client connects itself (only by request), but then, he joins a chat, so a socket is created and linked to the good channel.

I already used sockets so I understand how it works (C, C++, C#, Java), but what I want to make, with what I saw on the web, I think it's possible, but I don't understand how to handle it with golang.

I create a first server:

func main() {
    r := mux.NewRouter()
    r.HandleFunc("/", HomeHandler)
    r.HandleFunc("/products", ProductsHandler)
    r.HandleFunc("/articles", ArticlesHandler)
    http.Handle("/", r)
}

But for socket, I need another one?

package main

import "net"
import "fmt"
import "bufio"
import "strings" // only needed below for sample processing

func main() {

    fmt.Println("Launching server...")

    // listen on all interfaces
    ln, _ := net.Listen("tcp", ":8081")

    // accept connection on port
    conn, _ := ln.Accept()   

    // run loop forever (or until ctrl-c)
    for {     
        // will listen for message to process ending in newline (\n)
        message, _ := bufio.NewReader(conn).ReadString('\n')
        // output message received     
        fmt.Print("Message Received:", string(message))
        // sample process for string received     
        newmessage := strings.ToUpper(message)
        // send new string back to client     
        conn.Write([]byte(newmessage + "\n"))   
    } 
}

Thank for help !

5
  • 1
    Here is a great resource about socket level programming in Go: jan.newmarch.name/go/socket/chapter-socket.html Commented May 24, 2016 at 9:58
  • 1
    Depends on your scale. If you are only expecting a few sockets, then use goroutines to spin up a new socket - be careful with this approach as each goroutine takes up about 4KB of ram, so if you are spinning up a million you could have memory issues. If you planning this at scale, then yes have a separate service listening on a different port. Kind of odd to start a socket after a request to an API - in that case, you'd have to issue a command to start a new service each time or some other way to pass around the context. Commented May 24, 2016 at 13:32
  • 1
    Thank you both for your help. Simo your link is really interesting ! @eduncan911 Yep I knew that goroutines would take a bit of memory, but I didn't think it was so much.. Your last sentence is what I want to do, I juste don't understand how I can create a socket chat or something like that.. Does it have to be on the same server or I have to make 2 servers which share the same database? Multi-threading also have to work with mutex :/ Commented May 24, 2016 at 14:51
  • 1
    @MaximeGuittet Why not have a generic HTTP mux (like you have) and a generic Socker listener (running in a go routine)? you don't have to listen on a 1-to-1 ratio but use pooled connections instead. you store the context on the backend, and key it with a hash/encoded ID or something alike. That way, when you instruct the client to connect, you tell it to use this SessionID when making the connection. then the socket listener would take that sessionid and do whatever u want to do. Commented May 24, 2016 at 18:09
  • That is what I want to do at the end, but I need to keep my API for things as authentication by example, but I need a socket system like Skype or Messenger chat by example, at the same time :) Commented May 24, 2016 at 18:18

2 Answers 2

2

Based on our chat discussion.

OVERsimplified example with lots of pseudocode

import (
    "net"
    "encoding/json"
    "errors"
)

type User struct {
    name string
}

type Message {
    Action string
    Params map[string]string
}

type Server struct {
    connected_users map[*User]net.Conn
    users_connected_with_each_other map[*User][]*User
    good_users map[string]*User
}

func (srv *Server) ListenAndServe(addr string) error {
    ln, err := net.Listen("tcp", addr)
    if err != nil {
        return err
    }
    return srv.Serve(tcpKeepAliveListener{ln.(*net.TCPListener)})
}

func (srv *Server) Serve(l net.Listener) error {
    defer l.Close()

    for {
        rw, e := l.Accept()
        if e != nil {
            return e
        }

        // you want to create server_conn here with buffers, channels and stuff
        // to use async thread safe read/write from it
        go srv.serve_conn(rw)
    }
}

func (srv *Server) serve_conn(rw net.Conn) error {  
    dec := json.NewDecoder(rw)

    var message Message

    //read 1st message he sent, should be token to connect
    dec.Decode(&message)

    token := get_token(Message)

    user, ok := srv.good_users[token]

    if !ok {
        return errors.New("BAD USER!")
    }
    // store connected user
    srv.connected_users[user] = rw

    for {
        // async reader will be nice
        dec.Decode(&message)

        switch message.Action {
            case "Message":
                // find users to send message to
                if chats_with, err := users_connected_with_each_other[user]; err == nil {                   
                    for user_to_send_message_to := range chats_with {
                        // find connections to send message to
                        if conn, err := srv.connected_users[user_to_send_message_to]; err == nil {
                            // send json encoded message
                            err := json.NewEncoder(conn).Encode(message)
                            //if write failed store message for later
                        }
                    }
                }

            //other cases

            default:
                // log?
        }       
    }
}

func main() {
    known_users_with_tokens := make(map[string]*User)


    srv := &Server{
        connected_users: make(map[*User]net.Conn),
        users_connected_with_each_other: make(map[*User][]*User),
        good_users: known_users_with_tokens, // map is reference type, so treat it like pointer
    }
    // start our server
    go srv.ListenAndServe(":54321")


    ConnRequestHandler := function(w http.ResponseWriter, r *http.Request) {
        user := create_user_based_on_request(r)
        token := create_token(user)

        // now user will be able to connect to server with token
        known_users_with_tokens[token] = user
    }

    ConnectUsersHandler := function(user1,user2) {
        // you should guard your srv.* members to avoid concurrent read/writes to map
        srv.users_connected_with_each_other[user1] = append(srv.users_connected_with_each_other[user1], user2)
        srv.users_connected_with_each_other[user2] = append(srv.users_connected_with_each_other[user2], user1)
    }

    //initialize your API http.Server
    r := mux.NewRouter()
    r.HandleFunc("/", HomeHandler)
    r.HandleFunc("/products", ProductsHandler)
    r.HandleFunc("/articles", ArticlesHandler)
    r.HandleFunc("/connection_request", ConnRequestHandler) // added
    http.Handle("/", r)
}

Call ConnectUsersHandler(user1, user2) to allow them communicate with each other.

known_users_with_tokens[token] = user to allow user to connect to server

You need to implement async reader/writer for connections to your server. Usefull structs to keep good Users. Guard Server struct members and provide thread safe access to update it.

UDP

Looks like json.NewEncoder(connection).Encode(&message) and json.NewDecoder(connection).Decode(&message) is async and thread safe. So you can write simultaneously from different goroutines. No need to manual sync, YAY!

Sign up to request clarification or add additional context in comments.

Comments

1

default http server accepts connection on one "host:port" only

Answer depends on what protocol you are going to use to communicate via your sockets.

I suggest to do it this way: (much simplified)

  1. Leave http.Server alone to serve your API (it implements protocols HTTP 1.*/2 so you dont need to worry about it)

  2. Implement your own "MultiSocketServer", do to so:

    2.1 Implement GracefulListener (must implement net.Listener) (you need to shutdown your sockets when you dont need them anymore, right?)

    2.2 Implement MultiSocketServer.Serve(l GracefulListener) (hello http.Server.Serve() ) to serve individual connection (your protocol to communicate with client via sockets goes here. something like net.textproto will be easy to implement since you GracefulListener.Accept() will return net.Conn)

    2.3 Add methods MultiSocketServer.ListenAndServe(addr), MultiSocketServer.StopServe(l GracefulListener) to your MultiSocketServer

    type MultiSocketServer struct {
        listeners GracefulListener[] //or map?
    
        // lots of other stuff
    }
    
    
    // looks familiar? (http.Server.ListenAndServe) 
    func (s *MultiSocketServer) ListenAndServe(addr string) {
        ln, err := net.Listen("tcp", addr)
        graceful_listner = &GracefulListener(ln)
    
        s.listeners = append(s.listeners, graceful_listner)
    
        go s.Serve(graceful_listner)
        return graceful_listner
    }
    
    func (s *MultiSocketServer) StopServe(graceful_listner  GracefulListener) {
        graceful_listner.Stop()
        //pseudocode
        remove_listener_from_slice(s.listeners, graceful_listner)
    }
    

Ofcourse, you need to add error checking and mutex (propably) to guard MultiSocketServer.listeners to make it thread safe.

In your main() start your API http.Server, and initialize your MultiSocketServer. Now from your http.Handler/http.HandlerFunc of http.Server you should be able to call MultiSocketServer.ListenAndServe(addr) to listen and serve your sockets connections.

UPDATE based on question

however, I'm not sure to understand the part "In your main()". If I understand it good, you said I have my API, and after starting it, I initialize MultiSocketServer. But where? after the starting of my API? Or you mean it would be better that I use the logic of your code as an API? Every request trough a socket

BTW: updated MultiSocketServer.ListenAndServe to start Listen and return graceful_listner

func main() {
    //init MultiSocketServer
    multi_socket_server = &MultiSocketServer{} //nil for GracefulListener[] is fine for now, complex initialization will be added later
    // no listners yet, serves nothing

    // create new Handeler for your "socket requests"
    SocketRequestHandler := function(w http.ResponseWriter, r *http.Request) {
       // identify client, assign him an address to connect
       addr_to_listen := parse_request(r) //pseudocode

       listener := multi_socket_server.ListenAndServe(addr_to_listen)
       // TODO: handle errors
       // now your multi_socket_server listen to addr_to_listen and serves it with multi_socket_server.Serve method in its own goroutine
       // as i said MultiSocketServer.Serve method must implement your protocol (plaintext Reader/Writer on listener for now?)

       save_listener_in_context_or_whatever_you_like_to_track_it(listener) //pseudo
   }  

    SocketDisconnectHandler := function(w http.ResponseWriter, r *http.Request) {
      // identify client 
      some_client := parse_request(r) //pseudocode
      // get listener based on info
      listener := get_listener_from_context_or_whatever(some_client) //pseudo

      multi_socket_server.StopServe(listener)
      // TODO: handle errors
    }

    //initialize your API http.Server
    r := mux.NewRouter()
    r.HandleFunc("/", HomeHandler)
    r.HandleFunc("/products", ProductsHandler)
    r.HandleFunc("/articles", ArticlesHandler)
    r.HandleFunc("/socket_request", SocketRequestHandler) // added
    r.HandleFunc("/socket_disconnect", SocketDisconnectHandler) //added
    http.Handle("/", r)

    // it creates new http.Server with DefaultServeMux as Handler (which is configured with your http.Handle("/", r) call)
   http.ListenAndServe(":8080") // start serving API via HTTP proto
}

Actually, you may call multi_socket_server.ListenAndServe(addr_to_listen) and multi_socket_server.StopServe(listener) from any handler in your API server.

Every time you call multi_socket_server.ListenAndServe(addr_to_listen) it will create new listener and serve on it, you have to control it (dont listen on the same address more then once (i think it will error out anyway))

Your MultiSocketServer.Serve may looks like:

func (s *MultiSocketServer) Serve(l net.Listener) {
    defer l.Close()

    for {     
        // will listen for message to process ending in newline (\n)
        message, _ := bufio.NewReader(conn).ReadString('\n')
        // output message received     
        fmt.Print("Message Received:", string(message))
        // sample process for string received     
        newmessage := strings.ToUpper(message)
        // send new string back to client     
        conn.Write([]byte(newmessage + "\n"))   
    } 
}

Possible GracefulListener implementation github

Or are you trying to achieve something completely different? =)

8 Comments

Thank for your answer @Darigaaz :) however, I'm not sure to understand the part "In your main()". If I understand it good, you said I have my API, and after starting it, I initialize MultiSocketServer. But where? after the starting of my API? Or you mean it would be better that I use the logic of your code as an API? Every request trough a socket
Wow your update interesting me ! :) (I'm not at the work) If I understood what you said, it means that a client (C# currently) will make a request on /socket_request which will give me a listener (a socket). So if it's a request, I will get a response to my c# code, but I will not be connected as a TCP client, you know what I mean? In your answer above at my comment, you said about use go routine to handle the sockets, so it's not better to just have a double server running? I mean my handler, and in goroutine the TCP part? Sorry if I missunderstood something.. It's my first API !
client will make a request on /socket_request which will create a new socket and return an addr where its listening for your connections. after that you can connect to this addr with your tcp client. >it's not better to just have a double server running? what do you mean by "server"? http.Server or MultiSocketServer is just a multiplexor serving "connections", every connection runs in its own gouroutine eventually (check source code of http.Server.ListenAndServe). So yes, its like running two servers because http.Server is able to listen on one address only and use HTTP protocol to communicate
As you said "a client connects itself (only by request), but then, he joins a chat, so a socket is created and linked to the good channel".| Sorry for repeating. Send request to /socket_request and connect with your client to the returned addr. Exactly as you asked. Oh i think i got it now. Client joins API chat and asks server to create a socket to communicate? And server must reply with addr where it waits for his personal connections?
That's it ;) I run my handler which have routes, nothing complicated. It's kind of a service for customer ok? The seller connect itself as available to answer a customer about this service. So the seller, once he's connected, get a socket. (Seller == Client). when a customer (also a client) selects a service, then we give him a socket ;) After that, I think make something as TCP you know, with a sessionID like you also said :)
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.