0

I'm getting to make a web-based real-time game, which requires using websockets (please do not point me at Flash here), and so I need a backend server that also supports websockets. I decided to use Python for number of reasons, however I have trouble finding the right library. Specifically, it has to implement websockets on top of regular sockets, not as opaque black box. Because really, websockets are nothing more than conventional sockets, except with very specific handshake.

I've plowed through a number of Python websocket-supporting libraries, but none of them seem to even expose actual socket functionality, maybe I'm missing something but I just can't find anything. They all seem to implement the pattern where you define behavior of socket-resolvers (that are cornerstone of the whole app) and then launch library's main loop. This isn't suitable because 1) network loop isn't going to be main loop here and 2) sockets are only used to transport data back and forth and nothing more.

So is there such library that would allow me to use websockets like regular sockets and not abstract webserver data ports, or am I really better off just ripping the functionality in question out of existing library and carry on like usual?

Improve this question
7
  • But websockets aren't just sockets. Even after the handshake, websockets is a stream of framed messages, not a stream of bytes. Unless you're planning to do the framing (and TCP header manipulation) yourself, you can't just use the websocket socket as a TCP socket. Commented May 27, 2015 at 8:10
  • @abarnert From what I read, they are just sockets, with small extra header responsible specifically for websockets handling. Any game networking library implements similar thing. Nothing scary there. Commented May 27, 2015 at 8:25
  • Well, then you haven't read carefully enough. Websockets layers a framing mechanism on top of TCP. As the original documentation put it, it's "as close to just exposing raw TCP to script as possible, but no closer". If you just treat it as raw TCP, you will not be able to communicate with the browser. Commented May 27, 2015 at 8:29
  • @abarnet Yes I get the idea. It's just inherently there's nothing special about websockets that makes them anywhat different from regular sockets. If you put resolving handshake and the header under wrapper functions, regular sockets and websockets are indistinguishable usage-wise. Commented May 27, 2015 at 8:45
  • No, they really are distinguishable. You send and receive messages, not bytes. That's a fundamental difference. And that means just doing, e.g., an epoll or select to see if there's something to read isn't very helpful, because unless there's the final frame of a message, there's still nothing to process. Commented May 27, 2015 at 8:48

2 Answers 2

Reset to default
1

What you're asking for doesn't make sense.

You contend that, after the initial handshake, websockets are just TCP sockets.

But they're not. A websocket is a stream of messages, not a stream of bytes. As explained by Protocol Overview:

After a successful handshake, clients and servers transfer data back and forth in conceptual units referred to in this specification as "messages". On the wire, a message is composed of one or more frames. The WebSocket message does not necessarily correspond to a particular network layer framing, as a fragmented message may be coalesced or split by an intermediary.

A frame has an associated type. Each frame belonging to the same message contains the same type of data. Broadly speaking, there are types for textual data (which is interpreted as UTF-8 RFC3629 text), binary data (whose interpretation is left up to the application), and control frames (which are not intended to carry data for the application but instead for protocol-level signaling, such as to signal that the connection should be closed). This version of the protocol defines six frame types and leaves ten reserved for future use.

See Design Philosophy for why it works this way, and Data Framing for more complete details.

So, if you just take the TCP socket out from underneath a websocket and start sending unframed data, the other side will not understand you, and will just disconnect.

(You'll also break most proxies, since proxy support depends on frame masking if you're not using TLS, and clean-shutdown handling, but if you can't even do the most basic point-to-point communications, who cares?)

Of course you can build an API on top of websockets that exposes a connection-oriented and reliable (like TCP) but datagram-based (like UDP) socket interface and then pretend you're not using websockets… but if you're looking for real-time, low-latency networking, adding another layer that makes it hard to get as close to the metal as possible when you find things you need to optimize doesn't seem like a good idea. Or, if you do want that, I suspect you'd be better off building a tightly-optimized gateway that talks localhost UDP to your actual servers and websockets to your clients.

Meanwhile, you claim that something like asyncio or twisted

… isn't suitable because 1) network loop isn't going to be main loop here and 2) sockets are only used to transport data back and forth and nothing more.

  1. Both asyncio and twisted (although not most other networking libraries) allow you to drive their message loop from another loop, by just running one iteration of the loop manually instead of running forever.

  2. How else are sockets used, anywhere?

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

@RaidhoCoaxil: That's equivalent to saying that TCP is the same as bare IP except the RFC defines a set of basic headers and rules for you. There's a sense in which it's true, but not a useful one. And meanwhile: socket resolver objects are for DNS, which happens outside even the HTTP handshake level, so how is that even relevant here? Of course you can do your own DNS resolution is you want, but who cares? What you don't have are TCP-style recv and send, or selectors like select or epoll—in other words, they're not TCP sockets.
0

I think Twisted might be what you need :

http://twistedmatrix.com/trac/export/29073/branches/websocket-4173-2/doc/web/howto/websocket.xhtml

Here's an article about it in depth, anything you want to know about sockets and how to implement them :

http://www.fullstackpython.com/websockets.html

I'd suggest going over it and then deciding on how you want to go about it. I would say there's no need to reinvent the wheel unless you want to for academic purposes. Using existing stuff helps getting your work done faster. Also doing it manually is not taking advantage of the fact that Python has such powerful libraries.

Improve this answer

2 Comments

Twisted is among the libraries I've plowed through. Particularly, the example you given exhibits that exact pattern I was talking about - the socket resolver object is a cornerstone of the whole app. You obviously can't use this for a real-time video game server.
Maybe you could look into the asynchio library ?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.