Foray into websockets - pt 2

architecture

So, part 1 actually went much better than I would've expected. I had no idea the raw web apis were so friendly at least to get something basic working. Essentially once you have a connection you're off to the races. Following that up with a pub/sub topic subscription and firing off new notifications whenever they came was equally straightforward. Now, after I had that success I came away with some questions (which is always the big win of hacking on something new).

  1. What is this upgrade thing some docs reference as the "Web" in websockets (can't seem to find direct reference), but the MDN docs do reference this upgrade path as the way a browser creates them.
  2. What does it look like to turn this single connection into production worthy? a. What happenes when the network drops?
  3. How do you manage lots of persistent connections? a. Any established best-practice or "websocket servers" that manage all this for you?
  4. Where in the flow do we establish and link who is who for a new connection attempt to the server?

What is this http "upgrade" thing?

Direct quote from the MDN docs:

Establishing a WebSocket relies on the HTTP Upgrade mechanism, so the request for the protocol upgrade is implicit when we address the web server as ws://www.example.com or wss://www.example.com.

I've seen elsewhere this is the "web" in websockets. Primarily the fact that the initial request is regular http but is sent with the upgrade protocol which indicates to the server it would like to upgrade to a websocket connection.

Ew, some interesting info from the mdn Protocol Upgrade Mechanism:

This mechanism is optional; it cannot be used to insist on a protocol change. Implementations can choose not to take advantage of an upgrade even if they support the new protocol, and in practice, this mechanism is used mostly to bootstrap a WebSockets connection.

Note also that HTTP/2 explicitly disallows the use of this mechanism; it is specific to HTTP/1.1.

So I have a little digging to do to verify what happens when the initial connection is made over http/2.

An example upgrade request from the same MDN upgrade docs:

GET /index.html HTTP/1.1
Host: www.example.com
Connection: upgrade
Upgrade: example/1, foo/2

and a specific section for the upgrade to websocket protocal. I will say, the 101 Switching Protocols http response status code is not one I was familiar with. Interview questions, I guess.

What does production look like for websockets?

  • heartbeat?
  • wss connection
  • token expiration if the socket is open for more than token life span
  • does logging / apm agent actually monitor websockets? appears some don't.

Recommendations

  • Base autoscaling on number of open connections because things like CPU will often be misleading as open connections will often remain idle.
  • Limitations to be aware of at the instance level that get to the core of linux itself. Things like allowed connections for a given IP and port.

  • Create ping-pong to make sure the connection is good from both ends. Example code from the article was 30s between heartbeat checks, but not sure if that was meant to be followed as best practice or not.

    • This allows the server side to close up the connections so they don't get saturated and waste potential connections for a given machine.
  • Make sure to add auto-reconnect whenever a client detects that the socket has been killed from the server side for whatever reason.
  • Interesting point brought up about manually refreshing the connection if it's been open for a long time. Assumes that at the beginning of a connection you'll often do some additional work that may be stale after the connection has been open for some time. I feel like I'd do this through some other means like an "initialize" upon first connection, then maybe synchronize later instead of having this be a side-effect of the re-connection. This could be useful for token refresh, though, if that's how the initial connection was authorized.

Other links I used in building this.. that probably mean I had things to learn or lock into memory and needed to come back to.

Getting it going...

Production

Trevor Richardson
richardson-trevor on Profile