Tech Question: Curious about your low latency multi-region websocket architecture

Hi!

First of all: I know you code is closed-source, and I am not asking for specific details, but for a broad overview from one dev to another. Just an interesting dev conversation

My motivation is, that for a personal side-project I am working on a small card game. I've already set-up the core engine, that uses the websocket protocol during a game. What I am specifically interested is on how you are able to provide such low latency for players all over the world. Basically, how do you scale a game session running on websockets. And from what I can see, you are using web-sockets for the most part.

I don't think my project would need global matchmaking, and I am probably better of with a region-based matchmaking system. I am really just impressed on how fast your system is, and hope to be able to learn something from you!

For my project I am working with socket.io. They do show on how to scale a system running socket.io, using a load balancer to deploy multiple nodes with socket.io and an adapter to have them communicate between each other.

So for now I am imagining a architecture like so, for my project:

• Deployment on Region XY
  • Load balancer
    • Nodes (autoscaling)
  • Redis DB + Redis adapter

But as you can see, such a deployment would be limited Region XY. Of course I could replicate the whole setup for all kind of regions, but that would not allow for cross-region matchmaking. Players would need to select their mm region before searching for other players (as it is common for many online games).

So if there are some devs willing to share a broad overview on how you are able to match players against each other from around the world, I would absolutely love to hear it!

On that note, thank you for all the amazing you work you put into chess. I absolutely love chess.com from a player's point of view as well as a developers point of view!

I will try to pass this along to the right people.

/  I don't think your post belongs in "Off-Topic".

@justbefair Thanks!
I just saw there is a forum in the developer community. Should I have posted there? Feel free to move it if it doesn't fit here  

You don't need a fancy multi-region setup for worldwide play. The latency you get with WebSockets is perfectly acceptable, except maybe for first-person shooters and other reaction-based games.

For example, lichess has all their servers in France, and their lag is just fine. I would not be surprised if chess.com has a similar setup with one or two server locations, just shielded by a worldwide CDN.

What you may need, however, is lag compensation: Make the game feel fast, and minimize effects of lag. For chess, this means executing moves immediately locally, and accounting for the lag on the clock.

For your game, I would strongly recommend you go with a worldwide setup at first. For multiplayer games, a crucial problem – actually the crucial problem – is having enough players so people can find a game quickly.

Splitting it off into regions makes this problem much, much worse: The early bird in the NA region won't find anybody, then the early risers won't find opponents, and so on. Compare that to a worldwide lobby where somebody is awake at any time.

As you can see in your chess.com Insights, chess.com does this similarly: Your opponents' countries will be a rough reflection of chess players all over the globe, maybe weighted by the times you are playing.

That is only for the site's Public API.

@R_Doofus Well that’s just good news, that would make the whole setup much more simple and less expensive.

Now that I think about it, probably I am conditioned from my web developer as well as gaming years to think latency over a large distance would be an issue.

Because when playing FPS game, switching to Asia servers from Europe will make the game completely unplayable. And in the web dev world of course every millisecond counts too.

I think in my project even a lag compensation is probably not necessary, as it is not as tight on the clock as chess is. As long as the FE makes moves feel snappy.

I just subconciously accepted that connecting to a server on the otherside of the world is gonna be unplayable. But I am much less worried now.

I also did some load-testing memory wise and they looked quite promising too. Just a few nodes should be able to handle hundreds of players if not thousands.

This made me initially worry about WS scalabilty as it takes a stab at socket.io