502 ?? Server error. Our fault. We will look into it!

Dear chess community,

On December 31, we had seven million active members on Chess.com in a single day for the first time. On January 20, we had ten million active members. Traffic on Chess.com has nearly doubled since the beginning of December, and our servers are struggling, especially during peak hours, typically around noon to 4 p.m. ET. We are very sorry for the issues; we know it's super frustrating. We are all hands on deck to address these challenges, but sadly there isn't (yet) a simple button we can press to resolve these issues.

We want to address the two core questions we see from the community:

• What is going on with all of this chess growth?
• Why are the servers struggling?

Naturally, there are many reasons for both, but we'll try our best to summarize all that's going on.

What is going on with all this chess growth?

As we noted, traffic on Chess.com has nearly doubled since the beginning of December. Here are some statistics we can share showing just how remarkable the enthusiasm for chess has been.

• All but five days in January have set new site records for active members.
• The Chess.com app reached #2 in the Top Free Games section of the iOS app store in the US (#1 in some countries!) and #7 in the gaming section of the Play store in the US.
• Every day since December 5, one million people have solved the daily puzzle.
• On January 19, we had one million visits from Google for the first time.
• We had over 300,000 members join in a single day, over 100,000 more than at the peak of The Queen's Gambit.
• 31,700,000 games were played on January 20 alone, a site record, and we are now regularly seeing more than one million games an hour.

Why is chess growing so rapidly?

It's impossible to point to one reason why there is so much interest in chess so suddenly. During the chess boom in 2020 and 2021, the reasons were clear: lockdowns, Pogchamps, and The Queen's Gambit on Netflix. Right now, we believe that this rapid growth is a combination of many things driving interest and media attention for chess and combining into one big wave:

• The most popular social media post in 2022 featured Lionel Messi and Cristiano Ronaldo playing chess.
• Many celebrities are talking about their chess obsession.
• The cheating controversy and the resulting drama with lawsuits, ridiculous "device" allegations, and more.
• Amazing creators, streamers, coaches, and chess community members making awesome content
• Chessboxing.
• Chess.com's acquisition of the Play Magnus Group.
• Chessboard gifts and games over the holidays.
• Chess.com being featured in app stores and trending on top games lists.
• The incredible chess played by the best players in the world in so many amazing events.
• Mittens.

Why are Chess.com's servers struggling?

Back when chess first began booming during the COVID lockdown, our servers struggled to handle the traffic. We made a lot of investments in hardware and other improvements that allowed us to scale. When the Queen's Gambit boom happened, we experienced another massive increase in traffic, mostly without interruptions in service.

As discussed, today we are experiencing scaling issues on entirely new levels. When so many people use Chess.com's service, our database starts to spin out of control because it cannot "write" fast enough. What does that mean? 250,000+ new accounts are being created each day. People are playing games (16,000 chess moves per second on average). People are adding friends. They are commenting, chatting, and doing awesome chess things. All of this generates data that needs to be written to our databases. Sometimes our systems max out, and just as when someone exercises too hard and has to stop and catch their breath, our servers also become exhausted and need to recover. When that happens, they quit working, and our site and apps become unresponsive. It is a huge bummer, and unfortunately, there isn't anything "quick" we can do to resolve the issue.

Why can't Chess.com scale the systems?

We want to first assure you that every problem is being addressed as quickly as possible. They can be solved with more/better hardware both on-site and in the cloud (we have a hybrid infrastructure). We have shipments arriving with the most powerful possible live chess and database servers this week.

Unfortunately, it's not as easy as just adding more/bigger hardware. While that is part of the solution, there are bottlenecks, and once we solve one bottleneck, we grow until we find the next one. We are continuing to find the unscalable parts of Chess.com (both proactively and reactively) and work to make them scale more.

What is Chess.com's action plan?

To address the challenges our databases are experiencing, we are separating database tables, sharding databases, and putting services in memory. We are also working on cleaving off our most problematic database with users and gameplay. Each of these things takes time because there is SO MUCH DATA to move around.

We are also working on more "graceful" failures so that if things do go sideways and everything is exhausted, we can recover more quickly and with less interruption.

Everyone possible at Chess.com is focused on this, and we are also hiring as quickly as we can. Honestly, this sucks. We know you are here to play and enjoy chess, and it's very frustrating to be looking for a game and instead get a 502 (database connection) error, or have your game time out. We are not taking this lightly. We are implementing more short-term fixes today and, more broadly, expect to have a much better and more stable experience by later this week. Some major changes will be in place in 2-3 weeks that we hope will allow us to properly handle the next wave of chess enthusiasm.

It's never been a more exciting time to be a chess fan, but that's also why it's such a frustrating time to have service outages.

We love you, we feel you, we are sorry, and we are working as hard as we can to return to stability and provide the best possible experience—today and in the future, when we reach 15 million or even 20 million people playing chess in a day. Chess is incredible, and it's a joy to share this game with all of you.