The Evolution of Chess Engines
Ironically, the main task of chess software companies today is to find ways to make the program weaker, not stronger, and to provide options that any user can pick from different levels and the machine will try to make enough mistakes to give him a chance.
— Garry Kasparov
Thanks to @Matthew_Slater for helping me research because I had no internet while writing this! Don't ask me how! Just read the blog! Please!
Table of Contents
- Overview
- Pseudo-Automatons
- Kasparov vs. Deep Blue
- The Trolling of Crafty and Rybka
- The Big Boys
- Variants
- Conclusion
Ever since chess started becoming more popular in the early 18th and 19th centuries, players have always wondered how they can improve their skills and learn new techniques that they could employ against opponents in this game of masterminds. You had talents like Tal, Morphy, Fischer, etc., many of whom were strong players that reigned at the elite level for a very long time, some even storming their way up to the very top, becoming the World Chess Champion. But put these players against a computer from today and they would stand no chance. Not even Magnus could beat Stockfish. What are these machines that only started popping up recently, and where did these beasts emerge from?
Going back a couple centuries and you have the innovations of the first ever chess-playing "machines". You've got:
The Turk (1770):
The first example of an "automated" chess player that was actually
controlled secretly by a human player sitting inside the mechanism.
Ajeeb (1868):
Very similarly to The Turk, a hidden human operator would
relay moves to be played on the board above.
and Mephisto (1886, NOT the Marvel character):
Unlike The Turk and Ajeeb, Mephisto was not controlled by a hidden human player,
instead being remotely activated via electromagnetic signals.
In reality, these were just the "engines" of the 18th and 19th centuries. They didn't actually calculate moves or play them themselves: someone was controlling them from another place.
What is quite fascinating is that these automated players were once so novel that players around the world came to try and fight them in a match, but once people started realizing that these structures were hoaxes, things went downhill from there. These ancient marvels are now displayed in various museums across the globe, and are the roots of the true engines that formed in the next few centuries. In fact, Mephisto itself was reworked into a actual chess engine that battled its way up to the top spot on the so-called "leaderboard of chess engines", staying at the number one spot up until the late 1990s.
The Real Beginning:
Kasparov vs. Deep Blue
One of the most renowned and well-decorated chess players of all time, Garry Kasparov took his seat at the playing hall in the Philadelphia Convention Center.. Quite possibly the GOAT of chess—with the exception of Magnus, of course—Kasparov was a monster, defeating almost everyone he played against with such precision that at the time was nearly unfathomable. He had many strong opponents to face, such as Anatoly Karpov, Nigel Short, and Veselin Topalov, but nevertheless used his incredible skills and techniques to stay at the top of elite-level chess, defending his undisputed World Champion title for 8 whole years and then another 7 in classical chess before finally losing to Viswanathan Anand. However, as strong he may have been, there was one opponent that managed to outplay him in full-fledged match. The interesting part is, however, that his opponent...wasn't human.
From the depths of the ocean...enter Deep Blue, the machine that caused a revolution in the chess world. Developed by IBM's Murray Campbell and Feng-hsiung Hsu, it was the first computer program to defeat a world champion in a match under tournament regulations. In a broader spectrum, it was the first significant* chess engine to defeat a human player in an actual, full-blown chess match. It was the engine that started it all. No matter how strong its successors were, it will always be known to be the...MOAE? Mother of All Engines. I guess.
*Technically speaking, the true first engine to defeat a human player was MANIAC, developed by scientists at the Los Alamos Laboratory. It wasn't even a real chess game, however, since they played with simplified rules that mirror chess but don't exactly contract all the aspects of the game correctly.
While Kasparov won the first match—4-2, held in ol' Philly in 1996—the literally just a Stockfish ancestor Loch Ness of chess engines, Deep Blue—new and improved—struck back in a sleek rematch, played in downtown NYC the following year, with an overall match score of 3.5-1.5.
This result was monumental, as it marked the beginning of a new era and introduced the idea that artificial intelligence could one day outthink even the brightest humans that built them.
Man vs. Machine games are always interesting to watch, and these matches were no exception, garnering millions of spectators all over the world with live move-by-move coverage of the games that were being played, with the deadly super computer on one side and the man who was holding up humanity on the other.
- The original match was held in the Philadelphia Convention Center in Philadelphia, Pennsylvania, USA. The rematch took place at the Equitable Center in New York City, New York, USA.
- The matches were played under standard tournament conditions.
- The games were played with a classical time control. Each game lasted at least an hour, with games on average taking between one and a half to two hours.
Match 1, Game 1 | Philadelphia, PA, USA | 02/16/1996
This is the only game that Deep Blue won in the original match; the rest were draws or losses. However, I'm showcasing this game just to portray the incredible amount of power possessed by these computers, even at such an early age—enough to defeat a reigning world champion!
Kasparov completely demolished Deep Blue in the rest of the match, but still, what a game that was!
Match 2, Game 6 | New York City, NY, USA | 05/11/1997
That wasn't the end of Deep Blue, however. The team at IBM later offered a rematch, and Kasparov accepted. This time, however, the engine was much, much stronger. Garry actually struggled to defeat it in some games and ended up losing the match overall.
Although he won the first game, he lost the next one, proceeded to draw the next three, and lost the final game—quite potentially the most stunning game of all, with a flashy knight sacrifice on move 8. It was simply too much for Kasparov to handle. Had mankind finally lost its grasp over artificial intelligence?
Surprisingly, Kasparov actually thought that the team at IBM was...cheating? He alleged that they contracted assistance from a grandmaster. However, after further research and in-depth analysis of the games, he came to the conclusion that they weren't, in fact, cheating. Nevertheless, he seemed stubborn on the claim that neither his nor Deep Blue's play were of the highest quality.
I am not writing any love letters to IBM, but my respect for the Deep Blue team went up, and my opinion of my own play, and Deep Blue's play, went down.
— Garry Kasparov
The match itself inspired the filming of a documentary:
The world was shocked when the reigning champion went down to an engine! What could be more...depressing surprising and interesting...yet somehow saddening, than such an ending to a lively story?
But there was still a glimmer of hope. And what better way to assert dominance on these pathetic brute-force calculation mechanisms than by having the Bongcloud master himself, Bullet Chess Champion Hikaru Nakamura, troll them to shreds?
The Trolling of Crafty and Rybka
As we progress further along the timeline, the engines are getting stronger, but so are the humans.
In the later part of the decade between the 2000s and 2010s, during the era of the Internet Chess Club, Hikaru Nakamura decided to hop on the servers and start a new battle.
Hikaru vs Crafty | ICC Blitz, 2007
A relatively old chess engine, Crafty, was programmed and released to the general public in the 1990s. It steamrolled through players and ran over other engines of the time like a freight train (no pun intended?), though it wasn't necessarily stronger than Deep Blue; Crafty was estimated to have a rating of ~2600, while Deep Blue peaked at 2853. Nevertheless, as is during a new era of revolution, novelties and spectacles continue to appear, slowly evolving to become stronger as time goes on. For that reason, Crafty, while being one of the first, was not the most powerful.
Unfortunately for Crafty, that also meant that humans would have more strength relative to these engines. For Hikaru, however, that was a chance to show that humans are still capable of winning games over even some decently powerful engines.
His secret weapon was the fact that certain anti-computer moves and positions could be played, and the computer wouldn't be able to understand the dynamics of the given situations, moving into a "dead" state and shuffling endlessly while allowing him to advance the game into a smooth win. But Hikaru takes it a step further and trolls the CPU out of the engine. Instant classic right there.
The game itself wasn't all that interesting, but the 6 knight checkmate at the end was what turned this game into a meme of itself. As @SamCopeland puts it,
Watching this live, one was excited to see Nakamura securing a comfortable draw, but as the game progressed, it suddenly became clear he was after more as he revealed the engine's hubris and fallibly greedy programming.
—
Sam Copeland
Poor Crafty. If only it had known beforehand what drudgery it was getting into, only for it to be crushed into a fine powder by DEEZ KNIGHTS
Have a look yourself:
With a slow, positional squeeze that led to the inevitable asphyxiation of the White position, Hikaru once again shows his incredible prowess with his KID-like setup. Oh, the humility, the embarrassment the Crafty team must've faced. But oh no, Hikaru wasn't done yet...not at all!
Now, it's Rybka's turn.
Hikaru vs. Rybka | ICC Blitz, 2008
Only a year later, Hikaru steps up to the board once more, this time challenging Rybka to a game. Rybka was a much more recent release than was Crafty, first seen in its beta version on December 2, 2005, with the commercial version being released only a couple weeks later. Within a few years, it soon became the strongest chess engine to exist, before it was dethroned by Houdini in late 2011, as illustrated by Chess.com in this video.
Rybka's exact version number and rating at the time this game was played are unknown. However, it can be deduced that Hikaru either played Rybka 2 or 3 (the latter being much stronger, making this feat even more impressive), and its rating can be estimated to be somewhere between 3200 and 3250, which is much stronger than Hikaru nonetheless. Let's just say 3225 for simplicity's sake. Obviously, he was given "odds", with Rybka being forced to play the Grob Opening, but regardless it was a spectacle of a game:
Hikaru proved in this 270-move game just how weak an engine can be if placed in uncomfortable circumstances. After all, an engine is just a model of how chess should be played: a simulation at its core, built upon with basic hash functions, pruning methods, and calculatory techniques that make it stand out as a stronger computer, but not a human.
The year is 2008. Hikaru just demolished Rybka. Fritz, Shredder, Deep Blue, and many other chess engines are slowly falling off the list. But little did they know that they would go down much quicker than they expected, soon being lapped around the track, by none other than Stockfish, the current Computer Chess Champion and the highest rated engine in the world. In 2009, just a year after its birth, Stockfish suddenly shows up on the leaderboard, casually boasting a rating of 3000+ and shuffling between the third, fourth, and fifth highest rated spots, with Rybka still in first place.
Yet, it wouldn't be any fun if Stockfish just clapped all the other bots every single time, right?
We need some competition. After all, becoming the CCC (NOT the Civilian Conservation Corps passed into law by President Franklin Delano Roosevelt during the Great Depression) and TCEC takes more than just clean sweeping some boring machines no one's ever heard of!
So fast forward a couple years and we see the birth of even more 3000+ engines such as Komodo and Houdini, constantly fighting for the title of highest rated chess engine. Keep in mind that Stockfish is still a baby at this time, only ever reaching #1 for a brief period before being overtaken once more.
These engines are so unbelievably strong, however, that my analysis of any of their games would do injustice to the sheer performing capability they possess. Plus, I have to get this done in 10 days...well, 8 days now...with no access to internet and therefore no research except for basic asks from some friends of mine who were kind enough to provide me with the information I covered in this blog. So sue me. Also they're boring as FU So unfortunately, I won't be covering any games these engines played.
Okay, fine...here's ONE game.
Komodo vs. Stockfish TCEC
Season 2, Stage 2, Round 13 | 10/09/2023
As is with most engine games, this game features a standard opening, a boring-ish middlegame, and an even more boring endgame with lots of shuffling before one engine finally makes 5 inaccuracies in a row and loses. Tough.
I'm going to be honest with you...that was not an interesting game. I'm not the kind of guy to watch engines playing each other unless there are some brutal tactics involved. Because otherwise, what's the point? We know they play almost perfectly each time, so the game is obviously going to be very close to equal the whole way through. That's just me, though...let me know what you think in the comments!
There is an exception, however...
Ever heard of AlphaZero?
4 Hours, 1000 Games, 155 Wins | London, UK | 01/18/2018
The natural extension to this story of the gods of chess computers is the reign of AlphaZero. Of course it is!
Crafted by the DeepMind team at Google, this legendary monster was known for trapping its opponents pieces in their own positions, aggressive tactical attacks, and, most notably, its technical flank pawn pushes and structure-dissolving sacrifices.
What separates AlphaZero from traditional chess engines is that it is based on neural networks (NN), which is essentially how our own brains work; the engine learns through interconnected nodes, "neurons", in a process known as deep learning. While traditional engines learn through classical alpha-beta pruning and brute-force calculation, machines like AlphaZero learn through repeated (and I mean REPEATED) self-play, sometimes playing hundreds of millions of games against themselves.
Played in January of 2018, the revamped AlphaZero was originally given 9 hours to train itself from scratch. It prepared for and "studied" all the lore of chess, ranging from the wide variety of openings to the vast field of endgames. The results were tremendous. While it drew nearly 8 out of every 10 games it played against Stockfish, AlphaZero devastated it in 155 games, losing only six total. Imagine how powerful it could be today had DeepMind kept sharpening the blade. But enough talking...let's get to the game, baby!
A completely, brutally winning attack with multiple sacrifices ultimately led to an ill-timed draw by perpetual check. Such was the case in many of the 839 games that were drawn; AlphaZero ultimately proved to be better but was swindled out of a clean win by a cheeky perpetual check by Stockfish. Nonetheless, this was a great game, showing just how powerful AlphaZero was at the time of its prime playing ability, with the flank pawn pushing that we all know it for today, being one of its most prominent technical ideas.
The successor, almost the "heir" to AlphaZero, Leela Chess Zero is the second NN-based chess engine as of today. However, unlike its predecessor, Lc0 is open-source. Many players and analysists were unhappy that AlphaZero, being closed-source, was unable to be licensed or purchased, so they were in for a pleasant surprise when the project was announced in early 2018 as being the first and strongest NN-based chess engine available to the general public.
To be frank, Lc0 (to me at least) never had the same spark, the same flair that AlphaZero once possessed. While it also went for stunning sacrifices and had a prowess for positional play, it also tended to struggle a lot more against Stockfish. When compared to AlphaZero, which mauled SF in a lot of the games they played, Leela also defeated Stockfish but only by barely edging it out in a match, such as when it won the 13th CCCC, in a match of 200 games, with a final score of 106-94.
This enormous difference in strength can be attributed to the fact that Stockfish has become significantly stronger as well, gaining over 100 rating points since its 155 losses against AlphaZero in January 2018, but for an engine that is claimed to be the "strongest NN-based engine available to the public", I expected more from Leela. It's definitely tough to say, but I think it's safe to say Stockfish wins here.
13th Chess.com Computer Chess Championship, Finals Game 95 | 04/11/2020
Regardless, Lc0 is still a very powerful engine. While it may lose to Stockfish every now and then, it also is capable of crushing it.
Leela completely destroyed Stockfish in that game with a slow yet unstoppable pawn march and a nice rook sacrifice to enter the Black position.
But that was 2020. It's 2023 now, and Stockfish is back, better than ever before.
With the recent release of Stockfish 16, the SF team decided to put it up against Lc0, for testing purposes. But for us mere spectators, it's time for a proper rematch!
Stockfish the almighty GOAT reigns supreme once more. The true king has been crowned...at least, for the foreseeable future...
A Short Discussion:
Variants Engines
Engines that play regular chess aren't the only ones to exist. Multiple engines have been programmed to play chess variants such as crazyhouse and atomic, as well as unorthodox variants such as 4 player chess, and even completely different versions of chess like Shogi and Go.
These variants and their corresponding engines aren't as highly recognized as the many, many regular chess engines that we see today, nor are there that many variants engines in the first place, and hence there are no organizations that make them play each other like we see Chess.com do in their Computer Chess Championship or as we can observe in the TCEC. There are also so many variants that exist that it would be nearly impossible to compile a set of games played by said engines in said variants, and definitely impossible to do in such a short timespan. Therefore, I have dedicated this section to purely discussion.
The most notable variants chess engine is Fairy-Stockfish. Stockfish itself supports analysis and play of Chess 960 (Fischer Random), and has been forked by the community into variants-playing engines such as Fairy-Stockfish. This chess engine is able to analyze and play positions involving various board configurations and multiple "fairy" (unorthodox) chess pieces. While it doesn't support 4 player chess, it does support many different variants with some pretty crazy game rules, and it is able to understand positions with pieces such as the Amazon (Queen + Knight), Chancellor (Rook + Knight), Archbishop (Bishop + Knight), and more. Fairy-Stockfish is also able to play various historical chess variants such as Shogi, Makruk, Xiangqi, and more.
Other engines such as ChessV exist, but these are lesser known and less powerful than Fairy-Stockfish. Not to say that these computers aren't still perfectly viable for analysis and play, rather they are just not as optimal.
Lichess has its own version of Stockfish that it integrates within its user interface, allowing players to analyze and play against the engine in "simple", "basic", or "default" variants such as king of the hill and racing kings, as well as atomic and crazyhouse, and more. This version of Stockfish, however, does not support fairy pieces nor does it support any board configurations other than the standard 8x8 chess board.
There are also engines for 4 player chess, however these are still being developed and are not as optimally constructed as the strongest engines of regular chess. Another important fact is that engines so far have only been developed for teams, not free-for-all. While teams can be simplified into a 1v1 game, the more complicated FFA cannot be, along with the many luck-based and skill-based factors that determine the overall strategic preface of the game, and hence an engine hasn't been programmed yet due to all of these extra complications.
I have previously made a post on engines specifically built to play Teams 4 Player Chess. Click the button below to read it.
The evolution of chess engines over time has proved to be beneficial to humans in both an analytic and enterprising standpoint. Watching engine games not only provides instruction but also entertainment.
Yet, there is still much work to be done. Chess is a very complex game and, no matter how deep the analysis goes...
Chess is mathematically unsolvable. But at the end of the day, the machine doesn't have to solve the game. The machine has to win the game. And to win the game, it just has to make fewer mistakes than humans...
— Garry Kasparov
