PROOF That Chess.com's Engine is Weak!

So there have been arguments in the recent past on the forum about the strength of Chess.com's engine.  As an ICCF Correspondence player, which allows bots to be used, I can speak with first hand experience that Chess.com's engines are weak, and here is the proof!

I had a correspondence game that I won in 23 moves, but we are going to look at a position after White's 11th move, and compare two lines that Chess.com's engine assesses both as roughly -1.2, meaning almost winning for Black, but Viktor Moskalenko points out in his excellent book, "The Wonderful Winawer", that one is good and one is not, and so I had Chess.com's engine analyze and spit out Black's moves while using another engine for White.

Just look at how erratically the assessments change according to the Chess.com Engine after the moves made by White, and ultimately, White is winning.  The engine I used for White did not completely hold steady at one fixed number, and fluctuated within about a half-point range, but the glorious engine here that some seem to depend upon has erratically gone from -1.2 to almost +2 in a mere 8 moves.

It didn't take me long to realize that the engine here is garbage when trying to play Correspondence Chess on sites that allow artificial intelligence!  The engine here is so weak, Fritz 5 could probably beat it!

The human analysis of the actual game will be published later today in "The French Connection: Volume 37" on the blog site that I write articles on that many of you have read before from links I've posted in other threads.

Didn't know we required any proof that the engine here is weak. I thought that was axiomatic.

This line is also Bronstein's innovation (with Black) against Boleslavsky (White), from their Candidates play-off match.

Interesting - while familiar with the line, had no idea about the Bronstein - Boleslavsky match featuring this line.

In most cases, I would agree that "proof" is not needed, but since I have had myself attacked by a user that shall remain unnamed many times about how I play the game based strictly on the numbers of the engine here and not a single line of human analysis, once I found a game that I played that I knew the bots on here were trash at assessing, I ran with it and put this game on here.

By the way, in that match, did Black play 11...Rh4?  Or the inferior 11...Rg6? (though not inferior according to Chess.com's version of Stockfish 11).  Who won that game?  In the notes to Black's 11th move is the actual game that I played and won in 23 moves.

I automatically assumed it was strong.  Interesting.

https://old.chesstempo.com/gamedb/opening/712 

Well, chess.com's analysis engine is Stockfish 10, so I guess Stockfish 10 is weak, then?

The "human analysis" of that game is the first of the two games in the following article:

https://charlottechesscenter.blogspot.com/2020/05/the-french-connection-volume-37.html

Compared to you or me?  No!

Compared to other engines?  ABSO-FREAKING-LUTELY!

In the game (game 14 of the 1950 Candidates match in Moscow), White varied with 11. Nb5 and Black won, but Bronstein mentioned in his notes that on 11. Be2 he intended 11. ... Rh4 12. Qxh4 Qxh4 13. g3 Qh6! with advantage, and that this was the tactical point of Bf8. 

Humorously, Bronstein actually got a chance to play out the full line (out to 13. Qh6!) FORTY-THREE years later, in Escolana vs Bronstein, Oviedo 1993: https://www.chessgames.com/perl/chessgame?gid=1035079

It might have been in Bronstein's notes to that game that he mentioned his original home prep for the Boleslavsky game.

Wow!

Yeah, if White wants to make this line work, he has to play 8.Nge2.  8.O-O-O just doesn't work!

Yeah, it's really odd that he got a chance to "finish the game" forty-three years later.

It is proof that it is low depth.

Wow, and cheaters on Chess.com tried to use Stockfish just because they believe it's the best engine in the world, and it's always right.

Engine strength depends on hardware x time.

Their analysis is using approximately 0.1 seconds x1 cpu per position.

If you want serious analysis, download PGN, run into GUI, let your 4 core CPU run for 10 seconds per position, that will be x500  time stronger.

The one I started using for Correspondence is 20 core at 30 seconds, and Stockfish Dev Build 20200521-1211.  Blows 10 at 1 core CPU and 1 second out of the water!

But we'll let those that think the chess.com Stockfish build is so great continue to think that.  

Who do you think you are to call me weak compared to Stockfish 10, huh? 

That sounds like a fast machine you got there! What depth does Stockfish's analysis reach when you run an analysis of the position within your time constraint? When Chess.com's engine reached a depth somewhere in the mid to high thirties on my laptop, it agreed 11. ... Rh4 is significantly stronger than 11. ... Rg6. The evaluation fluctuated quite a bit as Stockfish gradually increased the depth of its analysis.

The screenshot below was taken while the engine was still running at a depth of 43. It took quite some time for my slower laptop to get that deep. I have no skin the game with respect to Stockfish, I am just amazed it took so long for the analysis on my laptop to settle on a somewhat consistent evaluation.

Set at Top 4 moves in 30 seconds:

11...Rh4, -1.11

11...Rg6, +0.00

11...Qxg2, +0.37

11...Rg7, +0.71

Depth - 33

Saying the engine is weak isn't really accurate.  It's Stockfish 10, which, according to the CCRL rating list, is currently the strongest engine in the world.

However - and this is what you're picking up on - the depth of analysis is important.  The depth of analysis is determined by the capabilities of the hardware and by the analysis time.

If you want deep analysis that would make you competitive in ICCF, using an online browser is simply not a good idea.  Download the engine to run it locally on your own hardware; you'll get much better results.

But if you just want a quick guide to where you blunders were and what tactics you missed, the online version is more than adequate.

The other point though is that what is often argued is what the bots here call inaccuracies are not inaccuracies.  It declares anything with a negative shift an inaccuracy.

It might claim 31.g3 is an inaccuracy because it thinks that 31.Rc5 is +4.13 and 31.g3 is +2.76.

A more tuned up engine will say 31.g3 is +3.56 and 31.Rc5 is +3.61.

Seriously, should he then be an inaccuracy?  If 31.a3 leads to +0.00, now you have an inaccuracy.

Who cares about +3 vs +5 vs +80?  Anything higher than something like +1.45 is considered "+-".  As long as it remains +- and not +/- or anything lower, how can you call it an inaccuracy?