Did the Chess.com Computer Blunder?

Here is the Chess.com computer analysis of a recent game of mine.  When the computer gave my 39th move (Kf4) a ?? was it correct?   (I don't think so).

I'm used to the Chess.com computer finding plenty of mistakes in my post mortems -- that's nothing new.    In this case however, I'm puzzled by what appears to be major error in the computer analysis.  The computer claims that 39. Kf4 was a blunder and gives Black's required response as 39... Rf2, with an evaluation of the position as drawn (see move list).  Significantly, the computer does NOT extend the variation beyond that single response by Black.

I had considered this reponse and concluded that if Black played it I could play either 40 b6 or 40 g5 with good winning chances.  So I put the game on Rybka 3 at home for a second opinion.

Here's the wierd thing.  Rybka ALSO thought 39  Kf4 was a blunder, even after having several hours to think about it, but as soon as I actually played Black's move (39 ...Rf2) Rybka immediately changed her mind and declared that White was just fine after all, winning easily, in fact (+5.00).  Sure enough, b6 or g5 were the top two suggested continuations.

What's going on?  (Computer geeks and Masters are especially invited to weigh in)  Is there some inexplicable Blind Spot in Rybka 3 that this game is exposing?  

And no, 39 Kf4 does NOT lead to a draw by 3-fold repitition.  Had Black played Rf2 it would have only been the second time that position would have been seen, not the third.  (See the Move List for all the variations)

Hehe. Crafty thinks that Kf4 is the BEST move. So there is definately something wrong with Rybka and the chess.com computer! Crafty calculates the position as 5.04 after a depth of 18 (about what Rybka evaluates after blacks move...) So there must be either a bug in both Rybka and Chess.com, OR crafty has some parameters in it that the position requires while the others do not! 

The chess.com computer does a very quick analysis, so the lines stop just a few moves in. If the line looks ok at that point, it's ok, as far as the computer is concerned. It's really pretty weak.

That can hardly be changed, since for chess.com it's a huge investment in computer power just to do that little bit of analysis, given the number of games people want to analyze.

 Maybe, but does that explain why the multi-processor version of Rybka 3 I have ran four hours and looked (supposedly) 29 moves deep and STILL claimed that White's move was a blunder?  I'm guessing that Chess.com is ALSO using Rybka and we have discovered a genuine bug in the software.

computer analysis here is not that amazing - it is mostly for blunder checking. it isn't perfect - it is only sd 14 with an engine that is not as powerful as rybka.

You have found a feature in these engines. Once they see a 2-fold repetition, they anticipate a 3-fold repetition automatically. (Of course, I can hardly say that's useful... but that's how it works.)

Sorry, I didn't read your actual post. Thought it was a random complaint about a blunder that wasn't one. Read it now.

I've seen many computers do this, decide the game is a draw if there's an opportunity for 2-fold repetition and the game is otherwise worse.

The reason, as far as I understand, is hash tables: once the computer has an evaluation for a certain position, it stores the evaluation for that position (using hash tables), and that evaluation can then be reused elsewhere in the move tree for that same position.

So first the computer generates possible moves to reach the second repetition; no immediate evaluation here and not at maximum depth yet, so it generates all possible moves after that; and then all possible moves after that. Aha! Now it reaches the position after the 3rd repetition, and it is an immediate draw (computers always claim, assuming they wouldn't have gone into the repetition if they were actually better). So now the draw evaluation is stored in the hash table for this position, and the computer backtracks. Reaching the position after the 2nd repetition, it re-uses this evaluation and decided that this position must also be a draw.

I hope that's clear. I really think it's the explanation.

So is that a bug? Probably not. To avoid it, the computer would have to compute and store with every position (of the millions it looks at while computing) how often it had already occurred in the game so far. That's slow! Adding that feature would mean the computer could look at fewer positions. Leaving it out is a trade-off that the programmer decided was worth it.

Especially since it probably is not a drawback at all in computer-computer play, which is where they get their benchmark ratings. Opponent computers wouldn't let them get a 2nd repetition of a position, because they use the same trick, so the situation wouldn't occur.

But when analyzing human games, it leads to this effect.

I really hope this is true because I made it up right now and it sounds logical, so if it's true I had an insightful moment today :-)

Edit: crap. it has to do all the work for deciding on _threefold_ repetition already. So it doesn't save any computation time. Ah well, the explanation of what happens still rings true, just not the reason why they do it :-)

Wow, that seems like a real problem.  Why would they do that?

rybka fritz and chess.com computers think kf4 is a draw simply because you allow black to repeat the position again. It thinks kf4 is a draw without even analysing in any depth, its just based on the possibility of black repeating it, even though it wouldnt be 3 fold repetition right away. kf4 is not a blunder (though it would be if you keep repeating till the 3-fold repetition, cause you have other moves that win also).

I'm utterly baffled by that. (Not doubting you, just baffled)  Why would these very sophisticated programs assume that a 2nd repitition would have to lead to a 3rd?  In this particuluar case my previous move WAS a mistake (I should have played b6 back on move 38) and allowing the possibility of a single repetition was necessary in order to get back on track -- but no way was I going to allow a 3rd repetition. 

Why on earth wouldn't the computer just take a few microseconds to actually look at the position after 39 ...Rf2 ?  Once the suggested move was actually played as a variation the computer changed its evaluation INSTANTLY.

You are on the track Scarblac, but you got out of your way in order to make this into more than a simple bug.

That an engine consider twofold repetition a draw is only logical, so do human chess players, if there is a better move than moving into the third repetition it is also there before the second repetition, and thus there is no need to consider the moves after the second repetition. That is however only true when the position hasn't been played yet, otherwise the repetiton is effectily a chance to undo the decision and choose a better move.

As a programmer I believe that fixing this flaw in Rybka is a minor task, and I don't think a correctly done fix would have any significant impact on performance.

Definitely a flaw in the analysis software. It automatically assumed that since that move was played the first time it will be played the second time. This is probably because computers never do this. Computers do not change their minds unless a random factor was deliberately induced or new input is provided.

Of course if those two moves were the best moves then the game is a draw. This is the wrong assumption took when checking for repitition. It was missed out that they are not the best moves.

 I'll have to disagree here. This depends on what Rybka would do if all other moves other than Kf4 would cause white to lose. Will it still be stubborn not going back to Kf4  and lose or will it decide to repeat for one more time and win. That's a whole game result we are talking about here, so its not that minor.

You have got no idea how a chess engine would be programmed, do you?

I just KNEW this thread would stir up some strong passions! 

 Well no need to get offensive, just answer my question. If all other moves were just other real blunders, will it still consider Kf4 a blunder recommending another move?

I evaluated the position with Rybka 3.0 at move 38, and it says Kf4 is the BEST move in this situation. But since it thought it was a blunder in your game, Scarblac must be right - it has something to do with the hash tables.

Does anybody know if this is a known problem with Rybka (and apparently Fritz as well) or should it be reported?

Do I have an excuse for emailing my hero, GM Larry Kaufman?  (Senior World Champion and member of the Rybka development team)

From what I can figure, Rybka as it is now will choose the repetition if all other moves are evaluated as less than equal, that is even if the repetition position is actually worse than some of the other options.

The most simple fix that comes to my mind is for Rybka simply to disregard any previous positions that have only appeared once.

when Rybka called Kf4 a blunder, it had already been analyzing your game until then, right? if you had Rybka analyze just the position at move 38, I think it would produce the correct analysis.

it seems to me that Rybka must have thought it had played 38. Ke4, and so assumed the best move after 38... Rb2 39. Kf4 Rf2 would still be 40. Ke4. Since Rybka thought it could win after 38... Rb2 and trusted its own "analysis" from 38. Ke4, it didn't want to play Kf4, otherwise the game would be a draw.

I'd say if I'm right it's a problem with Rybka if you're using it to analyze human games since we change our minds, make mistakes, etc. but not if it's actually playing the game.