Forums

Positions engines get wrong ( please contribute )

Sort:
watcha

I'm interested in positions the evaluation of which is clear to any human observer but one or more leading chess engines ( the likes of Houdini, Stockfish, Rybka ) completely get them wrong being blind to the obvious.

Have you ever encountered such a position? If you did please sumbit.

The position should be from a real game played between humans. Obscure puzzles and positions created with the sole purpose of fooling engines are not of interest here.

For a starter here is the final position of a correspondence chess game played at a facebook based correspondence chess site. White is a natural chess talent ( seems to have no fide title ), black is a very strong fide master ( rated nearly 2400 ). This position is valued +0.4 for white in single PV mode by Houdini 3. This can be attributed to the fact that black has doubles pawns and white king is better centralized in an endgame. But apart from that Houdini sees nothing decisive in the position. Since Houdini 3 is still the leading chess engine in the world it is surprising why would one resign such a position ( as black did ). In fact if you let Houdini play black and make moves for white which are obvious to a human Houdini quickly changes its evaluation and concludes that black is heavily losing:



LoekBergman

I am not sure if a decent chess engine will get it wrong, but the best chess engine I have, Sjeng, got it all wrong in this endgame.

The situation I tried to set the focus on is on move 59. That position is won for white. I can imagine that a decent chess engine will find out, but the one I have could not.

watcha
LoekBergman írta:

I am not sure if a decent chess engine will get it wrong, but the best chess engine I have, Sjeng, got it all wrong in this endgame.

The situation I tried to set the focus on is on move 59. That position is won for white. I can imagine that a decent chess engine will find out, but the one I have could not.

 

I have looked into it and both Houdini and Stockfish say without hesitation that the position is heavily winning for white. Black's uncoordinated minor pieces are just mere observers to how white's three passed pawns march toward promotion. In addition a rook theoretically is always better than some combination of minor pieces in an endgame. Especially in the hands of an engine a rook can win seamingly equal endgames against minor pieces with ease.

The position is very interesting though, thanks for posting.

watcha

The second position I would like to share is a real engine failure and not only by one engine. The game was played at a kentaur correspondence chess site. Black has just captured white's b2 pawn with his queen threatening to take the rook.

Not only Houdini thinks that black is definitely better but the position is included in the engine checked Rybka4 book. According to Houdini and the Rybka4 book white's best move is Qb3. Rybka4 book mentions the actual game continuation ( 8. e6 ) but with a valuation of -2.13 for white!

In fact 8. e6 is winning by force and the game did not last very long after this move was played.

Here is the full game:

So in this case two high quality engines and an extensive and good opening book collectively failed to correctly evaluate the position.

watcha
RogerOT wrote:

This is always a good test for engines and is from a real game.

White to move.

This looks to me as a well known puzzle. Could you please submit the game this position is from.

The_Cosmologist
The following position would be given as clear advantage for white by top engines.However it's a draw.
I don't know the source of this position but it's definitely not one of those bizarre positions especially meant to torment the engines(as you mentioned).I am pretty sure it is from a real game.
 
Well, let's see if you can do better than the computers.
 
Xilmi

Well, it also depends on how you read the engines output.
If the engine sees an advantage that never changes throughout it's analysis, in this case +3.08, then this means: "This is draw by 50 move rule."

Only if the advantage increases with longer calculation it means the engines "thinks" that someone's winning.

The_Cosmologist
Another one, and this satisfies your request.It's from a real game and that too from the elite.
Computers would never be able to win this position even though they would assess it as winning for black.
Try this, it's a brilliant game.
 
Hint:Even though black is two pawns up it's not easy to win(as is the case with opposite coloured bishops).If the white king manages to get to e3 it's a draw.
VLaurenT

Most brilliant move of all time !

MSC157

Ziryab

It's been awhile since I checked this position. Perhaps engines like Stockfish 4 do better than those I tested a few years ago.

Shirov Test

 
Alexei Shirov found the winning ideas in a position that often has stymied chess engines. The game was Topalov - Shirov, Linares 1998.

Black to move

Photobucket - Video and Image Hosting
Ziryab

Crap. I should have read the entries before posting mine. Sorry to duplicate. Here's one that did not occur in one of my games, but might have.

Here, I opted to keep my bishop on the board (42.Be8), thinking that it could restrain the advance of Black's pawns. Against Rybka this morning, I gave up the bishop and created a light-square fortress.

42.Bxc6 Kxc6 43.g4 Kd5 44.f3 Bc7 45.Nd3 g6 46.Nb2 f5 47.gxf5 gxf5 48.Nd3

Black to move


Black's extra pawn is of no value. Only the king can capture pawns, and there is no way to evict the knight from d3 when the bishop cannot cover all the dark squares to which the knight can hop. White's king finds safety on e2, where he is untouchable. Rybka played the position out to move 169, where the game was drawn by the Fifty Move Rule. Black's pawns advanced to f4 and h4 to avoid earlier draws by this rule. 

watcha

The Topalov - Shirov game is very interesting from an engine point of view.

Of course neither Houdini nor Stockfish finds the correct move in single PV mode. For 1. ... Bh3 to be included in the analysis you have to go up to multi PV 14 in the case of both engines. Initially 1. ... Bh3 will be the 14th best move for black with a valuation of cca. 0 for both engines. Both engines will realize that 1. ... Bh3 is a good move at depth 24.

When Houdini finds 1. ... Bh3 it immediately jumps to the top with a very clear margin:

Stockfish however when finding out the merits of 1. ... Bh3 does not rank it first. It is still in the middle of candidate moves at depth 24. Even at depth 35 it is only the second best move and there is no clear difference between the first four candidate moves:

So in this particular case Houdini has the better chance of playing the correct move but this chance is still not realistic since it is very untipical to make an engine play a competitive game running in multi PV 14 and thinking for 6 minutes on a single move.

watcha
Ziryab wrote:

Here, I opted to keep my bishop on the board (42.Be8), thinking that it could restrain the advance of Black's pawns. Against Rybka this morning, I gave up the bishop and created a light-square fortress.

Black's extra pawn is of no value. Only the king can capture pawns, and there is no way to evict the knight from d3 when the bishop cannot cover all the dark squares to which the knight can hop. White's king finds safety on e2, where he is untouchable. Rybka played the position out to move 169, where the game was drawn by the Fifty Move Rule. Black's pawns advanced to f4 and h4 to avoid earlier draws by this rule. 

I have put your position to simultaneous analysis by Houdini and Stockfish. Within the framework of the Arena GUI Stockfish believes black is definitely better with winning chances ( indicated by upward green arrow by Arena ) while Houdini thinks black is only slightly better but the position is basically drawish ( this is indicated by the yellow dot by Arena ). So in this case Houdini is closer to the truth, almost getting it.

watcha
The_Cosmologist wrote:
The following position would be given as clear advantage for white by top engines.However it's a draw.
I don't know the source of this position but it's definitely not one of those bizarre positions especially meant to torment the engines(as you mentioned).I am pretty sure it is from a real game.
Well, let's see if you can do better than the computers.

This is definitely an engine failure. In my two previous posts I could make at least one top engine to find out the truth ( or come very close to the truth ) of the position, but to find out that the position you posted is a draw ( or at least drawish ) seems hopeless for engines. Even Houdini thinks white has an advantage of more than +1.5 which can not be called a drawish evaluation.

watcha
MSC157 wrote (speaking of the starting position): Engine sees a small advantage for white, but in fact it's drawn. ;)
 

I don't agree. Top engines value the starting position cca. +0.2 which is an other way of saying that it is a draw:

It is a strong conviction of humans that chess is objectively a draw but as far as chess is not solved there is no absolute proof for this. If you would ask the question "Chess has been solved and it turned out to be a win for one side. Which side would you think it is?" vast majority of humans would answer "white". Also game databases show a slightly better performance by white which means that white has at least a very tiny practical advantage.

So the engine evaluation +0.2 for the starting position is not only absolutely correct but is the best evaluation possible since even humans can't come up with anything better.

The_Cosmologist

Here's another example of how useless computers are when the position demands clear strategic understanding(which usually occurs in the endgame).

The threat is Rh2, winning the f2 pawn with Whte's king cut off. Then Black will penetrate with his king via f5. Resistence seems hopeless.
It's a real test of your skills.Can you save white(White to move).The engines wouldn't for sure.
And make a note:It's from a real game.
The_Cosmologist

It seems that none of you could find the answer, don't run the computers, run your minds.

TheGreatOogieBoogie

I only got the Shirov move right because it was in one of my books and I remember the position.  Though yeah three passed pawns and more active king are typically decisive against a bishop since white is practically playing without pawns and white doesn't have the time to stop black's pawns.  The d and f pawns also promote on the bishop's opposite color, also critical. 

The_Cosmologist

Sorry, I didn't notice it first.It's not from a real game, it is a study by Chekhover.