Fixing New Analysis

Curiouser and Curioser...

Indeed it is. But the answers to this story lies in tbe coding, which we can't see. So only the dev guys responsibke for the engine functionality can fix it.

I've already posted quite a few examples here where alleged blunders and missed wins were nothing of that sort. Now, for a change, the opposite: I was already low on time (for my standards) and somewhat stressed out avoiding all the knight forks, so I simply missed a free knight, not to mention the strong continuation. Surprisingly though, the move I played instead was categorized as "good":

Ok, g4 was not losing, but to let slip a full piece in a game that (at amateur level) was not already a drop dead win should be at least a blunder, right?

https://www.chess.com/analysis/game/live/3995963956?tab=report

Hi all. Can you help me out here. I just finished analyzing a game I just finished. I gave it all away with a critical blunder - move 19... Nxe4?? The rather ironic fact was my move 18... Na3+!! was classed as "Brilliant". Not often I get one of those!! Boy Chess can be unforgiving! But I digress. I naturally wanted to understand what I did wrong at move 19... Nxe4?? I did an analysis using the Multi PV section (i.e. the" show lines function and section"). I had engine time limit at "unlimited". The suggested variation was to d = 31 which is a  is a substantial depth!! Below is a screenshot of the analysed variation- highlighted in yellow. As I reached the  end of the variation it reevaluated 4 of its own moves as blunders and 1 move as a mistake. Not the kind of result I was expecting! Any idea what is going on here? 

Here is a link to the game analysis (note there are many self analyses of mine still in the game list as well):

https://www.chess.com/analysis/game/daily/235816964

My opponent and friend jeff2E161 who is my chess mentor, naturally took quick advantage of my pathetic move 19... Nxe4?? and the game was his by the 29th move! 

@PawnstormPossie hi there

Not asking for game advice at all! I am pointing out that a variation of moves suggested by the analysis engine from line 1 of the MultiPV section - moves 19. to moves 43. which are highlighted in yellow in the game move list - when clicked on and re evaluated using the engine moves evaluations function (arrows, colours, icons etc) re evaluated 4 of its own engine moves as blunders and 1 as a mistake!

The moves I highlighted in yellow are engine suggestions and I have put a red line under the moves that the engine moves evaluation system re evaluated as blunders.

So I ain't asking anyone to comment on my strikingly brilliant play. I am pointing out the very bizarre situation of the engine suggesting blunders. And what makes it even more bizarre is that the depth indicator said the varation had been analysed to d= 31!   

Analysis varies for some reason. When I analyzed using analysis report the line suggested was considerably different from yours and the depth displayed was 20. 

versus

Side note: it is instructive to Show Lines. I set mine to 5.

@jas0501 just something I have picked up. That d= is showing the depth for the variation created with the engine evaluation system in your screenshot i.e. for the variation just under the Teal "brilliant" rating and the engine score of -4.71. It isn't reflecting the MultiPV depth. If you tick on "self analysis" the engine evaluation system is switched off and the d= indicator will then switch over to reflecting depth for the MultiPV area (i.e. the first line with a rating of -4.86 etc.)

My evaluation came from the MultiPV section, from a first line suggestion to d = 31

The indicator is just showing d=20 because I switched off "self analysis" to show how the engine evaluation system was re evaluating engine suggestions as blunders. The engine moves evaluation system only goes to d=20. 

@pawnstormpossi the engine moves highlighted in yellow where done using the MultiPV system (i.e. show lines ticked, self analysis ticked) and engine time limit set to unlimited. I stopped at d=31 as there wasn't any more line selection movement

The d indicator is only showing d= 20 because I switched on the engine move evaluation system by unticking "self analysis". I then just clicked on a move I knew the engine evaluation system had re evaluated as a blunder. My point is quite a simple one. Engine via MultiPV system to d= 31 suggests a variation of moves it considers favourable and one hopes are "best moves". However when I re evaluate these very same moves using the engine moves evaluation system (which only works to d = 20) it classes 4 moves as blunders and one as a mistake. It is this that is bizarre. I am going to ask a question here. Is it because I am now re evaluating moves worked out to d=31 at a d=20 level that these results are returning. In other words at d= 20 move 41...Kg7 is seen as a blunder but at d= 31 it is not? Could this be the reason? Essentially a horizon effect error. 

And this is a flaw in the system as a whole. 

You ask some questions:

Lessons learned? Don't re evaluate MultiPV lines if they were to a d > 20...I may see similar results.

What are you trying to accomplish and is this the best approach? I always do my own "self analysis" first to try understand why a move was classes as a blunder....i.e. I use my own brain first. Then I move to the engine to see what it suggested as the best move and variation and move through each move trying to understand the logic behind each move. I then actually put the game into a site that evaluates a position in English along a number of important dimensions: best line to play, threats, opportunities, move as part of an overall plan, concepts in play at the point when the blunder was made etc etc. There actually are even more variables it explains...but you get the picture. I only do this for the biggest mistake I made... the one that cost me the game...as I don't want to get analysis paralysis. I won't mention the site here...iI'l let you try decode what the site is

What will you do now, based on these results? PLAY BETTER!! Hopefully. Considering I would have learnt and understood the reasons for the blunder and the better moves to make and why they are better. Also....not to trust game analysis moves done by the engine too far forward.

I do believe the engine is getting more reliable as more work is done on it, and I am really looking forward to the additional features being worked on.

@pawnstormpossie:

24. Ke1 re-evaluated as a "best move". 

The variation I re-evaluated had  initially been suggested at d= 31

This variation was 25 moves long. It was suggested as line 1 in the MultiPV (show lines section). It was re evaluated using the engine move evaluation/quality system that evaluates only to d= 20.

Based on the feedback from @pawnstormpossie and some more thought my conclusions (please tell me if you agree or not):

1. Generally not a good idea to consider with a high degree of confidence the later moves in a 25 move variation given by the engine

2. Not a good idea to re-evaluate a suggested variation using the engine move evaluation/ quality system(the mode that gives the colours and arrows etc - AND EVALUATES TO A DEPTH OF 20!) if the variation was done to a depth greater than 20. This is  exactly what happened in this specific case. All of this really traces back to the horizon effect on move reliability/ viability.

When analyzed to d = 31 line 41..Kg7 was the suggested move, but when re-evaluated it was reclassified as a blunder. See screenshot below. Regarding getting those N's off the rim: move 25...Nf6 and 26...Nc4 were re-evaluated as "best moves" in the variation. All the earlier moves were given the green light.

More 24... Qg7 re-analyzed as a best move!

So in all the early moves of the variation there is, for want of a better word, agreement on the move to make, between an analysis done at d=20 and one done at  d=31

Q g6 ? I assume you mean 24...Qg7  ?

Yes 24... Qc5 is an excellent move! 

escapes capture by the white rook (Rd4)

allows playing Qxc2

threatens to play Nxc2+

Engine found 24... Qg7

Yes agreed. But thank you for the advise!  But it does clearly show how one must be very careful of accepting  the later engine moves in a long engine evaluation. 

I think this has been a very informative "investigation". Thanks @pawnstormpossie for your, as per usual, very insightful viewpoints.  In my post #486 the 2 points in orange basically put this investigation of the blunders I was seeing to bed. The current system must in certain cases be used with insight e.g. not a good idea to re-evaluate a  move suggested at d = 31 at a d = 20 level. So I hope what I have highlighted is of use to others as they use the new analysis system. 

I am looking forward to the new enhancements!!

 24...Qg7 re-evaluated as a "best move".

Sorry I saw a typo in my post# 498 it should say "engine found 24...Qg7. I blame my Apple Pencil entirely

My MultiPV set to unlimited time setting:

24...Qg7 score - 5.18

24...Qb6 score -5.42

24..Qc5 score -5.25

So the engine more evaluation system with all the arrows and colours chose 24... Qg7 as did my analysis done yesterday to d= 31. I officially give up. LOL  Clearly it's a close match on the right course to take.