New Analysis feature is not accurate enough.

Sort:
alexnode

First of all let me say that i like all the new graphs and colours, and in general how the new analysis looks and functions. But i have some issues with it. 

A. First of all it is very quick and quite inaccurate even for a 1700 hundred player. In complicated positions i already encountered best moves that are mistakes and mistakes presented as best moves. There is something inconsistent. I already encountered one retry puzzle too that was wrong. If i play a blitz game i would probably be happy with a draft analysis like this, but a diamond member who plays a game of daily chess for a whole month i believe that this is not good enough. 

B. Why you don't offload some deeper analysis to the client ? I understand that the backend costs could escalate with deep analysis but you can create a hybrid mode.  Ok some people play on mobile or very old machines but mainstream computing has 8 cores nowadays. That's a lot of Kn/s. 

C. Why there is no selection for the level of analysis we want ? When i play with 2000+ rated player i will study the analysis a lot when i play with a 1200, i won't study it at all. 

D. The way it is now we are doing manual analysis move by move , at least update the colours and the report when the draft analysis makes a mistake.  

E. What happened to the page announcing the new feature ? 

flashlight002

@alexnode I agree with you that this new analysis tool, while very sexy in design, does seem to have reliability issues. 

A real life case in point. I analysed a game where I had unfortunately made a number of blunders. I decided to try the "retry" feature. So I get the retry move correct, and under the green banner that appears when one gets it right is a move variations list from the point of the new correct move, going a number off moves deep. I tapped on this move list and it switched over to the analysis tab where this variations list was now showing. I began clicking each move in the list to see how the engine had made its choices. At the top of the screen I was interested to see that the engine was now also re evaluating its own moves. Suddenly one of the moves is classed as a blunder and another move is suggested!! I did a double take. How could the move be a blunder?? This is a move the engine has suggested and surely it should be classed as "best move"? So I go and play the suggested move, and what pops up in the evaluation at the top?...an alternative move is suggested...and this very move is the move that had just been flagged as a blunder!!! I will get screenshots and post here when I have them. It was the most bizzare situation. It makes me question the stability and accuracy of this new engine's output! In other parts where I retried other blunders and played through the supplied new line variations I encountered instances where what was supposed to naturally be a "best move" was re evaluated as a mistake or inaccuracy!!

In the engine settings there is a setting called "Engine Time Limit". I am not sure what this does exactly and if increasing the engine time limit will have an affect on output accuracy as the release article shows a picture of the settings dialogue box but does not explain this setting at all! Maybe someone with knowledge of chess engines can explain how this setting works and what it does exactly, and if it will improve engine outputs reliability. Because one minute saying a move is a best move, then re evaluating it as a blunder and then re evaluating it as an alternative best move is just downright nuts and does not leave me with a feeling that I can trust the results the engine is providing!

 

alexnode

Yes this engine time limit is confusing, it doesn't affect the main (draft) analysis but the analysis of individual moves. If you click a move then the engine will analyse for 5 minutes , or 10 seconds whatever you choose. Then you have to go manually move by move to actually see a more reasonable evaluation of each move. The blunders, mistakes and inaccuracies don't change at all, which is quite funny at times. 1- 2 points are nothing blunders turn to best moves and vice versa . 

flashlight002

@alexnode yes I also worked out that the "engine time limit" setting is only affecting new self analysis moves made in the analysis tab and the resulting computation of line variations based on the new move. If one has the "show lines" tick box ticked and has the default no of lines set at 3 then one can see that if one has say changed the engine time limit from the default 10 seconds to say 30 seconds then the engine makes an initial computation for all three lines a number of moves deep and continues to fine tune the moves over the longer period (one can see moves changing as the engine has more time to compute more variables).

However this setting does not seem to have any effect on how long it takes for the initial entire report to be generated. One of my chess.com buddies confirms that he can see no difference in the time it takes to generate the report, no matter what the "engine time limit" setting is set to. Hopefully a chess.com staffer can help us out here and explain this setting and also comment on the clearly strange anomalies we are seeing with some of the move suggestions. There clearly are issues. An engine cannot issue a move in a variation as a "best move" suggestion, and then when it is played again in the analysis tab re evaluate it as a blunder!! That is a major issue as far as I am concerned. One then, unfortunately has to question the accuracy of the new analysis tool at present. I will be contacting customer support and showing them a screen video of the issues I have highlighted. Hopefully this will help chess.com iron out bugs affecting the acccurac of the new analysis tool. 

If there are other users experiencing similar issues please post here so we can hear about it!

molemancoyote

This is frustrating for me too, especially because I paid for the Platinum membership which at the time promised deeper analyis, and now that feature has been taken away. It still needs to be fixed! It marks brilliant moves in GM games as blunders because it does not go deep enough.

jdcannon

a.) Its depth 20; (we have some deeper options coming) its very very fast because we split it between a network or machine to analysis game all at once. Much faster that way. Would take your client ages to get to the same depth our sever side can do in just a few seconds. 

b.) If you do self anlaysis, you can set depth to infinite and go as long as you are willing to wait. 

c.) on self-analysis you can choose a variety of possible depths in the settings. 

d.) I don't understand  I am sorry. 

e.) its still there. I will try to find. 

jdcannon

Choosing between a variety of engines will be a future option. 

flashlight002

@JDcannon wow that's a great feature!! Will the options below predetermined I. e. one chooses the engine to analyse from a drop down menu?

I am waiting with bated breathe for the options to run deeper depth game analyses beyond the current 20 ply. Do you have any idea of when that will be ready roughly?

UWannaFork

Today the engine is drunk again meh.png

flashlight002

@UWannaFork those 2 slides really made me laugh. Stockfish hitting the bottle alright! Are you a Beta member? If yes...then you will have the new depth choice features at your disposal. Now set full analysis depth to 26 via the drop  down menu. In the move analysis section you can choose an engine time limit that doesn't slow your machine down too much i.e. where it takes too long for a result to be returned on each move. Believe it or not setting it on unlimited is quite fast (faster,  believe it or not than some  of the other longer time settings for some bizarre reason I have found). Now do your moves again. A result will display when depth hits 26 on each move. Do you get the same result?