Fixing New Analysis

@dallin thank you for the update. Much appreciated. I did not know/was aware that Stockfish 10 actually had any issues. Do you know what these are? Have you had reports of these? 

@GhstlJrg1-0 my apologies for my last post were I said I wasn't following you regarding how you were showing 4 lines. I see now that the dev guys have increased the self analysis lines options to 5. So I get what you are now saying. Sorry!

I know that these calculations are all done on the client side using your own processor cores and hardware. The more lines one has under analysis the greater the computational resources required. On many GUIs one has the ability  to change the "Thread" and "hash" values...threads is the no of physical cores one wants to make available to the engine. Usually the default is 1 but I have increased it often to 2 and then computational power to the chess engine is greatly improved. However in the engine settings for chess.com there is no Thread or hash setting. So I don't know what the background instructions are asking of local client side resources. If you have other programs running at the same time that are using lots of processing resources then possibly this could be influencing your problem. Try shutting down other big resource hogging programs and see what happens.

I haven't run 4 lines yet as part of my analyses (I have mine on 3 ), but will do some testing and see if I experience any issues such as you have explained. My one device has 8 physical cores so I will test it on that device too.

Yes, I was talking about the UI. You think it is fast, I do not. My complaint makes no sense? That is your truth, mine is different.

The new analysis is faster, I agree with that, of course. By the way, I don't care at all, since it's VERY inaccurate. I've got some evidence it works at depth 14, I will post some screenshots in a few days.

 It's calculated at depth=20..   It  shows on the sidewhen it's analysing. The analysis wass  little sketchy at first, but now it's more accurate.

Yes, it says 20, but, as I said, I've got some evidence it calculates at depth 14.

Were analyzing analysis here. Yes. I would safely bet something is wrong. 

@dallin @jdcannon unfortunately I have to restart this conversation on the problems and accuracy issues with the new engine as I have just experienced major issues with it again tonight. I wish I could take a screen video to show you how bad it really is. On completion of today's puzzle https://www.chess.com/forum/view/daily-puzzles/5-4-2019-moravec-ceskoslovensky-sach-1952

I clicked the "analyse" button. First time I did this it just froze showing 70% in the green vertical bar and would go no further. Subsequent tests had it either freezing completely at 1% or taking very long to finish or completing in a second or 2. It was completely eratic. But unfortunately it gets worse. 

I had "lines" checked and had my engine time limit set to 30s initially. The 1st variation showed mate in 15. I clicked on this variation. The VERY first move in the variation was re evaluated as a blunder! I then ran multiple tests and the variations being returned by the engine were riddled with blunders! Even with my engine time limit set to 5 minutes the first variation in the 3 line list (I had it set to 3 lines) that ran from move 3....xxx to 17.xxx contained 5 blunders on re evaluation! The variation therefore consisted of 28 half moves. So 5 out of 28 proposed half moves supplied by the engine were re evaluated as blunders! Below is just one screenshot as proof relating to this particular test. I ran tests at various engine time limits and every single variation had multiple blunders in them. I did make sure I waited till the results had "settled down" and the variations were not altering before clicking on them, to make doubly sure sufficient time had been given for the engine to work and return a supposedly stable result. I also noticed that the depths being reached relative to what it says in the settings menu was quite different.  E.g. when set at 1min which says depths should be from 22 to 30 the "depth=" indicator showed for example "39".

In my testing the variations contained anything from 2 to 5 blunders. 

Can you explain what is going on? Clearly there is a problem. 

Hello? Anyone there? Surely I am not the only one experiencing these strange issues?? I wish I could have posted a screen video showing the bizarre behaviour of the engine. There must be something very wrong if a variation provided by the engine, and which one trusts are all "best" or excellent moves, on re evaluation by the engine itself of this variation finds 5 blunders in it! 

Has anyone else experienced similar problems? My aim is not to "moan"....I want to help fix a problem when I see one. I think the new system is great. But how can one trust it if it produces such nonsense? I really would appreciate input from chess.com support and other users so we can tackle this. I am quite disappointed as I really thought the new analysis system had been fixed. 

It's been a month now since the new analysis feature has been launched, and it's obviously still not working the way it's supposed to, so maybe you should consider some alternatives.

"please remove all analysis features" after the developers spent weeks programming them for your benefit? Just don't use them....jeez.

@EndgameStudier and @doyouacceptdraw thank you for your input, but its not helpful or constructive insights or help really. I am a paying diamond member. I should not have to be reduced to "considering some alternatives" which I will assume means "go find another site with an engine that works or use another program for analysis". Why must I? If you buy any product with your money don't you expect it to work properly? I think it's a pretty reasonable expectation actually. 

I am not a stranger to website construction and coding and as such understand the complexities involved. Sometimes it can take a long time to fix or isolate problems. Especially if the code is complex. But my viewpoint is that a feature should work... otherwise why is it there? Many of us use the analysis engine to learn and as such we rely on it being accurate and showing the correct moves.

@EndgameStudier I never called for the analysis engine to be removed. Quite the opposite actually (as can be seen in all my posts). I think it is a great tool...if and when it works properly! Which it does not. I just would like it to work properly. I AM SURE IT CAN BE DONE.

I have now made 2 screen videos showing clearly how the engine comes up with move variations that contain blunders in them. I did one test at depth 20, the standard analysis depth and another test with engine time at 1 minute. The variation I then re evaluated had been generated to depth=47! Yet it still, on re evaluation had a blunder move in it. I am sending these to customer support @dallin and @jdcannon for the team's review.

My intentions are to highlight that there are problems that need to be addressed by chess.com dev. They, and their support team are a great bunch of people...helpful and dedicated. I know that they would not want a sub par section on the site, that wasn't functioning properly... especially the analysis tool, probably one of the most important tools on the site.

I agree with you on that. Since you have a diamond membership it is perfectly reasonable to expect the analysis feature to work as it is supposed to, since you are paying for it. 

Thanks @doyouacceptdraw

I ran  it through analysis at depth=59 and it's +M17. I checked out move for move then I saw it shows one of the  move as a blunder.. BUT it's nota  blunder, I don't know why the analysis thinks that.Maybe it's just showing blunder when it's not?

 Btw am happy with the new analysis, some  minor improvements neededhere and there, but overall it works great and fast.

1) I asked to remove NEW analysis features, not ALL analysis features. Or, at least, to give us the possibility to choose between new and old analysis. Read carefully before commenting.

2) The developers spent week programming it for my benefit??? Please, explain me how replacing a good analysis engine with a broken one was for my benefit.

3) If I want to criticize, I don't see why I shouldn't. Something called "liberty of speech" exists. Why should I shut up? Because they worked hard for the new engine? Or maybe because my opinion differs from yours?

4) I don't blame you, I'm a serious chess player who looks into improving, something you clearly are NOT since you love a broken analysis engine, so you can't fully understand my point of view.

Keep using that and learn to make mistakes. Good luck.

This is a bit of old (I've been lazy, my bad), but it shows some evidence that engine worked at depth 14.

As you can see, the engine suggested Bg5 as best move.

Turning on self analysis, it suggested Bg5 too, but just in the first seconds of analysis, when the depth was 14. In the next seconds, when the analysis went up to 15-16, Bg5 wasn't even considered one of the best three moves anymore, and as the depth went higher this didn't change. I waited until depth went up to 22, no trace of Bg5 into best three moves. I can make a video, too, if you need even more evidence.

Now, since this is old I don't know if you improved engine since then, Chess.com, but I'd say there's some good evidence that, at the time, your engine worked at depth 14. Which is very low.

@Rasta_Jay thanks for this effort in helping to try understand what is going on. I agree the GUI and the various elements that make up the whole system (retry, histograms, themes, suggested lessons etc) are great. But this issue of the system re analysing/showing moves in its own suggested variations as blunder/s needs to be fixed and corrected by the chess.com development team. I have highlighted your points you raised above which are important. So basically you are saying you believe the moves are not blunders but for some reason the system thinks they are and therefore reclassifies the move as a blunder. Be this the case it still must be fixed as the system goes and then suggests another move to replace this "blunder". It would have been great if you could have shown a screenshot of the move classified as a blunder which you say was not. 

We must remember that users of all different levels use this tool and beginners will not be able to realize and refute a move classified as a blunder when it isn't one at all. They will "believe" the system. So it still must be fixed. 

This is the move that it showed as blunder 

As you can see in the table base, it's not.  

I think the initial analysis is done on chess. com servers which are more accurate, while the self analysis (move by move) is done on your computer.. Am just making an educated guess, maybe there are some other explanations..

@Rasta_Jay thank you very much for these screenshots. This is VERY helpful. Going to hand this extra evidence over to chess.com support. This certainly will help them isolate what to fix!

I gave up on the analysis function once I realized that the developers here dont know what they're doing. I dont enjoy being a paying beta tester. the developers should've had users who actually use the engine determine its value. anyone who has used it regularly would note how much better the old analysis was.