Fixing New Analysis

It also has some strange localization issues. Both moves suggested Rc1 and Tc1 are the same moves. Tc1 is just the German version of RC1, for "Turm". However it shows the same move twice in different languages.

@HenrikRueping have you sent this screenshot and reported this new bug to customer support? It is important that they know about this as well!

Did so now.

This is supposed to be a "missed win" for black? - LOL

Don't get this one either. (How does Q vs. R matter if Nxc8 is played anyway on the next move?)

Just out of curiousity, here are some examples of how Stockfish 10's evaluation changes with increasing depth:

(It's a sample of 28 positions taken from ~1200. Might do a more thorough analysis some day, to compare the evaluation at each depth with how the evaluation changes — at some fixed depth — after making the best move found at that depth.)

@fschmitz422 these are 2 very good examples of an inaccurate and unreliable engine analysis system that is churning out nonsense results for some reason! Why this is happening I can't say. I wish I knew the answers! It is up to the chess.com development team of coders to work out the "why" and then fix this new analysis system for us! I am sure they don't want their new system to be churning out these kinds of results! @dallin and @jdcannon should be seeing these new pieces of evidence and handing on to their dev team ASAP. 

@fschmitz422 have you sent these 2 examples to membersupport@chess.com ? I urge you to send these examples to them. They need to see this! The more evidence they have the more they will be able to figure out the "fixes" required. I have sent them my screen videos where the engine re classifies its own suggested moves as blunders, but your examples are very different in nature. 

I don't know if anyone else has also noticed the following 2 issues/bugs relating to the "retry" section:  

1. Not all of the mistakes and blunders one made in the game are shown in the retry section to retry. Why? Shouldn't all one's mistakes and blunders be in the retry section? 

2. When one gets the move right in the retry section directly under the green bar with the words "perfect" in it is a sequence of moves worked out by the engine following on from the correct move. If one clicks anywhere in this moves sequence (up to the last move in the sequence) then one is immediately taken to the analysis section, with this move sequence inserted (up until the move you clicked) into the game move list as a variation of moves in brackets. All perfect so far. However there is NO SAVE BUTTON at the bottom of the game move list! This is the bug. One cannot save this change to the game move list with this new variation. I did a test and added an annotation to one of the moves in this new "retry" variation. Then only did the save button come up.

Will others test it out and let me know if they also experience this behaviour. I have already asked one of my very good friends on this site and he experienced both of the above bugs/issues as well.

@chrka excuse my ignorance but what are the y axis variables? And what is the meaning of the 2 numbers in the heading of each graph e.g 1068:24 in the top left graph.

What's your conclusion from these graphs? Does it have a bearing on the problems we are seeing? 

Really?

Like I said, it was mainly an out-of-curiousity thing. What it shows is the evaulation at a certain depth. The y-axis is the score (in centipawns, so divide by a 100 to get what the analysis here shows), and the x-axis is the search depth. (I thought you could click to zoom images, but it doesn't seem like it — in case you're interested you can right-click on the plot and open it in a new window, otherwise the numbers are pretty hard to see.) The numbers in grey over each subplot is "game# : ply", only serves as in id of the position.

Without a more detailed analysis it's hard to say anything definite, but it looks as if there's usually not that big of a difference in evaluation between depth 20 and 30. (But this does not take the actual moves recommended by the engine into account.)

Hey @dopolian

What part of @notmtwain 's statements are you questioning? Is it the part about the new analysis going to greater depths than the old full analysis? 

Do you know to what depth the old full analysis went to? I can't remember it ever showing that data as part of the analysis info (but it would be interesting to know).

It's really the fact that depth 20 is the depth during the analysis that amazes me. SF 10 on my computer gets to depth 20 in a matter of seconds. Same story here, but why not up the depth if it doesn't take that long to do so?

@dopolian I have been thinking the exact same thing all along!! In many of the positions I have analysed with Stockfish on other GUIs it goes to d=24 or up to 27 or even higher etc. It's not ever just fixed on 20. Then with respect to the initial scan where it has been fixed on d=20 I agree with you...how much longer could it take to make the depth deeper and therefore more accurate? I don't think it would make it dramatically longer....but certainly more accurate. And then with respect to the "on the fly"  move by move engine suggestion system...it should also not be to just d=20. The system should allow Stockfish to go to the depth where it makes the right decision. It should not be curtailed. I really believe chess.com need to get their heads together and just figure out why these errors are happening, and just fix it. Make it a solid reliable product. We can all debate here what can "maybe" be done on a superficial level. But we don't have site of the nuts and bolts of this new system. Only chess.com dev do....and therefore the absolute responsibility lies with them to stabilize this system and fix it properly. Chess.com is supposed to be the no 1 chess site in the world. Well then it needs the no 1 analysis system....that works... consistently. In principle it's a very clever system. I love its features and benefits. I can't believe that there isn't the expertise available to fix this. To be honest I am also disappointed that we have not heard from any chess.com staff on this forum again, now that we are again seeing errors and proof is being presented. 

Our devs have looked at the issues reported here and are working to make sure everything we control is nailed down. Reports like this are helpful @flashlight002 — thank you! Stockfish 10 is still very new, the more we find the more we can either fix ourselves (if we have issues on our end) or pass on to Stockfish developers to improve the next release.

Just to share my POV...

SF is free, UCIs are free. If I want to do real analysis I use the software on my PC.

When I click a button on a website I'm usually wanting to check a tactic. Something super old like Fritz8 would work fine for that, and I'd be perfectly happy with it.

Two reasons server analysis is much lower in depth(much slower) than your offline analysis. 

1. They use 3 lines multi PV (so it takes nearly 3x resources to reach the same depth in single PV)

2. Web assembly Stockfish ( WASM) is about 30-70% slower than regular SF running on GUI.

@dallin hi hope you are doing well

Thank you for this feedback. It is really much appreciated! Our aim here is to be constructive in helping your guys to fix things so I hope all the screenshots, screen videos (I sent 2 to customer support as there is no way here to attach a vid file) and explanations of the bugs help in isolating the causes. I am holding thumbs your guys can crack most of these issues that we have documented so far here. Certain things are definitely nothing to do with Stockfish binary e.g. see my post 142 re the 2 "retry" issues I have found. Naturally if something is a Stockfish issue we rely on the Stockfish community to fix the issue.

@drmrboss thank you for this insight . Yes I have seen that Stockfish is slower with WASM. Another big chess site also has WASM implemented and one can see that on their "on the fly" analysis they actually show their Kn/s and it is running at around 90 or so which is naturally dramatically slower than the 800 kn/s I get on local usage. That is to be expected. However I have seen that initially at the start of the game, depths were around the 50 mark and then settled towards the 20s. There is also the option to allow limitless depth analysis...and then it was surpassing depth = 20 and this did not take very long actually (5 to 10 seconds or so...it all depended on the move), even with it being cloud driven. Our site should function speedwise the same using WASM.

In terms of MultiPV: on their site and this site that is configurable in engine setting and I have mine set on 2 to reduce computation time and improve accuracy as the engine has less to compute. As per an earlier post by @dallin he said all lines analyses (i.e. MultiPV) would be computed client side. So I am not following you regarding Multi PV being a factor in slowing down the initial scan run on the chess.com server. Why would an initial scan require 3 line MultiPV? 

Regardless...the time factors are not the issue right now actually. My posts and others highlight strange nonsense  move suggestions and re analyses of some of the engine's own variations as blunders! These issues, together with my post 142 regarding the retry functionality issues/bugs are the problems that need to be fixed presently. If they get these matters fixed we will have a more robust system we can trust.

Dallin, can you give me an explaination on my screenshots, too? Because they prove your engine works at depth 14.