Fixing New Analysis

@nmrugg hi there. Ok understood. Thanks for the feedback. Appreciated.

Let us know when you are confident you have solved the underlying problems and when/if you bring new features like different scan depth/time options for the initial game analysis online. Then we can run some of our games where we saw issues through the new improved engine to see what results it returns (that's why I thought a re-analyse function was really needed). I have a game I use as a "tester" as it is a game analysis that I know contains a patent error in it (as I ran it through a number of independent apps that proved the chess.com analysis was wrong). 

Anyways good luck fixing things  

@nmrugg maybe this can also help you...this  is the link to the game I am using as my game for testing the engine now:

https://www.chess.com/daily/game/225442842

I am using it as I know that move 41...Qc8 is not a blunder, even though the chess.com engine continually marks this move as a blunder in its game analysis. I know the result is wrong as I have analysed the game with 3 independent apps multiple times with Stockfish10 and Komodo 11, and all of these scans showed 41...Qc8 as ok with no issues (not even an inaccuracy!). 

What is more: in the analysis MultiPV section the second line suggestion by the chess.com engine is 41...Qc8!  (Test done with engine time limit set on 30s and 1 minute). So on the one hand it says it is a blunder, but on the other hand it suggests the move. So clearly the game analysis is wrong!

In addition the automatic quick post game blunder checker shows a vastly different result to the full game analysis result. The automatic game analysis said 2 mistakes and 3 blunders, while the full game analysis report says 1 mistake and 10 blunders. A VERY big discrepancy in results. Generally the automatic blunder checker never tallies with the full analysis but such a high difference in results points to real problems.

Here are the screenshots, followed by the game itself:

@dallin @nmrugg I forgot to add that the automatic blunder checker that runs after the game has finished gives a different result every time one reopens that game from one's archive. I just finished a game now and this was its results after I tested it by opening the game up a few times from my archive list:

1st time: 3? 8??

2nd time: 2? 7??

3rd time: 4? 7?? 1 missed win

4th time: 2? 7?? 1 missed win

5th time: 4? 6?? 1 missed win

Bizarre!! The automatic blunder checker is completely unreliable. One surely cannot believe its results as it changes its mind every single time it runs. 

No, as I mentioned, this is behind a paywall now. That's right, a simple arithmetic metric of one number divided by another is behind a paywall, which is petty and annoying. I can understand their "new" accuracy score which is more complex and depends on how your moves compare to their engine which is being programmed into the site, but removing this simple metric that we used to have and now putting it behind a paywall is just ridiculous.

@flashlight002, regarding post game summary, this is a super-fast blunder check taken to the depth of 8 using multithreaded processing. This quick summary is meant to give you an idea that is directionally correct. At this depth, using this processing method, inconsistencies are expected.

@dallin than you for this info and detail. Much appreciated. Makes sense now . So it's really just a rough "barometer" basically. Got it!

I hope you guys are making headway on the main engine analysis system itself.

@dallin @nmrugg hi. Have you made any progress on fixing the engine? Because it is still not working correctly.

Hi all. Can anyone else also confirm that the same thing is happening to them with the analysis system:

It is now taking VERY long for the analysis system to return an "on the fly" evaluation of a move (i.e. if it is considered good, excellent, best, an inaccurracy, mistake or blunder). Per move I am often waiting in excess of a minute. Some moves will take 20 to 30 seconds to return a value and then there are many that one waits about a minute before one gets a result. I have engine time limit on 30s, but I think, and please correct me if I am wrong, but that setting controls only the MultiPV section and not this section (which appears to go to d=20 before it returns a result). I even unchecked the MultiPV "show lines" function thinking it may speed up things...but it did not. I have attached a screenshot of the section I am talking about below, in case you misunderstand which functional area I am talking about. I can see the depth counter slowly inching to 20 on each move....and it has become significantly slower before d= 20 is reached on each move and a result finally is shown per move. When the engine was first launched it certainly didn't take this long. I can't understand it....considering that the MultiPV function is showing "best" moves and variations quite fast.

My friend who plays on a basic chess.com account has totally stopped playing here. An analysis depth=10 is completely rubbish. If you can't hold on to basic members, how will you convert them to premium?

@dallin and @nmrugg maybe if you can please help explain something that I am perplexed about please

Anyone else is welcome to follow my steps I followed below and also give me you 'take' on things.

Primarily what started this all off was that I noticed it was taking a very long time to return a result for the "move feedback" function, for each move I clicked when I was working through new variations of moves (e.g. a retry variation or a variation of my own even). So I did some 'testing'.

1. MultiPV Section (or the "show lines" function) in the Analysis tab. Open up an analysed game. Go to the analysis tab. Now disconnect your internet connection ,be this 4g, Wifi etc.

Now in the Analysis tab create a new variation of new alternative moves from any given point in the game list. Make sure the check box 'show lines' is checked. To make things less “confusing/straightforward” you can even tick the “self analysis” check box.... although it makes no difference.

Click any of your new moves. You will see the engine start calculating all the lines in the MultiPV section even though your internet connection is off! You will also see d=x setting in the right corner climb to 18, 20, 21 etc.

Now go to engine settings. From 10s to unlimited, try each one. They all function with Internet connection off. The lines also reach d=20 quite quickly and show best moves and variations stabilize fairly quickly too.

So MultiPV functions on the client side using our computer's hardware and it responds well. It doesn’t utilize chess.com’s servers at all in computing move variations.

One thing I also picked up is that the depths reached at various time settings don’t often agree with what chess.com has put next to that setting. E.g. sometimes a final move d will not go past 18 even though the engine time limit is set on 30s (depth=20-26)...so one would expect d to go to 20 at least. But it doesn’t sometimes. Maybe @nmrugg can explain why this is so.

2. The “on the fly” move feedback function (shows in coloured text if a move is an inaccuracy, mistake, blunder, good, excellent or “best” move together with arrows and icons on the board).
Still with your Internet connect off create another move variation from any point in your game while on the Analysis tab (or if you want, still use the new variation you used in point 1 above). The “self analysis” check box must obviously be unchecked.

Now click any of your new moves in your new variation. It takes about +- 15s to up to over a minute for a result to be returned. So this function also works on the client side using our computer's hardware! It isn’t using the chess.com's servers. But it’s taking much much longer to return a result and move variation list compared to using MultiPV for the very same move.

And this is what I am perplexed about. If both these functions are operating client side why is the move feedback feature taking so long to return its result? It is really sluggish!

And point 3 is just a nice to know for those who are interested:
3. Out of interest you may be interested to know that a portion of the retry function is also handled on the client side.

Ok, for the retry move feedback functionality to work, as you move pieces around to find the best move, you do now need an internet connection. So in this instance move feedback is being run on the chess.com’s servers. HOWEVER underneath the “perfect” bar you will see a variation of moves from the point of the correct move. This variation doesn’t need an internet connection to be calculated and is created using client side hardware! This is just an “interesting to know” fact I guess. The “best” move is already “stored” from the initial game scan, that was done on tbe chess.com servers when you first ran the game analysis. It isn’t calculated again. (The above is all easy to test e.g. with internet connection off, and on the retry tab click the “magnifying glass with star in it” icon. The best move is loaded on the board. Notice that to the right of the text d=20 you will see the move variation for this move changing and altering as the engine figures it out up to d=20 on your machine! Not on the server!)

@flashlight0002:

I'm sorry if this comes over as harsh, but with a 900 rating, you are wasting far too much time worrying about chess.com's computer analysis.

To improve your game (assuming that is what you want), there are resources like Puzzles/Tactics, Lessons, Videos etc, which, particularly as you are a Diamond Member, will benefit you so much more.

The fixation on helping chess.com with this half-arsed analysis product is a waste of your chess journey.

@Toire I am not fixating on anything. You actually surmise incorrectly what I am doing with my time in terms of improving myself at chess. That is because you never asked, but assumed. Always a dangerous thing to do @Toire, considering you know 0% about me. But since we are on the subject let me correct your misperceptions,.even though it has NOTHING to do with this forum topic in any way! (and my apologies to other serious contributors in having to read about me  lol).

Actually I am already doing tons of puzzles and tactics on this site and reading many chess books. I am currently reading, and almost finished with Weapons of Chess by Pandolphini as well as now reading Mastering the Chess Openings by John Watson. Both excellent books by the way. Plus I have completed close on 300 lessons on this site. I am already on master lessons. On the new lessons I have finished all intermediate lessons and am now on advanced lessons. I have just completed 2 Chess King courses. My ratings, for your interest are low at present because for many many many months I have been playing mostly unrated games. That is my choice as I learn. So actually I am doing A LOT besides just reviewing games with an analysis engine, or being "fixated", as you erroneously say, over the new engine.

I offer my time freely and with pleasure to chess.com in helping them try improve this engine for a number of good reasons, even though I don't need to justify myself actually to you at all! Primarily as a diamond member I expect the features to work properly. I actually have the time to help out and I do not mind helping chess.com improve their product and help with spotting bugs etc.

Yes you are quite right...your comments are harsh, cutting and somewhat supercilious in tone actually. You have offered nothing of value to this forum topic which I believe is called "fixing new analysis" (other than calling the.product "half arsed", which I am sure the dev team REALLY appreciate).

The topic of this forum is not called "attacking and belittling flashlight002". I don't appreciate your comments. Yes it's not working properly, but they probably put a great deal of time and effort into it, and the function of this forum thread is to help them fix and improve it...as the heading says..Attacking those members who only want to help is really not productive in the least. I will leave our interaction on this point at that @Toire.

So this is a "missed win"?

All instances of "brilliant" and "missed win" moves that I've seen so far were completely nonsensical. I  suggest these categories are removed.

Besides, I'm wondering how the client side engine (restricted to 5 secs) finds (at least) three different forced mates in 12, when the super-fast server side engine fails to find any.

@fschmitz422 check this out. I loaded your position into the analysis board with black to move. 1st a screenshot showing self analysis and MultiPV 3 lines. Engine time limit set to 30s:

Suggested move at d =40! is ....h4. Note that I had my engine time limit set for 30s which is supposed to give depths of 20 to 26. Yet I get a whopping 40. (Not that I am complaining....but rather confusing).

Then I made your move ....Kg2 and clicked "view analysis"....which is done by the chess.com server at d=20. Here is the bizarre screenshot of what the analysis feedback said:

What in the world???

@dallin @nmrugg what gives?

We are working to improve feedback in positions like this, @fschmitz422. We completely agree that this type of feedback in this context is non-sensical, and should be removed.

@dallin besides the initial result how can the engine one minute say it's a missed win and the next say it's a best move? Yet the client side MultiPV showed the best move was black h4. That is really bizarre! 

@dallin I know my post #205 was rather long winded (I would rather explain it properly so it turned out rather long!) but I hope you will look into why the "on the fly" move feedback is so sluggish... especially since it is being done client side just like the MultiPV readouts? Unless there is a valid reason of course.

Why in the world would anyone bother to analyze a position like this?

A raw beginner would know the game is won for black...in 10, or 11 moves?...who cares?

So the analysis board does not need to work properly then?

Oh my.