Fixing New Analysis

Retracing our steps and apologizing is a quality that few have, and a sign of intelligence. Well done. I changed my mind about you.

Next step: to avoid to be unrespectful/say stupid things, so you don't have to apologize after.

This isn't an easy task... sometimes, I say something stupid/am unrespectful too...

About the example I posted, if you read the posts in this thread, you'd know the one who posted that wasn't me.

By the way, glad it works on your machine, but for some reason on fschmitz's one it didn't, and the cause could be some bug.

This comments itself.

SMH @giancz91. Looking into how we can better treat situations like this in the endgame. Thanks again!
You guys have all been a huge help. The feedback has mostly been very constructive and has helped us find and fix many issues. I have changed the topic of this thread so that the feedback here is not quickly taken as negative or complaining. We are all working to make this better. We all want an Analysis tool we enjoy, trust, and can learn from for years to come. 

I'm the creator of the thread, and I say that changing its name was logical since we weren't discussing anymore about bringing back the old analysis (I insisted about that but it went nowhere, so now I'm providing feedback to fix the new one).

The thing I disagree with is the new name of the thread, "Fixing New Analysis" would be better, but they have to make their products as good looking as possible, so their words are sweeter than ours, and there's nothing wrong about that, it's normal they do that.

@dallin I actually went and ran a Stockfish 10 Scan to a whopping depth =35 on @giancz91's game and move 50.Kd6 was not marked as a bad move in any way. I then ran the same game through Komodo 11 at d=21 and it also found no issue with this move. So a clear issue!

I for one certainly share your sentiments that we need to work to make it a brilliant tool. It has a way to go, given what we are seeing, but I know your dev guys can get this working properly and reliably! In fact I am counting on it!! 

Can you give me feedback@dallin?:

@dallin did you see my questions to you in posts #163 and #161? If you could give me your views and feedback

Then:

I found a work around to force a rescan by loading the game pgn into the analysis board to see if the error shown in post #155 was no longer showing up as you have indicated that the dev guys have been working to fix  the problems...so I wanted to see if I got a better result. It is still returning that incorrect analysis for move 41...Qc8. I know it is not a blunder move as reported by the analysis engine as I have run the game through 2 independent programs also using stockfish10 and to a similar scan depth (I ran it at d=21 and d=23) and then also Komodo 11 too!...and both programs found nothing wrong with the move at all. So Houston we still have a problem.

9th Blunder said, "can't y'all just give us the best of both worlds? give us the new and modern gui with the option we once had of running a maximum analysis? I honestly never minded having to wait as long as 20 minutes to analyze a game. at least it was accurate and helpful. now the analysis mode is just prettier but crappy and unreliable."

I have not been much of an advocate of running chess engine to try and create a system for any given variation to any particular line. I guess my confusion lies in the fact that many lines are not used by GM's because they often question the validity of the line, even sometimes stated the any given line has no point to it due to how many counters exist against them. I am reminded of  quote I read from Bobby Fisher denouncing the validity of the French variations. Of course later he recanted only saying, ...maybe the French is a playable line after-all." (paraphrased) I have experienced the "benefit" of playing at the Denver Chess Club in Colorado against Brain Wahl. As well as some of his clones or those he spends time teaching. By benefit, I really mean. I don't remember ever winning a game using the French against him in all the time I have known him. I certainly believe there is more to the French than I am able to ascertain at current. But, I sure wish I could make heads or tails of engine after engine so that I could some day make the same claim that our U.S. GM Nakurmura made about chess in ten years will be obsolete, because of the computer age. Apropos moment, I more agree with a friend who said,"... as long as people play bad chess you will always have someone to play."

French Defense: La Bourdonnais variation, 2...d5

1.e4 e6 2.f4 d5 3.e5 c5 4.d3 Nc6 5.Be3 Qc7 6.Nd2 Nh6 7. g4 Bd7 8. Qf3 Be7 9.O-O-O d4 10.Bf2 Nb4

11.Kb1 Qa5 12.a3 Nxc2 13.Kxc2 Be6 14.Nc4 Qa4 15.Kb1 Bb5 16.Nxe2

@TimothyScottPuente I think your friend's quote is very funny....but rings true LoL  

It will be a sad day if chess is made "obsolete" due to engines. I don't think it will. This game is hundreds and hundreds of years old (in its current format that is) and I am sure in 200 years they will still be playing it....but maybe everyone will have chip sets wired into their brains to connect to the internet via a retina display and be playing chess.com version 100 in their heads!  . Funny enough I was hypothesising the other day that if we factor in Moore's Law we could see chess solved in about 130 odd years or so. All 1e+120 moves. Then engines would know it all. But there will still be bad chess players like so many of us lol.

Thanks for the support, @giancz91. At your suggestion, I have revised the topic to "Fixing New Analysis." The change was not political, @Toire, it was done to better align the topic with the conversation, and to help newcomers better understand that the intent of those who are continuing this discussion is to improve, not to remove.

The big issue with Giancz91's latest example is that there really is no good move here. To attribute any quality label to the move based on one losing score to another losing score is not helpful, and even misleading or confusing. Giving the move a Missed Win classification in a game that cannot be won is erroneous. We were already working on a better solution here, but Giancz91 added some good fuel to the fire.

Re #161 @flashlight002 - I think this is a good idea. Not a frequent need, but there are certainly some use cases where it would come in handy. For example, we had a few hardware issues in the past couple of weeks which resulted in poor analysis... I would not want to live with that. Perhaps a link to re-run analysis at the bottom of Details would be enough. Thanks for the suggestion!

Re #163 - we are still working on adding additional depths for premium members. The simple way to do this that I was hoping for (just add more numbers) did not result in a quality experience, so our developers asked to explore a different way to engineer our analysis to provide a better experience at deeper depths.

@dallin thanks for the update on developments. Glad you like and are in agreement with my "re-analyse" function suggestion.

I am really looking forward to positive developments .

Hell, if I was a hot shot ace programmer (I wish! lol)  I would gladly pitch in and help fix. But alas I am not. I am therefore really hoping (and holding thumbs) your dev team can crack all the issues, fix all the bugs we are experiencing, add the features under discussion (that will really add even more value) and we end up with a world class product that delivers on its expectations. Hoping and counting on it in fact!

I second @dallin's sentiments that we want an analysis system we are all proud of - the developers and the users alike.

To the dev guys who are working on this product: I hope the examples we have posted here (and continue to post as and when they arise) really help you see where the problems lie with the analysis system and make it easier for you to isolate and figure out what is going on. I know you can nail it and make this work! May the force be with you . (I appreciate this IS NOT simple programming, but probably quite complex work too!).

Make Analysis Great Again! Seriously, can y'all at least hook us up for paying to be your beta testers? I'm sorry, but again, if y'all would have bothered to test your new analysis tool this could have all been avoided.

@9thBlunder indeed I am in agreement that more beta testing should have been done, especially when I heard that these very same problems were being experienced in the beta phase. I don't think anyone could argue the fact really. I had not been a part of the beta group so I cannot pass judgement, but the chess.com dev team who built this new analysis interface must have believed they had ironed out all the problems and that it was "ready" for general usage. As they say "hindsight is 20/20 vision and it is clear to me that it looks like it required way more beta testing and UAT before release. But the engine is launched ....the cat is out the proverbial bag, but it is clear that chess.com believe they can fix it, and I get the distinct impression that they are serious that they want it to work properly, and @dallin has expressed as such.

So I am giving chess.com the time now to fix and enhance the product as follows (and at the risk of belabouring the point about all the things that need fixing or creating, I include them again all in one neat list):

1. Fix the unreliable and inaccurate post game analysis results we are seeing.

2. Fix the inaccurate "on the fly" analysis and variation computations (I am talking here of examples of situations where the engine is dishing up suggested variations that on re evaluation contain blunders). There is nothing wrong with MultiPV as I can use it to refute the blunders I find in the primary variations that the engine suggests.

3. Create a "choose your game analysis depth or run time option" menu before a game analysis runs so that one can choose from a range of accuracy types, from say a quick simple blunder checker to a extremely accurate scan ( e.g. that would go to d=26 or beyond, within the parameters of what Stockfish can reasonably accomplish - taking cognisance that there is no such thing yet as perfect answers but certainly exceptionally good answers). Dallin wanted a menu which listed different depth options (as stated in earlier posts) but his dev team were not happy with this idea. I can only surmise it was because time and depth are both required interdependent parameters to run an analysis properly. I know on my independent apps I have to set the per move scan time before starting a scan. I don't know what kind of options they will come up with that will allow us to control the initial scan accuracy levels. We will have to wait and see what is designed! With the old system there were adjectives to describe the type of scan like quick or deep and an estimated scan time in brackets. Maybe we will get a similar options set.

4. Include a re-analyse button so that one always has the option to redo a scan in cases where a) an updated engine has been announced and one now wants to therefore take advantage of a stronger engine to re analyse a game, and b)the system experienced hardware and/or software malfunctioning during the running of the scan and one then needs to redo the scan once the issues have been fixed

5. Fix the problem where one cannot save a "retry" variation when it is placed in the main game list. Right now if one gets a retry right then a variation of moves from that point is shown under the green bar. If one clicks say the last move in this variation it inserts this full variation in the game move list in brackets in the analysis section....HOWEVER NO SAVE BUTTON appears. Normally when one includes new variations in the move list a Save button is invoked. So there is a bug or oversight in the coding of this retry variation feature.

If they accomplish these 5 things then we will be in a MUCH MUCH better place. We now need to give the dev dev team some time to fix this list of issues. Yes I agree it should not have been launched in its current state....but it is launched...now it must just be fixed. 

@dallin maybe you can answer this question:

How come the retry section never lists all the blunders and mistakes? E.g. on a recent game which was an absolute blunderfest I had 8 blunders but the retry section only gave me 3 of the 8 to retry. On checking all my analyses with the new engine it is the same scenario in every analysis. The retry section doesn't give one all one's blunders. I asked one of my friends to check his game analyses and he reports the same thing. Is this a bug or is there some kind of logic to it? 

There is logic to what is being presented in the Retry section, @flashlight002. You may make the same blunder three moves in a row if you don't fight the right move and your opponent does not punish you for it, but you don't want to solve that same problem 3 times in Retry mistakes. The logic being applied eliminates duplicate or irrelevant problems.

@dallin thanks for clearing this one up . 

Did you see my remarks re the issue/bug of the variation in the retry section that, when one clicks on any move in it (I click the last move in the variation) gets automatically pasted into the move list but then doesn't bring up the save button? So one can't save it in the move list then unless one does something like make an arbitrary annotation on one of the variation moves (and delete it later) which then forces the save button to appear...enabling one to save the retry variation.

A bug report has been submitted for that issue, @flashlight002. Thanks!

@dallin thanks. Another bug bites the dust.

So we still can't get an accuracy score in our games unless we buy a membership. Why was this metric removed? It used to count the number of "excellent" moves out of all our moves. Now it's been changed to something else and the old "excellent" based number is gone. I've mentioned this before. It's easy enough to take the number of excellent moves and divide by the total number of moves, but it seems petty to remove the result of a grade-school arithmetic result for no reason.

@PkPum if you click the details tab, directly under the game info (players, elo, date etc) is a summary of all the accuracy metrics, including best move% . These are not in the game report. It was moved to the details tab. Here is an example from one of my game analyses. Best move% is there. Underneath the summary is an accuracy score also per piece:

Are you not able to access the details  tab?

@dallin there was one very important thing I wanted to ask you...which I should have actually asked in post no 185 which sets out in point form the things I believe we need to accomplish (and I do hope you are in agreement with them all). Far be it for me to dictate to you guys...I just like it when there is a goal, clear objectives and clear deliverables matched to an estimated timeframe for completion. Then everyone's expectations are managed....yours, chess.com management, the dev team and your customers .

My question is:

Do you have a date by when you believe we will have all the issues I have summarised in post 185 completed and tested and signed off by?

Hey @flashlight002. Thank you for all of your constructive feedback. It's hard to give estimates right now. But be assured, we're working on it!