Inconsistent Engine Evaluation

Sort:
konstantcheckov

When I use quick analysis on chess.com, I have noticed that the evaluation of the position changes after the best move has been played according to the engine. I don't understand the logic here: surely the evaluation should not change after the best move has been played according to the completed analysis. Any re-evaluation of the new position needs to be factored into the earlier evaluation, in order to accurately portray the equal value of both positions - before and after the determined best move. Developers - what am I missing?

KeSetoKaiba

I think "quick analysis" was the key. If you upgrade your chess.com account, then you can use the stronger analysis; these longer ones would be more accurate. Of course, just a "quick analysis" is enough for me to at least have my mistakes pointed out. The lines I can't view I might manually be able to calculate with time (since they at least give you the first move and line evaluation).

In short, I think a longer run of the analysis by a stronger engine setting will yield more precise information. My guess is that is what is being missed here wink.png 

konstantcheckov

Thanks for your responses.


@KeSetoKaiba Quick analysis is indeed not the key.

@Klauer, My main point is that there is no need to have a fluctuating evaluation on a completed analysis, since adjustments can be made after process completion. I cannot think of any justification to not update earlier evaluations when new information comes available during analysis. Either I am missing something, or this is an Engine weakness.

Farm_Hand

The only real evaluations are:
white mates in ____ number of moves
black mates in ____ number of moves
and draw.

Everything else is a guess. A highly educated guess, but still a guess.

And for example, when you play 1 move on the board, the engine can calculate one move deeper. It's normal for evaluations to fluctuate a little even when you download the engine (stockfish is free) and let it think for a long time.

konstantcheckov

@Farm_Hand I agree with everything you say. happy.png However this is not the same as live analysis: the analysis is completed at a specified depth within a limited time. A second pass through the results could correct the figures within milliseconds. I think this is a design flaw, although it could also be a design choice if you are specifically interested in evaluation fluctuations (for some reason). I don't have a problem with it personally, but I still think it's worthy of discussion: especially when I might learn something new.

Farm_Hand

I guess I don't know how different the results you're getting are.

If it's huge swings then yeah, that's not so good, but if it's only thinking for a few seconds then it's not totally unexpected.

konstantcheckov

The differences are not normally so significant - just confusing.

Farm_Hand

Oh, I see.

You can download an engine (and an interface that allows you to use it) both for free.

For example SCID and Arena are free interfaces.

And click on or google any green or blue engine from this link for example

http://www.computerchess.org.uk/ccrl/4040/

 

It may be a pain to set it up, but after that you can have a super strong engine analyze your game for as long as you want.

konstantcheckov

Thanks for the link. I'll get Stockfish - why not? happy.png

Farm_Hand

lol, I said green or blue, but blue is commercial.

Green or orange... but whatever, it doesn't matter, I'm sure you can figure it out tongue.png

Daniel1115

It's probably because quick analysis only checks x number of moves ahead, so move x+1 sways the evaluation

konstantcheckov

It seems most likely. I'm not entirely convinced that it needs to give multiple values for the main line once it has finished the analysis, but we are in agreement as to the likely cause. happy.png

congrandolor

I tend to think that quick analysis is not so accurate in complex positions, yet in my case is useful to find my mistakes

KeSetoKaiba
congrandolor wrote:

I tend to think that quick analysis is not so accurate in complex positions, yet in my case is useful to find my mistakes

+1

konstantcheckov

I think I've figured out what's going on now. In lines that are given as best alternatives, the evaluation appears to remain static after each move. I think that ought to be the expected output from a completed engine analysis. Adjustments must have been made on these separate lines to correct for discrepancies due to differences in ply depth - if my conclusions are correct. Otherwise one would expect fluctuations in those lines too - similar to those occurring in the actual game (best) move analysis. Mystery solved. I still find it interesting: a couple more lines of code and you could have the same behaviour in both cases. I'm not sure which is best though. Observing (outdated) evaluation history is also interesting. It's certainly good for debugging the software. wink.png