Computer Analysis

Does computer analysis, performed under controled conditions, indicate the objective strength of a player?

In other words, does a higher % match of moves to a very strong engine's (on a upscale system) top few moves over many games indicate a stronger player?

It indicates a cheater.

Yeah I think so generally. It wont be the most accurate though. Someone who favours more complicated positions will have a lower matchup even though they may in reality be a stronger player.

There are a lot of factors that mean it will probably have limited use in comparisons across generations, or even players who play in different tournaments. With deep opening theory you may have a high percentage matchup before putting in any original thought. Opposition will make a big difference. If you're forced to play out long winning games you can differ a lot from engine matchups while still playing winning, or even more logical moves. Forced moves will skew matchup rates.

It might underestimate someone who finds more challenging moves which have the same objective evaluation but are harder for a human to play against. 

A messy post but you get the idea.

So, you would say that modern computer-reared players should have a significantly higher match up than, say, 19th century who had no access to databases,  extenseive endgame theory, and even more so with pre-Steinitz era players?

"Given the fact that a chess position can have several good candidate moves, my conclusion is therefore that the % of matching moves with a chess engine does not always automatically indicate a stronger player."

My premise wasn't matching the top move but matching the engine's top several (it's 1st, 2nd, 3rd and 4th) move choices.

Due to engine analysed openings if nothing else this should be the case. I also believe more full time professionals and learning from the past has created stronger players.

However, I don't think it will be by as big a gap as expected as modern players tend to favour complicated positions which break principles etc and aren't as easy to play.

If complex position make an engine's choice less accurate, why do engine's consistenlty win againt unassisted GMs?  I just trying to understand this whole thing.

I'm talking about a similar process that's used to detect cheating, but used instead to compare different players' strength objectively.

That wasn't what I meant by that. In a complex position it's harder for a human to find the best move, or one of the best moves.

But now I've typed that I'm starting to doubt it. Perhaps it's harder when there's less going on and you have a lot of moves which look of similar strength.

Some people will make very good moves a high percentage of the time, but then blunder badly. This would skew the results of your study. I speak from experience. Bob

That's a good point. It's not just about the strength of your good moves, but the scale of your bad ones.

I think it's natural to expect a rough correlation between engine match-up rates and the strength of the player. Weaker players miss more.

As to what extent you can actually use such analysis to determine the intrinisic strength of the players who played a certain game, for example - I believe there are experts that do this. Remember reading a story about one of these experts engine-analysing 3 games from a tournament in Canada (USA?), where a 2100 beat 3 IMs in a row. The experts managed to conclude that the intrinsic strength of the moves the 2100 played during those 3 games was around 2100 - and it was in fact the IMs that had just all played very poorly those games.

Don't remember the source, sorry - it was somewhere on the forums I think.

It also sounds a little off-kilter at a high level.  Very strong players make very few very poor moves, just occasional ones. And one blunder wouldn't really skew anything would it, since each move it selected on it's own merit and not on the game result?  You can be playing a totally losing game but making the best moves in the process - and actually have a higher % match up.

edit: in reponse to:
Some people will make very good moves a high percentage of the time, but then blunder badly. This would skew the results of your study. I speak from experience. Bob

The short answer is probably not. At least, not reliably.

The main issue is that computers don't analyze moves the same way humans do. They often come up with lines that are somewhat cryptic even to the strongest players.

Also, computers dont have a skill level.

When computers analyze, they look at a position in depths. In laymen terms, they look N number of moves ahead.

This number is adjustable. You can set it very low, in which case, it is only going to look at a few possible variations on a  line. Or, you can set it very high and it is going to look at several variations and several variations' variation's.

So, the lower you have these variables, the more moves made by a player could be percieved as 'strong'. And, the higher you have it, the fewer moves will analyze as 'strong'.

In order to set-up controlled conditions, the software would need to be 'calibrated' to a level that is below or slightly higher than the players being analyzed. This would be quite a difficult task.

Mosai is sorta correct. If you have moves from a player that have a higher percentage of matches with computer lines that are very difficult to understand, then there is a good chance they are using a computer.

Part of (and I stress 'part of') my analaysis involves running my games through Fritz after I have analyzed them myself. The majority of my moves don't have any alternate suggestions by Fritz, so, I, assumably, have chosen the same moves Fritz would have.

This, of course, does not mean I am cheating or that I am particularly strong. These are moves that, because of my current skill level, are logical for the position.

Hope this helps.

What I'm try to arrive at is whether the % of match up between a player's moves and the top several computer choices indicates objectively the strength of a player. For my purposes the results don't matter, opening book moves don't matter, just the percentage of match-ups once the game is really afoot.

???

I think no. If Tal won, and the computer said the sac was wrong, why should we believe the computer rather than Tal?

The only thing that matters is who won. So what if the computer says it's wrong. Maybe the computer would have lost the game.and of course, as has been said, % of moves means nothing, it's % of critical moves. and who decides what's critical?

Computers may beat humans, but they are not qualified to judge human play.

I think this 'capablanca had x% of agreement with houdini therefore he's better than Alekhine with y%' is total and utter rubbish.

Does your IQ have 50 points to slip?

This totally misses the point of this thread.

I don't think so. Some moves may be objectively weaker, but harder to play against. Arguably the player who played the 'weaker' move is stronger.

If Tal's sacrifice is bad, but it takes a 17 move combination to defend against, it may be a good practical decision.