I am not arguing that they are sound. I see how you could claim these openings are unsound in general, but the Kings Indian is probably not unsound, yet engines give a clear advantage to white.
Can Engines properly assess openings?
Sort:
Aug 1, 2019
0
#63
drmrboss escribió:
nescitus wrote:
As an author of a chess engine (called Rodent, slightly above 3000 Elo on a CCRL scale) I must tell you that evaluations of strong engines are not objective - because they are not meant to. The goal of returning objective evaluation is different from the goal of returning evaluation that improves engine's play. One simple example: Stockfish has ridiculously high bonus for threatening to check enemy king. Please open Stockfish in a console mode (ie. open the engine exe file directly, so that a black window appears). In that window, please type the following line:
position startpos moves d2d4 g8f6 c2c4 g7g6 b1c3 d7d5 c4d5 f6d5 e2e4 d5c3 b2c3 c7c5
then press enter, then type "eval". With Stockfish 10 You will get score of over 1.5 pawn for white, meaning that Grunfeld defence is close to busted, which it obviously isn't. What the hell is going on?
I'll tell you. Stockfish gives insane bonus for the possibility of giving two checks to black king (Bb5 and Qa4). Why does it need such an absurd bonus? The answer is in the search. If Stockfish searches, say, 30 plies, then the chance of such eval being backpropagated right to the root of the search and returned as final positional evaluation are abysmally small, something like 1 to 100.000.000. What if they do get backpropagated to the root? It means that the engine cannot avoid these threats, or has to make big concessions in order to defuse them. Had these threats been avoidable (and in the Grunfeld line they will disappear once black castles short), search would avoid them and you would not see this high value as a final score. Final score returned by modern engines is a result of a struggle between two opposing forces: evaluation function trying to generate absurdly high score, and search trying to return the lowest common denominator, score that both black and white claim to be the best. Evaluation creates high values, search gravitates towards low scores, acceptable for both sides, unless these high scores are really unavoidable.
To summarize: engine evaluations aren't objective. They are numbers created by a function whose aim is to guide the search, not to inform user about true merits of the position. What's more, Stockfist with an evaluation that tries to be objective would be weaker by 200 to 400 Elo points.
As a member of talkchess for more than 5 years, I know most of the reasons where those evaluations come from. I dont get +1.5, instead +0.36. in grundfield. Here is analysis on my phone after 200 million nodes analysis( contempt 0, default is 24). with 5 men syzgy tablebase access.
That's a specific variation of the Grunfeld not the initial position of the Grunfeld, in any case, my computer gives it a 0.41 after searching 13384 million nodes. Contempt is at 24 for both sides and I'm using 6 men Syzygy tablebase.
The engine is "Stockfish 140719 64 BMI2".
Ah, I forgot to turn off analysis on my phone and this is the eval that I get after 4.6 billion nodes. 
As a side note. Today Stockfish is truly amazing, Stockfish already had 8 tablebase hits that means SF already know 8 endgame positions that ended as win or draw.
MARattigan wrote:
drmrboss wrote:
tmkroll wrote:
There's a line in the Traxler where a lot of people on this forum could play better than Stockfish. We were debating it here a few years back. Stockfish says White is winning until it sees black has a draw by repetition, then its evaluation goes to 0. Stockfish will take that draw but people who read that forum would castle Queenside as black. Eventually Stockfish sees black is better but it takes it a very long time. There's a line in the KG that at least five or six years back Fritz was similar, idk about now. Of course engines will never play into either of these lines if you don't make it do it because their opening books have been programmed by human players who have studied and know they are bad.
Which version of Stockfish you use and how many nodes SF searched for that position?
Do you mean analysis by this crappy chess.com server stockfish? In fact if SF search only a few hundreds nodes per move, her strength will be like 1200, but a few hundred million nodes per move will make her like 3500.
Show me the position, and I will analyse in 3 mins and show you how strong stockfish is. ( Let me see whether SF really played bad)
Try your SF out on this position. It's a well known win for White, but my SF can't play it for toffee.
As mentioned in a previous thread my SF evaluates the following position, which you yourself posted, as +6.40 no matter how long I leave it running, whereas Black has a very easy draw.
This one even gets +7.34.
Throw in an extra piece and it does no better. It evaluates this win for White at 0.16 at depth 30 both before and after it blows it on its second move.
In fact endgames tend to get more complicated the more men there are on the board. The maximum length forced mates with perfect play (no 50 move rule) are something like 28, 43, 127, 262 and 594 for 3,4,5,6 and 7 men respectively.
SF appears to play and evaluate 3 man positions perfectly (if you take +ve, 0 and -ve evaluations to mean wins draws and losses) but it already starts going awry with both evaluation and play with 4 pieces (it can't play KBNK accurately). With 5 or 6 pieces it starts losing half points.
Can you really believe that in spite of that, when it gets up to a 32 piece endgame, it starts to give accurate evaluations?
And what do the evaluations mean anyway - there's nothing in my SF documentation that tells me. Positions after all are either won for one side or drawn; there's nothing in between.
Do you use tablebase with your stockfish? Use 5-6 men tablebase in endgame positions.
And your posts are off topics, OP is about opening.
And also in any 7 men positions, you dont need to use engines.
And still in any endgame position, tell me a position that you followed stockfish moves and the result of the game changed from any of these goals
1. Win to draw, draw to lose etc!!
PawnstormPossie wrote:
Here's an opening position for your engines:
White to move.
How do you evaulate it?
How does your engine evaluate it?
What's the plan for both sides?
(White castled long 4 moves ago)
I know that game from Tcec final SF vs Leela.
here is leela evaluation on my phone. Black is slightly better.
Use of engine is like that,
1. e4 e5 I want to know whether king gambit is good? Then SF says bad, then it is very likely bad.Engines can filter 95-99% of bad opening choices from millions of opening variations.
Engines are better choices.
Play opening withiout engine assistant vs someone using engine assisant. You are likely lose majority of games.
Then why dont use your preparation with engines?
HolyCrusader5 wrote:
I think dmrboss believes that engines are superior to humans in openings because they are smarter. But hundreds of theoreticians using an engine analysis to assist them can play an opening better. Chess Engines like Leela can contribute to theory a lot, but they are not the basis of all theory. Engines that believe hypermodern openings are unsound believe that entirely because they like a space advantage. A "perfect engine" could possibly determine the soundness of openings with humans fact-checking. We should keep a close eye on AlphaZero in the case of which it evaluates openings. I doubt that this means that openings AlphaZero prefers to avoid are unsound.
I know how Leela Zero Evaluate opening position( I already mentioned it before).
Just to let you know Alpha Zero is an ancient 2 years old engine in 2017. Leela already surpassed Alpha Zero Level in 2018 December.
How do we know? Multiple testing of similar speed of Alpha Zero vs SF 8 results , and compared Leela Zero vs SF 8.( you can ask those results in Leela discord)
And now we have even much bigger size of Leela Neural Networks than Alpha Zero.. Leela training is extremely advancing project with average about 1 million games trained per day.
In short, this is the way to prepare with engines.
After 1. e4 e5, SF would like to play 2. Nf3 ( Then make your opening repertoire with Nf3).
You are not going to get wrong with Stockfish or Leela move. You are likely to win or you are playing the correct moves in majority of your game.
Is there anything better or stronger than engines moves in opening preparations? There can always be a minor exception only.
HolyCrusader5 wrote:
I am not arguing that they are sound. I see how you could claim these openings are unsound in general, but the Kings Indian is probably not unsound, yet engines give a clear advantage to white.
Sound means, good compensations/ counter play, and also 5 out of 10 possible moves are easily playable/findable.
Unsound means, not enough compensation, 8-10 out of moves are likely to lose. You have to find 1 out of 10 correct move and play precisely.
The question is " why would you like to intentionally put in crampy position, when there are other opening choices available".
For example. Since move 1, if you followed SF suggested moves, how many games did you get in trouble at move 10 , move 20 etc?
If someone is really serious in opening preparation , there are state of art computer prepared opening books.
1. Book by stockfish ( brainfish book)
2. Book by Leela ( Leela book).Leela is a better choice but the book size is small.
I would be really surprised if you can nick pick more than 1% of the flaw in those books. May be 0.1% flaw. Who knows
https://zipproth.de/Brainfish/download/
Aug 2, 2019
0
#71
drmrboss wrote:
MARattigan wrote:
drmrboss wrote:
tmkroll wrote:
There's a line in the Traxler where a lot of people on this forum could play better than Stockfish. We were debating it here a few years back. Stockfish says White is winning until it sees black has a draw by repetition, then its evaluation goes to 0. Stockfish will take that draw but people who read that forum would castle Queenside as black. Eventually Stockfish sees black is better but it takes it a very long time. There's a line in the KG that at least five or six years back Fritz was similar, idk about now. Of course engines will never play into either of these lines if you don't make it do it because their opening books have been programmed by human players who have studied and know they are bad.
Which version of Stockfish you use and how many nodes SF searched for that position?
Do you mean analysis by this crappy chess.com server stockfish? In fact if SF search only a few hundreds nodes per move, her strength will be like 1200, but a few hundred million nodes per move will make her like 3500.
Show me the position, and I will analyse in 3 mins and show you how strong stockfish is. ( Let me see whether SF really played bad)
Try your SF out on this position. It's a well known win for White, but my SF can't play it for toffee.
As mentioned in a previous thread my SF evaluates the following position, which you yourself posted, as +6.40 no matter how long I leave it running, whereas Black has a very easy draw.
This one even gets +7.34.
Throw in an extra piece and it does no better. It evaluates this win for White at 0.16 at depth 30 both before and after it blows it on its second move.
In fact endgames tend to get more complicated the more men there are on the board. The maximum length forced mates with perfect play (no 50 move rule) are something like 28, 43, 127, 262 and 594 for 3,4,5,6 and 7 men respectively.
SF appears to play and evaluate 3 man positions perfectly (if you take +ve, 0 and -ve evaluations to mean wins draws and losses) but it already starts going awry with both evaluation and play with 4 pieces (it can't play KBNK accurately). With 5 or 6 pieces it starts losing half points.
Can you really believe that in spite of that, when it gets up to a 32 piece endgame, it starts to give accurate evaluations?
And what do the evaluations mean anyway - there's nothing in my SF documentation that tells me. Positions after all are either won for one side or drawn; there's nothing in between.
Do you use tablebase with your stockfish? Use 5-6 men tablebase in endgame positions.
And your posts are off topics, OP is about opening.
And also in any 7 men positions, you dont need to use engines.
And still in any endgame position, tell me a position that you followed stockfish moves and the result of the game changed from any of these goals
1. Win to draw, draw to lose etc!!
You obviously missed the point of my post. It was about openings.
An opening is just a 32 piece endgame. SF unaided by any EGTB starts off perfect with a 3 piece endgame and gets progressively crappier as the number of pieces in the endgame increases. The common sense conclusion is that when it comes to a 32 piece endgame then without an opening book it will be playing really crap, so you should expect its evaluations to be crap in these endgames.
Similarly with a book if it evaluates positions out of its book.
Giving SF an EGTB would, of course have defeated the point of the exercise.
I gave an example Stockfish v Lomonosov in a 6 man endgame where SF takes two moves to turn a win into a draw. Here is another one against me, also two moves to turn a win into a draw in a 5 man endgame. Both played at G120+10.
It also managed to lose this win (on time) as White played with the same time controls (but I didn't save the game). You could try switching off its EGTB and playing the position yourself.
Tablebases are perfect chess( 100% correct) , it takes several years of computing power to build those. If you have little knowledge about science, there is no point comparing computing power of a few seconds/ minutes vs months to years of computing power, you will know which will better.
In other words, Stockfish will likely play those position properly( up to 95% correctly) if you give a few weeks/ months of position per move.
There are trillions of egtb positions, no human can play more better than an engine in randomly generated positions.
Aug 2, 2019
0
#73
drmrboss wrote:
Tabmebases are perfect chess, it takes several years of computing power to build those. If you have little knowledge about science, there is no point comparing computing power of a few seconds/ minutes vs months to years of computing power, you will know which will better.
In other words, Stockfish will likely play those position properly( up to 95% correctly) if you give a few weeks/ months of position per move.
But the Universe would end before it made a correct move in the opening (which as you pointed out is the topic).
No need. I know how those Tablebases are generated. Afaik, it took about 32 core pc for 1.5 years to generate 7 men TB( i can look at deatail, just guessing).
If you give chess engines close to approximate computing times, they can play similar moves.
Aug 2, 2019
0
#75
drmrboss wrote:
No need. I know how those Tablebases are generated. Afaik, it took about 32 core pc for 1.5 years to generate 7 men TB( i can look at deatail, just guessing).
If you give chess engines close to approximate computing times, they can play similar moves.
But see post #64 here https://www.chess.com/forum/view/endgames/k2n-vs-kp?page=4 . I don't think SF would do any better given years for its moves in the 5 man positions you refer to.
In fact it managed three moves without blowing the win in the first position shown at G20+3 but only two at G120+10.
MARattigan wrote:
drmrboss wrote:
No need. I know how those Tablebases are generated. Afaik, it took about 32 core pc for 1.5 years to generate 7 men TB( i can look at deatail, just guessing).
If you give chess engines close to approximate computing times, they can play similar moves.
But see post #64 here https://www.chess.com/forum/view/endgames/k2n-vs-kp?page=4 . I don't think SF would do any better given years for its moves in the 5 man positions you refer to.
In fact it managed 3 moves without blowing the win in the first position shown at G20+3 instead of two at G120+10.
As i told you several times, no engine or human is better than endgame table bases(they are 100% perfect mathematics) . It is a waste of time to tell you the same thing again and again!
Aug 2, 2019
0
#77
drmrboss wrote:
MARattigan wrote:
drmrboss wrote:
No need. I know how those Tablebases are generated. Afaik, it took about 32 core pc for 1.5 years to generate 7 men TB( i can look at deatail, just guessing).
If you give chess engines close to approximate computing times, they can play similar moves.
But see post #64 here https://www.chess.com/forum/view/endgames/k2n-vs-kp?page=4 . I don't think SF would do any better given years for its moves in the 5 man positions you refer to.
In fact it managed 3 moves without blowing the win in the first position shown at G20+3 instead of two at G120+10.
As i told you several times, no engine or human is better than endgame table bases(they are 100% perfect mathematics) . It is a waste of time to tell you the same thing again and again!
And I will tell you again, the topic is the opening. There are no 32 man EGTBs available and it would be pointless to extrapolate back from positions with few men using SF with an EGTB to SF in the opening with no EGTB.
It's not my fault if you repeatedly fail to grasp the argument.
By the way I can beat my computer program Wilhelm with attached Nalimov EGTBs from some positions in the tablebase where it fails to beat itself and I can win many positions quicker than the Syzygy tablebase.
And if you know how those tablebases are generated you'll know they give different sets of "optimal" moves in many positions. Can they both be perfect?
The 2 knight + king vs pawn + king endgame is a very rare endgame and the win is unusual. Therefore it is no surprise that engines may not be able to find the win. You can also construct plenty other positions with more pieces on the board where the engine doesn't find the win. However they do find good moves over 99% of the time. So you can definitely use them for openings.
Aug 2, 2019
0
#79
@Numquam
That they can be used for openings is of course true. The question is can they assess openings properly? A good move is only good until a refutation is found. SF's are clearly effective against other players. Whether they're accurate is a different question.
It's a strange coincidence that on endgames that I know how to play well with more than 3 men Stockfish appears to be slightly crappy to crappy, whereas on endgames I don't know how to play well it appears to be sh*t hot. I suspect its still as crappy. It's just that I'm even crappier.
I think the same is true of the human race in general for endgames with more men, including openings. It doesn't seem to be on the cards that SF would improve its accuracy in these, it's just that humans are worse.
As for KNNKP being rare, that's a function of how many people are acquainted with it. Troitsky could find only six recorded games when he analysed the ending, but a few years ago at a USCF tournament it was being played at two adjacent tables. Who could say what endgames would occur in perfect games?
Are the wins KNNKP so unusual, or just unusual compared with the endgames people generally feel comfortable with? I would have thought the win in the second diagram in post #82 wasn't much out of the ordinary. In any case SF already starts going wrong with KBNK (in terms of making the most accurate moves).
Engines can play openings accurately. However, humans should not say that the Alekhine or Pirc is unsound because an engine says so because that would be inaccurate. Overall, they are great at opening theory.
Let me tell you, alekhine and pircs are generally unsound openings, and most of the lines leads to busted for black. Some main lines need to play very accurately to get a draw.
As chess is a very drawish game with significant margin of error. You can play crappy opening and still get a draw. It is like fighting a boxing from the corner of the arena.
I dont see any benefits for playing those crappy opening. If white can play accurately, white has significantly more chances to win than other sound opening ( e.g berin , sicilian)
And I would be surprised if top 2700+ players still using those crappy opening today.