Chess will never be solved, here's why

Sort:
Avatar of ChessSBM

I want to learn how to debate. Any advice?

Avatar of MARattigan
playerafar wrote:

@MARattigan
I think your post #2948 was in response to my post # 2945 a few before.
This one:  https://www.chess.com/forum/view/general/chess-will-never-be-solved-heres-why?cid=68837185&page=148#comment-68837185
Looks like some posts have been deleted - I get an access violation on that link and #2948 is one of Elroch's.

A main point of mine in that post (among other points) is the relevance of strong to weak.
Much earlier in the forum I was talking about 13 to the 64th and factorials about 32 squares having to be empty and two Kings only.
Those posts of mine got good reaction at the time.  
Strong math - to get strong results.  
When the discussion 'converges' on weak solving - which its been doing for about 3000 posts - then 'strong' is still relevant.
Including as part of the process before 'weak' and also to contrast with 'weak'.  So that its clearer what's being omitted.  

The main point is a weak solution solves only the starting position but a strong solution solves all positions. But that is using the term "position" to mean all aspects of the situation that are relevant to possible contiuations - a diagram is only one aspect and doesn't determine the possible continuations.
Terminology and worry about 'positions' was not intended.
There's quite a lot of controversy here about 'games' versus 'positions'.
'Positions' can be defined according to the context.

That's true.

In the context of solving chess,  I would say "position" should be equivalent to a FEN less the ply count field if "chess" is taken to be the game under basic rules, but chess under competition rules is a different game with different solutions (weak or strong and possibly also ultra-weak) and, in the context of a forward search, "position" should be equivalent to a FEN less the ply count field + a list of positions with the same material that have previously occurred and how many times each has occurred. (The latter is also equivalent to a FEN with ply count 0 + the moves leading to the position, but in that case different specifications could represent the same position.)
If its relevant in the context whose move it is - then that's included.
If its not then it isn't.

Relevant in the context of solving either game.
In some of the discussion we even had a suggestion that positions that would have to be illegal - like adjacent Kings - should be allowed in advance.  And then cleaned up later. 
I disagreed with that.
But maybe that's valuable in the programming aspects of it. 

The point is that adjacent kings are just one case of the side not to move being in check. Those positions are all illegal and have to be removed anyway. (Note that I have to say "those positions" - the diagrams may or may not tell you if the situation is legal.)

Tromp deemed it easier or more efficient to leave that to the legality checker. The number of positions from which his sample was taken would be higher, but the sample would contain a higher fraction of illegal positions. That's OK for estimating the true number of legal positions; he's not trying to find an accurate upper bound.

(Though the legality checker couldn't be guaranteed to do a complete job, because the legality problem is not yet solved.)
Definitions according to context.  

By the way - when I went to the Tromp page you linked - there wasn't much there.  Something about 'complex formula'.  
Just now I went to a Wikipedia article about Claude Shannon (yes I believe Shannon was mentioned earlier) - he's got a neat formula that also ends up around 10 to the 44th.  
Saw this too:  But some of it is wrong:
https://chess.stackexchange.com/questions/5592/what-is-the-number-of-legal-positions-in-a-chess-game

I think Tromp's number is agreed to be the closest. This link and this link are the only links I provided for Tromp and both make the code available, so I don't see what more you would want. So far as I know the code for the improved upper bound of 10^45.888 has not been published, nor has the figure been firmly asserted by Tromp. 

Shannon's formula actually seems to give something between 10^42 and 10^43 but the formula itself could be distinctly more accurate. He only needed to show a 32 man tablebase was impracticable and it's already impracticable at 10^42.

 

Avatar of Elroch

Legality checking is easy.

Also desperately impractical.

(Just do retrograde analysis and keep adding to a tablebase of legal positions that get to the given position until you find the initial position or you reach an iteration with no new positions).

Avatar of playerafar
Elroch wrote:

Legality checking is easy.

Also desperately impractical.

(Just do retrograde analysis and keep adding to a tablebase of legal positions that get to the given position until you find the initial position or you reach an iteration with no new positions).

Good post.  Refers to Strength.  Mathematical objectivity.
But - (not really disagreement) - most of the 3000 posts have been about alternatives to such objectivity.  Because of the daunting nature of a 'solving strongly' project.
And - Strength is relevant though as well as part of 'weak' - therefore 'meaningful' to the discussion and can and should be included.  And it is.
And if and as the discussion heads for 4000 posts - strength and strong solving will probably get a minority share of that next 1000 posts.

Avatar of playerafar

@MARattigan - I tried that link just now ... it worked fine.
This one:
https://www.chess.com/forum/view/general/chess-will-never-be-solved-heres-why?cid=68837185&page=148#comment-68837185
I guess you know you can click on the paperclip button on any post to get a copy of its direct link into clipboard.
Not just to the page ... but to the post itself.

Regarding Shannon's algebraic formula for upper bound on positions -
its very neat. 
My computer isn't set up to copy algebraic expressions though.
Even algebraic x - which looks different from multiplication x ...
finding a user-friendly copy paste of it on the net - it was a no go for now.
I found instances of that 'x' but they refused to copy !

Regarding highlighting a phrase in bold blue - which I like to do ...  its hit me that people might think its a Link !   Lol.
There's green - but there was some staff issues with green I think.
And with red - people See Red. 
Is that why Mar likes Orange ?? grin.png

 "he's not trying to find an accurate upper bound."
Refers to Tromp.
But an 'accurate' upper bound has some usefulness I think.
a diagram is only one aspect and doesn't determine the possible continuations.
Of course.  But positions are part of the process.
Everybody will have their own take on it though.  Or their own non-take.
My take:  What you call 'diagram' and I call 'position' ...  is primary.
The process not only starts with that - it continues with diagram/position continuing to be the central feature.  
The static arrangement of the pieces on the board - is what people see first.
Even by itself - it has much meaning. 
And the pieces can't move themselves.
Intrinsically - each position in and of itself is not a 'variable'.  
It is primary - and other aspects of it like whose move - en passant and others are dependent on it.  They are dependent variables.
@Elroch said something about that in his post about 'states'.
Such posts should not be ignored.
And - regarding discussing terminology - that can be part of the whole forum too.
Again - I think that will get a minority share.  Too.
If the longterm status quo holds -
then @tygxc 's posts and discussion/responses to his views on 'weak' will get the biggest share of the posts and probably a majority.
But nobody (except one - and its not I and not ty and not you) is worried about that.
How does that happen?
Because some of the people learned enough in maths and computers want to refute his posts.  They see an opportunity to do so. 
Simple as that.
Puts him in charge of the forum.  😁

Avatar of MARattigan

Well Shannon's formula is obviously much less accurate than Tromp's approach. I think he was just looking for a first stab. Given that it put the possibility of storing a move for  each position completely out of the window there wouldn't have been much point in working to get a closer answer.

Tromp does both a pretty close upper bound and a good estimate of the actual number. One in each link.

I don't buy your idea of who has the move and whether castling rights or a possible en passant are available being dependent on the diagram. A diagram rarely determines any of those (though it can often rule out the last two).

I'd go along with what @Elroch's says about states here but I would also call a state a "position" (but see also my post #2999).

Terminology is, I suppose, here and there. The main thing is @tygxc can't offer to solve the competition rules game and base his search space size on basic rules positions. (Though he doesn't even do that; he ignores almost all of those as well.) 

Avatar of playerafar


@ Mar
"I don't buy your idea of who has the move and whether castling rights or a possible en passant are available being dependent on the diagram"
Well you don't have to !
But should  we 'argue' about it ?  Lol hahahhaha...
If no position ('diagram') then what would en passant apply to?  What would who 'has the move' apply to?  Or castling or anything else?
Suggestion - those things haven't got much application without a position to be applied to.
But the position can exist and be seen and have some meaning and chess identity even without knowning who is to move ! 
In chess books - you'll often see positions where each side to move is discussed separately.    

Suggestion: the whole subject of various professional experts on the subject of chess being 'solved' and related - whose names and work are available on the internet ...  can be discussed as a subject.  Shannon - Tromp - Gurion - den Herik  ...  the whole bunch of them.  The tablebase experts too.  
Bottvinnik should be included also.
And the history of all of that.  With timelines. 
Or better - timeline.  Singular.

"(Though he doesn't even do that; he ignores almost all of those as well.)"
Which is why/how @tygxc gets control of the forum.
Apparently - without even intending to.  

Avatar of playerafar
playerafar wrote:

Regarding computer chess -
one of the pioneers of that was Bottvinnik I believe.
And @tygxc used 'pioneer' to refer to himself didn't he?  
I've never clicked on his profile.  Will do that now.

From 'activity' there:
"Solve a few tactics puzzles each day to warm up. How much time do you need to solve a tactics puzzle? The time per move in a game should be more as in a game you do not know if there is a tactic so you have to assume there is one."

"There are an estimated 600,000,000 people that play chess. There are 1721 grandmasters.
Knowing opening lines is useless: tactics decide the game."

//////

I've been talking about those very things for months/years.
I basically agree with all of that.
Tactics puzzles are much more efficient in demonstating and teaching tactics insight than one's own games are.
Regarding studying tactics by looking at somebody else's games - the games of Tal may be best. 
Especially regarding the 'counter-intuitive'.   
But there are drawbacks in looking at entire master games.
Especially for players several classes below.

tygxc is a strong player too.
Regarding 'warming up' - one can also use blitz games for that too.
Which is better first - puzzles or blitz?
I don't know.  In my tournament days - I wasn't on the internet at all. 
No tactics puzzles.

Avatar of haiaku

@Optimissed

I referenced specifically that paper because it uses just van den Herik's definition of weak solution that @tygxc uses, and it explains the meaning of "any opposition". AFAIK no one interprets the concept in another way. You can find other works stating that a strategy must achieve the game-theoretic value against any possible opponent's move. There is consensus about that and the processes involved. An engine can, as you say, alternatively test White's strategy against all the possible Black's moves and Black's strategy against all the possible White's move, and it still would search less nodes than a strong solution requires; in fact, an engine can apply e.g. alpha-beta pruning, but the leaf nodes must have an exact value (win, draw or loss), not an estimation, to get a real solution.

Optimissed wrote:

If there's any doubt, you test a move. Where there's no doubt, you prune the move. You reject it. Then you repeat the processes, in logical order.

Great. If you can provide a strategy that indeed excludes useless lines without any doubt before values are given through a TBH (or 3-fold repetition, etc.), we are all ears. @tygxc 's method does not guarantee that.

In particular, it's necessary to analyse situations where a "strategic turning point" is encountered, to find what they have in common, in a mathematical sense. That in itself is an enormous project and it hasn't really been started as yet. I've mentioned my son ... well ... he thinks it's impossible. That means, to him, that it couldn't happen within the next few generations because the mathematics for it doesn't exist.

And in fact I totally agree with your son. A breakthrough in technology, allowing dramatically faster computations, is likely to happen earlier.

Avatar of tygxc

#3022
"If you can provide a strategy that indeed excludes useless lines without any doubt before values are given through a TBH (or 3-fold repetition, etc.), we are all ears."

There is not any doubt that 1 e4 e5 2 Ba6? is inferior, even loses for white.
There is not any doubt that 1 a4 is not superior to 1 e4 or 1 d4.

Likewise there is not any doubt that the opposite color bishop's ending and the endgame KRB vs. KRB with equal pawns on one wing are indeed draws, like the ICCF grandmasters agreed on.

So human knowledge allows to exclude these lines.
The 'good assistants' should take care of this pruning in the opening and in the endgame.

Avatar of Elroch

Thinking like a chess player not a game theorist. Strong guesses are fine for playing chess, just not good enough for game theory.

Avatar of tygxc

#3024

"Strong guesses are fine for playing chess, just not good enough for game theory."
++ Those are not guesses, that is knowledge.

Per game theorist van den Herik: "Next to brute-force methods it is often beneficial to incorporate knowledge-based methods in game-solving programs." section 5.2 page 303.
https://reader.elsevier.com/reader/sd/pii/S0004370201001527?token=3AD53819FEF603D72358044D0591F98280DF6ECF036B0E0EC2522C4D0100A55938CB7420B4E331271A5B8D357C0E0DFC&originRegion=eu-west-1&originCreation=20220413122447 

1 a4 being inferior to 1 d4 and 1 e4 is in figure 31 on page 55.
The title of that game-theoretic scientific paper has 'knowledge' in it, not 'guesses'.
https://arxiv.org/pdf/2111.09259.pdf 

Avatar of haiaku
tygxc wrote:

There is not any doubt that 1 e4 e5 2 Ba6? is inferior, even loses for white.
There is not any doubt that 1 a4 is not superior to 1 e4 or 1 d4 [ . . . ]

You contradict yourself, because it's you (and only you actually) who distinguishes "known" from "proven". You stated that "proven" is 100% sure, thus "known" cannot be 100% sure. If you say "there is no doubt", then it's 100% sure and thus proven. But that's not. That's actually what you have to prove; if you use that as a premise, it's actually "begging the question".

Heuristics are not used the way you intend to. Your interpretation is as usual very different from the general consensus.

As for A0's knowledge, I can safely say that I know that most of the times being a rook up is enough to win, but that does not mean that being a rook up is always enough to win. A0 generalizes from a limited sample: that's in fact an overgeneralization not suited for a solution. What A0 knows are averages.

Avatar of Elroch
tygxc wrote:

#3024

"Strong guesses are fine for playing chess, just not good enough for game theory."
++ Those are not guesses, that is knowledge.

No, they are guesses based on the experience of play that cannot be proven correct. Your confidence as a chessplayer does not strengthen the reasoning: it would not help you get such a claim peer-reviewed.

Per game theorist van den Herik: "Next to brute-force methods it is often beneficial to incorporate knowledge-based methods in game-solving programs." section 5.2 page 303.
https://reader.elsevier.com/reader/sd/pii/S0004370201001527?token=3AD53819FEF603D72358044D0591F98280DF6ECF036B0E0EC2522C4D0100A55938CB7420B4E331271A5B8D357C0E0DFC&originRegion=eu-west-1&originCreation=20220413122447 

1 a4 being inferior to 1 d4 and 1 e4 is in figure 31 on page 55.
The title of that game-theoretic scientific paper has 'knowledge' in it, not 'guesses'.
https://arxiv.org/pdf/2111.09259.pdf 

You really don't understand. I have access to the full text of your first link, and it has nothing in it that supports your amateur viewpoint that heuristic knowledge can form part of a solution of a game. It also excludes chess - a category 4 game - by pointing out that "knowledge-based methods" are suited to when the decision complexity is low. The decision complexity of chess is high, typically 40 options per move.

 

Your last post is about AlphaZero's "chess knowledge".

AlphaZero is a CHESS PLAYER. It plays chess based on the application of inductive learning and incomplete sampling of the analysis tree. Its knowledge is excellent for playing chess but it gets some tactics wrong and  loses some games to Stockfish (more now to the NNUE version for sure). And you assume it is infallible and think its knowledge is 100% reliable!

AlphaZero's "knowledge" has no different status to that of a chess player and is not certain for reasons which are very similar.

Bear in mind I have many years of experience with machine learning, some of it very similar to that used in AlphaZero, and I am entirely aware of the character of such "knowledge". It is entirely distinct from the knowledge involved in solving a game.

Avatar of tygxc

#3026

"If you say "there is no doubt", then it's 100% sure and thus proven."
++ When something is known, then there is no doubt, it is 100% sure, but the proof may not be available.

We know 1 e4 e5 2 Ba6? loses for white, there is no doubt, but there is no proof that works out all possible variations to checkmate. Proving it poses no problem, but it is not worthwhile to burn computer time on it, precisely because the outcome is not in any doubt.

Likewise we know 1 a4 is not superior to 1 d4 or 1 e4. Capablanca said so: 1 e4 and 1 d4 accomplish more than 1 a4. AlphaZero concurs. There is no doubt about it. It is 100% sure, but there is no proof available. Proving it is problematic in practice and nearly as demanding as proving 1 d4 and 1 e4 draw. However, once 1 d4 and 1 e4 are proven draws, then by the same methodology 1 a4 can be proven to draw too.

Likewise we know the opposite color bishop endgame is a draw and the ICCF grandmasters were right to agree on a draw. There would be no point in playing it out till a 3-fold repetition or a table base draw. If one of both had thought he had even the smallest chance of winning, then he would not have offered or declined the draw and they would have continued for months until all doubt was removed for both. 

Likewise we know the KRBPPPP vs. KRNPPPP endgame with symmetrical pawns on 1 wing is a draw and the ICCF grandmasters were right to agree on a draw. There would be no point in playing it out till a 3-fold repetition or a table base draw. If one of both had thought he had even the smallest chance of winning, then he would not have offered or declined the draw and they would have continued for months until all doubt was removed for both. 

Avatar of Elroch
tygxc wrote:

#3026

"If you say "there is no doubt", then it's 100% sure and thus proven."
++ When something is known, then there is no doubt, it is 100% sure, but the proof may not be available.

We know 1 e4 e5 2 Ba6? loses for white, there is no doubt, but there is no proof that works out all possible variations to checkmate. Proving it poses no problem, but it is not worthwhile to burn computer time on it, precisely because the outcome is not in any doubt.

Likewise we know 1 a4 is not superior to 1 d4 or 1 e4. Capablanca said so: 1 e4 and 1 d4 accomplish more than 1 a4. AlphaZero concurs.

No. AlphaZero outputs probabilities. It is uncertain about the outcome of every move (except where complete analysis to a result is possible). The move that it assesses to have a higher probability of a good result, it views as better. It is sometimes wrong (to the extent that its decisions lose).

And Capablanca is less reliable than AlphaZero.

 

Avatar of tygxc

#3027

"you assume it is infallible and think its knowledge is 100% reliable!"
++ No not at all, in fact I calculated its error rate at 60 h/move to be 1 error in 10^5 positions and thus concluded 4 white candidate moves are required to achieve 1 error in 10^20 positions.

"AlphaZero's "knowledge" has no different status to that of a chess player"
++ My point is that there are other sources of knowledge than computation. Chess knowledge by great human players and AlphaZero is valuable in its own right and per van den Herik is beneficial to incorporate in game solving. It is no match report of AlphaZero vs. Stockfish, it is a scientific paper that uses AlphaZero to acquire chess knowledge.

Likewise checkers was solved with the program Chinook.

Likewise humans have proven a lot about endgames. That is valuable knowledge and it is beneficial to incorporate that by truncating unnecessary calculations once a known drawn endgame >7 men is reached.

Weakly solving chess is a formidable task in itself. So all available help in reducing the calculation effort is needed. A purist attitude 'if it is not calculated by the computer then it is not true' is detrimental.

 

Avatar of MARattigan
tygxc wrote:

There is not any doubt that 1 e4 e5 2 Ba6? is inferior, even loses for white. #3023

We know 1 e4 e5 2 Ba6? loses for white, there is no doubt, ... #3028

The more people question it, the surer you get.

Avatar of tygxc

#3029

"The move that it assesses to have a higher probability of a good result, it views as better."
++ Yes, that is correct, but the paper went further to evolve several versions of AlphaZero so as to acquire more and more dependable knowledge.

"It is sometimes wrong (to the extent that its decisions lose)."
++ Yes, at 60 h/move it errs in 1 out of 10^5 positions.

"And Capablanca is less reliable than AlphaZero."
++ No, that is like saying Euler is less reliable than Ramanujan.
Capablanca remarks that 1 e4 and 1 d4 occupy and control the center and open diagonals for the bishops and the queen and no other move accomplishes that much. All of that is true and verifyable and his conclusion is logical and true. AlphaZero independently came to the same conclusion and ranks 1 d4 and 1 e4 above all 18 other possibilities, just like Capablanca said. Personally I would consider chess weakly solved if 1 e4 and 1 d4 are proven draws. To be extra sure I would also include 1 c4 and 1 Nf3 (top 3 and top 4 per AlphaZero). The other 16 possibilities however do nothing but needlessly burn computer time. There is no doubt and it is 100% sure that none of them is any better than 1 e4 or 1 d4.

The best first heuristic was also used in solving Losing Chess: some lesser lines were added later.

Avatar of tygxc

#3031
"The more people question it, the surer you get."
++ It baffles me how you can question that. Are you a beginner who does not know that a piece down without any compensation loses the game?