Chess will never be solved, here's why

Sort:
playerafar

I don't recall - but is the 50 move rule ignored in All of the tablebases?
Repetition?
I remember that en passant is factored somehow - but castling is skipped.
Hopefully they'll eventually fix that castling one - to ensure totally 'strong' solving of positions.

haiaku
Optimissed wrote:
haiaku wrote:

I have provided a paper that clearly contradicts the interpretation of "any opposition" you give.

But what tygxc is saying in #2986 is clearly correct. You should be relying on a "paper". If you can't explain in your own words and with your own ideas why your proposition is correct, you can't rely on a paper, since if you can't explain it in your own words then you probably wouldn't understand the paper.

I think I understand the paper and explained with simpler words its meaning.

tygxc wrote:

 If the strategy is proven against the stiffest opposition, then the weaker moves that do not even oppose to the draw become trivial.

I think it's not correct, because an opposition is considered "stiffest" according to a non exact result (not a tablebase hit, nor a 3-fold repetition or 50-moves rule).

Optimissed wrote:

It should be clear that <<We have to consider all moves for the max player>> is untrue. We have to consider all reasonable moves. We have to know what is "reasonable". If we don't know that or can't determine it, we're stuck once more in the strong solution

I cannot see why you consider checking all the possible replies for one player (the opponent) the same thing as cheking all the possible moves for both players. The first thing is required for a weak solution, the latter for a strong solution. There is quite a difference, indeed. For a weak solution you don't have to search all the legal positions, for a strong solution you do. To weakly solve checkers, around 10¹⁴ positions have been searched; to strongly solve checkers the entire search space has to be searched, around 5 ×10²⁰ positions.

Just don't fix on the meaning of "weak" and "strong" in common dictionaries. We could use "level 2" and "level 3", instead. Level 2 is enough for most players, because generally they don't want to know how to deal with every possible legal position; it is enough to know how to get the best from the initial position.

But if you consider only the "reasonable" moves for the opponent and you cannot prove beforehand that those reasonable moves cover all the possible exceptions, you cannot be sure that any "solution" (level 2 or level 3) is really a solution, there's no escape. I think that we will solve chess by searching all the possible opponent's moves before we solve (and I mean exactly) chess by your or @tygxc 's approach.

MARattigan
playerafar wrote:

I don't recall - but is the 50 move rule ignored in All of the tablebases?
Repetition?
I remember that en passant is factored somehow - but castling is skipped.
Hopefully they'll eventually fix that castling one - to ensure totally 'strong' solving of positions.

The fifty move rule is ignored in tablebases that are classified without a 50 on the end of the classification, so DTM (e.g Nalimov), DTC, DTZ ignore the 50 move rule and don't work for all positions under competition rules. DTZ50 (e.g. Syzygy) and DTM50 (some produced, but not generally available) work under both, but not for all positions under some previous FIDE variations of the 50 move rule.  

tygxc

#3001
"an opposition is considered "stiffest" according to a non exact result"
I also provided a paper saying that incorporating 'knowledge' is beneficial in solving games.
I also provided a paper with 'knowledge' in its title ranking 1 a4 not among the top moves.
I also consider it 'knowledge' that 1 e4 e5 2 Ba6 loses for white.
There is more 'knowledge' than a table base hit or a 3-fold repetition.
Forget the 50-moves rule: it does not happen with > 7 men.
I presented 2 examples of positions in the ICCF WC agreed to be draws with > 7 men: one with opposite colored bishops and one with rooks and pawns on one wing. That is applied 'knowledge' too.

playerafar

Opinion: (a method of many possible methods of going at the subject)
Start with positions defined by where each piece is on a square.  
Develop upper bounds on the numbers of such positions.  
Use computer enhancements to further develop (reduce) the upper bounds.
Factor in the tablebases.
After all the 'strong methods' have been exhausted to do so - (which does Not mean they're irrelevant - obviously - because something has been accomplished )  Mathematical reduction is part of the anatomy and phsyiology of 'solving'.  
then start considering alternatives to 'strong solving' - to be applied at that juncture.
Only factor in other information like whose move it is - en passant and other information - when timing is Ideal for each element and only after 'establishment' is complete for the stage concerned.
Horse before the cart in other words.  Walking before running.
That's apparently what they've done with castling in the tablebases.
They're not ready for it.  So they're doing something more basic first.

But the 'weakly solving' has to be factored in too.
A while back I mentioned computer evaluation numbers.
But @MARattigan rightly pointed out about the computers Pathetically unable to recognize drawn positions as draws.
Does that mean that sufficient/insufficient losing/drawing chances and the Whole Project are therefore Doomed?
No.  But its not good news for 'solved in 5 years'.   
😁😁😁

MARattigan
playerafar wrote:

Opinion: (a method of many possible methods of going at the subject)
Start with positions defined by where each piece is on a square.  
Develop upper bounds on the numbers of such positions.  
Use computer enhancements to further develop (reduce) the upper bounds.
Factor in the tablebases.
After all the 'strong methods' have been exhausted to do so - (which does Not mean they're irrelevant - obviously - because something has been accomplished )  Mathematical reduction is part of the anatomy and phsyiology of 'solving'.  
then start considering alternatives to 'strong solving' - to be applied at that juncture.
Only factor in other information like whose move it is - en passant and other information - when timing is Ideal for each element and only after 'establishment' is complete for the stage concerned.
Horse before the cart in other words.  Walking before running.
That's apparently what they've done with castling in the tablebases.
They're not ready for it.  So they're doing something more basic first.
...

But Tromp has already done that. He gives an upper bound on the number of legal positions on this page.

The only difference is that by "legal position" he means diagram + side to move + en passant possibility + castling rights that can be reached in a game by a series of legal moves.

That I think is the same definition that everyone here except yourself is using as the definition of "legal position" under basic rules chess.

It's true that many people use the term "position" to mean just the layout of the pieces on the board immediately after a move has been made, but I think everybody else would call that a diagram in this thread.

Tromp has gone further and estimated the actual number of legal positions here by sampling and testing for legality.

There doesn't seem to be a lot of point in reinventing the wheel.

playerafar

@MARattigan
I clicked on your link - 
Tromp admits he uses a 'more complex program'.
I chose to talk somewhat about how I arrived at a number in the 10 to the forties.
I don't understand why we have to use Tromp's definition all the time.
One could argue that Tromp's work is a 'paper' -
but now we see the other guy seeming to trash somebody's work of providing a paper to respond to somebody else's interpretation of a 'definition' by a notable 'ven den Herik' which could also be regarded as a 'paper'.
I've also seen 'Gurion number' and other numbers.
These references may be useful but we don't have to be bound by them all the time.
You asked me what I meant by 'positions' and I told you.
That can be seen as 'progress' rather than circles.
The discussion is different than it was a month ago.
But there's going to be references back to previous posts.
That's inevitable.
More surely than a bishop can do a 'diamond dance' around its seventh rank passed pawn ...  to drive off the defending bishop and promote for a win. 

MARattigan

@playerafar 

I don't think anybody has questioned the upper bounds on legal positions by Tromp or legal diagrams without promotions by Gurion (except that @tygxc has correctly pointed out, also accepted by all, that Gurion's figure relates to diagrams without excess promotions).

I would be awkward and assert that Tromp's figures relate only to current basic rules chess, not to chess under competition rules. 

When these things are generally agreed there seems to be no reason to discuss them further unless you think you have found a flaw.

You've said what you mean by "position", but it would be much more convenient if you could switch to the meaning in general use for this thread. It saves people from having to say anything about positions twice to accommodate two different views.

playerafar

@MARattigan
I've said what I meant by positions - in context.
The normal meaning of positions.  Arrangements of pieces on a board.
"I don't think anybody has questioned the upper bounds on legal positions by Tromp"
I don't think anybody has either.  I didn't.   I said 'if'.
But I didn't claim to the contrary either.
I question 'general use' though.
tygxc has had his own 'general use' and its been 'questioned' for about 3000 posts.  
Ideally - posts could be expressed so that newcomers to the forum could understand right away - instead of having to refer to a link concerning Tromp which then might lead to a link about ven den Herik and then yet another 'paper' where there's then a conversation 'going in circles' about interpretations of what ven den Herik is saying - where somebody then complains about 'circles'.
In other words - idea: - for the conversation to be about the subject - instead of about what somebody on an internet article might have meant by something.
Having said that - @Haiaku is still right to remind @tygxc about a paper that disagreed with ty's interpretation.  

Regarding 'competititon rules' I've questioned that terminology because somebody just entering the forum won't know what you mean by that.
So I've suggested - if you mean 50 move rule then say 50 move rule.
If you mean repetition of position say repetition.
Is a suggestion.  I won't get bent out of shape if you don't do it.

Regarding Gurion's number it seems somewhat arbitrary regarding 'excess'. 
Regarding who has been right and who has not ...
I have yet to see any incorrectness in @btickler 's posts or in @Haiaku's.
When you've been inaccurate (rarely) - well in the one instance I remember you simply deleted the post while admitting it was off.  That was recent.
Regarding Elroch's posts - I've had minor disagreement with some of them.
He rightfully keeps reminding @tygxc about mathematical standards of proof.
My 'minor disagreement' with that is that the term 'weakly' in relation to solving would seem to be in stark contrast to 'strongly' when you're talking about solving.  
When I mentioned that - ty did what he did before - he started talking about ven den Herik.  But with or without ven den Herik - that word 'weakly' is giving ty a lot of latitude.  So is 'heuristic'.  
I believe only one person is worried about 'leadership of the forum' and I believe its not you.  
Ironically - its @tygxc who has essentially led the forum.
Perhaps without the slightest intention of doing so.  happy.png
There's a lot of things that work that way ...
chase after something - it runs away.
But do better enough than 'chasing' - and that 'something' comes right along with you or to you.

playerafar

@MARattigan
I think your post #2948 was in response to my post # 2945 a few before.
This one:  https://www.chess.com/forum/view/general/chess-will-never-be-solved-heres-why?cid=68837185&page=148#comment-68837185

A main point of mine in that post (among other points) is the relevance of strong to weak.
Much earlier in the forum I was talking about 13 to the 64th and factorials about 32 squares having to be empty and two Kings only.
Those posts of mine got good reaction at the time.  
Strong math - to get strong results.  
When the discussion 'converges' on weak solving - which its been doing for about 3000 posts - then 'strong' is still relevant.
Including as part of the process before 'weak' and also to contrast with 'weak'.  So that its clearer what's being omitted.  
Terminology and worry about 'positions' was not intended.
There's quite a lot of controversy here about 'games' versus 'positions'.
'Positions' can be defined according to the context.
If its relevant in the context whose move it is - then that's included.
If its not then it isn't.
In some of the discussion we even had a suggestion that positions that would have to be illegal - like adjacent Kings - should be allowed in advance.  And then cleaned up later. 
I disagreed with that.
But maybe that's valuable in the programming aspects of it. 
Definitions according to context.  

By the way - when I went to the Tromp page you linked - there wasn't much there.  Something about 'complex formula'.  
Just now I went to a Wikipedia article about Claude Shannon (yes I believe Shannon was mentioned earlier) - he's got a neat formula that also ends up around 10 to the 44th.  
Saw this too:  But some of it is wrong:
https://chess.stackexchange.com/questions/5592/what-is-the-number-of-legal-positions-in-a-chess-game

playerafar

In spite of disapproval - @tygxc has led the forum it seems.
Doesn't let anything bother him.  
Regarding as to who here would qualify for the professional research projects on this subject ... four people here would seem to be up to it.
Regarding others ... well there'd have to be care regarding saying things like:
"lets take four candidate moves for each side on every ply - to do the weak solving".   That could disqualify. grin.png
As could 'no solution exists even though the number of positions is finite'

ChessSBM

I want to learn how to debate. Any advice?

MARattigan
playerafar wrote:

@MARattigan
I think your post #2948 was in response to my post # 2945 a few before.
This one:  https://www.chess.com/forum/view/general/chess-will-never-be-solved-heres-why?cid=68837185&page=148#comment-68837185
Looks like some posts have been deleted - I get an access violation on that link and #2948 is one of Elroch's.

A main point of mine in that post (among other points) is the relevance of strong to weak.
Much earlier in the forum I was talking about 13 to the 64th and factorials about 32 squares having to be empty and two Kings only.
Those posts of mine got good reaction at the time.  
Strong math - to get strong results.  
When the discussion 'converges' on weak solving - which its been doing for about 3000 posts - then 'strong' is still relevant.
Including as part of the process before 'weak' and also to contrast with 'weak'.  So that its clearer what's being omitted.  

The main point is a weak solution solves only the starting position but a strong solution solves all positions. But that is using the term "position" to mean all aspects of the situation that are relevant to possible contiuations - a diagram is only one aspect and doesn't determine the possible continuations.
Terminology and worry about 'positions' was not intended.
There's quite a lot of controversy here about 'games' versus 'positions'.
'Positions' can be defined according to the context.

That's true.

In the context of solving chess,  I would say "position" should be equivalent to a FEN less the ply count field if "chess" is taken to be the game under basic rules, but chess under competition rules is a different game with different solutions (weak or strong and possibly also ultra-weak) and, in the context of a forward search, "position" should be equivalent to a FEN less the ply count field + a list of positions with the same material that have previously occurred and how many times each has occurred. (The latter is also equivalent to a FEN with ply count 0 + the moves leading to the position, but in that case different specifications could represent the same position.)
If its relevant in the context whose move it is - then that's included.
If its not then it isn't.

Relevant in the context of solving either game.
In some of the discussion we even had a suggestion that positions that would have to be illegal - like adjacent Kings - should be allowed in advance.  And then cleaned up later. 
I disagreed with that.
But maybe that's valuable in the programming aspects of it. 

The point is that adjacent kings are just one case of the side not to move being in check. Those positions are all illegal and have to be removed anyway. (Note that I have to say "those positions" - the diagrams may or may not tell you if the situation is legal.)

Tromp deemed it easier or more efficient to leave that to the legality checker. The number of positions from which his sample was taken would be higher, but the sample would contain a higher fraction of illegal positions. That's OK for estimating the true number of legal positions; he's not trying to find an accurate upper bound.

(Though the legality checker couldn't be guaranteed to do a complete job, because the legality problem is not yet solved.)
Definitions according to context.  

By the way - when I went to the Tromp page you linked - there wasn't much there.  Something about 'complex formula'.  
Just now I went to a Wikipedia article about Claude Shannon (yes I believe Shannon was mentioned earlier) - he's got a neat formula that also ends up around 10 to the 44th.  
Saw this too:  But some of it is wrong:
https://chess.stackexchange.com/questions/5592/what-is-the-number-of-legal-positions-in-a-chess-game

I think Tromp's number is agreed to be the closest. This link and this link are the only links I provided for Tromp and both make the code available, so I don't see what more you would want. So far as I know the code for the improved upper bound of 10^45.888 has not been published, nor has the figure been firmly asserted by Tromp. 

Shannon's formula actually seems to give something between 10^42 and 10^43 but the formula itself could be distinctly more accurate. He only needed to show a 32 man tablebase was impracticable and it's already impracticable at 10^42.

 

Elroch

Legality checking is easy.

Also desperately impractical.

(Just do retrograde analysis and keep adding to a tablebase of legal positions that get to the given position until you find the initial position or you reach an iteration with no new positions).

playerafar
Elroch wrote:

Legality checking is easy.

Also desperately impractical.

(Just do retrograde analysis and keep adding to a tablebase of legal positions that get to the given position until you find the initial position or you reach an iteration with no new positions).

Good post.  Refers to Strength.  Mathematical objectivity.
But - (not really disagreement) - most of the 3000 posts have been about alternatives to such objectivity.  Because of the daunting nature of a 'solving strongly' project.
And - Strength is relevant though as well as part of 'weak' - therefore 'meaningful' to the discussion and can and should be included.  And it is.
And if and as the discussion heads for 4000 posts - strength and strong solving will probably get a minority share of that next 1000 posts.

playerafar

@MARattigan - I tried that link just now ... it worked fine.
This one:
https://www.chess.com/forum/view/general/chess-will-never-be-solved-heres-why?cid=68837185&page=148#comment-68837185
I guess you know you can click on the paperclip button on any post to get a copy of its direct link into clipboard.
Not just to the page ... but to the post itself.

Regarding Shannon's algebraic formula for upper bound on positions -
its very neat. 
My computer isn't set up to copy algebraic expressions though.
Even algebraic x - which looks different from multiplication x ...
finding a user-friendly copy paste of it on the net - it was a no go for now.
I found instances of that 'x' but they refused to copy !

Regarding highlighting a phrase in bold blue - which I like to do ...  its hit me that people might think its a Link !   Lol.
There's green - but there was some staff issues with green I think.
And with red - people See Red. 
Is that why Mar likes Orange ?? grin.png

 "he's not trying to find an accurate upper bound."
Refers to Tromp.
But an 'accurate' upper bound has some usefulness I think.
a diagram is only one aspect and doesn't determine the possible continuations.
Of course.  But positions are part of the process.
Everybody will have their own take on it though.  Or their own non-take.
My take:  What you call 'diagram' and I call 'position' ...  is primary.
The process not only starts with that - it continues with diagram/position continuing to be the central feature.  
The static arrangement of the pieces on the board - is what people see first.
Even by itself - it has much meaning. 
And the pieces can't move themselves.
Intrinsically - each position in and of itself is not a 'variable'.  
It is primary - and other aspects of it like whose move - en passant and others are dependent on it.  They are dependent variables.
@Elroch said something about that in his post about 'states'.
Such posts should not be ignored.
And - regarding discussing terminology - that can be part of the whole forum too.
Again - I think that will get a minority share.  Too.
If the longterm status quo holds -
then @tygxc 's posts and discussion/responses to his views on 'weak' will get the biggest share of the posts and probably a majority.
But nobody (except one - and its not I and not ty and not you) is worried about that.
How does that happen?
Because some of the people learned enough in maths and computers want to refute his posts.  They see an opportunity to do so. 
Simple as that.
Puts him in charge of the forum.  😁

MARattigan

Well Shannon's formula is obviously much less accurate than Tromp's approach. I think he was just looking for a first stab. Given that it put the possibility of storing a move for  each position completely out of the window there wouldn't have been much point in working to get a closer answer.

Tromp does both a pretty close upper bound and a good estimate of the actual number. One in each link.

I don't buy your idea of who has the move and whether castling rights or a possible en passant are available being dependent on the diagram. A diagram rarely determines any of those (though it can often rule out the last two).

I'd go along with what @Elroch's says about states here but I would also call a state a "position" (but see also my post #2999).

Terminology is, I suppose, here and there. The main thing is @tygxc can't offer to solve the competition rules game and base his search space size on basic rules positions. (Though he doesn't even do that; he ignores almost all of those as well.) 

playerafar


@ Mar
"I don't buy your idea of who has the move and whether castling rights or a possible en passant are available being dependent on the diagram"
Well you don't have to !
But should  we 'argue' about it ?  Lol hahahhaha...
If no position ('diagram') then what would en passant apply to?  What would who 'has the move' apply to?  Or castling or anything else?
Suggestion - those things haven't got much application without a position to be applied to.
But the position can exist and be seen and have some meaning and chess identity even without knowning who is to move ! 
In chess books - you'll often see positions where each side to move is discussed separately.    

Suggestion: the whole subject of various professional experts on the subject of chess being 'solved' and related - whose names and work are available on the internet ...  can be discussed as a subject.  Shannon - Tromp - Gurion - den Herik  ...  the whole bunch of them.  The tablebase experts too.  
Bottvinnik should be included also.
And the history of all of that.  With timelines. 
Or better - timeline.  Singular.

"(Though he doesn't even do that; he ignores almost all of those as well.)"
Which is why/how @tygxc gets control of the forum.
Apparently - without even intending to.  

playerafar
playerafar wrote:

Regarding computer chess -
one of the pioneers of that was Bottvinnik I believe.
And @tygxc used 'pioneer' to refer to himself didn't he?  
I've never clicked on his profile.  Will do that now.

From 'activity' there:
"Solve a few tactics puzzles each day to warm up. How much time do you need to solve a tactics puzzle? The time per move in a game should be more as in a game you do not know if there is a tactic so you have to assume there is one."

"There are an estimated 600,000,000 people that play chess. There are 1721 grandmasters.
Knowing opening lines is useless: tactics decide the game."

//////

I've been talking about those very things for months/years.
I basically agree with all of that.
Tactics puzzles are much more efficient in demonstating and teaching tactics insight than one's own games are.
Regarding studying tactics by looking at somebody else's games - the games of Tal may be best. 
Especially regarding the 'counter-intuitive'.   
But there are drawbacks in looking at entire master games.
Especially for players several classes below.

tygxc is a strong player too.
Regarding 'warming up' - one can also use blitz games for that too.
Which is better first - puzzles or blitz?
I don't know.  In my tournament days - I wasn't on the internet at all. 
No tactics puzzles.

haiaku

@Optimissed

I referenced specifically that paper because it uses just van den Herik's definition of weak solution that @tygxc uses, and it explains the meaning of "any opposition". AFAIK no one interprets the concept in another way. You can find other works stating that a strategy must achieve the game-theoretic value against any possible opponent's move. There is consensus about that and the processes involved. An engine can, as you say, alternatively test White's strategy against all the possible Black's moves and Black's strategy against all the possible White's move, and it still would search less nodes than a strong solution requires; in fact, an engine can apply e.g. alpha-beta pruning, but the leaf nodes must have an exact value (win, draw or loss), not an estimation, to get a real solution.

Optimissed wrote:

If there's any doubt, you test a move. Where there's no doubt, you prune the move. You reject it. Then you repeat the processes, in logical order.

Great. If you can provide a strategy that indeed excludes useless lines without any doubt before values are given through a TBH (or 3-fold repetition, etc.), we are all ears. @tygxc 's method does not guarantee that.

In particular, it's necessary to analyse situations where a "strategic turning point" is encountered, to find what they have in common, in a mathematical sense. That in itself is an enormous project and it hasn't really been started as yet. I've mentioned my son ... well ... he thinks it's impossible. That means, to him, that it couldn't happen within the next few generations because the mathematics for it doesn't exist.

And in fact I totally agree with your son. A breakthrough in technology, allowing dramatically faster computations, is likely to happen earlier.