Chess will never be solved, here's why

Sort:
Avatar of playerafar
haiaku wrote:

So many things...

@tygxc, you seem quite intelligent, and that's why I still don't understand whether you don't understand some crucial points, or you pretend to not understand. I would like to understand clearly: do you think you have proven that chess would be solved in 5 years, given adequate and reasonable resources, or you believe it, but you think it's a well educated opinion/theory?

That's also a good post.
'think you have proven' or just 'believe' ...
@tygxc doesn't seem to have made the mistake of the former ... but on the other hand it seems his position is more than just 'believe'.
Maybe its a grey area - somewhere 'between'.   So many things are.
In such situations it can be righteous to insist on not being pinned down to digital A or B ...

I suggest not using the term 'excess' in regards to chess pawn promotions and underpromotions.
Maybe Tromp used/uses it but that doesn't make it a valid term.
It is a word being used in a circular way -
just arbitrating what is 'excess' is like saying ...
'well we'll just say what is 'excess' - and then decide that excess is anything the computers don't have time to solve - and then we've proven that it can be done in five years if the money's right'
If there's only a month to do the project - and there's already money but its a very limited budget ...  then one could decide the 'excess' is whatever the computers don't have time for !  happy.png

Much better use of the word 'excess' would be something like:
"The computer program pre-excluded out all the adjacent Kings/both Kings in check/11 or more of any piece onboard/33 or more pieces on the board and similiar - before starting its 'deeper sorting' because all of those illegal positions are 'excess' and we don't want to bother with them. "
Although somewhat incomplete (doesn't say whether computer kept numerical counts of such 'excess' illegal positions) that would be Valid Use of the word 'excess'.
Becomes 'excess' only After being proven to be so !   

Contrast:  What computers can pre-exclude as illegal by Logic and what can only be found illegal by examination.  (too time consuming?  but its double edged - finding a position illegal by examination - is very final and much faster than 'solving.
Issue:  computer 'generation' of positions to be analyzed -
the computer eliminates illegal positions by simply not generating them in the first place !
That ought to be Very Very Fast !   Lol !  happy.pnggrin.png

And very easy !   No adjacent Kings.  Not even considered - so it doesn't take up any computer space nor time at all !

But not so easy with some other illegal positions ...
like for example - a King in Triple Check !  
going to be very 'consuming' for such positions to have to be 'detected' and only then excluded.

Avatar of Elroch

Not really. Theoretical values (and moves to mate) are generated for a tablebase by retrograde analysis using legal moves. Moves are generated for an opening book by forward analysis using legal moves.  It is perfectly reasonable for a tablebase to contain some orphan positions that turn out not to be reachable from the standard starting position. These can be removed if desired. Otherwise they don't have any influence.

Avatar of tygxc

#1561
The Gourion count 10^37 does not exclude promotions but excludes excess promotions.
Excess promotions are promotions to a piece not yet captured, i.e. promotions that need to borrow the promoted piece from another box of 32 chess men.

Avatar of playerafar
Elroch wrote:

Not really. Theoretical values (and moves to mate) are generated for a tablebase by retrograde analysis using legal moves. Moves are generated for an opening book by forward analysis using legal moves.  It is perfectly reasonable for a tablebase to contain some orphan positions that turn out not to be reachable from the standard starting position. These can be removed if desired. Otherwise they don't have any influence.

Hi @Elroch
You're saying that the 7-piece tablebases were generated from the opening position?
I don't think you're saying that.
By 'retrograde analysis' you're suggesting that the tablebase positions were generated by adding 'reverse legal captures' to lesser-material positions ?
Have you noted that that will not generate all legal positions of the higher class of positions?
I noted it here several weeks ago.
You cannot get to a lesser material position without a capture available -
so if no capture available - then there has to be play until it is ... 
and even there - it would only lead to a position with that particular piece added by 'reverse'.
If you're saying that the only good way to generate legal positions is by adding moves ...
well the figures we've been seeing from @tygxc -  so many of them Not generated that way.  happy.png

Avatar of Optimissed

playerafar is legislating on who is making good posts and on which ones they are. happy.png 

Avatar of playerafar

@tygxc
"Excess promotions are promotions to a piece not yet captured,"
That's Not Good.

One doesn't have to capture a knight to promote to a knight.
When players Queen promote to a second Queen - they sometimes turn a rook upside down that their opponent has captured ...
But that should not influence the project at all.

Avatar of playerafar
Elroch wrote:

Not really. Theoretical values (and moves to mate) are generated for a tablebase by retrograde analysis using legal moves. Moves are generated for an opening book by forward analysis using legal moves.  It is perfectly reasonable for a tablebase to contain some orphan positions that turn out not to be reachable from the standard starting position. These can be removed if desired. Otherwise they don't have any influence.

"Not really. Theoretical values (and moves to mate) are generated for a tablebase by retrograde analysis using legal moves"
@Elroch 
Do you see that such process would be incomplete?  It would fail to generate many legal positions that do in fact exist and could arise in a game.
In fact - it would fail to generate a majority of positions.
The 'doors' between positions of differing material - are only available in some of the positions.  A minority of them.

In considering how to 'not generate' positions with triple check in them -
that's not going to work if the computer is already considering them anyway.
Which apparently has been going on in some of the projects.

Suggestion:  When adding pieces to two Kings - its not necessary to do so by reverse legal capture. 
The computer should simply avoid both Kings in check - avoid triple check and higher. 
What else does it have to avoid?  
Counter:  if you generate positions that way - then how do you use the previous fewer-pieces tablebase to solve the next higher class of positions?
Answer:  You still can.  But the computer needs to show that you could or would get there - before trying to use the previous tablebase.  Otherwise they have to be solved on their own merits.

But its reversed with the so-called 'excess underpromotions' - where the use of the term 'excess' in the context concerned is now further shown to be invalid.   
You don't need to capture a piece nor to lose a piece to underpromote to that piece.
Nor to promote to two queens or whatever.

Avatar of Elroch
playerafar wrote:
Elroch wrote:

Not really. Theoretical values (and moves to mate) are generated for a tablebase by retrograde analysis using legal moves. Moves are generated for an opening book by forward analysis using legal moves.  It is perfectly reasonable for a tablebase to contain some orphan positions that turn out not to be reachable from the standard starting position. These can be removed if desired. Otherwise they don't have any influence.

"Not really. Theoretical values (and moves to mate) are generated for a tablebase by retrograde analysis using legal moves"
@Elroch 
Do you see that such process would be incomplete?  It would fail to generate many legal positions that do in fact exist and could arise in a game.

No.

In the generation of a tablebase, you can start with all the possible positions, including impossible ones (it would be more efficient to use some simple rules to avoid classes that are obviously impossible (such as white pawns on a2 to a7), but only a limited amount of this refinement helps much.

With the (reduced) set of positions that at least include all the legal ones in your class (eg all with 8 pieces), the first pass is to go through them and see which are checkmates on the board. You don't have to worry about which could be reached as mates - that will become obvious next pass.

The second pass adds the values for all positions that can get to a mate in one move. It can also provide values for stalemate positions (all those that have no legal move but are not mated). This will save a little time.  If you like you can also prune all the positions that were neither already resolved (mate or stalemate) and had no positions with a legal move to get to them.

As this continues, each pass you can add winning positions - those that can reach a winning position with a move; drawing positions - those that can reach a drawing position and have no unevaluated moves; and losing positions - those that have no move that does not reach a winning position for the opponent.

Anyhow, there is no way you can have a position in the original set that is not there!

Identifying the draws may not be as difficult as it seemed at first. A cunning way to achieve this is to keep doing passes to find more winning and losing positions for each side and, when you do a pass and find exactly zero new winning or losing positions, it must be that all the positions that have not been evaluated as a win or a loss are drawn!

The reason this is so is that for any position that has a definite result, there is a sequence of perfect play that gets there in the minimum number of moves, with the second player trying to survive as long as possible. The positions that are resolved in n half moves with perfect play are evaluated at pass n.  If there was such a position that has not been evaluated at pass n, it must take more than n half moves and in the line of perfect play there must be other positions that take every intermediate number of moves to be evaluated.

All clear?

[Checking, I find Syzygy doesn't even bother to omit blatantly impossible positions of the type I mentioned at the start. For example, here is one:

 

Avatar of playerafar

@Elroch - although your arguments there have considerable logic - I don't think you understood about the 'doors' between classes of positions. 
Or maybe you did.
But that's okay either way.

Avatar of playerafar


Regarding a good or rather better definition of 'excess promotions' - 
maybe there's a way to show that many promotion scenarios are logically impossible.

In order for a pawn to promote or underpromote (which I'll abbreviate to  just 'promote') - there has to be no pawn of the other side's in its way.
So therefore it can be concluded (in advance) that for just one promotion that has occurred in any position - there has to be have been at least one capture of a pawn or by a pawn. 
Of or by the other side's pawns in both cases.
But - that doesn't mean one has to go by captures or moves.
Instead - go by number of pieces on the board. 
Much much better.

This would mean its impossible to have any promotions in any position with 32 pieces.  Would it not?

So for one side to have two queens - there has to be a maximum of 31 pieces on board and a max of fifteen pieces for the other side.
Same with either side having two bishops moving on the same color squares.
(would also be covered by '3 bishops')
Or 3 knights or 3 rooks.

Such logic would be easy to pre-program into the computers would it not?
Idea:  don't even generate positions that are logically impossible in the first place ...

Extending this - would it ever be logically possible for all 16 pawns on board to promote?
Each side can only make seven captures of pieces that are not pawns.  
Is there a way to do it?
You can make 4 promotions by two captures.  
Black b-pawn takes a piece on a6 like in @tygxc opening example or in any old way - or takes onto any a-file square it can.
Plus white a-pawn takes on b3 or on any b-file square it can.
The resulting four doubled a and b pawns could then all promote on the a and b files. 
Extending that to the next two files and then across the board - you're only going to need 4 captures each to promote all pawns ... ?
So all 16 pawns could be promoted ...
but you're going to have a maximum of 24 pieces in that case.
A lot of crazy extra positions - but at least its a legitimate way to exclude a lot of positions.  Like maybe ten or fifteen powers of ten.
That's the kinda cutdown of many needed to make the project feasible.

Avatar of tygxc

#1574
It is necessary to distinguish between two different questions.
1) Assessing the feasibility of weakly or strongly solving chess.
2) Actually doing it: how to do it?
For 1) we know that chess can be solved weakly as well as strongly as each game ends in a finite number of moves and it can end only in a draw, a win, or a loss, so all positions including the initial position are either a draw, a win, or a loss. The only remaining question is how much time and storage is needed.
To strongly solve chess i.e. a 32-men table base needs 10^36 positions, that is 10^27 seconds and at least 10^36 bits of storage, which is unfeasible. 7-men table bases were generated from 6 men, those from 5 men, those from 4 men, those from 3 men, those from 2 bare kings. 
To weakly solve chess the most promising way is like Sveshnikov said from the opening towards the table base. That is how checkers and losing chess were solved.  Weakly solving chess requires far less positions than strongly solving chess. I arrived at 10^17 positions, which cloud engines can manage in 5 years, like Sveshnikov said.

Avatar of playerafar


'Weakly' solving chess has been going on for centuries.
Its whatever anybody wants it to be.
I think the game could be 'truly' solved eventually - (not 'weak' solving)
maybe even this century.
But:
1) got to take the sting out of the initial numbers by not even generating positions that couldn't be legal anyway.
2) besides pre-excluding logically impossible positions in advance -
the job needs to be done in phases.
Could the computers do auxiliary tasks beforehand that players might find useful even in just some very indirect way?
Computers did solve Queen against two bishops. 
It was found to be winnable.
Perhaps absolutely All five-piece positions were Absolutely solved.
Including with all castling/en passant stuff taken into account.
But have they solved for wins taking 100,000 moves?
How many moves (or games) would be possible with just five pieces?
Maybe its more than the number of electrons in ... your cup of coffee?

The researchers would need to get a handle on easier jobs.
But they probably are anyway. 
A lot of programmers like chess.  To start with.
The research is probably being used to help with chess software and programming. 
And with software and hardware for big business and even for government and the police and military.  

Avatar of DiogenesDue

As things sit now, even weakly solving chess requires 10^40+ positions (10^44.5 being the current best estimate) to actually achieve a proof that would be solid enough to withstand scrutiny.  Everything else is just conjecture.  Saying that excess promotions and "nonsensical" positions can be discarded is a shortcut that will not stand up to said scrutiny wink.png...and even if you created a set of criteria that would be acceptable for eliminating such positions, evaluating all those positions to determine if they should be "included" for deeper/full evaluation also requires timeframes with copious orders of magnitudes.  So it's not like that required effort just vanishes into thin air.

Avatar of playerafar


"Saying that excess promotions and "nonsensical" positions can be discarded is a shortcut that will not stand up to said scrutiny"
I agree.
But positions that would be pre-impossible because of chess rules and logic -
could be discarded in advance without even being generated in the first place.

Triple check is impossible under chess rules by the way ...
today - something not so far from it - but 'legal' - occurred to me. 
I posted it elsewhere.
Its discovered 'disappearance' double check by en passant.
Like a black king at e6 - white pawn on e5 ... w Queen goes Qg4ch  
black f7 pawn slides up to f5 to block - white pawn takes EP ... goes to f6.
Discovering white's rook at e1 for check plus 'disappearance' at f5 for renewal of Qcheck.  In a way - four pieces participate in the check happy.png

Avatar of CaydenZook

Chess ultimately cannot be solved, but computers are learning more every second, so it might in the future.

Avatar of Elroch
playerafar wrote:

@Elroch - although your arguments there have considerable logic - I don't think you understood about the 'doors' between classes of positions. 
Or maybe you did.
But that's okay either way.

There is fundamentally nothing but positions and legal moves between them. What a tablebase does is to infer the consequence of perfect play. But you start with ALL the positions. Eg if you wanted to make a tablebase of all positions with two kings and a queen, you'd first start with literally all positions with those pieces on the board: that's 64 * 63 * 62 positions. Then you start the process above. The only additional information is the 2 piece tablebase (which says any position with 2 pieces is a draw!).  For larger tablebases, they always refer to smaller tablebases as needed, as those are already complete (and much smaller).

Avatar of playerafar
Elroch wrote:
playerafar wrote:

@Elroch - although your arguments there have considerable logic - I don't think you understood about the 'doors' between classes of positions. 
Or maybe you did.
But that's okay either way.

There is fundamentally nothing but positions and legal moves between them. What a tablebase does is to infer the consequence of perfect play. But you start with ALL the positions. Eg if you wanted to make a tablebase of all positions with two kings and a queen, you'd first start with literally all positions with those pieces on the board: that's 64 * 63 * 62 positions. Then you start the process above. The only additional information is the 2 piece tablebase (which says any position with 2 pieces is a draw!).  For larger tablebases, they always refer to smaller tablebases as needed, as those are already complete (and much smaller).

I disagree with at least three of the points there.
Its quite clear there are many positions that temporarily have no available captures.
Therefore you can't directly transition to a lesser material situation in one move.
I'm sure that point is obvious to you - but I don't think you've considered the implications and related subjects and issues this generates in turn.

Plus you don't have to start 3 piece positions with literally all to start.
It doesn't follow.  Most of the illegals don't need to be generated at all at any point in the process.  Why would one 'have to' ?  It doesn't follow.

Plus - larger tablebases don't 'always' refer to lesser - and wouldn't - in many cases.
not if they end or are going to end in checkmate or stalemate or hopeless drawn positions.
But also not if you've got to 'solve' in situations with a gigantic number of non-capture moves - and maybe couldn't even get to particular 'lesser'.  

There then comes in a particular aspect ...
'only by obviously weaker play' is there a transition.
GM's often agree to a draw where there appears to still be a lot of opportunities to make mistakes.   
Perhaps that could be googled.
Famous instances of blundering draw offers or acceptances.
There are probably famous Resigns too.  That were blunders.
Some players might regard resign/draw offer/draw acceptance ...  as 'moves'.  They're certainly 'options'.  
Is there a way to precisely define as to when a draw offer/acceptance is a 'mistake' ?  Some would say - its only a mistake if whoever had a forced win.
I don't think so.
The game isn't just about 'mistakes'.
Its about 'more mistakes' too.

Avatar of Elroch
playerafar wrote:
Elroch wrote:
playerafar wrote:

@Elroch - although your arguments there have considerable logic - I don't think you understood about the 'doors' between classes of positions. 
Or maybe you did.
But that's okay either way.

There is fundamentally nothing but positions and legal moves between them. What a tablebase does is to infer the consequence of perfect play. But you start with ALL the positions. Eg if you wanted to make a tablebase of all positions with two kings and a queen, you'd first start with literally all positions with those pieces on the board: that's 64 * 63 * 62 positions. Then you start the process above. The only additional information is the 2 piece tablebase (which says any position with 2 pieces is a draw!).  For larger tablebases, they always refer to smaller tablebases as needed, as those are already complete (and much smaller).

I disagree with at least three of the points there.
Its quite clear there are many positions that temporarily have no available captures.
Therefore you can't directly transition to a lesser material situation in one move.

Nothing I said could be feasibly misinterpreted as claiming that. Yet you managed to.
I'm sure that point is obvious to you

Yes. And has no role in this discussion.

- but I don't think you've considered the implications and related subjects and issues this generates in turn.

I understand this well enough to implement it.

Plus you don't have to start 3 piece positions with literally all to start.

This was an illustrative example. For solving chess you might use a 16 piece tablebase or even larger, by analogy with checkers.
It doesn't follow.  Most of the illegals don't need to be generated at all at any point in the process.  Why would one 'have to' ?  It doesn't follow.

You don't seem to understand the algorithm. That was the purpose of the toy example of a small tablebase. Try studying it.

Plus - larger tablebases don't 'always' refer to lesser - and wouldn't - in many cases.
not if they end or are going to end in checkmate or stalemate or hopeless drawn positions.

I explained the rather neat way that draws can be determined in a tablebase. Try reading it again until you understand it.
But also not if you've got to 'solve' in situations with a gigantic number of non-capture moves - and maybe couldn't even get to particular 'lesser'. 

The algorithm is sound. Read it again.  For the 7 piece tablebase, the drawing positions can all be identified at pass 1098.

There then comes in a particular aspect ...
'only by obviously weaker play' is there a transition.
GM's often agree to a draw where there appears to still be a lot of opportunities to make mistakes.   
Perhaps that could be googled.
Famous instances of blundering draw offers or acceptances.
There are probably famous Resigns too.  That were blunders.
Some players might regard resign/draw offer/draw acceptance ...  as 'moves'.  They're certainly 'options'.  
Is there a way to precisely define as to when a draw offer/acceptance is a 'mistake' ?  Some would say - its only a mistake if whoever had a forced win.
I don't think so.
The game isn't just about 'mistakes'.
Its about 'more mistakes' too.

Tablebases (and solving chess) are about perfect play by both sides. The rest is inferior (and slower to win or faster to lose) sidelines.

 

Avatar of playerafar

Yes. And has no role in this discussion.
I disagree - it does have a role.
And 'the algorithm' however 'neat' it is - fails as an argument.
Because 'the algorithm' is obviously insufficient for the task.
It needs improvement - no matter how much its explained as to 'how it works'.
Considering positions even for less than a nanosecond -
that could be pre-excluded anyway beforehand by pure logic ...
would hardly be best.
Because with the numbers involved - wastes of nanoseconds get added up far too much ...
you end up with time in years needed that is much much more than Jeff Bezos will ever have in dollars in his bank accounts or stock worth.

Plus such wastes of nanoseconds - multiply more and more as more pieces are added. 

Got to add something else too ...
good chess is all about knowing about mistakes and players making them - and making 'more mistakes' too.  And not making them.
And finding or exploiting mistakes.  Or causing them.
But its also about how to play when somebody has not made a mistake too.

Avatar of Elroch
playerafar wrote:

Yes. And has no role in this discussion.
I disagree - it does have a role.
And 'the algorithm' however 'neat' it is - fails as an argument.
Because 'the algorithm' is obviously insufficient for the task.

It is sufficient for the task of generating a 32-piece tablebase, which contains all possible knowledge about chess. The only barrier is practical - the resources are too large. It is also sufficient for the task of generating a smaller tablebase as part of solving chess (by analogy with the solution of checkers)
It needs improvement - no matter how much its explained as to 'how it works'.

Your understanding requires improvement, not the algorithm! (Apart from tweaks for efficiency)
Considering positions even for less than a nanosecond -
that could be pre-excluded anyway beforehand by pure logic ...
would hardly be best.

Correct. But this is an efficiency issue. As I have pointed out, the largest tablebase generated  - Syzygy - contains large numbers of impossible positions, because this was not an undue burden and does no harm.
Because with the numbers involved - wastes of nanoseconds get added up far too much ...
you end up with time in years needed that is much much more than Jeff Bezos will ever have in dollars in his bank accounts or stock worth.

See above relating to Syzygy!  The very first step involves enumerating all possible positions. In doing so you can discard positions that are absolutely illegal (both sides in check) but there is limited return for more effort. Positions that can never be reached simply don't play a role later on, so cost no time. So all they are is a few bits of storage.

Plus such wastes of nanoseconds - multiply more and more as more pieces are added. 

Got to add something else too ...
good chess is all about knowing about mistakes and players making them - and making 'more mistakes' too.  And not making them.

The only part that plays a role in solving chess is avoiding mistakes. For example this is easy in a tablebase.
And finding or exploiting mistakes.  Or causing them.

Exploiting mistakes is no different to not making them. If there is a move that "exploits" a mistake, it means playing a move that fails to exploit it is a mistake.

For example, if you are in a winning position, a good move is one that wins (ideally as quickly as possible against more resilient defense). It doesn't matter if the position has been reached by a mistake (eg the opponent was in a drawing position and played an error, a losing move) or whether it was already winning the move before (the opponent was already losing so all his moves left the result the same). The same position could be reached in both ways. 
But its also about how to play when somebody has not made a mistake too.

It's always about playing a move that is optimal. Firstly that it achieves the best possible result against best possible play, secondly (to varying extent, depending on the context) that it wins most quickly (if the position is winning) or loses most slowly (if the position is losing), depending on what the best possible result against best defense is. [Of course when it's a draw, preserving the draw is all that matters].