The most beautiful puzzle that my engine can never solve

I muddied the waters unintentionally by saying "Stockfish" when I really meant "engines" generally or "computers" (although some of this discussion about Stockfish specifically has been interesting).

The question I intended to ask is whether there are positions computers can't solve, and I think the answer is no. Engines would include programs that search the entire width of the game tree, and such engines should be able to solve any problem within their horizon which admits to a definite outcome (1-0, 0-1, 1/2). Such engines should even be able to solve chess itself, albeit on impractical timescales.

Precisely! As a deterministic, 2-player, alternate move game, the solving algorithm can be written on a single sheet of paper (skipping the chess rules).

This is however not true for chess compositions which have a number of fundamental unknowns. Fundamental because the WFCC has specifically blocked the solvers access to that information. It concerns castling right, e.p. right, all the past repeatable positions, the state of the 50-move counter and who is on move. That's the reason why FENs cannot represent compositions - they pretend to know things which are untrue or undefined. To satisfy the unknowns one needs a mathematical model - or, as it is, a number of mathematical models! That's why the engines cannot solve a number of my compositions. Btw, the models do not only apply to retrograde problems, they are in the orthodox zone as well. For instance, If you could prove for any of the preceding diagrams that no proof game exists for it with "white on move" you are to assume that "black is on move" (irrespective of the FEN or anything the puzzle interface would force on you).

Only moves that are proven dominated can be truly pruned. Other moves can merely be given low priority.

Do you REALLY think that the Stockfish algorithms (infinite depth) stop analyzing a problem before they have found the solution (because, according to you, it has been pruned)?

If so it should be possible to find a mate in 2 or 3 it can't solve - just design it so the solution move is pruned. Moreover, it must stop analysing without finding the solution.

I assert neither is so.

Now engage your brain. If the analysis tree keeps growing at a finite rate it has to eventually solve the initial position (given unlimited room).

Your claim implies that the Stockfish analysis tree stops growing when there are moves that could logically change the evaluation and which only have a bare evaluation. Why??

No, it gives them low priority and gets round to them eventually. The key fact is that a bare evaluation does not imply a branch is inferior and can be ignored - it merely gives an indication of how likely it is to be irrelevant.

[There are some variations that can be permanently pruned - those that can be proven to not be superior to those that have been analysed. This requires at least exhaustive analysis of all legal moves by one side. For example, this is absolute minimum to prove a mate in N - every position reached by a legal move for black needs to be dealt with. That means both:

(1) analysing at least one move for white in such positions

and

(2) analysing all positions reachable by a legal move for black from a position white reaches by such moves.

Stockfish' relatively poor efficiency for solving puzzles is a matter of prioritisation. It can be busy deeply analysing more plausible looking moves when an implausible one is key. But if it finishes the plausible looking ones, Sherlock Holmes maxim comes into play.

Answer the following question if you want to have any claim to honesty.

Does the Stockfish algorithm stop analysing when it hasn't found a solution and still has a completely unanalysed candidate move?

Do you understand the point that an algorithm will either solve a problem or stop analysing and leave some node WITH JUST A BARE EVALUATION? Perhaps you have some confused belief that such a node can be pruned? Definitely false unless the problem is already solved.

If you do, why do you believe that Stockfish would fail to solve a mate in 2 by only analysing a subset of the legal first moves? Exactly the same can apply at any depth in an analysis tree, with the same conclusion.

Do you think a move can be pruned without doing any analysis of it? Why would you believe such a thing?

Why do you think the fact that when Stockfish has only analysed a few billion nodes it has not analysed enough of a tree that is much larger is relevant to the question about the algorithm with a quadrillion times more time and computational resources?

You confuse "an impractically long time" and "never".

I accept that this is beyond your capabilities to understand. You need to answer the much more enlightening question.

"When does Stockfish (in "unlimited" mode) stop analysing when there remains a candidate move that could change its opinion of the result?"

That is the one which has the answer: NEVER.

Actually I did answer it. Unlike you for my question.

More explicitly, in infinite analysis mode, if Stockfish manages say a million nodes per second and the number of nodes in the complete tree is N, the answer is:

In less than N / 1000000 seconds

This time can easily be aeons or more. That is irrelevant to the truth about the algorithm.

Now answer my question:

When does Stockfish (in infinite analysis mode) stop analysing when there is a completely unanalysed candidate move that can affect the result?

Question: When does Stockfish analyse every move (needed to determine the result)

Answer: Whatever time it needs to analyse the finite number of nodes needed.

It's not difficult (to me). I accept that it's beyond you.

You seem hopelessly confused between the algorithmic truth and what happens when you have a trillion times (or more) too little time to do the analysis.

Do you understand that there are only a finite number of legal positions in chess?

(Lack of answer will be taken as meaning you don't).

I will generously take that as a correct answer: there are indeed merely a finite number of positions in chess.

Next question:

Do you understand that while Stockfish is analysing it is constantly adding more analysed nodes?

[Assume the hash table is big enough so that it never gets full. Eg, big enough for all legal chess positions - impractical, but simple to the algorithm. This makes the answer clearer].

Now I am going for a cycle, but I will be back to see your ego-motivated pot-shot later. But do answer the question too.

You should understand that past a certain depth, no more nodes are added because they are all already there. (It's because the set of legal positions is finite (also there is a finite upper limit on the number of moves in a legal chess game).

So, no. After this depth has been reached , the percentage of moves that have been analysed obviously keeps going up from there. [More simply, the percentage of all legal positions in chess that have been analysed goes up until the analysis is complete].

The final question:

Does the Stockfish algorithm stop when there remain nodes that could affect the overall evaluation that have not been analysed at all?

Jeez. Everyone else can see I am NOT changing the subject.

Please answer the final question if you understand it.

Does the Stockfish algorithm (on "infinite analysis") stop when there remain nodes that could affect the overall evaluation that have not been analysed at all?

[Also, you presented an erroneous argument that the Stockfish algorithm does not analyse all positions that are relevant to a problem. The fact that you have already forgotten this from a few hours ago explains why you stay wrong.].

Again, do you not understand what a question is? Ask someone else to explain my last post to you if you don't.

The answers to the three questions I have asked you (you have got the first two right already) provide most of the proof you desire. There is only one more step once you answer the third question.

The proof is this.

Firstly, 3 simple facts. (You have stated the first two and would be unwise to dispute the third).

1. There are a finite number of positions

2. While the Stockfish algorithm is doing analysis it is continually adding new positions to the analysis tree

3. The Stockfish algorithm will not stop if it has sufficient computational resources and there is an position that has not been analysed and can be relevant.

Then one deduction (you should be able to understand this. If not the answer is to get able to do so).

Conclusion: given unlimited computing resources, the Stockfish algorithm finds the very best line.

This is an impractical result because most chess positions would take (much) more than a million years to solve using Stockfish on a fast computer, even one enhanced with the stupendous amount of memory to store a huge analysis tree. Experts are likely to have told you what happens in reasonable time. It's irrelevant what it could do in billions of years (but it is relevant to our question).

You are wise not to have been following.

But you have gone straight to the crux. A ridiculously large amount of time is not the same as an infinite amount of time.

While it is not logically impossible that Stockfish has code such that there is some evaluation of a position in a tree that means it will never look at it regardless of whether there are no other leaf nodes remaining, I don't believe this is plausible. It can certainly solve puzzles where a couple of queen sacrifices are necessary and that sort of thing. There would be no point in having a line drawn for when it is 3 queens down, or whatever. A move can be ignored if another move leads to a forced mate (although we probably all know (like I do) that Stockfish often updates its belief about the number of moves to mate, finding a long mate first, then finding a faster mate later, so it does not do this).

I am not sure that @DesperateKingWalk understands that "never in a million years" is not the same as "never".

No.

[And your reading comprehension is poor].

Do I need to tell you a third time that chess has a finite number of legal positions?

The answer to your question "does it?" is "yes, it does".

The only thing that stops Stockfish looking at every move is computational resources, especially TIME.

On multiple occasions you have forgotten that all the empirical evidence involves a minuscule amount of time. We are discussing what the algorithm would do with unlimited time and space. (The fact that it can't solve some problems with practical time has never been in question).

Let's see what Stockfish does when it is in a position to eliminate every node except one, but still has time to analyse. The answer is that it continues to analyse DEFINITELY INFERIOR BRANCHES. Given that, the belief that it might ever PERMANENTLY ignore branches that COULD BE BEST is ridiculous.

I don't know what StockFish currently does and how. But I do know I would program every chess engine - and I would be massively surprised if it isn't done already - to have a thread (threads) running doing a full tree search on the chance it strikes gold. Even if just running in the background as a garbage collector on 1% of CPU-time. When quantum computing gets fast enough (assuming it can) I would delete all the other code. The point is that there is a balance between the required complexity of the software and CPU speed plus memory. In the last century russian infomation theorists were considered brilliant in designing algorithms which they needed for instance in their space rockets since their computers were slow compared to those in the west.