Progress of 8 piece tablebase?

Sort:
MARattigan

Never mind the 8 man tablebases, when do we get the 7 man DTM tablebases back. Lomonosov appears to be permanently defunct.

Ziryab
TheWombat5 wrote:

Considering the incredible increasing rate of computational power and storage for computers, at all levels, over the last 50 years, that somehow an 8 man table base would be some sort of difficult task to either complete or store soon is a bit odd. By 2030 this will be seen as a rather easy task. I actually dread quantum computing getting involved in chess, an engine using a rather powerful quantum computer will very likely, this century, be able to tell you during the middle game that you have lost, won or drawn with correct play.

Nope. You greatly underestimate the complexity of chess.

cobra91
MARattigan wrote:

SF would usually tell me that I've lost pretty reliably quite early on anyway if I had it switched on.

Yes, because chess is already much closer to solved than many people realize.

MARattigan

Rather because I am further removed from solving chess than SF. I suspect we're both light years away.

cobra91
MARattigan wrote:

Never mind the 8 man tablebases, when do we get the 7 man DTM tablebases back. Lomonosov appears to be permanently defunct.

Lomonosov needed a $15 paywall just to help fund the continued maintenance of their enormous 7-man DTM tablebase server. The required funds were bound to dry up sooner or later. DTC and DTZ metrics are the way forward.

"The more pieces are in a position, the more DTM will likely increase relatively to DTC or DTZ without conveying any new insights, so I will not pursue DTM further."  -- Marc Bourzutschky

MARattigan

I was told some smegma sac infected the Lomonosov system with ransomware.

cobra91
MARattigan wrote:

Rather because I am further removed from solving chess than SF. I suspect we're both light years away.

My own opinion is that, if the best current engines were really lightyears away from playing perfect chess, then the TCEC format wouldn't have had to transition away from standard chess just to avoid the same fate as the World Correspondence Chess Championship.

cobra91
MARattigan wrote:

I was told some smegma sac infected the Lomonosov system with ransomware.

OK, I'll admit I was unaware of this.

My earlier point still stands, though. The Lomonosov approach is less efficient (and therefore less cost-effective) than, e.g., the Syzygy approach. So it isn't surprising the Syzygy tablebases proved to be more sustainable as well.

cobra91

Bourzutschky's 8-man tablebase results provide an interesting testing ground for chess engine progress toward eventual perfection. How close are top engines to GTO endgame play, at the moment? Here is one possible test case, among many:

The basic idea is to use some accuracy score (T1, T3, ACL, etc.), according to one's favorite engine (e.g. the latest/strongest Stockfish iteration) as a rough measure of how well modern chess AI approximates perfect play. Sadly, I don't have the hardware needed to get reliable results, but I would love to know how top engine moves compare to the optimal line above when allowed to analyze at high depth on powerful hardware.

tygxc

@30

"a rough measure of how well modern chess AI approximates perfect play"
++ Would you mean modern chess AI with or without 7-men endgame table base?
Without it is pretty bad as we know from e.g. KNN vs. KP.
In ICCF they reach perfect play, but with humans.

"I don't have the hardware needed to get reliable results"
++ You can do it with any hardware, it just takes more time to reach the same depth.

cobra91
tygxc wrote:

"a rough measure of how well modern chess AI approximates perfect play"
++ Would you mean modern chess AI with or without 7-men endgame table base?

Good question. I think it could be useful to have data for both unassisted and tablebase-assisted engine play. I guess the priority should be "total chess" (with 7-man tablebases used), but it may also be interesting to see how the latest chess programs fare on their own in some of the solved 8-man endings, just to see how much progress they've made with endgame strength.

Yes, in theory. But practically, even with my computer running continuously at maximum performance, it would take many days of calculation to reach any reasonable depth. It could easily overheat or crash during that time, and even without those risks, I can't really afford to devote most or all of my only PC's resources to this; I rely on it for too many other things, on an almost daily basis in some cases.

On top-of-the-line hardware, meanwhile, high-depth analysis can be done in a matter of hours.

tygxc

@32

In another thread I have formulated the conjecture that on an engine of 10^9 nodes/s running for 17 s the table base exact move is always among the top 4 engine moves.
I derived that by extrapolating from AlphaZero autoplay.
For a regular desktop of 10^6 nodes/s that corresponds to 17000 s = 4.7 h.
I expect that for 8 men only (not 26 where chess is most complex) even a shorter time suffices.

cobra91

Which thread? If there's a related discussion elsewhere, I'd certainly like to know where to find it!

Regarding your conjecture: based on how far the available chess software has progressed at this point, I'd tend to agree. Given that 7-man tablebases are unlikely to be very helpful in a 26-man position, I assume the conjecture is meant to apply even without tablebase support. I'm just curious why you also stated (post #31) that engines were "pretty bad" at playing KNN vs. KP without assistance. Does the main difficulty lie in distinguishing between the top 4 candidate moves to consistently make optimal choices?

magipi
cobra91 wrote:

chess is already much closer to solved than many people realize.

Yes. With 50 years of hard work, roughly 0.000000000000000000000001 percent of chess positions were solved. We are indeed very close to 100%.

tygxc

@34

"Which thread?" ++ The thread on solving chess

"7-man tablebases are unlikely to be very helpful in a 26-man position"
++ On the contrary, in TCEC the engines hit their 7-men endgame table base early

"the conjecture is meant to apply even without tablebase support" ++ Yes

"engines were "pretty bad" at playing KNN vs. KP without assistance"
++ The main difficulty is that the checkmate lies beyond the calculation horizon, and the evaluation being a modified count of material makes the engine happy to win the pawn, which leads to a draw. Keeping the enemi pawn alive is key to checkmate.

"distinguishing between the top 4 candidate moves to consistently make optimal choices?"
++ Yes indeed, the evaluation function is imperfect, but the engine can only play the move with the highest evaluation value and discard the 3 other candidate moves.

MARattigan
cobra91 wrote:

...

Regarding your conjecture: based on how far the available chess software has progressed at this point, I'd tend to agree. ...

I've posted half a dozen examples on the same thread where the tablebases show it's false for recent versions of Stockfish, so no, it's not a valid conjecture.

cobra91

@tygxc: Thank you for all the quick responses. I did not know about some of these technical details until now.

I will definitely check out the thread on solving chess. It appears to be a little more active than this one!

tygxc

@38
"It appears to be a little more active than this one!"
++ But it is infested with trolls who spam about their wives, mistresses, railways, and who dismiss all serious work with facts and figures with their own unsupported claims about white being in Zugzwang, 1 a4 winning for white in thousands of moves, 1 e4 e5 2 Ba6? drawing for black etc.

The argument is this:
From figure 2a of this scientific paper https://arxiv.org/pdf/2009.04374.pdf 
At 1 s/move: 88.2% draw, 11.8% decisive games, i.e. games with an odd number of errors
At 1 min/move: 97.7% draw, 2.3% decisive games, i.e. games with an odd number of errors
Extrapolating
At 1 h/move: 2.3% * 2.3% / 11.8% = 0.44% decisive games, i.e. games with an odd number of errors, i.e. 1 error in 227 games.
At 60 h/move: 0.44% * 2.3% / 11.8% = 0.087% decisive games, i.e. 1 error in 1144 games.
Thus top 2 moves: 1 error in 1144² = 1.3 * 10^6 games
Thus top 3 moves: 1 error in 1144³ = 1.5 * 10^9 games
Thus top 4 moves: 1 error in 1144^4 = 1.7 * 10^12 games
Assuming 100 positions/game: 1 error in 1.7 * 10^14 positions

cobra91
magipi wrote:
cobra91 wrote:

chess is already much closer to solved than many people realize.

Yes. With 50 years of hard work, roughly 0.000000000000000000000001 percent of chess positions were solved. We are indeed very close to 100%.

Chess can be solved without exploring the full set of legal positions. When checkers was solved in 2007, less than 0.0001% of positions were examined. This potential reduction of the effective complexity of a game tends to increase with the size of the game itself, which means it's possible that as few as 0.000000000001% of chess positions need to be covered in order to solve chess.

Additionally, chess is extremely well understood at the highest levels of correspondence play and analysis. Thanks to all of the powerful tools that are now available, decisive results have become practically nonexistent among the very best players, and there are no decisive games at all (to the best of my knowledge) that can defy the abilities of analysts to "restore order" by proving a draw from an earlier position.

There are also some advanced stats (a bit beyond the scope of this comment) which suggest very strongly that, essentially, there isn't much room left for reasonable doubt. The evidence all points to the conclusion that chess is theoretically drawn, and that current technology is already enough to navigate to that result virtually 100% of the time.

cobra91
MARattigan wrote:
cobra91 wrote:

...

Regarding your conjecture: based on how far the available chess software has progressed at this point, I'd tend to agree. ...

I've posted half a dozen examples on the same thread where the tablebases show it's false for recent versions of Stockfish, so no, it's not a valid conjecture.

I'll take a look at this when I have a bit more time. If the latest versions of Stockfish still perform poorly in tablebase endings, I want to see it!