Arena 3.5 and EGTB's (Part 2)

Crappov wrote:

HGM is indeed an authority ---->> Harm Geert Muller

With Nalimov and Gaviota tablebases there's nothing to think about.  They contain precise information as to the theoretically best move, i.e., the move that leads fastest to mate, and exactly how many moves that requires.

Syzygy doesn't contain that much info.  They tell the engine whether a move is winning, losing or drawing, but not necessarily which move is best.

Look at the graphic below (view in new window to enlarge).  It is whites move and white is clearly winning.

Look at the Gaviota tablebase info displayed by the GUI.  There's numerous moves that white could make and still be winning the game.  The Gaviota bases identify Qf2+ as the move which leads to the quickest mate.  If the engine is using Gaviota tablebases, that move would be played immediately.  The Syzygy bases don't contain info that says Qf2+ leads to the fastest mate, only that the move is a winning move - as are numerous other possible moves.

I'm not an authority on this by any means, this is just my laymans incomplete understanding of how the tablebases work.

I understand what you're saying Mr. Crappov but you didn't read all of my question

Why have Syzygy EGTB's if you can have precise Nalimov/Gaviota EGTB's?

By the way, concerning that Arena picture, did you actually install 5 men Gaviota TB's? That's the only way to have Arena GUI show that information in the movelist

While the Syzygy bases can't give distance-to-mate info, I think the advantages are that they do incorporate the 50-move rule, and the bases don't take up as much hard drive space.

Gaviota only has up to 5-man support and the full 6-man Nalimov takes up 1.2 TB or so.

So that's the reason most people use Syzygy.  Support for up to 6-man and only takes up 151GB of space.

The total space taken up by 5 men Syzygy is about 1Gb.

Vs. 7Gb to store Nalimov/Gaviota

??? I'm not sure

But 50-move is very important and should be incorporated into EGTB's sooner or later. No doubt about that... 

For me chess programming is just a hobby. But it is a hobby I might spend 80 hours a week on.

Note that what was written somewhere above is not entirely accurate. EGTs do not contain moves, they just contain the distance (in moves) to a 'winning event'. For a given position the engine can generate all legal moves, and then look up the positions they lead to in the EGT, to see which of them gets you fastest to the winning event (or with which it can postpone it maximally when losing). It can do this in unmeasurably short time.

That holds both for Syzygy, as for Nalimov/Gaviota.

The difference is that in Nalimov and Gaviota the 'winning event' is the checkmate, (DTM = Distance To Mate) while Syzygy tabulates the distance to zeroing of the 50-move counter. i.e. capture or Pawn push (Distance To Zeroing = DTZ).

Playing the move with the best DTM will make you arrive at the checkmate fastest. Unfortunately this can cause the game to end in a draw, because the program would not want to 'waste' time on pushing a Pawn when it would delay the mate. So engines that use Nalimov or Gaviota EGTs tend to blunder away difficult end-games.

With Syzygy you don't have that problem: playing the move leading to the position with the best DTZ will always reset the 50-move counter as fast as you can without spoiling the theoretical result. Unfortunately this often leads to 'silly' play, like forcing the opponent to accept a Queen sac in KQBNK, followed by a lengthy KBNK mate. Some engine programmers think this so embarrasing that they don't want to play the DTZ-wise best move instantly, but rather let the engine think to come with its own move.

So the short answer is: Nalimov/Gaviota will win in fewer moves, Syzygy will win more certainly.

On unrelated subject... Is there a way in Arena 3.5 to copy/move large number of games from one PGN file to another? Or combine large PGN files? Theoretically, one can open game list associated with the PGN file, right click on it to open little icon, and then select necessary game or choose "Select all" to copy the whole file. Then you have to click "Copy selected games", open another file, and right click "Insert games" in it's game list. But I have never tried aforementioned process and I remember somebody saying on a forum that Arena has difficulty copying large number of games and can crash.... ???

I always use the command line to combine PGN files. E.g. on Linux the command

cat *.pgn > XXX

would put all PGN files in the current directory in the file XXX. Couldn't be much simpler. Better not let XXX end in .pgn, though, or it would try to copy itself...

Well, very nice, but I use Windows 7 Pro and my computer is Dell laptop. I guess I must ask those other guys (Crappov , Eschere-whatever)

Not me, I think Arena sucks for that. 

BTW, did the Talkchess mods ever send you a link to complete your registration?

They did. Today, in fact, after making me wait for 10 days. So, now I can ask questions on TalkChess.com too. You still didn't answer my question, do you have Gaviota 5 men TB's installed in your Arena? I'm just curious, it takes about 7 Gb as I understand and your engines won't use those TB's. It's only for Arena GUI use. But, of course, it is nice to have this feature on the computer, being able to see all those endgame moves in the movelist...

Even on Windows there exist commands to display a text file. Try:

type *.pgn > XXX

I'm not sure, I understand... display file to do what? I need to combine two (or more) pgn files. Besides, how do I use this command? Open some "run command" window?

Yes, I have Gaviota TB's installed - Nalimov, too, for engines that use them.  I have also the Aquarium 2015 GUI, which lets me query the seven piece Lomonosov tablebases in Moscow.  :) Doesn't help me play better chess, but it's fun ---->>

Which engines use Gaiota TB's except Gaviota engine? And which ones use Nalimov TB's?

By the way I'm Russian and lived in Moscow for 6 years. That name (Lomonosov) is the name of Moscow University. Well, first, it's the name of a famous Russian scientist who lived in the 18th century and laid foundation of Russian science

I guess I can try PGN Mentor to deal with files. I looked at Aquarium website, seems too expensive for me, I'm not a professional chess player. First, I'll try using Arena...

Yes, be there...

I have to agree with Crappov about being reluctant to use Arena for moving games between very large PGN files. I'd only feel comfortable using Arena to move one game at a time between small PGN files. Anything beyond that, and I think you'd be asking for trouble.

I might be willing to try using Arena to only view games in very large PGN files.

Regarding using DOS commands to merge large PGN files; It's really strange, I went to Wikipedia, and it said that the "type" command could do this, but it wasn't reliable with large text files. Wikipedia stated that the "copy" command should be used for large text files.

However, I just tried both commands, and I got exactly the opposite result. The "copy" command sometimes wouldn't fully merge large PGN files, but the "type" command seemed to work just fine.

You first have to open up a DOS command line, then change the DOS command line path to point to the group of PGN files to be merged. Then, similar to HGM's advice, you type a DOS command like:

type *.pgn > Total.txt

then hit the enter key. After the merging is finished, rename the "Total.txt" back to "Total.pgn", and you're done.

If you're going to be working with large game files often, you might consider using something like Scid vs. PC. This program can handle PGN files as read-only files, but loading is slow, and if you're going to be modifying the PGN files, you're better off just converting the PGN files to Scid file format. Once in Scid format, loading is fast, and editing database files is fast and reliable.

http://scidvspc.sourceforge.net/index.html

Somewhere in the forums there's a very long support/info thread for Scid vs. PC.