Using Rybka for Top 3 match analysis with ChessAnalyse

gwhuebner
gwhuebner
Aug 17, 2010, 11:53 PM |
8

Some issues have been reported (http://www.chess.com/groups/forumview/fritz-xi-top-3-matchup-benchmark-stats , #133) in connection with the use of Rybka for constant time/ply analysis with ChessAnalyse, which have to be investigated further and might result in a change of ChessAnalyse's engine control.

To investigate the behavior of Rybka in Top 3 match analysis, I will perform some tests, analysing the engine's output and compare it to the uci specification and another engine (Houdini).
Let's do a manual (Command prompt) top 3 match analyis of the following (book) position:

 

 

 

 

 

 

 

 

 

 

 

 

 

1.) Starting Deep Rybka 4 w32 in a command shell (cmd.exe) and entering the following uci commands:
ucinewgame
// indicates, that a new game has to be analysed)
setoption name MultiPV value 3
// this sets MultiPV mode to 3)
position fen rn1q1rk1/pbp1bppp/1p1ppn2/3P4/2P5/P1N1P1N1/1P2BPPP/R1BQK2R b KQ - 1 9
// defines the starting position for analysis as a FEN string
go movetime 30000
// analyse exactly 30 seconds; this is the same command as launched by ChessAnalyse, when the parameters Time sec = 30 and Depth plies = 0.

Rybkas output is as follows:
... (first few hundred lines have been omitted) ...
info currmove b7c8 currmovenumber 27
info currmove f6e8 currmovenumber 28
info currmove h7h5 currmovenumber 29
info depth 12 time 7380 nodes 546967 nps 74114
info depth 13
info currmove c7c6 currmovenumber 1
info time 8112 nodes 604803
info time 9126 nodes 682393
info time 10140 nodes 755009
info multipv 1 depth 13 score cp -25 time 10344 nodes 770950 nps 74531 pv c7c6 e3e4 b8d7 e1g1 c6d5 c4d5 a8c8 c1e3 f8e8 d5e6 f7e6 a1c1 d7e5 d1a4 f6g4 a4a7 g4e3
info multipv 2 depth 12 score cp -17 time 4712 nodes 324823 nps 68935 pv b8d7 e1g1 c7c6 d5e6 f7e6 f2f4 d6d5 d1c2 a7a5 c1d2 a8c8 b2b3 d8c7 g3h5 f6h5 e2h5 d7c5
info multipv 3 depth 12 score cp -23 time 5040 nodes 349838 nps 69412 pv e6d5 c4d5 c7c6 e3e4 c6d5 e4d5 b8d7 e1g1 a8c8 c1e3 g7g6 a1c1 d7e5 d1d4 c8c3 c1c3 f6d5 c3c1 d5e3 d4e3
info currmove b8d7 currmovenumber 2
info time 11154 nodes 821657
info time 12168 nodes 896263
info multipv 1 depth 13 score cp -25 time 10344 nodes 770950 nps 74531 pv c7c6 e3e4 b8d7 e1g1 c6d5 c4d5 a8c8 c1e3 f8e8 d5e6 f7e6 a1c1 d7e5 d1a4 f6g4 a4a7 g4e3
info multipv 2 depth 13 score cp -25 time 12809 nodes 937644 nps 73201 pv b8d7 e1g1 c7c6 e3e4 c6d5 c4d5 a8c8 c1e3 f8e8 d5e6 f7e6 a1c1 d7e5 d1a4 f6g4 e3d4
info multipv 3 depth 12 score cp -23 time 5040 nodes 349838 nps 69412 pv e6d5 c4d5 c7c6 e3e4 c6d5 e4d5 b8d7 e1g1 a8c8 c1e3 g7g6 a1c1 d7e5 d1d4 c8c3 c1c3 f6d5 c3c1 d5e3 d4e3
info currmove e6d5 currmovenumber 3
info time 13182 nodes 965895
info multipv 1 depth 13 score cp -25 time 10344 nodes 770950 nps 74531 pv c7c6 e3e4 b8d7 e1g1 c6d5 c4d5 a8c8 c1e3 f8e8 d5e6 f7e6 a1c1 d7e5 d1a4 f6g4 a4a7 g4e3
info multipv 2 depth 13 score cp -25 time 12809 nodes 937644 nps 73201 pv b8d7 e1g1 c7c6 e3e4 c6d5 c4d5 a8c8 c1e3 f8e8 d5e6 f7e6 a1c1 d7e5 d1a4 f6g4 e3d4
info multipv 3 depth 13 score cp -27 time 13542 nodes 995587 nps 73518 pv e6d5 c4d5 c7c6 e3e4 c6d5 e4d5 b8d7 e1g1 g7g6 c1e3 a8c8 a1c1 d7e5 d1d4 c8c3 b2c3
info currmove h7h6 currmovenumber 4
info time 14196 nodes 1036522
info time 15210 nodes 1103169
info currmove a7a5 currmovenumber 5
info time 16224 nodes 1169817
info currmove b8a6 currmovenumber 6
info time 17238 nodes 1236465
info currmove f8e8 currmovenumber 7
info currmove c7c5 currmovenumber 8
info currmove d8d7 currmovenumber 9
info currmove f6d7 currmovenumber 10
info currmove b8c6 currmovenumber 11
info currmove f6d5 currmovenumber 12
info currmove b7d5 currmovenumber 13
info currmove f6e4 currmovenumber 14
info currmove f6g4 currmovenumber 15
info currmove b7c6 currmovenumber 16
info currmove b7a6 currmovenumber 17
info currmove e6e5 currmovenumber 18
info currmove g8h8 currmovenumber 19
info time 18252 nodes 1299134
info currmove b6b5 currmovenumber 20
info time 19266 nodes 1398608
info time 20280 nodes 1471224
info time 21294 nodes 1549809
info time 22308 nodes 1624415
info time 23322 nodes 1720905
info time 24336 nodes 1805458
info time 25350 nodes 1892995
info time 26364 nodes 1966606
info time 27378 nodes 2043201
info currmove a7a6 currmovenumber 21
info currmove g7g6 currmovenumber 22
info currmove f6h5 currmovenumber 23
info currmove g7g5 currmovenumber 24
info currmove d8c8 currmovenumber 25
info currmove d8e8 currmovenumber 26
info currmove b7c8 currmovenumber 27
info currmove f6e8 currmovenumber 28
info currmove h7h5 currmovenumber 29
info depth 13 time 28222 nodes 2095655 nps 74256
info depth 14
info currmove c7c6 currmovenumber 1
info time 28392 nodes 2108854
info time 29406 nodes 2195397
info multipv 1 depth 13 score cp -25 time 29984 nodes 2241155 nps 74745 pv c7c6
e3e4 b8d7 e1g1 c6d5 c4d5 a8c8 c1e3 f8e8 d5e6 f7e6 a1c1 d7e5 d1a4 f6g4 a4a7 g4e3
bestmove c7c6 ponder e3e4

Obviously the engine works in the following way:
Every possible move (29 in the position above) is analysed to a certain ply depth.
The analysis begins with the line
info depth [newDepthLevel]
and ends with the line
info depth [newDepthLevel] time ... nodes ... nps ...
Within such a depth level section there are one or more blocks displaying the information for the best 3 moves. They start with
info multipv [k] depth ... score ... cp time ... nodes ... nps ... pv ...
where [k] is 1, 2 and 3.
As one can see from the last lines of the output, the engine has just switched to start analysing depth level 14 for the first move, when the 30 sec were spent.
But, instead of giving all 3 multipv informations, as claimed in the uci specification (http://download.shredderchess.com/div/uci.zip where it reads:
"...in k-best mode always send all k variants in k strings together..."), only the information for multipv 1 is shown, with slightly different (higher) values for time, nodes and may be even different score cp, which may lead to the effects described in the link in the beginning, since ChessAnalyse always takes the last output for each multipv line.

The same effect occurs in an analysis to a certain ply depth. The corresponding uci command would be:
go depth 11
for example. Trying this, I noticed that an analysis to depth 13 for the above position last a significant time longer than the 30 sec analysis, which has already reached depth level 13, but it seems to be that all 29 moves are analysed to depth 13 with this latter command, while with the movetime command only the best 3 moves are analysed to this level.

There is still another possibility to do a 30 sec analysis with an uci engine. By typing the following command
go infinite
an infinite search is launched, which has to be interrupted with the command
stop
Unfortunately Rybka seems to not responding to the stop command in any mode...
(may be anybody could try this also and post something here..)

I did the same commands with the "Houdini 1.03a w32" engine and found everything fine.
Note, that the ending line of a ply depth section:
info depth [newDepthLevel] time ... nodes ... nps ...
is missing in Houdini's output, but the 3 multipv infos are always shown together and the engine reacts to the stop command immediately.