Decided I'd go along with it
openings to test engines ???
Sort:
Nov 7, 2013
0
#3
I'm not an official tester or anything, but I've done some informal testing just for fun. Here's my two cents:
First off, a ten-game match is completely worthless; It tells you almost nothing. I'd say you should consider at least 200 to 300 games between two engines before you start getting fairly accurate ratings. (The closer the two engines are in strength, the more games you'll need to play.) There's a command-prompt program called Bayeselo that you can use on a completed tournament to get errors of margin for the ratings. (This program isn't easy to use - It'll take some learning to get proficient with it.)
On the opening book, I wouldn't worry too much about the types of openings in the book. I'd just try to find a decent size book for the GUI you're using, but I'd really try to find a "neutral" book, which is preferred for testing.
I'm not a fan of handicap games, since I'm not sure how you'd go about adjusting the rating for the handicap. Instead, I'd use a reference engine that's listed on a rating list (like the CCRL 40/4 or CCRL 40/40) and that has a rating fairly close to the engine you're testing.
I'm talking about testing unrated engines so that seems unesseccary
I've recently picked backed up the hobby of rating chess engines that have no elo. I do the by means of chess test such as tactical quizzes and "guess the move/solitare chess" also by means of having the engine in question play series of handicap games with my strongest engines or a mixture of the 2 methods and many and I mean many other methods but those are the main 2. My latest idea deals with a having the engine question play a series of 10 game matches using pre set openings (2 symmetrical, 2 gambits, 2 open based, 2 closed based and 2 busted openings) the starting opponent will be a 2000 rated engine/comp. Where winning the series of matches I will and by how convincingly won will I will pit it against a higher rated engine and so on. the 2 games will be alternating colors and WITHOUT the same opening being played so that the computer wont be rellying on the hash tables so much and the 1 busted opening is having the computer plays the busted openings to test its skills in playing in a slightly worse and worse position. What do you guys think ?