Update: I stickied this thread at @hicetnunc's request to make benchmarks easier to find.

So I've run a few scans with my new scanner, which I believe has no known outstanding bugs left:

Interestingly, they seem to play worse in losing positions than in equal positions (I don't track similar stats for winning positions). However, the losing positions sample size is fairly small, since they tend to resign relatively quickly once they're significantly behind.

St. Louis 2014 (30 games):

EQUAL POSITIONS
Positions: 1593
T1: 470/999; 47.05% (std error 1.58)
T2: 449/626; 71.73% (std error 1.80)
T3: 445/545; 81.65% (std error 1.66)
Blunders: 20/1593; 1.26% (std error 0.28)
Avg centipawn loss: 11.10 (std deviation 126.90)

LOSING POSITIONS
Positions: 120
T1: 20/64; 31.25% (std error 5.79)
T2: 29/47; 61.70% (std error 7.09)
T3: 18/26; 69.23% (std error 9.05)
Blunders: 13/120; 10.83% (std error 2.84)
Avg centipawn loss: 45.52 (std deviation 148.07)

WCC 2013 (10 games):

EQUAL POSITIONS
Positions: 646
T1: 215/462; 46.54% (std error 2.32)
T2: 242/335; 72.24% (std error 2.45)
T3: 245/288; 85.07% (std error 2.10)
Blunders: 2/646; 0.31% (std error 0.22)
Avg centipawn loss: 5.94 (std deviation 23.12)

LOSING POSITIONS
Positions: 5
T1: 1/4; 25.00% (std error 21.65)
T2: 2/4; 50.00% (std error 25.00)
T3: 3/3; 100.00% (std error 0.00)
Blunders: 1/5; 20.00% (std error 17.89)
Avg centipawn loss: 146.20 (std deviation 271.19)

WCC 2014 (11 games)

EQUAL POSITIONS
Positions: 739
T1: 258/553; 46.65% (std error 2.12)
T2: 287/409; 70.17% (std error 2.26)
T3: 282/355; 79.44% (std error 2.15)
Blunders: 5/739; 0.68% (std error 0.30)
Avg centipawn loss: 6.83 (std deviation 35.74)

LOSING POSITIONS
Positions: 14
T1: 0/3; 0.00% (std error 0.00)
T2: 0/1; 0.00% (std error 0.00)
T2: 0/0
Blunders: 4/14; 28.57% (std error 12.07)
Avg centipawn loss: 101.43 (std deviation 173.36)

These were all run with my current default settings. Any suggestions on better defaults? Particularly scan time and depth (in plies), and book depth (which is in moves, not plies). I'm assuming for investigating a suspected cheater, I'll probably chop the time settings for an initial scan, and maybe do a deeper scan if the quick scan suggests it's merited.

Fair Play Board

Super Admins

Admins