Chess960 computer analysis and cheat detection
(To download PGN Spy Chess960 version scroll down to the bottom of the page)
As most of modern chess engines support Chess960 for some time already, obviously they can be used also to play chess here on chess.com and gain an unfair advantage.
We all (or almost all) know about great computer analysis tool written and published by chess.com member @MGleason - PGN Spy. Originally, it doesn't support Chess960 games, as one part of it lacked support and there was less interest in Chess960 games analysis.
So, first let's see what's inside of PGN Spy software pack, and what happens when analysis is started.
- PGN Extract (Official version do have chess960 support now)
Converts game(s) PGN into UCI format, - UCI Analyser
Sends starting position and moves to UCI compatible engine, generates an XML formatted output file(s), Inits the engine and sets hash, depth, threads to be used and game type, analyses positions for best moves and also move played, - UCI compatible engine (Stockfish for example)
Makes popcorn and sometimes icecream - PGN Spy (This is link to standard version)
Handles everything and calculates statistics after parsing XML output generated by UCI-Analyser .
To add Chess960 support, only small intervention was actually required. At that time, PGN-Extract couldn't process chess960 games properly, namely castling was handled wrong, due to trivial error in source code (game tag "chess 960" vs "chess960").
What we have done was to modify code so it treats all games as chess960 games, thus completely discarding any need for type tag.
Now, we have correctly converted PGN into UCI format, and rest was quite simple - modify PGN Spy sources so it sends Chess960 UCI command to engine, along with other settings (hash size, threads and depth). Here, like with PGN-Extract, we decided to treat all games as Chess960 games as destination squares for King and Rook are always the same, both in chess960 and standard chess.
So, this version of PGN Spy is capable of analyzing both Chess960 and standard chess PGN files.
Next, some tuning was done to PGN Spy, so it can use more CPU power per single game. In official version, single game is being analyzed using single CPU core (or thread) while in this version it uses two CPU cores per game, which allows reaching greater depths required for very complicated positions.
There is also option to use remote engine, via Putty Plink. Some of us do have access to powerful multi core servers and ability to use it's power while in idle state is useful. Generally, server CPU-s do have more power and can do the job much faster.
Dual CPU Xeon based server can run 24 threads, or analyze 12 games at once with this version 
Simply save profile for destination server ("server" in screenshot above) and use \\plink.exe -load profile -l user -pw password "engine" command line arguments.
Now, there are two types of settings which could be used for analysis.
Shallow and quick analysis, and deep but slow, which of course provides more accurate results.
For quick analysis it's enough to modify only min time per move (set it to 20 or 50 ms) and Min search depth (15 or 16 plies)
For deep analysis, settings on above screenshot will do fine.
Book depth can be set to 0 as there is no databases, other than those played by amateurs. However, knowing that some chess servers do have Chess960 "book" games can be manually checked and book depth set accordingly.
Downloads:
Stockfish v8 and v9, if compiled for Windows, will produce inaccurate results, while Linux versions both will work properly, if used as remote engine. Errors happens in number of positions but also in T stats.
PGN Spy archive can be extracted to any folder, and it's settings after saving wont affect official version settings due to different names and thus different keys in registry.
Analysis samples:
Players who's account is closed for cheating:
Player 1
UNDECIDED POSITIONS
Positions: 427
T1: 184/367; 50.14% (std error 2.61)
T2: 243/345; 70.43% (std error 2.46)
T3: 285/335; 85.07% (std error 1.95)
=0 CP loss: 252/427; 59.02% (std error 2.38)
>0 CP loss: 175/427; 40.98% (std error 2.38)
>10 CP loss: 105/427; 24.59% (std error 2.08)
>25 CP loss: 35/427; 8.20% (std error 1.33)
>50 CP loss: 10/427; 2.34% (std error 0.73)
>100 CP loss: 2/427; 0.47% (std error 0.33)
>200 CP loss: 0/427; 0.00% (std error 0.00)
>500 CP loss: 0/427; 0.00% (std error 0.00)
CP loss mean 7.21, std deviation 14.10
LOSING POSITIONS
Positions: 0
--
Player 2
UNDECIDED POSITIONS
Positions: 391
T1: 163/333; 48.95% (std error 2.74)
T2: 211/307; 68.73% (std error 2.65)
T3: 230/294; 78.23% (std error 2.41)
=0 CP loss: 223/391; 57.03% (std error 2.50)
>0 CP loss: 168/391; 42.97% (std error 2.50)
>10 CP loss: 103/391; 26.34% (std error 2.23)
>25 CP loss: 53/391; 13.55% (std error 1.73)
>50 CP loss: 22/391; 5.63% (std error 1.17)
>100 CP loss: 7/391; 1.79% (std error 0.67)
>200 CP loss: 1/391; 0.26% (std error 0.26)
>500 CP loss: 0/391; 0.00% (std error 0.00)
CP loss mean 11.04, std deviation 24.75
LOSING POSITIONS
Positions: 0
--
GM Grischuk, Alexander, 10 Rapid games (Obviously clean )
UNDECIDED POSITIONS
Positions: 363
T1: 96/247; 38.87% (std error 3.10)
T2: 105/176; 59.66% (std error 3.70)
T3: 120/162; 74.07% (std error 3.44)
=0 CP loss: 227/363; 62.53% (std error 2.54)
>0 CP loss: 136/363; 37.47% (std error 2.54)
>10 CP loss: 91/363; 25.07% (std error 2.27)
>25 CP loss: 47/363; 12.95% (std error 1.76)
>50 CP loss: 14/363; 3.86% (std error 1.01)
>100 CP loss: 4/363; 1.10% (std error 0.55)
>200 CP loss: 2/363; 0.55% (std error 0.39)
>500 CP loss: 0/363; 0.00% (std error 0.00)
CP loss mean 9.66, std deviation 24.55
LOSING POSITIONS
Positions: 2
Please feel free to ask any questions on how to interpret the results.
Please keep in mind of no naming and shaming policy on chess.com, so do not post user names of any suspects in Your comments.
Thank You.
