Chess analysis on Linux - command line addicts heaven :)

Mar 13, 2018, 6:25 PM |

(Download links at the bottom of the page )

So, some guys here asked to be self-tortured, namely by analyzing chess games for engine correlation using only command line tools on Linux. Yes, such people still exist.

Now, let's see:

Standard chess analysis requirements:

PGN file with games, SAN format
A "book" containing millions of games to be used for cross checking above games for "book moves". (book.bin)
A tool to do that cross check and cut the book moves from the games (rm_bookmoves.py)
PGN-Extract to convert PGN into UCI format
UCI compatible engine (Stockfish)
UCI Analyser to send games to engine for analysis and create analysis output
A tool to parse analysis output and calculate "stats". (Analyzer.py)
Python 3.6 with pexpect and some other libs which I can't remember now. It will probably cry out "i can't run because xyz is missing

So far so good.

Chess960 analysis requirements:

Everything as above, but without "book".

Let's say You rushed and downloaded required files, and You have it already installed on Your Linux box. You have pgn containing games where player You want to analyse is white or black.

You have created directory "games" and placed "book.bin", "rm_bookmoves.py" "analyzer.py"and "games.pgn" in that dir. So, You're here now:

/users/penguin/games

First, we run book moves cross check by python script:
(Skip this for Chess960 games)

python rm_bookmoves.py --book book.bin --minmoves 10 --pgn games.pgn

This will create "games.pgn" but in /users/penguin/games/output directory. You'll get some statistics and depending on --minmoves arg value games with less than "value" moves will not be included:

python3.6 rm_bookmoves.py --pgn games.pgn
Games processed: 12
Average book moves per game: 22.58
Minimum book moves in a game: 12
Maximum book moves in a game: 37
--- Finished in 0.19 seconds ---

(these are half moves, of course)

Now, we cd to /output and we will use PGN-Extract:

pgn-extract -Wuci -oout.pgn games.pgn

Now we have UCI formatted "out.pgn".

We start UCI-Analyser:

For white part:

analyse --engine sf7 --setoption UCI_Chess960 false (true for chess960 games) --setoption hash 512 --setoption threads 4 --searchdepth 20 --bookdepth 0 --whiteonly out.pgn > out.xml

For black part:

analyse --engine sf7 --setoption UCI_Chess960 false (true for chess960 games) --setoption hash 512 --setoption threads 4 --searchdepth 20 --bookdepth 0 --blackonly out.pgn > out.xml

Now You'll have to wait for job to be finished. Depending on number of positions and of course CPU power it can take very long time . . . Boring part

Boring

Now, it's done ! We have "out.xml" and we cd back to cp anylzer.py to /output and call it to parse the data and generate "out.txt" report file:

cd..
cp analyzer.py output/analyzer.py

cd output

python analyzer.py out.xml - this is done quickly

And now - lets peak inside:

tail -n 20 out.txt

Total moves: 716

Unfiltered:

Average CP loss: 17.75
T1 moves: 396 (55.31%)
T2 moves: 150 (76.26%)
T3 moves: 96 (89.66%)

Filtered:

Number of moves: 528
Average CP loss: 11.02
T1 moves: 303 (57.39%)
T2 moves: 96 (75.57%)
T3 moves: 72 (89.20%)
========================================

(That is actually analysis of Candidates 2018 tournament, all players)

Note that these stats aren't compatible with PGN Spy stats. I'll add some reference data to get some clue on what is humanly achievable and what not.

Downloads:

Credits:

Python scripts written by @jugojov, admin of Team Republika Srpska and lousy chess player who gets around with simple code but can't write anything complicated

Authors of pgn-extract, uci-analyser and Stockfish can be found at respective pages. Those are actual heroes

Chess analysis on Linux - command line addicts heaven :)

Anti cheating advices