Chess analysis on Linux - command line addicts heaven :)

Chess analysis on Linux - command line addicts heaven :)

Avatar of e4_guy
| 6

(Download links at the bottom of the page )

So, some guys here asked to be self-tortured, namely by analyzing chess games for engine correlation using only command line tools on Linux. Yes, such people still exist. grin.png

 

Now, let's see:

Standard chess analysis requirements:

  • PGN file with games, SAN format
  • A "book" containing millions of games to be used for cross checking above games for "book moves". (book.bin)
  • A tool to do that cross check and cut the book moves from the games (rm_bookmoves.py)
  • PGN-Extract to convert PGN into UCI format
  • UCI compatible engine (Stockfish)
  • UCI Analyser to send games to engine for analysis and create analysis output
  • A tool to parse analysis output and calculate "stats". (Analyzer.py)
  • Python 3.6 with pexpect and some other libs which I can't remember now. It will probably cry out "i can't run because xyz is missing grin.png

So far so good.

Chess960 analysis requirements:

  • Everything as above, but without "book".

Let's say You rushed and downloaded required files, and You have it already installed on Your Linux box. You have pgn containing games where player You want to analyse is white or black.

You have created directory "games" and placed "book.bin", "rm_bookmoves.py" "analyzer.py"and "games.pgn" in that dir. So, You're here now:

/users/penguin/games

First, we run book moves cross check by python script:
(Skip this for Chess960 games)

python rm_bookmoves.py --book book.bin --minmoves 10 --pgn games.pgn

This will create "games.pgn" but in /users/penguin/games/output directory. You'll get some statistics and depending on --minmoves arg value games with less than "value" moves will not be included:


python3.6 rm_bookmoves.py --pgn games.pgn
Games processed: 12
Average book moves per game: 22.58
Minimum book moves in a game: 12
Maximum book moves in a game: 37
--- Finished in 0.19 seconds ---

(these are half moves, of course)

Now, we cd to /output and we will use PGN-Extract:

pgn-extract -Wuci -oout.pgn games.pgn

Now we have UCI formatted "out.pgn".

 

We start UCI-Analyser:

For white part:

analyse --engine sf7 --setoption UCI_Chess960 false (true for chess960 games) --setoption hash 512 --setoption threads 4 --searchdepth 20 --bookdepth 0 --whiteonly out.pgn > out.xml

For black part:

analyse --engine sf7 --setoption UCI_Chess960 false (true for chess960 games) --setoption hash 512 --setoption threads 4 --searchdepth 20 --bookdepth 0 --blackonly out.pgn > out.xml

Now You'll have to wait for job to be finished. Depending on number of positions and of course CPU power it can take very long time . . . Boring part happy.png

.

.

.

.

.

Boring

.

.

.

.

Now, it's done ! We have "out.xml" and we cd back to cp anylzer.py to /output and call it to parse the data and generate "out.txt" report file:

cd..
cp analyzer.py output/analyzer.py

cd output

python analyzer.py out.xml - this is done quickly happy.png

And now - lets peak inside:

tail -n 20 out.txt

 

Total moves: 716

Unfiltered:

Average CP loss: 17.75
T1 moves: 396 (55.31%)
T2 moves: 150 (76.26%)
T3 moves: 96 (89.66%)


Filtered:

Number of moves: 528
Average CP loss: 11.02
T1 moves: 303 (57.39%)
T2 moves: 96 (75.57%)
T3 moves: 72 (89.20%)
========================================

 (That is actually analysis  of Candidates 2018 tournament, all players)

Note that these stats aren't compatible with PGN Spy stats. I'll add some reference data to get some clue on what is humanly achievable and what not.

 

Downloads:

Credits:

Python scripts written by @jugojov, admin of Team Republika Srpska and lousy chess player grin.png who gets around with simple code but can't write anything complicated grin.png

Authors of pgn-extract, uci-analyser and Stockfish can be found at respective pages. Those are actual heroes happy.png

Blogs

Anti cheating advices

e4_guy
Baron Muncshausen
Република Српска (Republic of Srpska)

Tools and instructions required to detect and "exterminate" cheaters happy.png