Game accuracy not included in PGNs, how do I extract this value from downloaded PGNs?

Sort:
Ndrizky

Hello peeps,

Have some pgns and I would like to get the corresponding accuracy (as produced by the engine from chess.com) for each game. However, this is not included in the pgns. Is there any way I can programmatically collect this to correspond to the specific games? Or do I have to individually feed the pgns into stockfish and analyze in order to obtain the accuracy values of both players?

Input needed please.

Thanks

VirtualKnightJoakim

I see it when in the PNG?

 

jas0501

It isn't included. One approach is to scrape it from the archive listing, though you don't seem to analyze any games. Here is one I analyzed:


Another approach is to scrape it from the analyzed game:

Hyper-Dragon

The problem with scraping it is the you have to be logged in (this was a fairly recent change) and then deal with a Cloudflare captcha. It is a pity that the API never seems to keep up with site changes.  It would be great to have the analysis included in the comments too.

jas0501
Hyper-Dragon wrote:

The problem with scraping it is the you have to be logged in (this was a fairly recent change) and then deal with a Cloudflare captcha. It is a pity that the API never seems to keep up with site changes.  It would be great to have the analysis included in the comments too.

Assuming you have run the analysis in the first place.

Suggestion I'm submitting via:

When a game has been analyzed include in the PGN a comment or comments containing:

  • the accuracy numbers
  • some kind of CAPS version number
  • analysis depth
  • analysis engine and engine version
  • timestamp of analysis
Ndrizky
jas0501 wrote:

It isn't included. One approach is to scrape it from the archive listing, though you don't seem to analyze any games. Here is one I analyzed:


Another approach is to scrape it from the analyzed game:

Thank you Jason for your comments. So here, I am actually not talking about specifically my own games. I am interested in knowing seeing this for games also that are not mine. So, I actually want to obtain it without analyzing any game. I guess the only way might be to use an external algorithm and feed the pgns as input?

 

Ndrizky
Ndrizky wrote:
jas0501 wrote:

It isn't included. One approach is to scrape it from the archive listing, though you don't seem to analyze any games. Here is one I analyzed:


Another approach is to scrape it from the analyzed game:

Thank you Jason for your comments. So here, I am actually not talking about specifically my own games. I am interested in knowing seeing this for games also that are not mine. So, I actually want to obtain it without analyzing any game. I guess the only way might be to use an external algorithm and feed the pgns as input?

so in which case the engine analyzes and outputs the accuracy for each player for each game in pgn. Done anything like this in python? Any idea if there is some version of the code available that I can refine to do what I want? Thanks a bunch buddy

 

NefariousNebula

I've looked for a solution to the same question and I'll pass along what I found. I've been trying find where I originally stumbled across this, whether it was somewhere on the site, on Github, or elsewhere, but it certainly wasn't something I found on my own. There is an undocumented way to access accuracy scores. If you construct a URL for a user and a page from their game archive, you can return JSON that includes the accuracy. The two fields ("user1Accuracy" for White and "user2Accuracy" for Black) are rather buried in the JSON, but with a bit of work I'm sure you can access. I never took the time to access it so I don't have any Python scripts to pass along unfortunately.

Example URL: https://www.chess.com/callback/user/daily/archive?userId=59264884&all=1

  • The "userID" value can be extracted via https://api.chess.com/pub/player/{username}. The example is my own.
  • The "all" value is the page number from the game archives, with 1 being the most recent.
jas0501

The accuracy numbers need to be computed. CAPS accuracy on chess.com is a proprietary algorithm so you would need to invoke the analysis report on each game. and scrape it from the archive or the analysis report Report tab.

Hyper-Dragon
NefariousNebula wrote:

There is an undocumented way to access accuracy scores. If you construct a URL for a user and a page from their game archive, you can return JSON that includes the accuracy.

 

A brilliant find, thanks so much for posting this. 

Ndrizky
NefariousNebula wrote:

I've looked for a solution to the same question and I'll pass along what I found. I've been trying find where I originally stumbled across this, whether it was somewhere on the site, on Github, or elsewhere, but it certainly wasn't something I found on my own. There is an undocumented way to access accuracy scores. If you construct a URL for a user and a page from their game archive, you can return JSON that includes the accuracy. The two fields ("user1Accuracy" for White and "user2Accuracy" for Black) are rather buried in the JSON, but with a bit of work I'm sure you can access. I never took the time to access it so I don't have any Python scripts to pass along unfortunately.

Example URL: https://www.chess.com/callback/user/daily/archive?userId=59264884&all=1

  • The "userID" value can be extracted via https://api.chess.com/pub/player/{username}. The example is my own.
  • The "all" value is the page number from the game archives, with 1 being the most recent.

Amazing, sounds exactly like what I need. I will try and see what I can get. Thank you so much for your input. 

Ndrizky
jas0501 wrote:

The accuracy numbers need to be computed. CAPS accuracy on chess.com is a proprietary algorithm so you would need to invoke the analysis report on each game. and scrape it from the archive or the analysis report Report tab.

I definitely get your point @Jas0501. What if the analysis is invoked at game point backend, in which case it's actually saved but you need to run it to see it. Anyways, it doesn't hurt to try his suggestion. Thank you for your input.

Hyper-Dragon
Ndrizky wrote:
NefariousNebula wrote:
  • The "all" value is the page number from the game archives, with 1 being the most recent.

Amazing, sounds exactly like what I need. I will try and see what I can get. Thank you so much for your input. 

 

If only life was so simple.  I have been playing about with this today and have the (pre-calculated) CAPS scores from the .../daily/archive... URL.  Unfortunately, whilst better than nothing, the 'all' parameter does not seem to work.  Irrespective of the number I use I still have the same 20 records returned.   If anyone has any ideas on how to get this endpoint to return more data your thoughts would be much appreciated.

jas0501

The url has a page number and there up to 100 pages retuned via live games archive query. As in https://www.chess.com/games/archive?gameOwner=my_game&gameType=live&timeSort=desc&;page=1

or for other user_name's games

https://www.chess.com/games/archive/user_name?gameOwner=other_game&gameType=live&timeSort=desc&page=1

 

Hyper-Dragon
jas0501 wrote:

The url has a page number and there up to 100 pages retuned via live games archive query. As in https://www.chess.com/games/archive?gameOwner=my_game&gameType=live&timeSort=desc&;;;page=1

or for other user_name's games

https://www.chess.com/games/archive/user_name?gameOwner=other_game&gameType=live&timeSort=desc&page=1

 

I already had code to scrape the CAP's scores from the archive URL's but then chess.com, in its wisdom, moved the pages behind a login screen.

The endpoint suggested by NefariousNebul works well because you can hit it without login but it only ever seems to return the same 20 records regardless of the value of the 'all' parameter.

 

NefariousNebula

This is an old thread I'm resurrecting, but for those that might come back to this, CDC now offers the CAPS/accuracy scores as part of the API. Not sure when it was added, but they can be found in the monthly archive call via https://api.chess.com/pub/player/{username}/games/{YYYY}/{MM}.

Hyper-Dragon
NefariousNebula wrote:

This is an old thread I'm resurrecting, but for those that might come back to this, CDC now offers the CAPS/accuracy scores as part of the API. Not sure when it was added, but they can be found in the monthly archive call via https://api.chess.com/pub/player/{username}/games/{YYYY}/{MM}.

Back from the dead again...

For anyone using the C# API at https://github.com/nullablebool/ChessDotComSharp, i've added the CAPS scores and sent a pull request (now accepted).

 

Not sure why the changelog does not seem to be updated anymore.  Very frustrating 😤.

 

 

I