Is bulk downloading games possible?

Sort:
dawgisgreat

I am looking for ways to bulk download game PGNs, but the closest thing I found is using the countries api to get 10k users per country and download all of their games using the player archive API. However this is absurdly slow and ideally I'd like to download several billion games. Am I missing some official way to do this? Does chess.com sell that kind of data for reasonable prices?

stephen_33

No, that's the only method of doing that the site provides.

dawgisgreat

I found a list of 2 million users, but fetching their games would take like 100 days :/

owenldyer

https://api.chess.com/pub/player/<username here>/games/archives

this gives links to a player's monthly games. from there you can get all of the games that they have played in each month. I have been trying to figure out a way to bulk download my games with the chess.com analysis in the pgn comments but have been unsuccessful so far.

dawgisgreat
owenldyer hat geschrieben:

https://api.chess.com/pub/player/<username here>/games/archives

this gives links to a player's monthly games. from there you can get all of the games that they have played in each month. I have been trying to figure out a way to bulk download my games with the chess.com analysis in the pgn comments but have been unsuccessful so far.

too slow, this would take over 100 days

owenldyer
dawgisgreat wrote:
owenldyer hat geschrieben:

https://api.chess.com/pub/player/<username here>/games/archives

this gives links to a player's monthly games. from there you can get all of the games that they have played in each month. I have been trying to figure out a way to bulk download my games with the chess.com analysis in the pgn comments but have been unsuccessful so far.

too slow, this would take over 100 days

... I don't think so. If you had player's usernames, you could easily get thousands, if not tens of thousands, games per minute. I'm not sure if you are talking about writing a script to automate this, or if you want to do it by hand.

Ximoon

Even with script, due to speed limit without the oauth I get less than 2 requests per second served. Is it really that much more with the oauth?

I mean yes you'd get all those games, but to go through all the 2 millions users you'd need at least 20 days.