Error 429

I was not checking my scripts for some time and today I noticed that from 9th Oct, all my script requests fail with 429. For example, there is the list of latest errors. If I try the same URL from the browser that it works properly.

I am accessing data sequentially (not parallerly).

Any idea, what is wrong?

Thanks.

I just noticed slight change in behaviour

First call to https://api.chess.com/pub/club/sk-slavia-praha/members is successull, and than two following requests fails.

I know purpose of this error, but my script gets this response now for all requests although I send just one request for one URL.

I have the same problem Dale00007 is reporting, error 429 is there even in the first call to a resource:

8884 players.

005german Got an error:  429

007bottom Got an error:  429

00simon00 Got an error:  429

01067774180 Got an error:  429

Just to help debug this problem, here is the code I was using (trying to get the list of players for a country, and then call sequentially for stats for each of them)

https://gist.github.com/alopezo/fff05dbd8298cfff4f645eeea165f110

I saw the other thread with the proposed workaround, it doesn't seem to work for me on the same machine or using a different machine that uses the same public ip as the first machine. I don't initially have the problem if I run it from a machine with another public ip (e.g. a virtual machine in the internet) -no parallel/multithreaded procesing, so it could explain error 429 - moving to serial processing did not solve the issue because of the cache I guess - perhaps I should add a delay. But soon the error 429 is there again when I am looping to get the stats or the profile for each player.

Thanks for the effort, lots of interesting things you are enabling us to do to facilitate club management and team match monitoring. Very much appreciated.

Thanks for the info, guys. I had suspected that there was a bug in the rate limiting code, and your evidence supports my suspicions so I will dig in and see what I can fix even without a concrete bug-reproduction procedure. I've cleared the tracking log, which should fix things for you now. I've made a ticket for me to fix this, but this week is the annual Chess.com meet-up where we have a lot of meetings and so code is expected to only trickle out to production. Please alert me if you get blocked again for any period longer than 10 minutes. If you keep logs of your requests, you might be able to unblock yourself by re-requesting any URL that returned a 202 or 500 response code.

It still doesn§t work. Or at least it failed at 8pm CEST:

On the other hand I have tried club/members API and country API and those works now. So the issue with only player API ... 

As of now I am limiting to call this API only to update 200 players in one slot each hour.

Petr,

The rate limiting is not per endpoint, but per client, so limiting specifically the player endpoints won't help directly. I cannot see anything that is blocking your access to the endpoints you show above. When I reviewed the access logs for "/pub/player/maxgluteus", I saw the first request on October 16 resulted in a 429, and the second (3 hours later) gave a 200 response with full data. I think things cleared up for you — I see about 25,000 requests in 3 days, 58% have data (200 code) and 40% validated the cache (304 code), and most of the rest are 404s with only 87 responses with a 429 code. We are running a new access log index, and I'm curious if these numbers match what you are seeing.

From what I see, it was the rate-limiter bug that caught you, and my workaround should keep things mostly clear until I return to my office and can fix it properly. Let me know if you still see problems!

It already works. Thanks.

I started to read /stats and as part of that process it seems I crashed again something :-)

I am getting 429 on all player, player/games, player/stats calls - for example:

Hi @Dale00007, I was encouraged by your report but have been unable to get the stats to work consistently. For example, if I get a list of country member links to call their stats some get retrieved, but most of them give 429. 

I also think that a list of players with flattened stats might probably work better when processed server side, rather than querying for each resource in a long list.

My use case is retrieving players from country Argentina that have logged in recently (<7 days), remove the ones that already are members of team Argentina or have a high timeout rate, consider the ones with a number of ongoing daily chess games and get their daily and 960daily rating and number of played games. 

Hope to see this working somewhat soon or I will have to retrieve the stats in manual mode until it is available.

@dale00007, your work looks very interesting to club administrators, hope the new public API would make it easier to progress.

Best regards,

Guillermo

New rate limiting! The servers are happy with the amount of traffic so we have increased the limits. Now up to 3 simultaneous requests for non-cached data are allowed per connection. You should be receiving far fewer 429s now. If you still get some, let me know! We will continue to increase the number of allowed simultaneous non-cached requests as our servers prove that they can support the traffic without gameplay suffering.

Thanks. But good was that I haven't seen 429 for a few weeks already.