Help me with chess-web-api errors

EDIT

Should have looked at the exception, it's because of usage limits:

[Error [WebapiError]: Too Many Requests] { statusCode: 429 }

Where can I find out about the usage limits?

I'm trying to figure out the number of cheaters per country, so Ive written a small application to do just that which connects to the chess-web-api. Basically, it gets all the member names per country (in this case Pakistan). Then it sends a request for player information for each member and checks the members status. If the member has a status of 'closed:fair_play_violations' then it is counted as a cheater

It works for small numbers (e.g. 100 players).. but I get about 500 errors if I go for say  3000 members. The repeated call to  getPlayer fails about 500 times if I have 3000 members. see code screenshot below and actual code pasted below that. Ive added a member into the list who is a known cheater just for test purposes (ajtheboss1709)

Anyone know why I get errors? Is there a quota limit that Im failing? Ideas how to solve? Written in javascript

var ChessWebAPI = require('chess-web-api');

var chessAPI = new ChessWebAPI();
countCheaters().then(() => {
console.log('Finito')
})

async function countCheaters() {
const res = await chessAPI.getCountryPlayers('PK')
const players = res.body.players

const subPlayers = players.slice(0, 3000)
const cheaters = await getCheaters(subPlayers)

console.log('Number of cheaters = ', cheaters.length);
}

async function getCheaters(subPlayers) {
subPlayers.push('ajtheboss1709')
const res = await Promise.all(subPlayers.map(async sp => {
try {
const player = await chessAPI.getPlayer(sp)
return {player: sp, cheater: player.body.status === 'closed:fair_play_violations' ? 'Y' : 'N'}
} catch (ex) {
return {player: sp, cheater: 'UNKNOWN'}
}
}))

console.log('unknown = ' + res.filter(r => r.cheater == 'UNKNOWN').length)

return res.filter(r => r.cheater == 'Y')
}

Should have looked at the exception:

[Error [WebapiError]: Too Many Requests] { statusCode: 429 }

Where can I ffind out about the usage limits?

Ok Ive found my own answer.. if I make it synchronous it shouuld work (although will be slow!)

from https://www.chess.com/news/view/published-data-api#pubapi-general-rate-limits

Rate Limiting
Your serial access rate is unlimited. If you always wait to receive the response to your previous request before making your next request, then you should never encounter rate limiting.

However, if you make requests in parallel (for example, in a threaded application or a webserver handling multiple simultaneous requests), then some requests may be blocked depending on how much work it takes to fulfill your previous request. You should be prepared to accept a "429 Too Many Requests" response from our server for any non-serial request that you make.

In some cases, if we detect abnormal or suspicious activity, we may block your application entirely. If you supply a recognizable user-agent that contains contact information, then if we must block you application we will attempt to contact you to correct the problem.

What's the point of this script? Those accounts that have been closed for fair play, are closed.

Have you tried approaching a member of staff about this? @bcurtis is the SA of this club, so you might message him to ask if he'll post some limits on downloads here.

Failing that, you're probably best advised to split up your api requests into smaller bundles. Say, 100 requests per batch?

look at post #3 - that solved the problem

You are trying to check the status for every player account in each country? Thats a huge amount of work.

yes a huge amount of work (for a computer) - I will likely be drinking a frappucino whilst it does all the hard work!

By the way not intending to run it for every country.. just a few. Pakistan for instance only has about 10K members

O/k but you didn't make it clear at first that you were making multiple, parallel requests?

That does make a difference.

didn't make it clear? I posted the code! I thought this was a community of developers

"I posted the code! I thought this was a community of developers"

My preferred language is Python & users of the api use a range of different means to access it. If you expect all members of this club to be able to understand every piece of code that's posted, you have a little to learn.

hmm well TBH I wouldn't expect you to get involved if you're not familiar with javascript. No offence. I'd probably steer clear of any python questions - although TBH I could read it quite easily

Also, before you even posted I'd identified and posted what the problem was and the solution! So yes, I think it was pretty clear!

By the way the native language for the chess-web-api is javascript. Alll their README and examples are in JS

"Also, before you even posted I'd identified and posted what the problem was and the solution!"

Yes, that was strange because it wasn't there when I wrote my post - I only spotted it afterwards.

I thought you'd edited an earlier post or maybe our posts simply crossed?

But on the issue of 'specialised knowledge', I think non-specialists are still free to offer generalised advice such as contacting the admin of the club, which is all I was doing.

This may seem pedantic but more correctly all the API endpoints are organised in Javascript Object Notation (JSON), which a wide range of applications are able to decode & read.

No one needs to be fluent in Javascript itself to be able to use the API.

You do need to know javascript to use the chess-web-api (which is different from the chess.com public api)

Agree with you, that the usage of the chess.com public api is language independent (as are all API's). In any case will wind my neck in. Think there were some crossed wires

I agree that we've probably done this to death but I'm not clear what you mean by 'chess-web-api'? The site provides & maintains just the one API surely?

What application are you using?

This is the chess-web-api https://www.npmjs.com/package/chess-web-api

It's a nice library that you can plug into any nodeJS app which makes it easier and more convenient to call the API

It's name checked on the main developer page (which is why I used it)

https://www.chess.com/club/chess-com-developer-community

sounds interesting, did you finish the analysis ?

yes, it is quite an interesting topic. Unfortunately I hit a stumbling block. The end point that gives you players per country only give you active players.. so you dont get usernames of closed accounts!

However, I sampled a few countries and got all their active player list as of 10 Jan.. so I will run the test in a few months to identify the proprtion of cheaters from that different sample. That discussion is taking place here - but I must warn you Ive taken a lot of heat on the cheaters forum because of this controversial topic:

https://www.chess.com/clubs/forum/view/profile-of-a-cheater