Error with https://api.chess.com/pub/country/{ID}/players

Sort:
Tricky_Dicky

Until recently I have had no problem with this end point for 'GB' although I am aware it will not work for 'US' due to the dataset size.

Is this now becoming an issue for all larger country login member lists? Typically I would get between 130k - 170k logins in the dataset and that has been OK until now.

https://api.chess.com/pub/country/GB/players

Error 524

Ray ID: 6ba55c8fff3e7556 • 2021-12-08 10:39:39 UTC

A timeout occurred

hippobaguette

That's interesting, the endpoint seems to work for me

 

curl https://api.chess.com/pub/country/GB/players

Gives me 

StatusCode : 200
StatusDescription : OK
Content : {"players":["---k---","-shinobi-","-winner-","0-michael-0","0-town","000000001a","000fox","000hello
world000","000isaac000","000mr000","000pdh","000sarah","001001011001","005s","007ajhughes","007bon"
,"...
RawContent : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
X-Chesscom-Matched: pubapi_country_players
Access-Control-Allow-Origin: *
Link: <https://api.chess.com/con.
Forms : {}
Headers : {[Transfer-Encoding, chunked], [Connection, keep-alive], [Vary, Accept-Encoding],
[X-Chesscom-Matched, pubapi_country_players]...}
Images : {}
InputFields : {}
Links : {}
ParsedHtml : mshtml.HTMLDocumentClass
RawContentLength : 1923254

 

Tricky_Dicky

Interesting indeed. I am not using curl however. I had an issue with EXCEL spreadsheet upload and with firefox browser.

Just tried that in a command window and it did seem to work.

Nevfy

It works with Firefox for me.

Tricky_Dicky

It has just worked for me with EXCEL and firefox. Think it might be a server issue.

The time out error possibily a result of not enough resource to service the request.

But it did seem to apply only to larger datasets. had the same issue with 'CA' this AM. Had it a few times in recent days for 'GB'

All small datasets were fine.

YankeeBastid

I am running the

https://api.chess.com/pub/country/{ID}/players

for the US and getting timeout error 502. Is this a fixable error or is the site unable to handle this request?

 

YankeeBastid

Just tried it in CHROME and it worked fine for the US

 

Tricky_Dicky

Know issue. dataset too big

Tricky_Dicky

Similar problems again today.

This is from Cloudflare troubleshoot:

Error 524: a timeout occurred

Error 524 indicates that Cloudflare successfully connected to the origin web server, but the origin did not provide an HTTP response before the default 100 second connection timed out. This can happen if the origin server is simply taking too long because it has too much work to do - e.g. a large data query, or because the server is struggling for resources and cannot return any data in time.

YankeeBastid

Is it possible to change the 100 second connection timeout limit? I will be needing to retrieve large data in the future...

 

Tricky_Dicky

Last three days. Same error. Time out and cloudflare web page HTML code returned as end point.

Regardless of data size end points shoul never return anything other than data or an error code. Definitely not html code.

YankeeBastid

I think they turned it off because I was pulling large numbers of members.

 

Tricky_Dicky

it still works, but intermittently. probably a resource issue.

But the end point should never return HTML code. That's just wrong.

bcurtis

Hey there. The sudden growth of activity on the site about a year ago caused some of these larger result sets to take too long — right at the same time our developers were needed to handle other parts of the site. Also, as we have enhanced our security to protect the site from DDoS attacks and hardware failure, sometimes Cloudflare mis-categorizes unusual requests like these and initiates a response themselves; since these responses are delivered to you before our server can respond, Cloudflare does not know that it should not deliver HTML. You might be able to avoid this by setting the appropriate "Accept" header, specifying the JSON data type.

A year is a long time for a problem like this to exist. You have my personal apology, and please know that I want to improve our response times. We have re-prioritized this issue, and hope to find some developer time soon. The most likely solution will be to reduce page sizes, which means that you will need to make multiple requests to obtain complete lists. This may be straight forward to adapt to for programs, but for simple scripting like in Excel it may pose a problem — but no more of a problem than the current "no data" response.

YankeeBastid

bcurtis, thanks for the quick response. I do not want to get on the wrong side of the organization, so would like an opportunity to explain what 4teamchess.com is about and what I am going to need to implement it. Should I do it here in a public forum, or can we discuss it privately?

erik did not know I would be pulling the complete chess.com membership into my table. I have already retrieved ~ a million players. I have stopped until I am sure what I am doing does not go against the understanding of chess.com's best interests. what say you?

Tricky_Dicky

Now getting internal DB error for the larger datasets instead of cloudflare time out.

https://api.chess.com/pub/country/GB/players

"An internal error has occurred. Please contact Chess.com Developer's Forum for further help https://www.chess.com/club/chess-com-developer-community ."

Tricky_Dicky

Seems there may be some changes happening to the Country endpoint

For larger dataset, it now returns a string with a message at the end saying data restricted.

comment "Results are temporarily …for performance reasons"
Martin_Stahl

I asked someone about it. Haven't heard anything yet but I linked them to this topic.

Tricky_Dicky

Thanks Martin

RainPiper

Any news about this issue?

We experience the same problem at:

https://api.chess.com/pub/country/NL/players

It says: "Results are temporarily reduced to the first 10000 players for performance reasons". Would be nice to hear how long this "temporarily" should be expected to last.