Error 403 in member profile

I'm still getting 403 errors for every request.

Yes my request works in R, in the browser if i copy the request url. So it seems my IP is not banned at all. Maybe some update version problem in python? My python version is: Python 3.10.12.

If I had to guess, the default python user agent with the requests library (looks like python-requests/version number) is probably blocked (causes a 403), as its very likely lots of python was requesting info from the API. Lots of libraries will set a default user agent string if one is not provided.

R may not have as many users so the user agents may not have been blocked yet?

I'm still in 3.9.7 but you're also having the problem using 3.10.12, so perhaps we both need to update but I'd prefer to avoid that until we have a definitive answer from the staff.

That's interesting information but it doesn't explain my problem because I've been providing a personalised User-Agent since I started using the Python Requests module....

specifically:

import os, requests, time, json, keyboard

....

headers = {'User-Agent': 'Chess.py (Python 3.9) (username: stephen_33; contact: swilliamsx33@gmail.com)'}

r = requests.get(url, headers)

....

#

Changing to this: headers = {'User-Agent': 'swilliamsx33@gmail.com)'} hasn't fixed it.

* I'm starting to wonder if maybe I've been defaulting to the Python default User-Agent all this time and it's only just become a problem - is that likely?

I correct my self, you can check this by running:

r.request.headers["User-Agent"]

i get this response even though i specify my user-agent:

'python-requests/2.25.1'

But in R it doesnt seem to be a problem since i get this:

* useragent: libcurl/7.81.0 r-curl/4.3.2 httr/1.4.6

By the way, I just dropped my email from the headers and it still worked. Perhaps they're only strict about it in busy hours or something.

>>> import requests

>>> session = requests.session()

>>> session.headers["User-Agent"] = "username: wallace_wang, email: wallace.dev@proton.me"

>>> session.get("https://api.chess.com/pub/player/wallace_wang")

<Response [200]>

>>> session.headers["User-Agent"] = "wallace.dev@proton.me"

>>> session.get("https://api.chess.com/pub/player/wallace_wang")

<Response [200]>

>>> session.headers["User-Agent"] = "wallace_wang"

>>> session.get("https://api.chess.com/pub/player/wallace_wang")

<Response [200]>

Ok i solved my problem using the correct parameter in the requests function.

I was using r.requests.get(url, paramas = {'User-Agent': 'username: myaccount, email: my@emaill'}.

But using headers instead of paramsfixed the problem:

r.requests.get(url, headers = {'User-Agent': 'username: myaccount, email: my@emaill'}

Now i get r.request.headers["User-Agent"] updated correctly and response 200.

i guess your problem is that you are using params instead of headers, since the request function uses params as default if you don't specify the paramater: requests.get(url, params=None, **kwargs)

try this r = requests.get(url, headers=headers) so you pass headers as kwargs.

I asked about IP in case it was a problem with a poor performing script, but @AfraidToShootStrangers may be right, and it's possible that Python isn't passing the User-Agent information the way the site expects and something may have changed in those regards on the site.

I asked about it but haven't heard back yet

This has been my constant for many months now. Combination of literal and variables. Hasn't been an issue so far.

.setRequestHeader "User-Agent", ModName & ", username: " & UNameWindows & "; contact: me.name@hotmail.com"

I have tested in python:

url = "https://api.chess.com/pub/player/" + user
params = {'User-Agent': 'username: fgawkfiv, email: myapp@gmail.com'}
r = requests.get(url,params=params)

It still returns 403. I tested navigator and python code from the same machine (same IP).

Yeah, I'm starting to see what the problem is and also the solution ...

>>> help(requests.get)
Help on function get in module requests.api:

get(url, params=None, **kwargs) Sends a GET request.

param url: URL for the new :class:`Request` object.
param params: (optional) Dictionary, list of tuples or bytes to send
in the query string for the :class:`Request`.
param \*\*kwargs: Optional arguments that ``request`` takes.
return: :class:`Response <Response>` object
rtype: requests.Response

#

I remember having a conversation with other members here about five years ago when I had no idea what a 'User-Agent' was and someone offered a suggestion for a suitable format and that's what I copied into my requests routine. It was one of those things that I planned to go back and check later but since it worked, and has done ever since, I never got around to it.

Curious though that so many other users have experienced exactly the same problem if mine was caused by an incorrectly coded request?

Another thing that puzzles me is this - if I've been including a second positional argument in my requests calls, why hasn't Python been raising an error? Normally, the number of positional arguments presented must match exactly the number of positional parameters in the function. Isn't that true in every language?

Anyway, good news - it's fixed for me. I'm now using the following ....

requests.get(url, headers = {'User-Agent': 'Chess.py (Python 3.9) (username: stephen_33; contact: swilliamsx33@gmail.com)'})

Many thanks for help and feedback 😊

As i reccomended to stephen_33, you have to use the key_word headers (not params).

try :

url = "https://api.chess.com/pub/player/" + "yourusername"

user_agent = {'User-Agent': 'username: fgawkfiv, email: myapp@gmail.com'}

r = requests.get(url, headers = user_agent)

look at the structure of the function request.get:

get(url, params=None, **kwargs)

you only need to pass the url argument, params is optional.

If you pass 2 arguments without specifying the corresponding key_word, the first one is url, the second one is params.

If you want to pass other arguments, i.e. headers, you have to specify the key_word headers.

Examples:

request.get(arg1, arg2) takes the first as url and the second as params

request.get(arg1, headers = arg2) takes the second one as header.

Thanks stephen_33 and AfraidToShootStrangers! Now, it works for me.

This is worth a read for Python users:-

https://requests.readthedocs.io/en/latest/user/quickstart/#passing-parameters-in-urls

Also this:-

https://requests.readthedocs.io/en/latest/user/quickstart/#custom-headers

Custom Headers¶

If you’d like to add HTTP headers to a request, simply pass in a dict to the headers parameter.

For example, we didn’t specify our user-agent in the previous example:

>>> url = 'https://api.github.com/some/endpoint'
>>> headers = {'user-agent': 'my-app/0.0.1'}

>>> r = requests.get(url, headers=headers)

Note: Custom headers are given less precedence than more specific sources of information. For instance:

• Authorization headers set with headers= will be overridden if credentials are specified in .netrc, which in turn will be overridden by the auth= parameter. Requests will search for the netrc file at ~/.netrc, ~/_netrc, or at the path specified by the NETRC environment variable.

• Authorization headers will be removed if you get redirected off-host.

• Proxy-Authorization headers will be overridden by proxy credentials provided in the URL.

• Content-Length headers will be overridden when we can determine the length of the content.

Furthermore, Requests does not change its behavior at all based on which custom headers are specified. The headers are simply passed on into the final request.

Note: All header values must be a string, bytestring, or unicode. While permitted, it’s advised to avoid passing unicode header values.

#

It's that (in red) that I should have gone back and checked later! 😄

Found out there was a block added yesterday for the default Python User-Agent. Changing how the headers are passed must change that as well.

More brilliant communication from the devs.

It wasn't from the devs. There was an issue with some DBs getting excessive traffic and that was put in to stem that as a quick stop-gap

I just found out about it this morning.