Updated Python Version & Having Trouble with API Downloads Now!

Sort:
stephen_33

My laptop crashed after a routine download this week - long story short - I had to reinstall Python which gave me the chance to download the latest version: 3.7.2

So far so good but I've started getting errors when accessing endpoints on this site. The pertinent one (I think) is this:-

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>

I can run test codes to download API data from other sites but not chess.com. This is the cod I'm running:-

import urllib.request, json

url = 'https://api.chess.com/pub/match/885716'

def response(url):
____with urllib.request.urlopen(url) as response:
________return response.read()

res = response(url)
print(json.loads(res))

.
If for example I substitute 'https://hacker-news.firebaseio.com/v0/topstories.json?print=pretty' for the url, it works normally & other sites I've tried work fine too but not this one.

I've spent hours searching online for a solution but without success. If possible I'd like to continue using the Python urllib.request module.

chrka

OS and Python distribution would be helpful but I guess you're on macOS? 

If so, it looks as if you might have to install appropriate certificates. Check this search for example.

Short story ( I use Anaconda myself so I can't check) if you are using an installer from Python.org, there might be a "Install Certificates.command" file in "/Application/Python\ 3.7" (or thereabouts) which you can run to get them to install.

stephen_33

My OS is Windows 10 but Python 'distribution' - what's that chrka?

I found something about installing appropriate certificates online but if that's the problem then why can I download API data from some other sites, that have 'https' url's - i.e. also secure sites?

But I'll search through the download package for any files of that kind - thanks  happy.png

chrka

There are several places from where you can download Python, often with some packages and tools pre-installed. These bundles are often referred to as "distributions". (There's a list of some of them at https://wiki.python.org/moin/PythonDistributions.)

I figured you might have been using macOS since most people having that problem seems to be on macs. Unfortunately I don't know why it works on some sites but not on chess.com (I just tried out your example on 3.7.2 on my system (macOS, Anaconda Python) and it worked fine for me).

Hope you manage to get it sorted!

stephen_33

I see. I downloaded it from the Python website itself & once you click on the downloader, the rest is automatic. I can't remember any other information being given.

Strange thing but I can't find a folder marked Pythonxx anywhere on my PC. The old one (Python3.3) was placed in the root directory but I haven't a clue where the new one is.

chrka

Maybe try this and see if it helps (I have the latest version on my system, but I think you should have it already too, but worth a try): 
pip install --upgrade certifi

(See https://pypi.org/project/certifi/ — "Certifi is a carefully curated collection of Root Certificates for validating the trustworthiness of SSL certificates while verifying the identity of TLS hosts. It has been extracted from the Requests project.")

stephen_33

Much as I hate throwing in the towel on these things, I've had to put Python 3.7.2 to one side for now & download a fresh copy of vers. 3.3. That's working fine with my existing scripts.

It feels like a backward step but there are about 80 teams waiting for me to post the round 2 draws in the TMCL tournament, so I'm under a certain amount of time pressure!

Back soon to try & sort this out .....

NichtGut

Of no help whatsoever but perhaps funny.

I thought you had Linux Mint for a second and had uninstalled the Python 2.7 it comes with. I was just gonna get started with the prayers.

stephen_33

'Linux Mint'? Never heard of it. From what I'm reading Python2.x is fast becoming redundant & users are being encouraged to migrate to vers. 3.x

There're marked differences between the two that make even simple scripts incompatible. For example, a basic print statement is executed in 2.x with print "string" but in 3.x it's used much as a function: print("string").

High time the language was fully standardised.

stephen_33
NichtGut wrote:

I was just gonna get started with the prayers.

No, I've tried that already & it didn't work!  wink.png

Odd thing though - when I re-installed Python3.3 it was automatically placed in my C: drive's root directory where I'd expect it to go. But I still haven't found a folder named Python3.7.2. What I did find yesterday, after a lot of searching, were the Python application files (3 in total) but they've been put in the Windows folder by the Python 3.7.2 installer.

It looks really strange & I can't see any associated library files or other supporting documentation files in the same folder. When I open the 3.7.2 version it works perfectly normally except for this SSL-certificates problem.

stephen_33

From the reading I've done of other people's problems with SSL certification, it looks as if I need to use a slightly different way of making my endpoint requests. I'm going to install the Requests module which is a non-standard module from a third party but is highly regarded.

Some people have said it solved their problem immediately because it handles certification automatically & more securely than the standard Python module commands. Only thing is that I first need to instal pip & that doesn't seem entirely straightforward - working on that ...

chrka

@stephen_33 Pip is installed by default (since 3.4 IIRC).

stephen_33

"Pip is installed by default (since 3.4 IIRC)." - so why can't my Windows find it when I try to run it?

I assume it's meant to be opened like a normal application?

chrka

I don't have any experience with Python on Windows sad.png but pip is usually used from the command line and I'd assume it's the same on Windows? (Sorry I can't be of more help)

If that's what you're doing, then maybe your path isn't set up properly? (Or if it's just the "pip" command it doesn't find, you should be able to use "python -m pip <arguments>" instead.)

Some python distributions do come with a graphical user interface though which you can use to install packages (and create virtual environments and so on) (eg., https://www.anaconda.com/download/#windows)

stephen_33

I've found references to pip being bundled with all versions of Python after 3.4 (my first version was 3.3 so doesn't have it). One thing I've managed to establish at least is that the latest version (3.7.2) was installed in my C:\Users\Stephen\AppData\Local\Programs\ folder, so I can at least search through all the folders within that.

I haven't managed to find any file or app that looks like pip though. What I have found is a package within the Python library called pip but when I import it, I can't do anything with it.

stephen_33

Got there!  happy.png

Managed to find the exact location after some searching & invoked pip from the command line with an 'install Requests' command & it worked. I'm now able to import the requests module, so I should be able to use that instead of the code I had been using. The requests module is said to be more up to date & functional, so hopefully it will solve the problem.

Thanks for the tips. (Time for dinner - I'll test that later ...)