Python URLLib will not accept Let's Encrypt Websites (from Europe only)

[Naleksuh]

Hello, I have what seems to be the strangest issue. Python's URLLib library gives an error authenticating the certificate of any website using Let's Encrypt, both NFO-based websites and non-nfo websites that use Let's Encrypt (such as catchpoint.com).

When trying to use Python's URLLib to read the contents of this webpage (through

Code: Select all

import urllib

then

Code: Select all

print(urllib.urlopen("https://catchpoint.com").read())

substituting anything else with Let's Encrypt, I get an error looking like this: IOError: [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727) . However, this only happens from my Europe VPS, from either of my United States VPS this does not happen, I have already verified both versions of Python and all software as being the same on both. Any particular reason that Let's Encrypt may not be being verified properly from Europe VPS only? I have already asked NFO Support about this but it seems the problem is not through NFO and I would like to ask here for general community support

[Edge100x]

Are you certain that you're running the latest OS updates? Each OS has a certificate store that contains root certificates, and that's where Let's Encrypt should be authorized as a CA. If your authentication is failing, that suggests that the certificate for LE may be missing. It should have been included in an OS update.

[Naleksuh]

Are you certain that you're running the latest OS updates? Each OS has a certificate store that contains root certificates, and that's where Let's Encrypt should be authorized as a CA. If your authentication is failing, that suggests that the certificate for LE may be missing. It should have been included in an OS update.

Yes, I've tried updating the OS, I have also tried freshly reinstalling it. See also this ticket: https://www.nfoservers.com/support/tick ... ?id=343940

[Naleksuh]

Before, I said the problem is neither the fault of my configuration nor NFO, as I am able to do the exact same procedure in both VPSes with different results, and the only differing factor being regions.

However, I recently tried on my own personal computer (United States) and my friend in the United Kingdom, and it works in both cases there. It is most likely that this problem is the fault of Internap

[Edge100x]

I'm happy to report that this problem turned out to be unrelated to NFO, or INAP, or Europe. It was due to a software quirk with Python and how it interfaces with Windows.

Windows includes a very limited set of root certificates by default and installs the rest on-the-fly when an application using its standard SCHANNEL API (such as any web browser) requests them. Python uses the Windows certificate store, but it does not use the standard API. This means that if you use Python to load a URL that has never been loaded before, the corresponding root certificate may not be trusted yet, and Windows doesn't automatically load it up -- causing the request to fail with "SSL: CERTIFICATE_VERIFY_FAILED".

The workaround is pretty easy, thankfully: Just manually visit the desired website in a browser window first, and it will load properly in Python from there on out.

https://bugs.python.org/issue36011 talks about another workaround for installing an entire bundle of certificates.

In this case, the customer noticed the problem with a certain VDS precisely because it was freshly installed: He hadn't visited his own site yet on that VDS in his web browser.