bpo-31399: Let OpenSSL verify hostname and IP address (#3462)

bpo-31399: Let OpenSSL verify hostname and IP

The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and
X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses.

* Remove match_hostname calls
* Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host()
* Add documentation for OpenSSL 1.0.2 requirement
* Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform.
* Add hostname_checks_common_name

Signed-off-by: Christian Heimes <christian@python.org>

2 files changed, 72 insertions, 4 deletions

diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
index 4c44ffa..aa1075d 100644
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -146,9 +146,10 @@ Functions, Constants, and Exceptions
 
 .. exception:: CertificateError
 
-   Raised to signal an error with a certificate (such as mismatching
-   hostname).  Certificate errors detected by OpenSSL, though, raise
-   an :exc:`SSLCertVerificationError`.
+   An alias for :exc:`SSLCertVerificationError`.
+
+   .. versionchanged:: 3.7
+      The exception is now an alias for :exc:`SSLCertVerificationError`.
 
 
 Socket creation
@@ -430,8 +431,14 @@ Certificate handling
       of the certificate, is now supported.
 
    .. versionchanged:: 3.7
+      The function is no longer used to TLS connections. Hostname matching
+      is now performed by OpenSSL.
+
       Allow wildcard when it is the leftmost and the only character
-      in that segment.
+      in that segment. Partial wildcards like ``www*.example.com`` are no
+      longer supported.
+
+   .. deprecated:: 3.7
 
 .. function:: cert_time_to_seconds(cert_time)
 
@@ -850,6 +857,14 @@ Constants
 
    .. versionadded:: 3.5
 
+.. data:: HAS_NEVER_CHECK_COMMON_NAME
+
+   Whether the OpenSSL library has built-in support not checking subject
+   common name and :attr:`SSLContext.hostname_checks_common_name` is
+   writeable.
+
+   .. versionadded:: 3.7
+
 .. data:: HAS_ECDH
 
    Whether the OpenSSL library has built-in support for Elliptic Curve-based
@@ -1075,6 +1090,12 @@ SSL sockets also have the following additional methods and attributes:
       The socket timeout is no more reset each time bytes are received or sent.
       The socket timeout is now to maximum total duration of the handshake.
 
+   .. versionchanged:: 3.7
+      Hostname or IP address is matched by OpenSSL during handshake. The
+      function :func:`match_hostname` is no longer used. In case OpenSSL
+      refuses a hostname or IP address, the handshake is aborted early and
+      a TLS alert message is send to the peer.
+
 .. method:: SSLSocket.getpeercert(binary_form=False)
 
    If there is no certificate for the peer on the other end of the connection,
@@ -1730,6 +1751,17 @@ to speed up repeated connections from the same clients.
    The protocol version chosen when constructing the context.  This attribute
    is read-only.
 
+.. attribute:: SSLContext.hostname_checks_common_name
+
+   Whether :attr:`~SSLContext.check_hostname` falls back to verify the cert's
+   subject common name in the absence of a subject alternative name
+   extension (default: true).
+
+   .. versionadded:: 3.7
+
+   .. note::
+      Only writeable with OpenSSL 1.1.0 or higher.
+
 .. attribute:: SSLContext.verify_flags
 
    The flags for certificate verification operations. You can set flags like
@@ -2324,6 +2356,10 @@ in this case, the :func:`match_hostname` function can be used.  This common
 check is automatically performed when :attr:`SSLContext.check_hostname` is
 enabled.
 
+.. versionchanged:: 3.7
+   Hostname matchings is now performed by OpenSSL. Python no longer uses
+   :func:`match_hostname`.
+
 In server mode, if you want to authenticate your clients using the SSL layer
 (rather than using a higher-level authentication mechanism), you'll also have
 to specify :const:`CERT_REQUIRED` and similarly check the client certificate.
diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst
index 133975a..1ece6a3 100644
--- a/Doc/whatsnew/3.7.rst
+++ b/Doc/whatsnew/3.7.rst
@@ -568,6 +568,32 @@ can be set within the scope of a group.
 ``'^$'`` or ``(?=-)`` that matches an empty string.
 (Contributed by Serhiy Storchaka in :issue:`25054`.)
 
+ssl
+---
+
+The ssl module now uses OpenSSL's builtin API instead of
+:func:`~ssl.match_hostname` to check host name or IP address. Values
+are validated during TLS handshake. Any cert validation error including
+a failing host name match now raises :exc:`~ssl.SSLCertVerificationError` and
+aborts the handshake with a proper TLS Alert message. The new exception
+contains additional information. Host name validation can be customized
+with :attr:`~ssl.SSLContext.host_flags`.
+(Contributed by Christian Heimes in :issue:`31399`.)
+
+.. note::
+   The improved host name check requires an OpenSSL 1.0.2 or 1.1 compatible
+   libssl. OpenSSL 0.9.8 and 1.0.1 are no longer supported. LibreSSL is
+   temporarily not supported until it gains the necessary OpenSSL 1.0.2 APIs.
+
+The ssl module no longer sends IP addresses in SNI TLS extension.
+(Contributed by Christian Heimes in :issue:`32185`.)
+
+:func:`~ssl.match_hostname` no longer supports partial wildcards like
+``www*.example.org``. :attr:`~ssl.SSLContext.host_flags` has partial
+wildcard matching disabled by default.
+(Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in
+:issue:`31399`.)
+
 string
 ------
 
@@ -1120,6 +1146,12 @@ Other CPython implementation changes
   emitted in the first place), and an explicit ``error::BytesWarning``
   warnings filter added to convert them to exceptions.
 
+* CPython' :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl.
+  OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer
+  supported. LibreSSL is temporarily not supported as well. LibreSSL releases
+  up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs.
+
+
 Documentation
 =============