bpo-32726: macOS installer changes for 3.6.5

Backport the new 10.9+ installer variant from 3.7.  This variant features
more modern options; like 64-bit only (Apple is deprecating 32-bit support
in future macOS releases); a built-in version of Tcl/Tk 8.6.8; built with
clang rather than gcc-4.2.  For 3.6.5, the 10.9+ variant will be offered
as an additional alternative to the traditional 10.6+ variant in earlier
3.6.x releases.  Binary extension modules (including wheels) built for
earlier versions of 3.6.x with the 10.6 variant should continue to work
with either 3.6.5 variant without recompilation.

In addition, both installer variants have updated 3rd-party libraries:
OpenSSL 1.0.2m -> 1.0.2n
XZ 5.2.2 -> 5.2.3
SQLite 3.21.0 -> 3.22.0

Also the 10.6 variant now sets CC=gcc instead of CC=gcc-4.2 and does not
search for the outdated 10.6 SDK.  The variant is built with the same
compiler as before.  As before, for extension module builds, the CC can
be overridden with the CC env variable and an SDK can be specified
with the SDKROOT env variable (see man xcrun).  These minor changes
should be transparent to nearly all users.

Author: ned-deily

Mac/BuildScript/build-installer.py

@@ -0,0 +1,20 @@
+{\rtf1\ansi\ansicpg1252\cocoartf1561\cocoasubrtf200
+{\fonttbl\f0\fswiss\fcharset0 Helvetica;\f1\fnil\fcharset0 LucidaGrande-Bold;\f2\fnil\fcharset0 LucidaGrande;
+\f3\fnil\fcharset0 Monaco;}
+{\colortbl;\red255\green255\blue255;}
+{\*\expandedcolortbl;;}
+\margl1440\margr1440\vieww10540\viewh8400\viewkind0
+\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
+
+\f0\fs28 \cf0 Congratulations!
+\fs24   
+\f1\b\fs28 Python $FULL_VERSION for macOS $MACOSX_DEPLOYMENT_TARGET
+\f2\b0  was successfully installed.
+\fs24 \
+\
+One more thing: to verify the identity of secure network connections, this Python needs a set of SSL root certificates.  You can download and install a current curated set from {\field{\*\fldinst{HYPERLINK "https://pypi.org/project/certifi/"}}{\fldrslt the Certifi project}} by double-clicking on the 
+\f3 Install Certificates
+\f2  icon in {\field{\*\fldinst{HYPERLINK "file://localhost/Applications/Python%20$VERSION/"}}{\fldrslt the Finder window}}.  See {\field{\*\fldinst{HYPERLINK "file://localhost/Applications/Python%20$VERSION/ReadMe.rtf"}}{\fldrslt the 
+\f3 ReadMe
+\f2  file}} for more information.\
+}

Mac/BuildScript/issue19373_tk_8_5_15_source.patch

@@ -1,43 +1,44 @@
-{\rtf1\ansi\ansicpg1252\cocoartf1504\cocoasubrtf750
+{\rtf1\ansi\ansicpg1252\cocoartf1561\cocoasubrtf200
 {\fonttbl\f0\fswiss\fcharset0 Helvetica;\f1\fmodern\fcharset0 CourierNewPSMT;}
 {\colortbl;\red255\green255\blue255;}
 {\*\expandedcolortbl;;}
 \margl1440\margr1440\vieww13380\viewh14600\viewkind0
 \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
 
-\f0\fs24 \cf0 This package will install Python $FULL_VERSION for Mac OS X $MACOSX_DEPLOYMENT_TARGET for the following architecture(s): $ARCHITECTURES.\
+\f0\fs24 \cf0 This package will install Python $FULL_VERSION for macOS $MACOSX_DEPLOYMENT_TARGET for the following architecture(s): $ARCHITECTURES.\
 \
 \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
 
-\b \cf0 \ul \ulc0 Which installer variant should I use?
+\b \cf0 \ul \ulc0 Which installer variant should I use? [CHANGED in 3.6.5]
 \b0 \ulnone \
 \
 \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
 
 \b \cf0 **NEW**
-\b0  For Python 3.6, the python.org website now provides only one installer variant for download: one that installs a 
+\b0  With Python 3.6.5, the python.org website now provides two installer variants for download: one that installs a 
+\i 64-bit-only 
+\i0 Python capable of running on 
+\i macOS 10.9 (Mavericks)
+\i0  or later; and one that installs a 
 \i 64-bit/32-bit Intel
 \i0  Python capable of running on 
-\i Mac OS X 10.6 (Snow Leopard)
-\i0  or later.  This ReadMe was installed with the 
+\i macOS 10.6 (Snow Leopard)
+\i0  or later.  (This ReadMe was installed with the 
 \i $MACOSX_DEPLOYMENT_TARGET
-\i0  variant.  By default, Python will automatically run in 64-bit mode if your system supports it.  The Python installed by this installer is built with private copies of some third-party libraries not included with or newer than those in OS X itself.  The list of these libraries is included at the end of the License.rtf file.
-\b \ul \
-\
-Certificate verification and OpenSSL\
+\i0  variant.)  Previous Python 3.6.x releases only provided the 10.6 or later installer. If you are running on macOS 10.9 or later and if you have no need for compatibility with older systems, use the 10.9 variant.  Use the 10.6 variant if you are running on macOS 10.6 through 10.8, if you need to maintain compatibility with previous 3.6.x releases, or if you want to produce standalone applications that can run on systems from 10.6.  The Pythons installed by these installers are built with private copies of some third-party libraries not included with or newer than those in macOS itself.  The list of these libraries varies by installer variant and is included at the end of the License.rtf file.\
 \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
 
-\b0 \cf0 \ulnone \
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
+\b \cf0 \ul \ulc0 \
+Certificate verification and OpenSSL\
 
-\b \cf0 **NEW**
-\b0  This variant of Python 3.6 now includes its own private copy of OpenSSL 1.0.2.  Unlike previous releases, the deprecated Apple-supplied OpenSSL libraries are no longer used.  This also means that the trust certificates in system and user keychains managed by the 
+\b0 \ulnone \
+This variant of Python 3.6 now includes its own private copy of OpenSSL 1.0.2.  Unlike previous releases, the deprecated Apple-supplied OpenSSL libraries are no longer used.  This also means that the trust certificates in system and user keychains managed by the 
 \i Keychain Access 
 \i0 application and the 
 \i security
 \i0  command line utility are no longer used as defaults by the Python 
 \f1 ssl
-\f0  module.  For 3.6.0, a sample command script is included in 
+\f0  module.  A sample command script is included in 
 \f1 /Applications/Python 3.6
 \f0  to install a curated bundle of default root certificates from the third-party 
 \f1 certifi
@@ -49,16 +50,17 @@ The bundled
 \f1 pip
 \f0  included with the Python 3.6 installer has its own default certificate store for verifying download connections.\
 \
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
 
-\b \cf0 \ul Update your version of Tcl/Tk to use IDLE or other Tk applications
+\b \ul Using IDLE or other Tk applications [NEW/CHANGED in 3.6.5] 
 \b0 \ulnone \
 \
-To use IDLE or other programs that use the Tkinter graphical user interface toolkit, you need to install a newer third-party version of the 
+The 10.9+ installer variant comes with its own private version of Tcl/Tk 8.6. It does not use system-supplied or third-party supplied versions of Tcl/Tk.\
+\
+For the 10.6+ variant, you continue to need to install a newer third-party version of the 
 \i Tcl/Tk
-\i0  frameworks.  Visit {\field{\*\fldinst{HYPERLINK "https://www.python.org/download/mac/tcltk/"}}{\fldrslt https://www.python.org/download/mac/tcltk/}} for current information about supported and recommended versions of 
+\i0  8.5 (not 8.6) frameworks to use IDLE or other programs that use the Tkinter graphical user interface toolkit.  Visit {\field{\*\fldinst{HYPERLINK "https://www.python.org/download/mac/tcltk/"}}{\fldrslt https://www.python.org/download/mac/tcltk/}} for current information about supported and recommended versions of 
 \i Tcl/Tk
-\i0  for this version of Python and of Mac OS X.  For the initial release of Python 3.6, the installer is still linked with Tcl/Tk 8.5.\
+\i0  for this version of Python and of macOS.\
 
 \b \ul \
 Other changes\

Mac/BuildScript/openssl_sdk_makedepend.patch

@@ -1,34 +1,24 @@
-{\rtf1\ansi\ansicpg1252\cocoartf1504
+{\rtf1\ansi\ansicpg1252\cocoartf1561\cocoasubrtf200
 \cocoascreenfonts1{\fonttbl\f0\fswiss\fcharset0 Helvetica;}
 {\colortbl;\red255\green255\blue255;}
-{\*\expandedcolortbl;\csgray\c100000;}
+{\*\expandedcolortbl;;}
 \paperw11905\paperh16837\margl1440\margr1440\vieww12200\viewh10880\viewkind0
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0
+\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
 
 \f0\fs24 \cf0 This package will install 
 \b Python $FULL_VERSION
 \b0  for 
-\b Mac OS X $MACOSX_DEPLOYMENT_TARGET
+\b macOS $MACOSX_DEPLOYMENT_TARGET
 \b0 .\
-\
+\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0
+\cf0 \
 
-\b Python for Mac OS X
-\b0  consists of the Python programming language interpreter, plus a set of programs to allow easy access to it for Mac OS X users including an integrated development environment 
+\b Python for macOS
+\b0  consists of the Python programming language interpreter, plus a set of programs to allow easy access to it for macOS users including an integrated development environment 
 \b IDLE
 \b0 .\
 \
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0
-
-\b \cf0 NEW:
-\b0   There are important changes in this release regarding network security and trust certificates.  Please see the ReadMe for more details.\
-\
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\partightenfactor0
 
-\b \cf0 IMPORTANT:
-\b0  
-\b IDLE
-\b0  and other programs using the 
-\b tkinter
-\b0  graphical user interface toolkit require specific versions of the 
-\b Tcl/Tk
-\b0  platform independent windowing toolkit.  Visit {\field{\*\fldinst{HYPERLINK "https://www.python.org/download/mac/tcltk/"}}{\fldrslt https://www.python.org/download/mac/tcltk/}} for current information on supported and recommended versions of Tcl/Tk for this version of Python and Mac OS X.}
+\b NEW in 3.6.5: 
+\b0 two installer variants (10.9+ 64-bit-only, 10.6+ 64-/32-bit), built-in Tcl/Tk 8.6 support in the 10.9+ variant (no additional third-party downloads!)\
+}

Mac/BuildScript/resources/Conclusion.rtf

@@ -12,6 +12,7 @@ SHARE_DOCDIR_TO_FWK="../../.."
 # make link in /Applications/Python m.n/ for Finder users
 if [ -d "${APPDIR}" ]; then
     ln -fhs "${FWK_DOCDIR}/index.html" "${APPDIR}/Python Documentation.html"
+    open "${APPDIR}" || true  # open the applications folder
 fi
 
 # make share/doc link in framework for command line users

Mac/BuildScript/resources/ReadMe.rtf

@@ -0,0 +1,18 @@
+Fix build failure with +quartz variant on OS X 10.8 and 10.9.
+Even though Gestalt was deprecated in OS X 10.8, it should work fine
+through OS X 10.9, and its replacement NSOperatingSystemVersion was
+not introduced until OS X 10.10.
+
+Patch from MacPorts project and reported upstream:
+https://trac.macports.org/ticket/55649
+--- tk8.6.8/macosx/tkMacOSXXStubs.c.orig	2017-12-06 09:25:08.000000000 -0600
++++ tk8.6.8-patched/macosx/tkMacOSXXStubs.c	2018-01-06 19:34:17.000000000 -0600
+@@ -175,7 +175,7 @@
+     {
+ 	int major, minor, patch;
+ 
+-#if MAC_OS_X_VERSION_MIN_REQUIRED < 1080
++#if MAC_OS_X_VERSION_MIN_REQUIRED < 101000
+ 	Gestalt(gestaltSystemVersionMajor, (SInt32*)&major);
+ 	Gestalt(gestaltSystemVersionMinor, (SInt32*)&minor);
+ 	Gestalt(gestaltSystemVersionBugFix, (SInt32*)&patch);

Mac/BuildScript/resources/Welcome.rtf

@@ -0,0 +1,4 @@
+Provide an additional, more modern macOS installer variant that supports
+macOS 10.9+ systems in 64-bit mode only. Upgrade the supplied third-party
+libraries to OpenSSL 1.0.2n, XZ 5.2.3, and SQLite 3.22.0. The 10.9+
+installer now links with and supplies its own copy of Tcl/Tk 8.6.8.

Mac/BuildScript/scripts/postflight.documentation

@@ -1491,7 +1491,8 @@ Optional Packages:
   --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
   --with-universal-archs=ARCH
                           select architectures for universal build ("32-bit",
-                          "64-bit", "3-way", "intel", "intel-32", or "all")
+                          "64-bit", "3-way", "intel", "intel-32", "intel-64",
+                          or "all")
   --with-framework-name=FRAMEWORK
                           specify an alternate name of the framework built
                           with --enable-framework
@@ -7345,6 +7346,11 @@ $as_echo "$CC" >&6; }
                LIPO_32BIT_FLAGS=""
                ARCH_RUN_32BIT=""
                ;;
+            intel-64)
+               UNIVERSAL_ARCH_FLAGS="-arch x86_64"
+               LIPO_32BIT_FLAGS=""
+               ARCH_RUN_32BIT="true"
+               ;;
             3-way)
                UNIVERSAL_ARCH_FLAGS="-arch i386 -arch ppc -arch x86_64"
                LIPO_32BIT_FLAGS="-extract ppc7400 -extract i386"
@@ -7355,11 +7361,14 @@ $as_echo "$CC" >&6; }
                ;;
             esac
 
-            CFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${CFLAGS}"
-            LDFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${LDFLAGS}"
             if test "${UNIVERSALSDK}" != "/"
             then
+                CFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${CFLAGS}"
+                LDFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${LDFLAGS}"
                 CPPFLAGS="-isysroot ${UNIVERSALSDK} ${CPPFLAGS}"
+            else
+                CFLAGS="${UNIVERSAL_ARCH_FLAGS} ${CFLAGS}"
+                LDFLAGS="${UNIVERSAL_ARCH_FLAGS} ${LDFLAGS}"
             fi
         fi
 

Mac/BuildScript/tk868_on_10_8_10_9.patch

@@ -214,7 +214,7 @@ fi
 AC_SUBST(LIPO_32BIT_FLAGS)
 AC_MSG_CHECKING(for --with-universal-archs)
 AC_ARG_WITH(universal-archs,
-    AS_HELP_STRING([--with-universal-archs=ARCH], [select architectures for universal build ("32-bit", "64-bit", "3-way", "intel", "intel-32", or "all")]),
+    AS_HELP_STRING([--with-universal-archs=ARCH], [select architectures for universal build ("32-bit", "64-bit", "3-way", "intel", "intel-32", "intel-64", or "all")]),
 [
 	UNIVERSAL_ARCHS="$withval"
 ],
@@ -1782,6 +1782,11 @@ yes)
                LIPO_32BIT_FLAGS=""
                ARCH_RUN_32BIT=""
                ;;
+            intel-64)
+               UNIVERSAL_ARCH_FLAGS="-arch x86_64"
+               LIPO_32BIT_FLAGS=""
+               ARCH_RUN_32BIT="true"
+               ;;
             3-way)
                UNIVERSAL_ARCH_FLAGS="-arch i386 -arch ppc -arch x86_64"
                LIPO_32BIT_FLAGS="-extract ppc7400 -extract i386"
@@ -1792,11 +1797,14 @@ yes)
                ;;
             esac
 
-            CFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${CFLAGS}"
-            LDFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${LDFLAGS}"
             if test "${UNIVERSALSDK}" != "/"
             then
+                CFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${CFLAGS}"
+                LDFLAGS="${UNIVERSAL_ARCH_FLAGS} -isysroot ${UNIVERSALSDK} ${LDFLAGS}"
                 CPPFLAGS="-isysroot ${UNIVERSALSDK} ${CPPFLAGS}"
+            else
+                CFLAGS="${UNIVERSAL_ARCH_FLAGS} ${CFLAGS}"
+                LDFLAGS="${UNIVERSAL_ARCH_FLAGS} ${LDFLAGS}"
             fi
         fi