Building Python wheels with modern C++ standards (C++11 and later) requires a few tricks.
Python 2.7 and C++17¶
The Python 2.7 header files use the register
keyword, which is reserved and unused from C+17 onwards. Compiling a wheel for Python 2.7 with the C++17 standard is still possible to allow usage of register
using proper flag -Wno-register
for gcc/clang and /wd5033
for MSVC.
manylinux1 and C++14¶
The default manylinux1
image (based on CentOS 5) contains a version of GCC and libstdc++ that only supports C++11 and earlier standards. There are however ways to compile wheels with the C++14 standard (and later): pypa/manylinux#118
manylinux2010
and manylinux2014
are newer and support all C++ standards (up to C++17).
macOS and deployment target versions¶
OS X/macOS allows you to specify a so-called "deployment target" version that will ensure backwards compatibility with older versions of macOS. One way to do this is by setting the MACOSX_DEPLOYMENT_TARGET
environment variable.
However, to enable modern C++ standards, the deploment target needs to be set high enough (since older OS X/macOS versions did not have the necessary modern C++ standard library).
To get C++11 and C++14 support, MACOSX_DEPLOYMENT_TARGET
needs to be set to (at least) "10.9"
. By default, cibuildwheel
already does this, building 64-bit-only wheels for macOS 10.9 and later.
To get C++17 support, Xcode 9.3+ is needed, requiring at least macOS 10.13 on the build machine. To use C++17 library features and link against the C++ runtime library, set MACOSX_DEPLOYMENT_TARGET
to "10.13"
or "10.14"
(or higher) - macOS 10.13 offers partial C++17 support (e.g., the filesystem header is in experimental, offering #include <experimental/filesystem>
instead of #include <filesystem>
); macOS 10.14 has full C++17 support.
However, if only C++17 compiler and standard template library (STL) features are used (not needing a C++17 runtime) it might be possible to set MACOSX_DEPLOYMENT_TARGET
to a lower value, such as "10.9"
. To find out if this is the case, try compiling and running with a lower MACOSX_DEPLOYMENT_TARGET
: if C++17 features are used that require a more recent deployment target, building the wheel should fail.
For more details see https://en.cppreference.com/w/cpp/compiler_support, https://en.wikipedia.org/wiki/Xcode, and https://xcodereleases.com/: Xcode 10 needs macOS 10.13 and Xcode 11 needs macOS 10.14.
Windows and Python 2.7¶
In previous years, Microsoft distributed a compiler toolchain called 'Visual C++ for Python 2.7', which was a distribution of MSVC 2008 that was created to make it easier to build Python 2.7 extensions on Windows, because it was fully compatible with the toolchain that built Python 2.7.
This toolchain does not support modern C++ standards (i.e., C++11 and later). And it is hard to find this toolchain these days, since Microsoft removed the download for the required Visual Studio 2008 needed to build a native extension in April, 2021. So, by default, cibuildwheel does not attempt to build Python 2.7 extensions on Windows.
There is an optional workaround, though: the pybind11 project argues and shows that it is possible to compile Python 2.7 extension with a newer compiler and has an example project showing how to do this: pybind/python_example. The main catch is that a user might need to install a newer "Microsoft Visual C++ Redistributable", since the newer C++ standard library's binaries are not included by default with the Python 2.7 installation.
Forcing distutils
or setuptools
to use a more recent version of MSVC that supports modern C++ can be done in the following way:
- Set the environment variables
DISTUTILS_USE_SDK=1
andMSSdk=1
. These two environment variables will telldistutils
/setuptools
to not search and set up a build environment that uses Visual C++ for Python 2.7 (aka. MSVC 9). - Set up the build Visual Studio build environment you want to use, making sure that e.g.
PATH
containscl
,link
, etc.- Usually, this can be done through
vcvarsall.bat x86
orvcvarsall.bat x64
. The exact location of this file depends on the installation, but the default path for VS 2019 Community (e.g. used in AppVeyor'sVisual Studio 2019
image) isC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat
. Note:vcvarsall.bat
changes the environment variables, so this cannot be run in a subprocess/subshell and consequently runningvsvarsall.bat
inCIBW_BEFORE_BUILD
does not have any effect. - In Azure Pipelines, a
VSBuild
task is available - In GitHub Actions, the default shell is powershell, so you'll need to use
shell: cmd
to source a.bat
file, and the directory is"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvarsall.bat"
. Or you can use an action, such as theilammy/msvc-dev-cmd@v1
action to setup the environment (see example below).
- Usually, this can be done through
- Next, call
cibuildwheel
. Unfortunately, MSVC has separate toolchains for compiling 32-bit and 64-bit, so you will need to runcibuildwheel
twice: once withCIBW_BUILD=*-win32
after setting up thex86
build environment, and once withCIBW_BUILD=*-win_amd64
in ax64
environment (see previous step).
GitHub Action example with ilammy/msvc-dev-cmd:
- uses: ilammy/msvc-dev-cmd@v1
- name: Build 64-bit wheel
run: python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: cp27-win_amd64
DISTUTILS_USE_SDK: 1
MSSdk: 1
- uses: ilammy/msvc-dev-cmd@v1
with:
arch: x86
- name: Build 32-bit wheel
run: python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: cp27-win32
DISTUTILS_USE_SDK: 1
MSSdk: 1