Add Python wheel packaging and CI wheel builds#1
Merged
Conversation
Add a scikit-build-core PEP 517 build (pyproject.toml) that drives the existing CMake build to produce an installable wheel containing only the vcellmbsolver_py extension and the vcellmbsolver wrapper. The wheel build limits CMake to the vcellmbsolver_py target and the "python" install component, and forces BUILD_TESTING/OPTION_TARGET_MESSAGING off. python/CMakeLists.txt installs to a relative dir under SKBUILD (wheel) and to Python3_SITEARCH otherwise, tagging the python install rules with a COMPONENT so the C++ lib/binary/headers stay out of the wheel. Add .github/workflows/wheels.yml: cibuildwheel across Linux/macOS/Windows plus an sdist, uploaded as artifacts; a publish job does PyPI trusted publishing on v* tags only (dormant until PyPI is configured). Document install/build/publish in README and add a packaging note to CLAUDE.md. Wheel version is scraped from the CMake project() version. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Match the sibling vcell-fvsolver / pyvcell_fvsolver convention: ship the
bindings as a package whose compiled extension is the private submodule
pyvcell_mbsolver._core, and whose __init__.py is the high-level wrapper.
- pyvcellmbsolver.cpp: PYBIND11_MODULE(vcellmbsolver_py) -> (_core)
- python/vcellmbsolver.py -> python/pyvcell_mbsolver/__init__.py; imports
the extension via `from . import _core as _mb` and re-exports the API
- python/CMakeLists.txt: build target `_core`, assemble the package under
build/bin/pyvcell_mbsolver/ (extension + copied __init__.py) so it imports
with PYTHONPATH=build/bin; install into pyvcell_mbsolver/ (relative under
SKBUILD, else ${Python3_SITEARCH}/pyvcell_mbsolver)
- pyproject.toml: distribution name pyvcell_mbsolver, build target _core,
import-check + pytest configured with --import-mode=importlib so the source
tree does not shadow the built/installed package
- tests: import `from pyvcell_mbsolver import _core as _mb` / the wrapper API
- README/CLAUDE.md updated to the new import names
Verified locally: wheel builds, installs into a clean venv, imports, and the
40-case pytest suite passes against the installed wheel; the native build-tree
ctest (PyVcellMbSolver) also passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Runner matrix now matches vcell-fvsolver: macos-15-intel + macos-15, ubuntu-latest + ubuntu-24.04-arm, windows-latest (drops the deprecated/ scarce macos-13 and macos-14 Intel/arm labels that were stuck queued). - Use the proven separate-step submodule auth from build-and-release.yml (checkout with token, configure insteadOf with SUBMODULE_TOKEN, then submodule update) instead of a single condensed step. - cibuildwheel pinned to v3.4.0 (matches fvsolver) with CIBW_ARCHS=native; add manylinux_2_28 aarch64 image for ubuntu-24.04-arm. - Python range aligned to 3.10-3.13 (requires-python + classifiers + build). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First real CI run surfaced one root cause per platform: - Linux: find_package(Python3 ... Development) failed because manylinux ships no shared libpython. Switch to Development.Module (headers/SOABI only, the correct policy for extension wheels). HDF5/Boost via dnf already worked. - macOS: configure/compile succeeded but delocate rejected the wheel because Homebrew's HDF5 requires a newer minimum macOS than the 11.0 target. Raise MACOSX_DEPLOYMENT_TARGET to 15.0 (macOS wheels now require macOS 15+). - Windows: vcpkg succeeded but CMake's "Visual Studio 17 2022" generator could not find VS on windows-latest (windows-2025). Pin the runner to windows-2022. Verified locally: cmake configure resolves Development.Module (CMake 3.29). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The windows-2022 pin fixed VS detection but surfaced two more issues: - Replace vcpkg (which built HDF5 + full Boost from source, ~40 min) with Conan: prebuilt HDF5 binary from Conan Center and header-only Boost (nothing compiles). full_deploy lays packages out predictably; CMake module-mode finds are pointed at them via HDF5_ROOT/BOOST_ROOT and the Conan toolchain, and the HDF5 DLL dir is added to PATH so delvewheel vendors it. Matches the Conan approach used by vcell-fvsolver. - find_package(Python3 ... Development.Module) does not resolve under cibuildwheel's build venv on Windows, so request only Interpreter there and let pybind11 locate the Python import library (Linux/macOS keep Development.Module). Linux + macOS already build green; this commit only touches the Windows path and the WIN32 branch of the Python find. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conan Center ships no prebuilt HDF5 binaries (recipe-only), so it would compile HDF5 from source. conda-forge provides genuinely prebuilt HDF5 + Boost for Windows in ~1-2 min, in a standard <env>/Library layout that the project's module-mode find_package(HDF5/Boost) consumes via HDF5_ROOT/ BOOST_ROOT. The Library/bin dir is added to PATH so delvewheel vendors the HDF5/zlib/aec DLLs. Uses the Miniconda preinstalled on windows-2022. Linux + macOS already build green; this only changes the Windows dep setup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Passing -DCMAKE_PREFIX_PATH=<conda> replaced the prefix scikit-build-core sets to locate pybind11, so find_package(pybind11) failed. HDF5_ROOT and BOOST_ROOT are sufficient for the module-mode HDF5/Boost finds; drop the CMAKE_PREFIX_PATH override so pybind11 is still discoverable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ools)
pybind11 v3 uses FindPython "new tools" mode, whose pybind11_add_module calls
python3_add_library — a command only defined when the Development.Module
component is found. The earlier Windows-only Interpreter fallback therefore
broke pybind11_add_module ("Unknown CMake command python3_add_library").
The original Windows Development.Module failure was caused by the vcpkg
toolchain interfering with scikit-build-core's Python hints; now that Windows
uses conda-forge (no toolchain file), Development.Module resolves like Linux/
macOS, so require it unconditionally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bundled FronTier front-tracking library is Unix/GCC code that does not compile under MSVC (POINT/boolean clash with the Windows SDK, GCC anonymous struct tags), and there is no MSVC equivalent of the -fpermissive flag used for it on Linux/macOS. (vcell-fvsolver builds on Windows only because it does not depend on FronTier.) Porting FronTier to MSVC is a separate, substantial effort, so remove Windows from the wheel matrix for now. Wheels are built for Linux (x86_64 + arm64) and macOS (x86_64 + arm64). Remove the Windows conda/cibuildwheel setup and document the limitation in README and CLAUDE.md; Windows users can build under WSL via the Linux wheels. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Python packaging path so the moving-boundary solver can be distributed as installable wheels (and, when you choose to enable it, published to PyPI). Inspired by the sibling
vcell-fvsolversetup.pyproject.toml—scikit-build-corePEP 517 backend that drives the existing CMake build into a wheel. The wheel contains only thevcellmbsolver_pyextension + thevcellmbsolverpure-Python wrapper (C++ static lib, CLI binary, and headers are excluded via apythoninstall component + avcellmbsolver_py-only build target).BUILD_TESTINGandOPTION_TARGET_MESSAGINGare forced off for wheel builds, and the wheel version is scraped from theproject(VCellMovingBoundary VERSION …)line so there's a single source of truth.python/CMakeLists.txt— installs to a relative dir underSKBUILD(wheel) vs.Python3_SITEARCHfor a plaincmake --install; tags the Python install rules withCOMPONENT python. Plaincmake --installbehavior is unchanged..github/workflows/wheels.yml—cibuildwheelacross Linux (manylinux_2_28), macOS (x86_64 + arm64), and Windows (vcpkg) plus an sdist, all uploaded as downloadable artifacts. Apublishjob does PyPI trusted publishing onv*tags only — dormant until PyPI is configured.Verified locally (macOS arm64)
python -m build --wheelproducesvcellmbsolver-1.0.0-cp312-cp312-macosx_*.whlcontaining exactly the extension + wrapper (no C++ lib/binary/headers).import vcellmbsolver_py, vcellmbsolverworks (enums/exceptions re-export correctly).python -m build --sdistproduces a self-contained sdist — scikit-build-core bundles thevcell-messagingandvcell-expressionparsersubmodule sources and vendoredFronTierLib.Needs a CI run to validate (cannot be tested locally)
hdf5-devel+boost-develviadnf(EPEL) insidemanylinux_2_28. If those packages aren't available/complete (esp. HDF5 C++ bindings), this is the most likely thing to need adjustment — the fallback is a custom manylinux image or Conan, asvcell-fvsolverdoes.CIBW_ENVIRONMENT_WINDOWS. Riskiest platform; may need iteration.SUBMODULE_TOKENsecret pattern frombuild-and-release.yml.Not included
build-and-release.yml(native binary artifacts) — this workflow is additive and wheel-only.🤖 Generated with Claude Code