PLUME-64: NWP Emulator Core Python API - #29
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #29 +/- ##
===========================================
- Coverage 70.72% 70.64% -0.09%
===========================================
Files 91 91
Lines 3816 3826 +10
Branches 343 345 +2
===========================================
+ Hits 2699 2703 +4
- Misses 1117 1123 +6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
44dd231 to
61ff8ee
Compare
There was a problem hiding this comment.
Pull request overview
This PR introduces a new pynwp_emulator Python package backed by a pybind11 native extension, enabling programmatic use of the NWP emulator core from Python (single-shot execution as well as step-by-step lifecycle control), along with tests, documentation, and CI integration.
Changes:
- Adds a
pybind11module (nwp_emulator_bindings) exposing the C++NWPEmulatorCoresurface, plus a pure-Python wrapper (pynwp_emulator) implementing context-manager and iterator protocols. - Adds binding-surface, integration, and MPI integration tests for the Python API and wires them into CTest via CMake.
- Adds Sphinx documentation pages for installation/API and a dedicated GitHub Actions workflow for building/testing the Python interface.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
tests/pynwp_emulator/test_nwp_emulator.py |
Binding surface/unit tests for importability and Python API shape. |
tests/pynwp_emulator/test_nwp_emulator_mpi.py |
MPI integration tests verifying rank/topology behavior and iterator lifecycle under MPI. |
tests/pynwp_emulator/test_nwp_emulator_integration.py |
Single-process integration tests exercising CONFIG data source and optional Plume runs. |
tests/pynwp_emulator/CMakeLists.txt |
Registers the new Python tests (including MPI) with CTest and sets required env vars. |
tests/nwp_emulator/data/plume_config_simple.yml |
Minimal Plume config used by integration tests. |
tests/CMakeLists.txt |
Conditionally includes tests/pynwp_emulator when Python bindings are enabled. |
src/pynwp_emulator/README.md |
User-facing README for building and using the Python interface. |
src/pynwp_emulator/nwp_emulator.py |
Pure-Python wrapper adding context-manager/iterator semantics over the C++ core binding. |
src/pynwp_emulator/CMakeLists.txt |
Stages the Python package into the build staging directory via symlink. |
src/pynwp_emulator/_internal/__init__.py |
Loads dependencies and imports/initializes the native extension, re-exporting core types. |
src/pynwp_emulator/__init__.py |
Defines the public Python package surface (__all__) and re-exports API symbols. |
src/CMakeLists.txt |
Adds bindings/ and pynwp_emulator/ subdirs when Python interface is enabled. |
src/bindings/nwp_emulator.cc |
Implements pybind11 bindings for enums, options/results structs, and core methods. |
src/bindings/CMakeLists.txt |
Builds and stages the nwp_emulator_bindings extension linked to available precision. |
docs/requirements.txt |
Adds sphinx-autoapi dependency for generating Python API docs. |
docs/nwp_emulator/installation.rst |
New documentation for building/using the Python interface. |
docs/nwp_emulator/index.rst |
New doc index for the NWP emulator Python interface section. |
docs/nwp_emulator/api.rst |
New API reference page for the Python interface. |
docs/index.rst |
Adds the NWP emulator Python docs section to the main docs toctree and updates authors list. |
docs/conf.py |
Enables AutoAPI/autodoc extensions and configures AutoAPI scanning for src/pynwp_emulator. |
CMakeLists.txt |
Adds CMake option to enable/disable building the Python NWP emulator interface. |
.github/workflows/pynwp_emulator.yml |
New workflow to build/test the Python bindings across Python versions, with/without MPI tests. |
.github/ci-hpc-config.yml |
Disables Python interface in the HPC CI config matrices. |
.github/ci-config.yml |
Disables Python interface in the default CI config. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
A nice and clean implementation!
I mentioned some issues I found in the review. I just want to state that I'm not super familiar with the use-case of plume and what a user can/should/is allowed to do. Take that in consideration for my comments ;)
Another thing I didn't state is the test coverage: Would be nice if there are tests for the comments I left, if those comments are things you want to implement, at all.
Think about adding a section to the existing docs, regarding the python interface :)
I know this is a quite verbose review and to state this explicitly: The PR is super nice and my comments should be considered as nit-picky :)
3da4a7a to
6fdc906
Compare
bfdaed0 to
04b75f2
Compare
5fada71 to
7572353
Compare
Description
This PR adds Python bindings to the NWP emulator using pybind11 following the ADR-003. It exposes the main
executemethod so that the emulator can be used from python in a similar fashion as the CLI, and the broken down emulator core functionalities for integration in step by step pipelines. This change is stage 2 of PLUME-64 to build a small plume emulator app.Tests cover:
Contributor Declaration
By opening this pull request, I affirm the following: