Skip to content

PLUME-64: NWP Emulator Core Python API - #29

Merged
cducher merged 1 commit into
developfrom
feature/emulator-core-py-api
Jul 17, 2026
Merged

PLUME-64: NWP Emulator Core Python API#29
cducher merged 1 commit into
developfrom
feature/emulator-core-py-api

Conversation

@cducher

@cducher cducher commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR adds Python bindings to the NWP emulator using pybind11 following the ADR-003. It exposes the main execute method so that the emulator can be used from python in a similar fashion as the CLI, and the broken down emulator core functionalities for integration in step by step pipelines. This change is stage 2 of PLUME-64 to build a small plume emulator app.

Tests cover:

File Type
test_nwp_emulator.py Unit / binding surface
test_nwp_emulator_integration.py Integration (real config file + C++ core)
test_nwp_emulator_mpi.py Integration + system (real config + MPI launcher)

Contributor Declaration

By opening this pull request, I affirm the following:

  • All authors agree to the Contributor License Agreement.
  • The code follows the project's coding standards.
  • I have performed self-review and added comments where needed.
  • I have added or updated tests to verify that my changes are effective and functional.
  • I have run all existing tests and confirmed they pass.

@cducher cducher self-assigned this Jun 26, 2026
@codecov-commenter

codecov-commenter commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 45.45455% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.64%. Comparing base (d519f3b) to head (7572353).
⚠️ Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
src/nwp_emulator/nwp_emulator_core.cc 45.45% 6 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop      #29      +/-   ##
===========================================
- Coverage    70.72%   70.64%   -0.09%     
===========================================
  Files           91       91              
  Lines         3816     3826      +10     
  Branches       343      345       +2     
===========================================
+ Hits          2699     2703       +4     
- Misses        1117     1123       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cducher
cducher force-pushed the feature/emulator-core-py-api branch 7 times, most recently from 44dd231 to 61ff8ee Compare June 26, 2026 13:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new pynwp_emulator Python package backed by a pybind11 native extension, enabling programmatic use of the NWP emulator core from Python (single-shot execution as well as step-by-step lifecycle control), along with tests, documentation, and CI integration.

Changes:

  • Adds a pybind11 module (nwp_emulator_bindings) exposing the C++ NWPEmulatorCore surface, plus a pure-Python wrapper (pynwp_emulator) implementing context-manager and iterator protocols.
  • Adds binding-surface, integration, and MPI integration tests for the Python API and wires them into CTest via CMake.
  • Adds Sphinx documentation pages for installation/API and a dedicated GitHub Actions workflow for building/testing the Python interface.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tests/pynwp_emulator/test_nwp_emulator.py Binding surface/unit tests for importability and Python API shape.
tests/pynwp_emulator/test_nwp_emulator_mpi.py MPI integration tests verifying rank/topology behavior and iterator lifecycle under MPI.
tests/pynwp_emulator/test_nwp_emulator_integration.py Single-process integration tests exercising CONFIG data source and optional Plume runs.
tests/pynwp_emulator/CMakeLists.txt Registers the new Python tests (including MPI) with CTest and sets required env vars.
tests/nwp_emulator/data/plume_config_simple.yml Minimal Plume config used by integration tests.
tests/CMakeLists.txt Conditionally includes tests/pynwp_emulator when Python bindings are enabled.
src/pynwp_emulator/README.md User-facing README for building and using the Python interface.
src/pynwp_emulator/nwp_emulator.py Pure-Python wrapper adding context-manager/iterator semantics over the C++ core binding.
src/pynwp_emulator/CMakeLists.txt Stages the Python package into the build staging directory via symlink.
src/pynwp_emulator/_internal/__init__.py Loads dependencies and imports/initializes the native extension, re-exporting core types.
src/pynwp_emulator/__init__.py Defines the public Python package surface (__all__) and re-exports API symbols.
src/CMakeLists.txt Adds bindings/ and pynwp_emulator/ subdirs when Python interface is enabled.
src/bindings/nwp_emulator.cc Implements pybind11 bindings for enums, options/results structs, and core methods.
src/bindings/CMakeLists.txt Builds and stages the nwp_emulator_bindings extension linked to available precision.
docs/requirements.txt Adds sphinx-autoapi dependency for generating Python API docs.
docs/nwp_emulator/installation.rst New documentation for building/using the Python interface.
docs/nwp_emulator/index.rst New doc index for the NWP emulator Python interface section.
docs/nwp_emulator/api.rst New API reference page for the Python interface.
docs/index.rst Adds the NWP emulator Python docs section to the main docs toctree and updates authors list.
docs/conf.py Enables AutoAPI/autodoc extensions and configures AutoAPI scanning for src/pynwp_emulator.
CMakeLists.txt Adds CMake option to enable/disable building the Python NWP emulator interface.
.github/workflows/pynwp_emulator.yml New workflow to build/test the Python bindings across Python versions, with/without MPI tests.
.github/ci-hpc-config.yml Disables Python interface in the HPC CI config matrices.
.github/ci-config.yml Disables Python interface in the default CI config.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/pynwp_emulator.yml
Comment thread src/pynwp_emulator/_internal/__init__.py
Comment thread src/pynwp_emulator/nwp_emulator.py
Comment thread docs/nwp_emulator/installation.rst Outdated
Comment thread docs/nwp_emulator/installation.rst Outdated
Comment thread docs/nwp_emulator/api.rst Outdated
Comment thread src/pynwp_emulator/README.md Outdated
@cducher
cducher requested a review from tweska June 26, 2026 14:23
Comment thread docs/nwp_emulator/installation.rst Outdated
Comment thread src/pynwp_emulator/README.md Outdated

@tbkr tbkr left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nice and clean implementation!

I mentioned some issues I found in the review. I just want to state that I'm not super familiar with the use-case of plume and what a user can/should/is allowed to do. Take that in consideration for my comments ;)

Another thing I didn't state is the test coverage: Would be nice if there are tests for the comments I left, if those comments are things you want to implement, at all.

Think about adding a section to the existing docs, regarding the python interface :)

I know this is a quite verbose review and to state this explicitly: The PR is super nice and my comments should be considered as nit-picky :)

Comment thread src/bindings/nwp_emulator.cc
Comment thread src/bindings/nwp_emulator.cc
Comment thread src/pynwp_emulator/nwp_emulator.py Outdated
Comment thread src/bindings/nwp_emulator.cc
Comment thread src/bindings/nwp_emulator.cc Outdated
Comment thread src/pynwp_emulator/nwp_emulator.py
Comment thread src/bindings/nwp_emulator.cc
Comment thread src/bindings/nwp_emulator.cc
Comment thread src/bindings/nwp_emulator.cc
Comment thread tests/pynwp_emulator/test_nwp_emulator_integration.py
@cducher
cducher force-pushed the feature/emulator-core-py-api branch from 3da4a7a to 6fdc906 Compare July 10, 2026 14:03
@tbkr
tbkr requested a review from Copilot July 10, 2026 14:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 2 comments.

Comment thread src/nwp_emulator/nwp_emulator_core.cc Outdated
Comment thread tests/pynwp_emulator/test_nwp_emulator.py Outdated
@cducher
cducher force-pushed the feature/emulator-core-py-api branch from bfdaed0 to 04b75f2 Compare July 10, 2026 15:44
@cducher
cducher requested a review from tbkr July 10, 2026 15:47

@tbkr tbkr left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Clara, just some final remarks :)

Comment thread src/pynwp_emulator/nwp_emulator.py Outdated
Comment thread src/pynwp_emulator/nwp_emulator.py Outdated
Comment thread src/nwp_emulator/nwp_emulator_core.h Outdated
Comment thread src/pynwp_emulator/nwp_emulator.py
Comment thread src/bindings/nwp_emulator.cc
@cducher
cducher requested a review from tbkr July 13, 2026 12:49

@tbkr tbkr left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nice work! :)

@cducher
cducher force-pushed the feature/emulator-core-py-api branch from 5fada71 to 7572353 Compare July 17, 2026 17:30
@cducher
cducher merged commit 7572353 into develop Jul 17, 2026
140 checks passed
@cducher
cducher deleted the feature/emulator-core-py-api branch July 17, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants