maintenance/improve test coverage #213

greenc-FNAL · 2025-12-19T15:59:39Z

Non-test changes:

1. Python/C++ Interoperability Enhancements

Files: modulewrap.cpp, lifelinewrap.cpp

Feature: RAII for Python Objects
- Change: Introduced PyObjectPtr (a std::shared_ptr alias with a custom PyObjectDeleter) to manage Python object reference counts.
- Rationalization: Manual reference counting (Py_INCREF/Py_DECREF) is error-prone, especially in the presence of C++ exceptions. If an exception is thrown, manual decrements might be skipped, leading to memory leaks.
- Resolution: PyObjectPtr ensures that Py_DECREF is called automatically when the pointer goes out of scope, even during stack unwinding.
Fix: Robust Annotation Parsing
- Change: Rewrote the argument parsing logic in parse_args to iterate over the __annotations__ dictionary using PyDict_Next and explicitly skip the "return" key.
- Root Cause: The previous implementation relied on PyDict_Values, which returns all values including the return type annotation. Depending on dictionary iteration order (which can vary or be insertion-ordered), the return type could be mistakenly interpreted as an input argument type.
- Diagnosis: Likely diagnosed by observing type mismatch errors when Python functions had return type annotations.
Fix: Flexible Input Conversion (List vs. NumPy)
- Change: Replaced rigid macro-based vector converters with explicit implementations (py_to_vint, py_to_vuint, etc.) that accept both Python list and NumPy ndarray objects.
- Root Cause: The previous converters strictly expected NumPy arrays. Users passing standard Python lists would cause runtime errors or type mismatches.
- Resolution: The new converters check the input type (PyList_Check vs PyArray_Check) and handle data extraction accordingly.
Fix: Memory Safety in Cyclic GC
- Change: Added PyObject_GC_UnTrack(pyobj) in ll_dealloc (lifelinewrap.cpp).
- Root Cause: Python objects that support cyclic garbage collection must be untracked before deallocation to prevent the GC from visiting invalid memory. Missing this can lead to segfaults during interpreter shutdown or garbage collection cycles.
Fix: Type String Matching
- Change: Replaced brittle fixed-offset string comparisons (e.g., inp_type.compare(pos, ...)) with robust substring searching (suffix.find(...)). Corrected a typo where double64]] was checked instead of float64]].
- Root Cause: The fixed-offset logic assumed a specific string format for type signatures, which could break if the format changed slightly. The typo prevented float64 arrays from being correctly identified.

2. Build System & CI Improvements

Files: CMakeLists.txt, CMakeLists.txt

Enhancement: Reduced Build Dependencies
- Change: Removed the dependency on the external packaging Python module in CMakeLists.txt.
- Rationalization: The build system previously used packaging.version to check module versions. This required the packaging library to be installed in the build environment.
- Resolution: Implemented a lightweight, inline version parser (splitting strings by .) to perform the check using only the standard library.
Fix: GCC 14+ Warning Suppression
- Change: Added -Wno-maybe-uninitialized to compile options for GCC 14.1+.
- Root Cause: Newer GCC versions have more aggressive static analysis that produces false positives for uninitialized variables in complex C++ templates used by the project.

3. Documentation

Files: copilot-instructions.md

New Feature: Added a comprehensive instructions file for GitHub Copilot.
Rationalization: To standardize the behavior of AI assistants working in the repository, ensuring they follow project-specific coding standards (formatting, error handling) and workflow guidelines.

Test code changes and additions:

1. New Tests: `py:vectypes` and `py:types`

Files: vectypes.py, test_types.py, pyvectypes.jsonnet, pytypes.jsonnet, verify_extended.py

Rationale:
- The existing tests primarily covered basic integer and string types.
- There was a gap in coverage for:
  - Floating point types (float, double).
  - Unsigned integers (unsigned int, unsigned long).
  - 64-bit integers (long, int64_t).
  - NumPy array interoperability (passing vectors from C++ to Python as NumPy arrays).
- These tests were added to verify the robustness of the new modulewrap.cpp converters.
Coverage Improvements:
- py:types: Validates scalar type conversion between C++ and Python for float, double, and unsigned int.
- py:vectypes: Validates vector/array conversion. It tests:
  - Creation of NumPy arrays from scalar inputs (collectify_*).
  - Summation of NumPy arrays back to scalars (sum_array_*).
  - Handling of all major numeric types: int32, uint32, int64, uint64, float32, float64.
- verify_extended.py: Introduces specialized verifiers (VerifierFloat, VerifierUInt, etc.) that handle type-specific assertions (e.g., epsilon comparison for floats).
Problems Exposed:
- Integer Overflow/Underflow: The py:vectypes test exposed a logic error in source.cpp where large 64-bit hashes were being used in arithmetic (100 - id), causing underflow for unsigned types and wrapping for signed types. This was fixed by introducing modulo arithmetic to keep values small and predictable.
- Type Mismatches: The strict type checking in the new tests likely exposed the need for the robust annotation parsing and explicit type converters implemented in modulewrap.cpp.
Regression Detection:
- Type Conversion Breakages: These tests will fail if future changes to modulewrap.cpp break the mapping between C++ types (like std::vector<int>) and Python types (like numpy.ndarray or list).
- Precision Loss: The float/double tests will catch regressions where 64-bit precision is accidentally truncated to 32-bit.
- Sign Errors: The unsigned integer tests will detect if unsigned values are incorrectly cast to signed values (e.g., treating UINT_MAX as -1).

2. Test Infrastructure Enhancements

Files: CMakeLists.txt, source.cpp

Rationale:
- To support the new tests and ensure the test environment is consistent with real-world usage.
- To fix flaky or incorrect test data generation.
Changes:
- CMakeLists.txt:
  - Added py:vectypes and py:types to the test suite.
  - Enhanced PYTHONPATH setup to explicitly include Python_SITELIB and Python_SITEARCH. This ensures tests running in embedded environments (like Spack) can find installed packages.
  - Replaced the external packaging dependency with a simple inline version parser for the module check.
- source.cpp:
  - Expanded the C++ data provider to generate all required types (float, double, uint, int64, uint64).
  - Fix: Changed data generation logic from id.number() to id.number() % N to prevent integer overflow and ensure deterministic summation results.

3. Code Quality & Modernization

Files: adder.py, all_config.py, reducer.py, sumit.py, verify.py

Rationale:
- To comply with the project's stricter linting rules (ruff, mypy) introduced in this commit.
Changes:
- Formatting: Applied standard Python formatting (whitespace, indentation).
- Linting: Fixed issues like:
  - Comparison to False (changed == False to is False or kept as is with # noqa if intentional for testing).
  - Missing docstrings or blank lines.
  - Unused imports.
- Type Hinting: Added or corrected type hints to satisfy mypy.
Regression Detection:
- Static Analysis: By enforcing these standards, the CI pipeline can now detect syntax errors, undefined variables, and type inconsistencies before tests are even run.

greenc-FNAL · 2025-12-19T16:03:04Z

@phlexbot format

github-actions · 2025-12-19T16:03:32Z

Automatic cmake-format fixes pushed (commit e60d811).
⚠️ Note: Some issues may require manual review and fixing.

greenc-FNAL · 2025-12-19T16:11:19Z

@phlexbot format

github-actions · 2025-12-19T16:11:44Z

No automatic cmake-format fixes were necessary.

github-actions · 2025-12-19T16:11:45Z

Automatic clang-format fixes pushed (commit 2aa7957).
⚠️ Note: Some issues may require manual review and fixing.

greenc-FNAL · 2025-12-19T16:13:17Z

@phlexbot python-fix

github-actions · 2025-12-19T16:13:41Z

Automatic Python linting fixes pushed (commit 092dec4).
⚠️ Note: Some issues may require manual review and fixing.

codecov · 2025-12-19T16:22:51Z

Codecov Report

❌ Patch coverage is 65.87537% with 115 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
plugins/python/src/modulewrap.cpp	65.77%	73 Missing and 42 partials ⚠️

❌ Your patch check has failed because the patch coverage (65.87%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

@@            Coverage Diff             @@
##             main     #213      +/-   ##
==========================================
+ Coverage   75.87%   78.70%   +2.82%     
==========================================
  Files         124      124              
  Lines        2898     3156     +258     
  Branches      506      576      +70     
==========================================
+ Hits         2199     2484     +285     
+ Misses        483      429      -54     
- Partials      216      243      +27

Flag	Coverage Δ
unittests	`78.70% <65.87%> (+2.82%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
plugins/python/src/lifelinewrap.cpp	`53.33% <100.00%> (+3.33%)`	⬆️
plugins/python/src/modulewrap.cpp	`68.91% <65.77%> (+22.88%)`	⬆️

... and 1 file with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e74780...90d2593. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

test/python/verify_extended.py

greenc-FNAL · 2025-12-19T18:34:40Z

@phlexbot format

github-actions · 2025-12-19T18:35:00Z

No automatic clang-format fixes were necessary.

github-actions · 2025-12-19T18:35:07Z

Automatic cmake-format fixes pushed (commit bb954c8).
⚠️ Note: Some issues may require manual review and fixing.

greenc-FNAL · 2025-12-19T18:35:46Z

@phlexbot python-fix

github-actions · 2025-12-19T18:36:09Z

Automatic Python linting fixes pushed (commit cfdb09a).
⚠️ Note: Some issues may require manual review and fixing.

test/python/verify_extended.py

greenc-FNAL · 2025-12-21T17:33:56Z

@phlexbot format

github-actions · 2025-12-21T17:34:24Z

Automatic clang-format fixes pushed (commit b145a22).
⚠️ Note: Some issues may require manual review and fixing.

greenc-FNAL · 2025-12-21T17:35:10Z

@phlexbot format

github-actions · 2025-12-21T17:35:36Z

Automatic cmake-format fixes pushed (commit cc928e7).
⚠️ Note: Some issues may require manual review and fixing.

github-actions · 2025-12-21T17:35:36Z

No automatic clang-format fixes were necessary.

github-actions · 2026-01-14T19:06:58Z

Automatic cmake-format fixes pushed (commit 38a38fe).
⚠️ Note: Some issues may require manual review and fixing.

github-actions · 2026-01-14T19:07:09Z

No automatic clang-format fixes were necessary.

test/python/test_callbacks.py

greenc-FNAL · 2026-01-14T20:23:04Z

@phlexbot python fix

greenc-FNAL · 2026-01-14T20:24:40Z

@phlexbot format

github-actions · 2026-01-14T20:25:01Z

No automatic clang-format fixes were necessary.

github-actions · 2026-01-14T20:25:04Z

Automatic cmake-format fixes pushed (commit 1354192).
⚠️ Note: Some issues may require manual review and fixing.

greenc-FNAL · 2026-01-14T20:25:25Z

@phlexbot python-fix

github-actions · 2026-01-14T20:25:49Z

Automatic Python linting fixes pushed (commit 6b1f634).
⚠️ Note: Some issues may require manual review and fixing.

greenc-FNAL · 2026-01-14T22:49:14Z

@phlexbot python-fix

greenc-FNAL · 2026-01-14T22:49:24Z

@phlexbot format

github-actions · 2026-01-14T22:49:41Z

Automatic Python linting fixes pushed (commit b1c55bf).
⚠️ Note: Some issues may require manual review and fixing.

github-actions · 2026-01-14T22:49:46Z

No automatic clang-format fixes were necessary.

greenc-FNAL · 2026-01-14T22:50:28Z

@phlexbot format

github-actions · 2026-01-14T22:50:51Z

No automatic clang-format fixes were necessary.

github-actions · 2026-01-14T22:50:53Z

Automatic cmake-format fixes pushed (commit f8a3c8d).
⚠️ Note: Some issues may require manual review and fixing.

greenc-FNAL force-pushed the maintenance/improve-test-coverage branch from 38fc738 to 47661fa Compare December 19, 2025 17:19

github-advanced-security bot found potential problems Dec 19, 2025

View reviewed changes

greenc-FNAL force-pushed the maintenance/improve-test-coverage branch 3 times, most recently from c9ccf06 to 1159796 Compare December 21, 2025 17:30

greenc-FNAL marked this pull request as ready for review December 21, 2025 17:35

greenc-FNAL requested review from Copilot, knoepfel, marcpaterno and wlav December 21, 2025 17:35

Copilot started reviewing on behalf of greenc-FNAL December 21, 2025 17:36 View session

Apply cmake-format fixes

38a38fe

More tests to fill gaps

690cb26

github-advanced-security bot found potential problems Jan 14, 2026

View reviewed changes

test/python/test_callbacks.py Fixed Show fixed Hide fixed

test/python/test_callbacks.py Fixed Show fixed Hide fixed

Apply cmake-format fixes

1354192

Apply Python linting fixes

6b1f634

Address remaining ruff issues

b0b30b1

greenc-FNAL force-pushed the maintenance/improve-test-coverage branch from d48f0ff to b0b30b1 Compare January 14, 2026 20:32

greenc-FNAL added 3 commits January 14, 2026 14:50

Per Gemini 3 Pro, get GIL when updating ref count

3c3043c

Attempt to address CI hangs in py:badbool and py:raise tests

90d2593

More coverage improvement

ed5e23e

Apply Python linting fixes

b1c55bf

Apply cmake-format fixes

f8a3c8d

greenc-FNAL added 3 commits January 14, 2026 17:19

Silence inapposite complaints; remove unused class

a9d5d70

More hang protection

3aeb5c5

Extra diagnostics to debug hangs during testing

4ea5f61

maintenance/improve test coverage #213

Are you sure you want to change the base?

maintenance/improve test coverage #213

Uh oh!

Conversation

greenc-FNAL commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Non-test changes:

1. Python/C++ Interoperability Enhancements

2. Build System & CI Improvements

3. Documentation

Test code changes and additions:

1. New Tests: py:vectypes and py:types

2. Test Infrastructure Enhancements

3. Code Quality & Modernization

Uh oh!

greenc-FNAL commented Dec 19, 2025

Uh oh!

github-actions bot commented Dec 19, 2025

Uh oh!

greenc-FNAL commented Dec 19, 2025

Uh oh!

github-actions bot commented Dec 19, 2025

Uh oh!

github-actions bot commented Dec 19, 2025

Uh oh!

greenc-FNAL commented Dec 19, 2025

Uh oh!

github-actions bot commented Dec 19, 2025

Uh oh!

codecov bot commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greenc-FNAL commented Dec 19, 2025

Uh oh!

github-actions bot commented Dec 19, 2025

Uh oh!

github-actions bot commented Dec 19, 2025

Uh oh!

greenc-FNAL commented Dec 19, 2025

Uh oh!

github-actions bot commented Dec 19, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greenc-FNAL commented Dec 21, 2025

Uh oh!

github-actions bot commented Dec 21, 2025

Uh oh!

greenc-FNAL commented Dec 21, 2025

Uh oh!

github-actions bot commented Dec 21, 2025

Uh oh!

github-actions bot commented Dec 21, 2025

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

Uh oh!

Uh oh!

greenc-FNAL commented Jan 14, 2026

Uh oh!

greenc-FNAL commented Jan 14, 2026

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

github-actions bot commented Jan 14, 2026

Uh oh!

greenc-FNAL commented Dec 19, 2025 •

edited

Loading

1. New Tests: `py:vectypes` and `py:types`

codecov bot commented Dec 19, 2025 •

edited

Loading