Reorganize CI to separate upstream-dev signal from PR gating#700
Conversation
# Conflicts: # CHANGELOG.md
|
While I am at it I decided to fix some failures of the CI. This and #681 (the DynamicTableRegion HDMF 5 update) should then fix the CI. The fixes are: 1. read_nwbfile(url, method="ros3", backend_kwargs={"aws_region": "us-east-1"})This keeps the helper a thin pass-through, and future upstream kwarg requirements do not need an API change here. 2. Windows While in there I also did a small API consolidation: deprecated |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## dev #700 +/- ##
==========================================
+ Coverage 80.01% 82.81% +2.80%
==========================================
Files 48 48
Lines 1886 1897 +11
==========================================
+ Hits 1509 1571 +62
+ Misses 377 326 -51
🚀 New features to boost your workflow:
|
rly
left a comment
There was a problem hiding this comment.
This all looks good to me. Thanks @h-mayorquin !
The PR-gating workflow currently runs three jobs that depend on upstream development branches. Before describing the change, here is what each of those three jobs is for:
pynwb-dev-tests.ymltest-pynwb-devdevbranch (installed from GitHub). Catches upstream pynwb changes that would break nwbinspector.dandi-dev.ymltest-dandi-devdandi-climaster and runs DANDI's own test suite against the current nwbinspector, withDANDI_TESTS_NONETWORK=1so tests requiring network are skipped. Catches local-only DANDI integration breakage.dandi-dev-live-services.ymltest-dandi-dev-livetest-dandi-devbut with network enabled, so the tests can hitdandiarchive.org, the staging sandbox, and S3. Catches integration breakage that only surfaces against live infrastructure.When any of those three upstream branches break for reasons unrelated to this repository, every nwbinspector PR turns red, which blocks review on changes that have nothing to do with the upstream churn. This PR adopts the three-tier CI pattern I have been using in neuroconv:
deploy-tests.ymlruns only stable jobs and gates PRs,dailies.ymlruns the same stable jobs on a daily cron, and a newdev-dailies.ymlruns the three upstream-dev jobs on a daily cron. Maintainers still receive notifications about upstream failures via daily email, but those failures no longer add noise to every unrelated PR's CI signal. In other words, the split separates upstream churn from day-to-day development while still monitoring it.deploy-tests.yml(PR gating)run-tests,run-streaming-tests,run-read-nwbfile-tests,test-dandi-latest,test-pynwb-dev,test-dandi-dev,test-dandi-dev-liverun-tests,run-streaming-tests,run-read-nwbfile-tests,test-dandi-latestdailies.yml(daily cron)run-daily-tests,run-daily-doc-link-checks,test-dandi-latest,test-dandi-dev,test-dandi-dev-liverun-daily-tests,run-daily-doc-link-checks,test-dandi-latestdev-dailies.yml(new daily cron)test-pynwb-dev,test-dandi-dev,test-dandi-dev-liveThe daily failure emails are now split into per-workflow notification jobs, each with a subject line that names the failing workflow ("NWB Inspector Daily Failure - pynwb dev branch", "NWB Inspector Daily Failure - DANDI dev branch (offline)", etc.). Previously a single generic email fired if any daily job failed, so the recipient had to open the Actions tab to see which one. The per-workflow pattern makes the email itself actionable.
The Python and OS version matrices for
testing.ymlare now centralized intoall_python_versions.txtandall_os_versions.txt, loaded by aload_python_and_os_versionsjob that exposes them as workflow outputs. Bumping the supported Python or OS sweep is now a one-file edit. The streaming, read-nwbfile, and DANDI-release workflows intentionally use different matrices (skipping macOS with aTODOcomment, or pinning to a specific Python) so I have left them on their hardcoded matrices to avoid silently re-enabling broken combinations.I also removed Steph from the failure email recipient list since she is no longer on the team.