Lift and shift `monsoon` assessment area by pdearnshaw · Pull Request #361 · MetOffice/CMEW

pdearnshaw · 2026-02-09T11:34:46Z

This is a draft Pull Request to get a pre-review to make sure the direction of travel is as expected.

Closes #343 .

PR creation checklist for the developer

Has <issue_number> above ☝️ been replaced with the issue number?
Has main been selected as the base branch?
Does the feature branch name follow the format <issue_number>_<short_description_of_feature>?
Does the text of the PR title exactly match with the text (not including the issue number) of the issue title?
Have appropriate reviewers been added to the PR (once it is ready for review)?
Has the PR been assigned to the developer(s)?
Have the same labels as on the issue (except for the good first issue label) been added to the PR?
Has the Climate Model Evaluation Workflow (CMEW) project been added to the PR?
Has the appropriate milestone been added to the PR?

Definition of Done for the developer

Does the change in this PR address the above issue / have all acceptance criteria been met?
Does the change in this PR follow the requirements in the wiki: Developer Guide (including copyrights)?
Have new tests related to the change been added?
Do all the GitHub workflow checks pass?
Do all the tests run locally and pass? (Note: the tests are not run by the GitHub workflow, see wiki: Run the tests locally)
Has the API documentation (e.g. docstrings in Python modules) related to the change been updated appropriately?
Has the user documentation (i.e. everything in the doc directory) related to the change been updated appropriately, including the Quick Start section?
Do the HTML pages render correctly? (See wiki: Build the documentation locally)

PR creation checklist for the reviewer

Has <issue_number> above ☝️ been replaced with the issue number?
Has main been selected as the base branch?
Does the feature branch name follow the format <issue_number>_<short_description_of_feature>?
Does the text of the PR title exactly match with the text (not including the issue number) of the issue title?
Have appropriate reviewers been added to the PR (once it is ready for review)?
Has the PR been assigned to the developer(s)?
Have the same labels as on the issue (except for the good first issue label) been added to the PR?
Has the Climate Model Evaluation Workflow (CMEW) project been added to the PR?
Has the appropriate milestone been added to the PR?

Definition of Done for the reviewer

Does the change in this PR address the above issue / have all acceptance criteria been met?
Does the change in this PR follow the requirements in the wiki: Developer Guide (including copyrights)?
Have new tests related to the change been added?
Do all the GitHub workflow checks pass?
Do all the tests run locally and pass? (Note: the tests are not run by the GitHub workflow, see wiki: Run the tests locally)
Has the API documentation (e.g. docstrings in Python modules) related to the change been updated appropriately?
Has the user documentation (i.e. everything in the doc directory) related to the change been updated appropriately, including the Quick Start section?
Do the HTML pages render correctly? (See wiki: Build the documentation locally)

…n area

…REAS a python_list

alistairsellar · 2026-02-09T17:27:25Z

CMEW/inc/autoassess.cylc

+
+[scheduling]
+    [[graph]]
+        R1 = """


Noddy question: does this overwrite, or append to, the R1 = ... in line 16 of the main flow.cylc file?

It appends to. Use this quite a bit in our model workflows.

ehogan

This looks great @pdearnshaw, thank you! 🥳

I have some minor comments to add when you are ready for them, but my biggest question is whether we want to utilise Rose (i.e. use optional configuration based on the autoassess_area and use a single command in the rose-app.conf file) rather than defining the script within the autoassess.cylc file, for consistency with the rest of CMEW? 🤔 If this feels like a lot of work we can defer this, so we can showcase what works at the demo on Wednesday? 🤔

I have made some suggestions on what I needed to change to get CMEW to run. With these changes, CMEW fails for me as follows:

[FAIL] file:~/cylc-run/CMEW/run55/share/src/AutoAssess=source=git:https://github.com/MetOffice/AutoAssess.git::./::45_update_for_cmew: ls-remote: could not locate 'https://github.com/MetOffice/AutoAssess.git':
[FAIL]     fatal: could not read Username for 'https://github.com/': No such device or address
2026-02-16T09:10:12Z CRITICAL - failed/ERR

I assume this is because the AutoAssess repository is private and I don't have access. We might need to consider how best to solve this (edited to add: we could use a mirror?) / add something to the prerequisites.

ehogan · 2026-02-16T09:06:44Z

CMEW/inc/autoassess.cylc

+                --data-dir $DATA_DIR            \
+                --suite-id $CYLC_TASK_PARAM_suite_id
+        """
+    {%- endfor %}


I had to remove this line to get CMEW to run.

This line has been removed.

ehogan · 2026-02-16T09:07:04Z

CMEW/inc/autoassess.cylc

+                --ancil-dir $ANCIL_PATH
+        """
+
+    [[run_area<autoassess_area=australia>]]


I had to remove this section to get CMEW to run.

I would have suggested removing this section anyway, since the issue is for the monsoon assessment area only (but it's great to see that other areas work!) 😊

The suggestion of using rose apps has resolved this problem. There is only one task defined with parametrisation so whatever logic that prevented this from working with this will have gone away,

ehogan · 2026-02-16T09:07:10Z

CMEW/inc/autoassess.cylc

+                --ncpu 1
+        """
+
+    [[run_area<autoassess_area=mjo>]]


I had to remove this section to get CMEW to run.

I would have suggested removing this section anyway, since the issue is for the monsoon assessment area only (but it's great to see that other areas work!) 😊

The suggestion of using rose apps has resolved this problem. There is only one task defined with parametrisation so whatever logic that prevented this from working with this will have gone away,

ehogan · 2026-02-16T09:11:47Z

CMEW/inc/autoassess.cylc

+            END_DATE = {{ (START_YEAR+NUMBER_OF_YEARS-1)|string + '/12/01' }}
+
+    [[install_autoassess]]
+        inherit = AUTOASSESS, SITE_AA_INSTALL


To remove the error CYLC_TASK_PARAM_autoassess_area: unbound variable:

Suggested change

inherit = AUTOASSESS, SITE_AA_INSTALL

inherit = SITE_AA_INSTALL

I have re-engineered [[AUTOASSESS]] to not use the parametrised variable so this family can still be used by install_autoassess

…e local mirror

pdearnshaw · 2026-03-03T12:16:08Z

In response to Emma's review I have the following comments, and have tried to reply to specific issues in the code review comments.

The issue over accessing AutoAssess from the GitHub repository has potentially been resolved. The URL used was the HTTPS version due to a misconfigured user authentication key (not authorised for SSO). Once this was resolved the SSH version of the repository URL worked. The Met Office's local mirror bot has also been added to the collaborator list so the AutoAssess repository is also available via local mirror. I have tested both of these and they "work for me"!

I have implemented the AutoAssess scripts as rose apps (all prefixed aa_). As suggested these can make use of optional configurations where appropriate, and this also tidies up the inc/autoassess.cylc file by removing the [[run_area<autoassess_area=some_area>]] sections. By using an "optional" optional configuration (use of brackets to define the optional configuration) we can supply an area assessment's alternative calling command if it exists and leave the default if it doesn't.

I think this PR is ready for another look if and when you are ready Emma?

pdearnshaw · 2026-03-03T13:36:16Z

One thing that has cropped up is that the AutoAssess code will not work with the esmvaltool community version of scitools due to the missing mo_pack library. Once this is included in version 2.13 then it should work fine. In the interim, the best choice is to switch to use scitools/default-current.

pdearnshaw added 6 commits February 3, 2026 14:04

#343. Added infrastructure to run AutoAssess and set up to run monsoo…

5a92e09

…n area

#343. Made workflow work for mjo and australia, and made AUTOASSESS_A…

9c8d9db

…REAS a python_list

#343. Fixed metadata file, and switched AutoAssess by default

2168ead

#343. Rewrote graph to be more parametrised

7ea47e7

#343. Made all AutoAssess families uppercase.

e8f3358

#343. Finished uppercasing families.

10fa549

pdearnshaw changed the title ~~343 autoassess monsoon~~ Lift and shift monsoon assessment area Feb 9, 2026

pdearnshaw self-assigned this Feb 9, 2026

pdearnshaw added this to the v0.5.0 (single AutoAssess science area) milestone Feb 9, 2026

pdearnshaw requested a review from alistairsellar February 9, 2026 11:37

alistairsellar reviewed Feb 9, 2026

View reviewed changes

ehogan reviewed Feb 16, 2026

View reviewed changes

pdearnshaw added 4 commits February 20, 2026 17:19

Attempt to put AutoAssess commands in Rose Apps

552c7be

Fix to output path for code run through aa_run_area

38d57b7

Swapped URL for AutoAssess to be SSH version and added support for th…

fddeda2

…e local mirror

Configured to use local mirror

2032246

pdearnshaw added 2 commits March 3, 2026 12:18

Fixed formatting issue flagged up by actions

fa9d32c

Fixed family used to define MASS queue, so actually used

b28e82d

	inherit = AUTOASSESS, SITE_AA_INSTALL
	inherit = SITE_AA_INSTALL

Conversation

pdearnshaw commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR creation checklist for the developer

Definition of Done for the developer

PR creation checklist for the reviewer

Definition of Done for the reviewer

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ehogan left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pdearnshaw commented Mar 3, 2026

Uh oh!

pdearnshaw commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pdearnshaw commented Feb 9, 2026 •

edited

Loading

ehogan left a comment •

edited

Loading

pdearnshaw commented Mar 3, 2026 •

edited

Loading