Background
Investigation ticket #315 identified limitations in the CMEW workflow where REF/EVAL handling was hard-coded, serialised, and not extensible beyond two runs. The investigation proposed a parameterised, run-agnostic workflow design enabling parallel execution and future extension to N runs.
Scope
Implement the investigated design in CMEW by:
- Introducing explicit task-level parameterisation over run
- Decoupling REF/EVAL logic from scripts and embedding it in the workflow graph
- Supporting per-run CDDS request generation and workflow execution
- Ensuring backward compatibility during transition where possible
Implementation details
- Add
dataset as a Cylc task parameter (e.g. dataset=u-ab123, dataset=u-az513).
- Creation of the request and variables files should also be parallelised
- Parameterise:
- o
configure_standardise<recipe,dataset>
- o
standardise_model_data<dataset>
- Generate per-run CDDS request files (
request_<dataset>.cfg) and variables files (variables_<dataset>.cfg)
Out of scope
- Changes to CDDS internals
- Anything involving
model_runs.yml
Acceptance criteria
- Given
SUITE_ID and REF_SUITE_ID template variables in rose-suite.conf
- When
configure_standardise runs
- Then the request and variables files are made in parallel for each dataset
- And when
standardise_mdoel_data runs
- Then two CDDS workflows run in parallel
Related tickets
• Investigation: #315
Background
Investigation ticket #315 identified limitations in the CMEW workflow where REF/EVAL handling was hard-coded, serialised, and not extensible beyond two runs. The investigation proposed a parameterised, run-agnostic workflow design enabling parallel execution and future extension to N runs.
Scope
Implement the investigated design in CMEW by:
Implementation details
datasetas a Cylc task parameter (e.g.dataset=u-ab123,dataset=u-az513).configure_standardise<recipe,dataset>standardise_model_data<dataset>request_<dataset>.cfg) and variables files (variables_<dataset>.cfg)Out of scope
model_runs.ymlAcceptance criteria
SUITE_IDandREF_SUITE_IDtemplate variables inrose-suite.confconfigure_standardiserunsstandardise_mdoel_datarunsRelated tickets
• Investigation: #315