Update 5 Jan 2026: AC2 and AC4 now covered by other issue & PR (linked below), so this issue reduces to testing for bash scripts. Question is whether / how to enhance testing for bash, noting that scripts as a whole are covered by the workflow integration testing. Note that #315 may lead to a reduction of the bash code in the workflow.
User Story
As a CMEW developer,
I want unit tests that validate the new dual-run (REF and EVAL) standardisation workflow introduced in ISSUE #286,
So that regressions or incorrect behaviour can be detected early and automatically.
AC1 – configure_standardise.sh correctly emits REF and EVAL metadata
Given the workflow has REF_* and base (EVAL) model metadata defined in flow.cylc and passed into configure_standardise.sh,
When the script generates per-run request files,
Then unit tests must verify that:
• REF_* variables are exported correctly during the REF pass.
• MODEL_ID / SUITE_ID / CALENDAR / VARIANT_LABEL (EVAL run) are exported correctly during the EVAL pass.
• Missing required variables trigger a failure (e.g., set -u behaviour).
AC2 – create_request_file.py configures the variables that should differ between runs
AC2 completed with #309
Given create_request_file.py reads metadata from environment variables,
When it is invoked
Then tests must assert that:
• model_id, variant_label, suite_id, and calendar are set correctly
• The resulting file contains valid and complete CDDS metadata sections.
AC3 – standardise_model_data.sh triggers two CDDS runs
Given dual request files exist in ${CYLC_WORKFLOW_SHARE_DIR}/etc,
When standardise_model_data.sh executes,
Then tests must validate (via mocking) that:
• The REF cdds_convert call is issued first with request_ref.cfg.
• The EVAL cdds_convert call is issued second with request_eval.cfg.
• Both calls use the appropriate environment (e.g., cmew-standardise-env wrapper).
• The script fails cleanly if either request file is missing.
AC4 – Tests integrate with CMEW’s unittest workflow
AC4 covered by #165
Given the CMEW unittest execution path (cylc vip … -O unittest),
When the new tests are added under app/configure_standardise/bin and app/standardise_model_data/bin,
Then they must:
• Run successfully under pytest during the Cylc unittest run.
• Run successfully in GitHub Actions.
• Avoid modifying runtime logic (tests rely solely on mocks and environment injection).
Update 5 Jan 2026: AC2 and AC4 now covered by other issue & PR (linked below), so this issue reduces to testing for bash scripts. Question is whether / how to enhance testing for bash, noting that scripts as a whole are covered by the workflow integration testing. Note that #315 may lead to a reduction of the bash code in the workflow.
User Story
As a CMEW developer,
I want unit tests that validate the new dual-run (REF and EVAL) standardisation workflow introduced in ISSUE #286,
So that regressions or incorrect behaviour can be detected early and automatically.
AC1 –
configure_standardise.shcorrectly emits REF and EVAL metadataGiven the workflow has REF_* and base (EVAL) model metadata defined in
flow.cylcand passed intoconfigure_standardise.sh,When the script generates per-run request files,
Then unit tests must verify that:
• REF_* variables are exported correctly during the REF pass.
• MODEL_ID / SUITE_ID / CALENDAR / VARIANT_LABEL (EVAL run) are exported correctly during the EVAL pass.
• Missing required variables trigger a failure (e.g., set -u behaviour).
AC2 –create_request_file.pyconfigures the variables that should differ between runsAC2 completed with #309
Given create_request_file.py reads metadata from environment variables,
When it is invoked
Then tests must assert that:
•
model_id,variant_label,suite_id, andcalendarare set correctly• The resulting file contains valid and complete CDDS metadata sections.
AC3 –
standardise_model_data.shtriggers two CDDS runsGiven dual request files exist in
${CYLC_WORKFLOW_SHARE_DIR}/etc,When
standardise_model_data.shexecutes,Then tests must validate (via mocking) that:
• The REF
cdds_convertcall is issued first withrequest_ref.cfg.• The EVAL
cdds_convertcall is issued second withrequest_eval.cfg.• Both calls use the appropriate environment (e.g.,
cmew-standardise-envwrapper).• The script fails cleanly if either request file is missing.
AC4 – Tests integrate with CMEW’s unittest workflowAC4 covered by #165
Given the CMEW unittest execution path (cylc vip … -O unittest),
When the new tests are added under app/configure_standardise/bin and app/standardise_model_data/bin,
Then they must:
• Run successfully under pytest during the Cylc unittest run.
• Run successfully in GitHub Actions.
• Avoid modifying runtime logic (tests rely solely on mocks and environment injection).