Skip to content

Conversation

kaituo
Copy link
Collaborator

@kaituo kaituo commented Sep 27, 2025

Description

This PR adds history and frequency parameters to anomaly detectors.

UI

  • Add Frequency, and History fields in Operation setting section. We recently added frequency in the backend: feat: add frequency scheduling in real time anomaly-detection#1562
  • Add an assisted Operation setting setup via suggestions. Move Operation Settings panel from Define detector page to Configure Model page after Feature and categorical fields definition as we can call Suggest API. Suggest API works better when we know features and categorical fields.
  • Validate frequency as a multiple of interval; add validateMultipleOf util.
  • Show OperationalSettings under model configuration section in Review & Create as have moved them in creating detector pages.
  • Replace BASE_DOCS_LINK usages with AD_DOCS_LINK; add ALERTING_DOCS_LINK.
  • Add error in detector status if any. Otherwise, if a detector is stopped, users won't know why unless they know how to run get detector API with required parameter.

Server/Redux

  • Add POST /detectors/_suggest/{suggestType}[/{dataSourceId}] route.
  • Add cluster client method ad.suggestDetector.
  • Add suggestDetector Redux action.

Models/Helpers

  • Extend Detector with optional frequency and history.
  • Map interval/windowDelay/frequency/history in formikToDetector and model helpers (conditionally persisted).
  • Pass operational settings through preview/sample anomalies flow.

Test

  • Update snapshots and tests due to UI changes.
  • public/pages/DetectorConfig/containers/tests/Features.test.tsx imports featureQuery1/featureQuery2 straight from ./DetectorConfig.test. That re-evaluates the entire DetectorConfig test module when the Features tests run. This causes the flaky Features test failure when DetectorConfig.test fails. This PR extracts featureQuery1/featureQuery2 into public/pages/DetectorConfig/containers/tests/utils/featureQueries.ts and import them from there; this keeps test files isolated and avoids re-running other test suites on import.

Testing done

  1. Fixed broken IT, snapshots and UT. Will publish IT fix in functional test repo.
  2. Manual test.

demo:

frequency.mov

show stopped detector error:

stop error

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@kaituo
Copy link
Collaborator Author

kaituo commented Sep 29, 2025

Cypress IT failed with:

====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✖ plugins/anomaly-detection-dashboard 01:22 1 - 1 - - │
│ s-plugin/create_detector_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 00:25 1 1 - - - │
│ s-plugin/create_forecaster_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ plugins/anomaly-detection-dashboard 02:46 2 - 2 - - │
│ s-plugin/create_remote_detector_spe │
│ c.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 01:27 1 1 - - - │
│ s-plugin/create_remote_forecaster_s │
│ pec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 01:26 1 1 - - - │
│ s-plugin/custom_flattened_forecaste │
│ r_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 00:51 1 1 - - - │
│ s-plugin/daily_forecaster_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 00:26 8 8 - - - │
│ s-plugin/dashboard_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ plugins/anomaly-detection-dashboard 01:18 5 - 1 - 4 │
│ s-plugin/detector_configuration_spe │
│ c.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 00:34 11 11 - - - │
│ s-plugin/detector_list_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ plugins/anomaly-detection-dashboard 00:48 1 - 1 - - │
│ s-plugin/forecaster_configuration_s │
│ pec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 00:11 1 1 - - - │
│ s-plugin/forecaster_list_spec_mock. │
│ js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 01:02 1 1 - - - │
│ s-plugin/forecaster_list_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 01:35 6 6 - - - │
│ s-plugin/historical_analysis_spec.j │
│ s │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 00:16 4 4 - - - │
│ s-plugin/overview_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 00:16 2 2 - - - │
│ s-plugin/real_time_results_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 00:31 3 3 - - - │
│ s-plugin/sample_detector_spec.js │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ plugins/anomaly-detection-dashboard 00:38 1 1 - - - │
│ s-plugin/top_forecaster_spec.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✖ 4 of 17 failed (24%) 15:59 50 41 5 - 4

@kaituo
Copy link
Collaborator Author

kaituo commented Sep 29, 2025

Manually run fixed IT (will create another PR):

====================================================================================================

(Run Finished)

   Spec                                              Tests  Passing  Failing  Pending  Skipped

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ plugins/anomaly-detection-dashboard 01:05 1 1 - - - │
│ s-plugin/forecaster_configuration_s │
│ pec.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 01:05 1 1 - - -

====================================================================================================

(Run Finished)

   Spec                                              Tests  Passing  Failing  Pending  Skipped

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ plugins/anomaly-detection-dashboard 00:29 5 5 - - - │
│ s-plugin/detector_configuration_spe │
│ c.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 00:29 5 5 - - -

✨ Done in 53.02s.

────────────────────────────────────────────────────────────────────────────────────────────────────

Running: plugins/anomaly-detection-dashboards-plugin/create_detector_spec.js (1 of 1)
Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db

Why you should do it regularly:
https://github.com/browserslist/browserslist#browsers-data-updating

Create detector workflow
✓ Full creation - based on real index (28616ms)

1 passing (29s)

(Results)

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 1 │
│ Passing: 1 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: true │
│ Duration: 29 seconds │
│ Spec Ran: plugins/anomaly-detection-dashboards-plugin/create_detector_spec.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘

(Video)

  • Started processing: Compressing to 32 CRF
    ====================================================================================================

(Run Finished)

   Spec                                              Tests  Passing  Failing  Pending  Skipped

┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ plugins/anomaly-detection-dashboard 01:23 2 2 - - - │
│ s-plugin/create_remote_detector_spe │
│ c.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 01:23 2 2 - - -

✨ Done in 117.11s.

…e Model

This PR centralizes operational settings in the model step as we can call suggest API. Suggest API works better when we know features and categorical fields. This PR also adds an assisted setup via suggestions, and corrects documentation links.

Specifically

  - UI
      - Add SuggestParametersDialog to recommend detection interval, frequency, history, and window delay; apply
  suggestions to the form.
      - Introduce Settings panel in Configure Model with Interval, Frequency, Window delay, and History fields.
      - Validate frequency as a multiple of interval; add validateMultipleOf util.
      - Move interval/window delay out of Define Detector (remove DefineDetector/Settings and tests).
      - Show read-only OperationalSettings in Review & Create; remove interval/window delay display from
  DetectorDefinitionFields.
      - Replace BASE_DOCS_LINK usages with AD_DOCS_LINK; add ALERTING_DOCS_LINK.
      - Add error in detector status if any. Otherwise, if a detector is stopped, users won't know why unless they know how to run get detector API with required parameter.
      - Keep frequency in sync when the interval changes (only when they started equal). This reduces UX friction for users who don't care to tweak frequency while editing interval.

  -
  Server/Redux
      - Add POST /detectors/_suggest/{suggestType}[/{dataSourceId}] route.
      - Add cluster client method ad.suggestDetector.
      - Add suggestDetector Redux action.
  -
  Models/Helpers
      - Extend Detector with optional frequency and history.
      - Map interval/windowDelay/frequency/history in formikToDetector and model helpers (conditionally persisted).
      - Pass operational settings through preview/sample anomalies flow.
  -
  Test
      - Update snapshots and tests due to UI changes.
      - public/pages/DetectorConfig/containers/__tests__/Features.test.tsx:33 imports featureQuery1/featureQuery2 straight from ./DetectorConfig.test. That re-evaluates the entire DetectorConfig test module when the Features tests run. This causes the flaky Features test failure when DetectorConfig.test fails. This PR extracts featureQuery1/featureQuery2 into public/pages/DetectorConfig/containers/__tests__/utils/featureQueries.ts and import them from there; this keeps test files isolated and avoids re-running other test suites on import.

Testing done:
1. fixed broken IT and UT. Will publish IT fix in functional test repo.
2. manual test.

Signed-off-by: kaituo <[email protected]>
@kaituo kaituo merged commit 9fc2400 into opensearch-project:main Sep 30, 2025
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants