Skip to content

Conversation

kaituo
Copy link
Contributor

@kaituo kaituo commented Oct 2, 2025

Description

This PR:

  • Add new page _observing-your-data/ad/managing-anomalies.md covering how to alert on anomalies with Alerting monitors, including a JSON example, rationale table, and sample alert.
  • Expand _observing-your-data/ad/index.md:
    • Separate timestamp selection from operational settings.
    • Add guidance on detector interval, frequency, window delay, and history, with trade-off explanations.
  • Cross-link Step 6 to the Managing anomalies page.
  • Include a frequency vs. window delay timeline diagram.
  • Add assets:
    • images/anomaly-detection/window-delay-vs-frequency.png
    • images/anomaly-detection/alerting_editor.png

Issues Resolved

Closes #11145

Version

3.3+

Frontend features

If you're submitting documentation for an OpenSearch Dashboards feature, add a video that shows how a user will interact with the UI step by step. A voiceover is optional.

frequency.mov

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…add diagrams

  - Add new page _observing-your-data/ad/managing-anomalies.md covering how
  to alert on anomalies with Alerting monitors, including a JSON example,
  rationale table, and sample alert.
  - Expand _observing-your-data/ad/index.md:
      - Separate timestamp selection from operational settings.
      - Add guidance on detector interval, frequency, window delay, and history,
      with trade-off explanations.
  - Cross-link Step 6 to the Managing anomalies page.
  - Include a frequency vs. window delay timeline diagram.
  - Add assets:
      - images/anomaly-detection/window-delay-vs-frequency.png
      - images/anomaly-detection/alerting_editor.png

Signed-off-by: kaituo <[email protected]>
Copy link

github-actions bot commented Oct 2, 2025

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

@kolchfa-aws kolchfa-aws added v3.3.0 Tech review PR: Tech review in progress release-notes PR: Include this PR in the automated release notes labels Oct 2, 2025
@kaituo
Copy link
Contributor Author

kaituo commented Oct 6, 2025

@kolchfa-aws The PR is ready for doc review.

@kolchfa-aws kolchfa-aws added Doc review PR: Doc review in progress and removed Tech review PR: Tech review in progress labels Oct 6, 2025

## Alert on anomalies

You can create an [Alerting monitor]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/) using either the Anomaly detector editor or the Extraction query editor. When you want to monitor an individual anomaly detector's results and notification condition thresholds on anomaly grade and confidence, use the Anomaly detector editor. Otherwise, use the Extraction query editor to monitor multiple detectors' results or write complex queries/trigger conditions.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaituo Can you use any of the 5 Monitor types to create an alerting monitor for anomalies or does it have to be a per query monitor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They can use any of the five options. The two I mentioned are the most common use cases. However, it’s ultimately up to the users how they choose to use them.

Signed-off-by: Fanit Kolchina <[email protected]>
kolchfa-aws and others added 2 commits October 7, 2025 18:08
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: kaituo <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>

<img src="{{site.url}}{{site.baseurl}}/images/anomaly-detection/alerting_editor.png" alt="Alerting editor" width="800" height="800">

For anomaly alerting, in **Monitor type**, select **Per query monitor** (this is the only type that supports anomaly detection). Then, in **Monitor defining method**, choose one of these methods to define your monitor:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaituo I'm assuming that the per query monitor is required because it's the only one that supports anomaly detection. Please confirm that this is accurate. Thanks!

Copy link
Contributor Author

@kaituo kaituo Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not. Anomaly detection writes to an index and can be monitored just like other indexes. We have a dedicated AD UI in query monitor that simplify configuration.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - removed this text. How about this statement "Anomaly detection is available only if you are defining a per query monitor." on the per query monitor page: is this not accurate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it meant anomaly detection UI is available in query monitor. We don't have anomaly detection UI in other types of monitors. But they can write query to fetch anomaly results and define trigger for it in other types of monitors.

@kolchfa-aws kolchfa-aws added Editorial review PR: Editorial review in progress and removed Doc review PR: Doc review in progress Awaiting response labels Oct 9, 2025
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Editorial review


- **`"size": 1`** in the search input: Retrieves a single document so you can reference `ctx.results.0.hits.hits.0` in the notification to identify which entity (such as `host` or `service`) triggered the alert.

- **`execution_end_time` range `"{{period_end}}||-2m"``"{{period_end}}"`**: Filters results based on detector `execution_end_time`---the time the detector finishes running and indexes the result. Because OpenSearch operates in near-real-time (results are not immediate), indexing and refresh operations introduce a delay before a document becomes searchable. To account for this write-to-search latency, this example includes a small overlap (`-2m`). Specify the overlap based on your system's worst-case delay. Avoid using `data_end_time` (the bucket’s logical end), which can miss results that arrive later.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **`execution_end_time` range `"{{period_end}}||-2m"``"{{period_end}}"`**: Filters results based on detector `execution_end_time`---the time the detector finishes running and indexes the result. Because OpenSearch operates in near-real-time (results are not immediate), indexing and refresh operations introduce a delay before a document becomes searchable. To account for this write-to-search latency, this example includes a small overlap (`-2m`). Specify the overlap based on your system's worst-case delay. Avoid using `data_end_time` (the buckets logical end), which can miss results that arrive later.
- **`execution_end_time` range `"{{period_end}}||-2m"``"{{period_end}}"`**: Filters results based on detector `execution_end_time`---the time the detector finishes running and indexes the result. Because OpenSearch operates in near real time (results are not immediate), indexing and refresh operations introduce a delay before a document becomes searchable. To account for this write-to-search latency, this example includes a small overlap (`-2m`). Specify the overlap based on your system's worst-case delay. Avoid using `data_end_time` (the bucket's logical end), which can miss results that arrive later.


- **`"max_anomaly_grade"` aggregation**: Detects the most severe anomaly in the time window. You can use any field in the anomaly result index for aggregation. For additional fields, see the [Anomaly result mapping]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/result-mapping/).

- **Monitor schedule every 2 minutes**: Evaluates results every two minutes to detect anomalies quickly. Combined with a 2-minute alert throttle, this avoids duplicate notifications for the same event.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Monitor schedule every 2 minutes**: Evaluates results every two minutes to detect anomalies quickly. Combined with a 2-minute alert throttle, this avoids duplicate notifications for the same event.
- **Monitor schedule every 2 minutes**: Evaluates results every 2 minutes to detect anomalies quickly. Combined with a 2-minute alert throttle, this avoids duplicate notifications for the same event.

Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@natebower natebower removed the Editorial review PR: Editorial review in progress label Oct 9, 2025
@natebower natebower merged commit 5385f4e into opensearch-project:main Oct 9, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-notes PR: Include this PR in the automated release notes v3.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DOC] Frequency/history in anomaly detection

3 participants