-
Notifications
You must be signed in to change notification settings - Fork 122
Description
Tell us about the problem you're trying to solve
Have an absolute or a relative threshold to trigger the alert. Right now, using z_score or the rest of the anomaly metrics might not be always meaningful. For example, if freshness is usually 2 minutes and today's data shows a freshness of 2 hours, then, even though this is a big anomaly, we only would like to be alerted after 4 hours. Thus, there should be possible to put the value of 4 hours as an upper bound, together with the anomaly detection
Similarly, the alerts message show a percentage increase or decrease, which should also be an input for different columns to be able to react to the alerts and put a lower or upper bound.
Describe the solution you’d like
In this example, the sum for the amount should only alert the upper bound hits 1000, even though in terms of z_score it could already go over 3. Similarly, we can express this as percentage_upper_bound = 0.3
so only when it gets 30% higher.
models:
- name: pending_orders
config:
re_data_monitored: true
re_data_time_filter: created_at
re_data_columns:
- amount
- status
re_data_metrics_groups:
- table_metrics
re_data_metrics:
column:
amount:
- sum:
upper_bound: 1000
re_data_anomaly_detector:
name: modified_z_score
threshold: 3
Describe the alternative you’ve considered or used
I have replaced the re_data_alerts
with my own model so I could write my own filters