You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on May 29, 2024. It is now read-only.
Copy file name to clipboardExpand all lines: docs/architecture/alerting.markdown
+15-6
Original file line number
Diff line number
Diff line change
@@ -24,14 +24,17 @@ end
24
24
subgraph AM["Alerting Manager"]
25
25
alertingRelay --> |Alert|EL
26
26
EL[eventLoop] --> |Alert SUUID|AS["AlertStore"]
27
-
AS --> |Alert Destination|EL
28
-
EL --> |Submit alert|AD[Destination]
29
-
AD --> |Slack|SH["Slack Handler"]
30
-
AD --> |counterParty|CPH["Counterparty Handler"]
27
+
AS --> |Alert Policy|EL
28
+
EL --> |Submit alert|SR["SeverityRouter"]
29
+
SR --> SH["Slack"]
30
+
SR --> PH["PagerDuty"]
31
+
SR --> CPH["CounterParty Handler"]
31
32
32
33
end
33
34
CPH --> |"HTTP POST"|TPH["Third Party API"]
34
35
SH --> |"HTTP POST"|SlackAPI("Slack Webhook API")
36
+
PH --> |"HTTP POST"|PagerDutyAPI("PagerDuty API")
37
+
35
38
</div>
36
39
{% endraw %}
37
40
@@ -49,9 +52,12 @@ An alert destination is a configurable destination that an alert can be sent to.
49
52
#### Slack
50
53
The Slack alert destination is a configurable destination that allows alerts to be sent to a specific Slack channel. The Slack alert destination will be configured with a Slack webhook URL. The Slack alert destination will then use this URL to send alerts to the specified Slack channel.
51
54
52
-
**NOTE: As of now Pessimism can only post alerts to a single slack channel**
53
55
54
-
### Cooldown
56
+
#### PagerDuty
57
+
The PagerDuty alert destination is a configurable destination that allows alerts to be sent to a specific PagerDuty services via the use of integration keys. Pessimism also uses the SUUID associated with an alert as a deduplication key for PagerDuty. This is done to ensure that PagerDuty will not be spammed with duplicate or incidents.
58
+
59
+
60
+
### Alert CoolDowns
55
61
To ensure that alerts aren't spammed to destinations once invoked, a time based cooldown value (`cooldown_time`) can be defined within the `alert_params` of a heuristic session config. This time value determines how long a heuristic session must wait before being allowed to alert again.
56
62
57
63
An example of this is shown below:
@@ -73,3 +79,6 @@ An example of this is shown below:
73
79
}
74
80
}
75
81
```
82
+
83
+
### Alert Messages
84
+
Pessimism allows for the arbitrary customization of alert messages. This is done by defining an `message` value string within the `alerting_params` of a heuristic session bootstrap config or session creation request. This is critical for providing additional context on alerts that allow for easier ingestion by downstream consumers (i.e, alert responders).
0 commit comments