Skip to content

feat(GrafanaAlertRuleGroup): add support for Grafana-managed recording rules #1881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions api/v1beta1/grafanaalertrulegroup_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ type AlertRule struct {
// +kubebuilder:validation:Enum=Alerting;NoData;OK;KeepLast
NoDataState *string `json:"noDataState"`

Record *Record `json:"record,omitempty"`

// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=190
// +kubebuilder:example="Always firing"
Expand All @@ -101,6 +103,14 @@ type NotificationSettings struct {
RepeatInterval string `json:"repeat_interval,omitempty"`
}

type Record struct {
// +kubebuilder:validation:Required
From string `json:"from"`

// +kubebuilder:validation:Required
Metric string `json:"metric"`
}

type AlertQuery struct {
// Grafana data source unique identifier; it should be '__expr__' for a Server Side Expression operation.
DatasourceUID string `json:"datasourceUid,omitempty"`
Expand Down
20 changes: 20 additions & 0 deletions api/v1beta1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,16 @@ spec:
required:
- receiver
type: object
record:
properties:
from:
type: string
metric:
type: string
required:
- from
- metric
type: object
title:
example: Always firing
maxLength: 190
Expand Down
6 changes: 6 additions & 0 deletions controllers/grafanaalertrulegroup_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,12 @@ func (r *GrafanaAlertRuleGroupReconciler) reconcileWithInstance(ctx context.Cont
RepeatInterval: rule.NotificationSettings.RepeatInterval,
}
}
if rule.Record != nil {
apiRule.Record = &models.Record{
From: &rule.Record.From,
Metric: &rule.Record.Metric,
}
}
for idx, q := range rule.Data {
apiRule.Data[idx] = &models.AlertQuery{
DatasourceUID: q.DatasourceUID,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,16 @@ spec:
required:
- receiver
type: object
record:
properties:
from:
type: string
metric:
type: string
required:
- from
- metric
type: object
title:
example: Always firing
maxLength: 190
Expand Down
10 changes: 10 additions & 0 deletions deploy/kustomize/base/crds.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,16 @@ spec:
required:
- receiver
type: object
record:
properties:
from:
type: string
metric:
type: string
required:
- from
- metric
type: object
title:
example: Always firing
maxLength: 190
Expand Down
41 changes: 41 additions & 0 deletions docs/docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -368,6 +368,13 @@ AlertRule defines a specific rule to be evaluated. It is based on the upstream m
<br/>
</td>
<td>false</td>
</tr><tr>
<td><b><a href="#grafanaalertrulegroupspecrulesindexrecord">record</a></b></td>
<td>object</td>
<td>
<br/>
</td>
<td>false</td>
</tr></tbody>
</table>

Expand Down Expand Up @@ -528,6 +535,40 @@ relative time range
</table>


### GrafanaAlertRuleGroup.spec.rules[index].record
<sup><sup>[↩ Parent](#grafanaalertrulegroupspecrulesindex)</sup></sup>





<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>from</b></td>
<td>string</td>
<td>
<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>metric</b></td>
<td>string</td>
<td>
<br/>
</td>
<td>true</td>
</tr></tbody>
</table>


### GrafanaAlertRuleGroup.status
<sup><sup>[↩ Parent](#grafanaalertrulegroup)</sup></sup>

Expand Down
13 changes: 13 additions & 0 deletions examples/alertrulegroups/resources.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@ spec:
admin_user: root
admin_password: secret

## Below, we enable Grafana-managed recording rules, which is an opt-in feature used in the GrafanaAlertRuleGroup below
feature_toggles:
# Disabled in OSS / Enterprise Grafana by default
grafanaManagedRecordingRules: "true"
recording_rules:
enabled: "true"
url: http://prometheus:9090/api/prom/push

## Starting from Grafana 11.3.0, it is possible to enforce
## a limit on how many alert rule versions should be stored
## in a database (including the current version of the rule).
Expand Down Expand Up @@ -94,3 +102,8 @@ spec:
noDataState: NoData
title: Temperature below zero
uid: 4843de5c-4f8a-4af0-9509-23526a04faf8

## Grafana-managed recording rule (opt-in feature)
record:
from: A
metric: "weather:temperature:celsius"
6 changes: 6 additions & 0 deletions tests/e2e/example-test/00-create-grafana.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,9 @@ spec:
security:
admin_user: root
admin_password: secret
## Required to test creation of Grafana-managed recording rules
feature_toggles:
grafanaManagedRecordingRules: "true"
recording_rules:
enabled: "true"
url: http://prometheus:9090/api/prom/push
38 changes: 21 additions & 17 deletions tests/e2e/example-test/08-alert-rule-group.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ spec:
- datasourceUid: grafanacloud-demoinfra-prom
model:
datasource:
type: prometheus
uid: grafanacloud-demoinfra-prom
type: prometheus
uid: grafanacloud-demoinfra-prom
editorMode: code
expr: weather_temp_c{location="Toronto"}
instant: true
Expand All @@ -41,22 +41,22 @@ spec:
- datasourceUid: __expr__
model:
conditions:
- evaluator:
params:
- 10
type: gt
operator:
type: and
query:
params:
- C
reducer:
params: []
type: last
type: query
- evaluator:
params:
- 10
type: gt
operator:
type: and
query:
params:
- C
reducer:
params: []
type: last
type: query
datasource:
type: __expr__
uid: __expr__
type: __expr__
uid: __expr__
expression: A
intervalMs: 1000
maxDataPoints: 43200
Expand All @@ -70,3 +70,7 @@ spec:
noDataState: NoData
title: test rule from operator updated
uid: 4843de5c-4f8a-4af0-9509-23526a04faf8
## Grafana-managed recording rule, which is an opt-in feature that must be first enabled and configured in a Grafana instance
record:
from: A
metric: "weather:temperature:celsius"
Loading