Skip to content

Commit f9c9f70

Browse files
committed
feat(ta): allowNamespaces and denyNamespaces
When the target allocator is configured to watch Prometheus custom resources in the cluster to discover targets, it is currently hard-coded to require a ClusterRole with a policy rule of listing namespaces. This prevents usage in environments where configuring ClusterRoles is not permitted i.e. in namespace-as-a-service setups where only Roles can be created. This change introduces two fields in the prometheusCR specification to allow configuring the namespaces that can be interacted with by the target allocator. - allowNamespaces is a comma-separated list of namespaces for the target allocator to watch. If set to an empty string, it will list all list all namespaces in the cluster. This is mutually exclusive with denyNamespaces. Defaults to an empty string. - denyNamespaces is a comma-separated list of namespaces for the target allocator to not watch. If set to an empty string, it will not exclude any namespaces. This is mutually exclusive with allowNamespaces. Defaults to an empty string. Fixes: #3086 Signed-off-by: Charlie Le <[email protected]>
1 parent a39bb7f commit f9c9f70

33 files changed

+482
-192
lines changed

.chloggen/namespace-ta.yaml

+3-2
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ component: target allocator
66

77
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
88
note: |
9-
Add support for `WATCH_NAMESPACE` environment variable in the target allocator.
9+
Add support for setting the allowNamespaces and denyNamespaces in the target allocator.
1010
1111
# One or more tracking issues related to the change
1212
issues: [3086]
@@ -15,4 +15,5 @@ issues: [3086]
1515
# These lines will be padded with 2 spaces and then inserted directly into the document.
1616
# Use pipe (|) for multiline entries.
1717
subtext: |
18-
This variable can be set to an empty string to watch all namespaces, or to a comma-separated list of namespaces to watch.
18+
allowNamespaces can be set to an empty string to watch all namespaces (default) or to a comma-separated list of namespaces to watch.
19+
denyNamespaces can be set to an empty string to deny watching any namespaces (default) or to a comma-separated list of namespaces to deny watching.

apis/v1beta1/targetallocator_types.go

+6
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,12 @@ type TargetAllocatorPrometheusCR struct {
1212
// Enabled indicates whether to use a PrometheusOperator custom resources as targets or not.
1313
// +optional
1414
Enabled bool `json:"enabled,omitempty"`
15+
// AllowNamespaces Namespaces to scope the interaction of the Target Allocator and the apiserver (allow list). This is mutually exclusive with DenyNamespaces.
16+
// +optional
17+
AllowNamespaces string `json:"allowNamespaces,omitempty"`
18+
// DenyNamespaces Namespaces to scope the interaction of the Target Allocator and the apiserver (allow list). This is mutually exclusive with AllowNamespaces.
19+
// +optional
20+
DenyNamespaces string `json:"denyNamespaces,omitempty"`
1521
// Default interval between consecutive scrapes. Intervals set in ServiceMonitors and PodMonitors override it.
1622
//Equivalent to the same setting on the Prometheus CR.
1723
//

bundle/community/manifests/opentelemetry.io_opentelemetrycollectors.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -7889,6 +7889,10 @@ spec:
78897889
type: object
78907890
prometheusCR:
78917891
properties:
7892+
allowNamespaces:
7893+
type: string
7894+
denyNamespaces:
7895+
type: string
78927896
enabled:
78937897
type: boolean
78947898
podMonitorSelector:

bundle/community/manifests/opentelemetry.io_targetallocators.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -2257,6 +2257,10 @@ spec:
22572257
type: string
22582258
prometheusCR:
22592259
properties:
2260+
allowNamespaces:
2261+
type: string
2262+
denyNamespaces:
2263+
type: string
22602264
enabled:
22612265
type: boolean
22622266
podMonitorSelector:

bundle/openshift/manifests/opentelemetry.io_opentelemetrycollectors.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -7889,6 +7889,10 @@ spec:
78897889
type: object
78907890
prometheusCR:
78917891
properties:
7892+
allowNamespaces:
7893+
type: string
7894+
denyNamespaces:
7895+
type: string
78927896
enabled:
78937897
type: boolean
78947898
podMonitorSelector:

bundle/openshift/manifests/opentelemetry.io_targetallocators.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -2257,6 +2257,10 @@ spec:
22572257
type: string
22582258
prometheusCR:
22592259
properties:
2260+
allowNamespaces:
2261+
type: string
2262+
denyNamespaces:
2263+
type: string
22602264
enabled:
22612265
type: boolean
22622266
podMonitorSelector:

cmd/otel-allocator/README.md

+4-6
Original file line numberDiff line numberDiff line change
@@ -269,8 +269,8 @@ rules:
269269

270270
#### Namespace-scoped RBAC
271271

272-
If you want to have the TargetAllocator watch a specific namespace, you can set the WATCH_NAMESPACE environment variable
273-
in the TargetAllocator's deployment. This is useful if you want to restrict the TargetAllocator to only watch Prometheus
272+
If you want to have the TargetAllocator watch a specific namespace, you can set the allowNamespaces field
273+
in the TargetAllocator's prometheusCR configuration. This is useful if you want to restrict the TargetAllocator to only watch Prometheus
274274
CRs in a specific namespace, and not have cluster-wide access.
275275

276276
```yaml
@@ -279,13 +279,11 @@ CRs in a specific namespace, and not have cluster-wide access.
279279
serviceAccount: opentelemetry-targetallocator-sa
280280
prometheusCR:
281281
enabled: true
282-
env:
283-
- name: WATCH_NAMESPACE
284-
value: "foo"
282+
allowNamespaces: foo
285283
```
286284

287285
In this case, you will need to create a Role and RoleBinding instead of a ClusterRole and ClusterRoleBinding. The Role
288-
and RoleBinding should be created in the namespace specified in the WATCH_NAMESPACE environment variable.
286+
and RoleBinding should be created in the namespace specified by the allowNamespaces field.
289287

290288
```yaml
291289
apiVersion: rbac.authorization.k8s.io/v1

cmd/otel-allocator/internal/config/config.go

+2
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,8 @@ type Config struct {
5353

5454
type PrometheusCRConfig struct {
5555
Enabled bool `yaml:"enabled,omitempty"`
56+
AllowNamespaces string `yaml:"allow_namespaces,omitempty"`
57+
DenyNamespaces string `yaml:"deny_namespaces,omitempty"`
5658
PodMonitorSelector *metav1.LabelSelector `yaml:"pod_monitor_selector,omitempty"`
5759
PodMonitorNamespaceSelector *metav1.LabelSelector `yaml:"pod_monitor_namespace_selector,omitempty"`
5860
ServiceMonitorSelector *metav1.LabelSelector `yaml:"service_monitor_selector,omitempty"`

cmd/otel-allocator/internal/watcher/promOperator.go

+20-11
Original file line numberDiff line numberDiff line change
@@ -54,21 +54,30 @@ func NewPrometheusCRWatcher(ctx context.Context, logger logr.Logger, cfg allocat
5454
return nil, err
5555
}
5656

57-
// Check env var for WATCH_NAMESPACE and use it if its set, else use v1.NamespaceAll
58-
// This is to allow the operator to watch only a specific namespace
59-
watchNamespace, found := os.LookupEnv("WATCH_NAMESPACE")
57+
if cfg.PrometheusCR.AllowNamespaces != "" && cfg.PrometheusCR.DenyNamespaces != "" {
58+
return nil, fmt.Errorf("both allow and deny namespaces are set, only one should be set")
59+
}
60+
6061
allowList := map[string]struct{}{}
61-
if found {
62-
logger.Info("watching namespace(s)", "namespaces", watchNamespace)
63-
for _, ns := range strings.Split(watchNamespace, ",") {
62+
if cfg.PrometheusCR.AllowNamespaces != "" {
63+
logger.Info("watching namespace(s)", "namespaces", cfg.PrometheusCR.AllowNamespaces)
64+
for _, ns := range strings.Split(cfg.PrometheusCR.AllowNamespaces, ",") {
6465
allowList[ns] = struct{}{}
6566
}
6667
} else {
68+
logger.Info("cfg.PrometheusCR.AllowNamespaces is unset, watching all namespaces")
6769
allowList = map[string]struct{}{v1.NamespaceAll: {}}
68-
logger.Info("the env var WATCH_NAMESPACE isn't set, watching all namespaces")
6970
}
7071

71-
factory := informers.NewMonitoringInformerFactories(allowList, map[string]struct{}{}, mClient, allocatorconfig.DefaultResyncTime, nil) //TODO decide what strategy to use regarding namespaces
72+
denyList := map[string]struct{}{}
73+
if cfg.PrometheusCR.DenyNamespaces != "" {
74+
logger.Info("excluding namespace(s)", "namespaces", cfg.PrometheusCR.DenyNamespaces)
75+
for _, ns := range strings.Split(cfg.PrometheusCR.DenyNamespaces, ",") {
76+
denyList[ns] = struct{}{}
77+
}
78+
}
79+
80+
factory := informers.NewMonitoringInformerFactories(allowList, denyList, mClient, allocatorconfig.DefaultResyncTime, nil)
7281

7382
monitoringInformers, err := getInformers(factory)
7483
if err != nil {
@@ -114,7 +123,7 @@ func NewPrometheusCRWatcher(ctx context.Context, logger logr.Logger, cfg allocat
114123
logger.Error(err, "Retrying namespace informer creation in promOperator CRD watcher")
115124
return true
116125
}, func() error {
117-
nsMonInf, err = getNamespaceInformer(ctx, allowList, promLogger, clientset, operatorMetrics)
126+
nsMonInf, err = getNamespaceInformer(ctx, allowList, denyList, promLogger, clientset, operatorMetrics)
118127
return err
119128
})
120129
if getNamespaceInformerErr != nil {
@@ -164,7 +173,7 @@ type PrometheusCRWatcher struct {
164173
store *assets.StoreBuilder
165174
}
166175

167-
func getNamespaceInformer(ctx context.Context, allowList map[string]struct{}, promOperatorLogger *slog.Logger, clientset kubernetes.Interface, operatorMetrics *operator.Metrics) (cache.SharedIndexInformer, error) {
176+
func getNamespaceInformer(ctx context.Context, allowList, denyList map[string]struct{}, promOperatorLogger *slog.Logger, clientset kubernetes.Interface, operatorMetrics *operator.Metrics) (cache.SharedIndexInformer, error) {
168177
kubernetesVersion, err := clientset.Discovery().ServerVersion()
169178
if err != nil {
170179
return nil, err
@@ -180,7 +189,7 @@ func getNamespaceInformer(ctx context.Context, allowList map[string]struct{}, pr
180189
clientset.CoreV1(),
181190
clientset.AuthorizationV1().SelfSubjectAccessReviews(),
182191
allowList,
183-
map[string]struct{}{},
192+
denyList,
184193
)
185194
if err != nil {
186195
return nil, err

config/crd/bases/opentelemetry.io_opentelemetrycollectors.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -7875,6 +7875,10 @@ spec:
78757875
type: object
78767876
prometheusCR:
78777877
properties:
7878+
allowNamespaces:
7879+
type: string
7880+
denyNamespaces:
7881+
type: string
78787882
enabled:
78797883
type: boolean
78807884
podMonitorSelector:

config/crd/bases/opentelemetry.io_targetallocators.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -2254,6 +2254,10 @@ spec:
22542254
type: string
22552255
prometheusCR:
22562256
properties:
2257+
allowNamespaces:
2258+
type: string
2259+
denyNamespaces:
2260+
type: string
22572261
enabled:
22582262
type: boolean
22592263
podMonitorSelector:

controllers/builder_test.go

+28-14
Original file line numberDiff line numberDiff line change
@@ -1500,6 +1500,8 @@ config:
15001500
target_label: instance
15011501
filter_strategy: relabel-config
15021502
prometheus_cr:
1503+
allow_namespaces: ""
1504+
deny_namespaces: ""
15031505
enabled: true
15041506
pod_monitor_selector: null
15051507
probe_selector: null
@@ -1537,7 +1539,7 @@ prometheus_cr:
15371539
"app.kubernetes.io/version": "latest",
15381540
},
15391541
Annotations: map[string]string{
1540-
"opentelemetry-targetallocator-config/hash": "286a5a4e7ec6d2ce652a4ce23e135c10053b4c87fd080242daa5bf21dcd5a337",
1542+
"opentelemetry-targetallocator-config/hash": "780df8ce12aa86e4c0c159a7550b74f12b0bede90a64b448af05b35205291f01",
15411543
},
15421544
},
15431545
Spec: corev1.PodSpec{
@@ -1669,7 +1671,7 @@ prometheus_cr:
16691671
"app.kubernetes.io/version": "latest",
16701672
},
16711673
Annotations: map[string]string{
1672-
"opentelemetry-targetallocator-config/hash": "286a5a4e7ec6d2ce652a4ce23e135c10053b4c87fd080242daa5bf21dcd5a337",
1674+
"opentelemetry-targetallocator-config/hash": "780df8ce12aa86e4c0c159a7550b74f12b0bede90a64b448af05b35205291f01",
16731675
},
16741676
},
16751677
Spec: policyV1.PodDisruptionBudgetSpec{
@@ -1959,6 +1961,8 @@ config:
19591961
target_label: instance
19601962
filter_strategy: relabel-config
19611963
prometheus_cr:
1964+
allow_namespaces: ""
1965+
deny_namespaces: ""
19621966
enabled: true
19631967
pod_monitor_selector: null
19641968
probe_selector: null
@@ -1996,7 +2000,7 @@ prometheus_cr:
19962000
"app.kubernetes.io/version": "latest",
19972001
},
19982002
Annotations: map[string]string{
1999-
"opentelemetry-targetallocator-config/hash": "286a5a4e7ec6d2ce652a4ce23e135c10053b4c87fd080242daa5bf21dcd5a337",
2003+
"opentelemetry-targetallocator-config/hash": "780df8ce12aa86e4c0c159a7550b74f12b0bede90a64b448af05b35205291f01",
20002004
},
20012005
},
20022006
Spec: corev1.PodSpec{
@@ -2128,7 +2132,7 @@ prometheus_cr:
21282132
"app.kubernetes.io/version": "latest",
21292133
},
21302134
Annotations: map[string]string{
2131-
"opentelemetry-targetallocator-config/hash": "286a5a4e7ec6d2ce652a4ce23e135c10053b4c87fd080242daa5bf21dcd5a337",
2135+
"opentelemetry-targetallocator-config/hash": "780df8ce12aa86e4c0c159a7550b74f12b0bede90a64b448af05b35205291f01",
21322136
},
21332137
},
21342138
Spec: policyV1.PodDisruptionBudgetSpec{
@@ -2463,6 +2467,8 @@ https:
24632467
tls_cert_file_path: /tls/tls.crt
24642468
tls_key_file_path: /tls/tls.key
24652469
prometheus_cr:
2470+
allow_namespaces: ""
2471+
deny_namespaces: ""
24662472
enabled: true
24672473
pod_monitor_selector: null
24682474
probe_selector: null
@@ -2500,7 +2506,7 @@ prometheus_cr:
25002506
"app.kubernetes.io/version": "latest",
25012507
},
25022508
Annotations: map[string]string{
2503-
"opentelemetry-targetallocator-config/hash": "3e2818ab54d866289de7837779e86e9c95803c43c0c4b58b25123e809ae9b771",
2509+
"opentelemetry-targetallocator-config/hash": "fbe8a4926da66590ccd42266083a98c88ac1f4970e1c662afc4a5841deca023f",
25042510
},
25052511
},
25062512
Spec: corev1.PodSpec{
@@ -2658,7 +2664,7 @@ prometheus_cr:
26582664
"app.kubernetes.io/version": "latest",
26592665
},
26602666
Annotations: map[string]string{
2661-
"opentelemetry-targetallocator-config/hash": "3e2818ab54d866289de7837779e86e9c95803c43c0c4b58b25123e809ae9b771",
2667+
"opentelemetry-targetallocator-config/hash": "fbe8a4926da66590ccd42266083a98c88ac1f4970e1c662afc4a5841deca023f",
26622668
},
26632669
},
26642670
Spec: policyV1.PodDisruptionBudgetSpec{
@@ -3532,6 +3538,8 @@ config:
35323538
target_label: instance
35333539
filter_strategy: relabel-config
35343540
prometheus_cr:
3541+
allow_namespaces: ""
3542+
deny_namespaces: ""
35353543
enabled: true
35363544
pod_monitor_selector: null
35373545
probe_selector: null
@@ -3569,7 +3577,7 @@ prometheus_cr:
35693577
"app.kubernetes.io/version": "latest",
35703578
},
35713579
Annotations: map[string]string{
3572-
"opentelemetry-targetallocator-config/hash": "f80c054419fe2f9030368557da143e200c70772d1d5f1be50ed55ae960b4b17d",
3580+
"opentelemetry-targetallocator-config/hash": "d03c739e0ad848160f39e963b358fa11c33abe4bb28be11e496a539314bc4b82",
35733581
},
35743582
},
35753583
Spec: corev1.PodSpec{
@@ -3701,7 +3709,7 @@ prometheus_cr:
37013709
"app.kubernetes.io/version": "latest",
37023710
},
37033711
Annotations: map[string]string{
3704-
"opentelemetry-targetallocator-config/hash": "f80c054419fe2f9030368557da143e200c70772d1d5f1be50ed55ae960b4b17d",
3712+
"opentelemetry-targetallocator-config/hash": "d03c739e0ad848160f39e963b358fa11c33abe4bb28be11e496a539314bc4b82",
37053713
},
37063714
},
37073715
Spec: policyV1.PodDisruptionBudgetSpec{
@@ -3807,6 +3815,8 @@ config:
38073815
target_label: instance
38083816
filter_strategy: relabel-config
38093817
prometheus_cr:
3818+
allow_namespaces: ""
3819+
deny_namespaces: ""
38103820
enabled: true
38113821
pod_monitor_selector: null
38123822
probe_selector: null
@@ -3844,7 +3854,7 @@ prometheus_cr:
38443854
"app.kubernetes.io/version": "latest",
38453855
},
38463856
Annotations: map[string]string{
3847-
"opentelemetry-targetallocator-config/hash": "f80c054419fe2f9030368557da143e200c70772d1d5f1be50ed55ae960b4b17d",
3857+
"opentelemetry-targetallocator-config/hash": "d03c739e0ad848160f39e963b358fa11c33abe4bb28be11e496a539314bc4b82",
38483858
},
38493859
},
38503860
Spec: corev1.PodSpec{
@@ -3976,7 +3986,7 @@ prometheus_cr:
39763986
"app.kubernetes.io/version": "latest",
39773987
},
39783988
Annotations: map[string]string{
3979-
"opentelemetry-targetallocator-config/hash": "f80c054419fe2f9030368557da143e200c70772d1d5f1be50ed55ae960b4b17d",
3989+
"opentelemetry-targetallocator-config/hash": "d03c739e0ad848160f39e963b358fa11c33abe4bb28be11e496a539314bc4b82",
39803990
},
39813991
},
39823992
Spec: policyV1.PodDisruptionBudgetSpec{
@@ -4132,6 +4142,8 @@ config:
41324142
target_label: instance
41334143
filter_strategy: relabel-config
41344144
prometheus_cr:
4145+
allow_namespaces: ""
4146+
deny_namespaces: ""
41354147
enabled: true
41364148
pod_monitor_selector: null
41374149
probe_selector: null
@@ -4169,7 +4181,7 @@ prometheus_cr:
41694181
"app.kubernetes.io/version": "latest",
41704182
},
41714183
Annotations: map[string]string{
4172-
"opentelemetry-targetallocator-config/hash": "286a5a4e7ec6d2ce652a4ce23e135c10053b4c87fd080242daa5bf21dcd5a337",
4184+
"opentelemetry-targetallocator-config/hash": "780df8ce12aa86e4c0c159a7550b74f12b0bede90a64b448af05b35205291f01",
41734185
},
41744186
},
41754187
Spec: corev1.PodSpec{
@@ -4301,7 +4313,7 @@ prometheus_cr:
43014313
"app.kubernetes.io/version": "latest",
43024314
},
43034315
Annotations: map[string]string{
4304-
"opentelemetry-targetallocator-config/hash": "286a5a4e7ec6d2ce652a4ce23e135c10053b4c87fd080242daa5bf21dcd5a337",
4316+
"opentelemetry-targetallocator-config/hash": "780df8ce12aa86e4c0c159a7550b74f12b0bede90a64b448af05b35205291f01",
43054317
},
43064318
},
43074319
Spec: policyV1.PodDisruptionBudgetSpec{
@@ -4431,6 +4443,8 @@ https:
44314443
tls_cert_file_path: /tls/tls.crt
44324444
tls_key_file_path: /tls/tls.key
44334445
prometheus_cr:
4446+
allow_namespaces: ""
4447+
deny_namespaces: ""
44344448
enabled: true
44354449
pod_monitor_selector: null
44364450
probe_selector: null
@@ -4468,7 +4482,7 @@ prometheus_cr:
44684482
"app.kubernetes.io/version": "latest",
44694483
},
44704484
Annotations: map[string]string{
4471-
"opentelemetry-targetallocator-config/hash": "3e2818ab54d866289de7837779e86e9c95803c43c0c4b58b25123e809ae9b771",
4485+
"opentelemetry-targetallocator-config/hash": "fbe8a4926da66590ccd42266083a98c88ac1f4970e1c662afc4a5841deca023f",
44724486
},
44734487
},
44744488
Spec: corev1.PodSpec{
@@ -4626,7 +4640,7 @@ prometheus_cr:
46264640
"app.kubernetes.io/version": "latest",
46274641
},
46284642
Annotations: map[string]string{
4629-
"opentelemetry-targetallocator-config/hash": "3e2818ab54d866289de7837779e86e9c95803c43c0c4b58b25123e809ae9b771",
4643+
"opentelemetry-targetallocator-config/hash": "fbe8a4926da66590ccd42266083a98c88ac1f4970e1c662afc4a5841deca023f",
46304644
},
46314645
},
46324646
Spec: policyV1.PodDisruptionBudgetSpec{

0 commit comments

Comments
 (0)