observability.metrics.DisablePrometheusAnnotations does not work

[verejoel]

Component(s)

No response

What happened?

Description

Setting the new DisablePrometheusAnnotations config option to true in the latest version of the operator does not work.

Steps to Reproduce

Operator version: 0.93.0
CRDs have been updated on K8s for this version.

Add the following stanza to an existing OpenTelemetryCollector and observe that the deployment is not rolled out, nor are annotations removed.

spec:
  observability:
    metrics:
      DisablePrometheusAnnotations: true

I also tried redeploying the operator, to no avail.

Expected Result

The prometheus scrape annotations should not be present on the deployment if this option is enabled.

Actual Result

Prometheus scrape annotations are present, operator does not perform reconciliation

Supplemental

Can we also fix the naming of this resource in the OpenAPI spec to not start with a capital letter?

Kubernetes Version

1.28.3

Operator version

1.93.0

Collector version

Custom build

Environment information

No response

Log output

No response

Additional context

No response

[verejoel]

I tested this also with the --feature-gates=operator.observability.prometheus flag enabled with the same result

[jaronoff97]

Are there any error logs from the operator?

[pavolloffay]

The implementation of this feature is very simple

opentelemetry-operator/internal/manifests/collector/annotations.go

Line 31 in 769c984

if !instance.Spec.Observability.Metrics.DisablePrometheusAnnotations {

Could you pleae paste collector deployment with DisablePrometheusAnnotations: true

[rivToadd]

I just ran a validation

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: my-collector
spec:
  mode: statefulset
  replicas: 1
  observability:
    metrics:
      enableMetrics: true
      DisablePrometheusAnnotations: true
  config: |
    receivers:
      jaeger:
        protocols:
          grpc:
    processors:

    exporters:
      debug:

    service:
      pipelines:
        traces:
          receivers: [jaeger]
          processors: []
          exporters: [debug]

give it a go with this spec. I was successful with it

[jaronoff97]

an aside, I do think we should rename DisablePrometheusAnnotations to be lowercased to match all of the other fields we have.

[NickAnge]

Hey everyone. 👋 We are facing the issue.
Defining DisablePrometheusAnnotations: true, doesn't seem to remove the annotations. Is there a plan to get this fixed ?

Thanks in advance

[jaronoff97]

@NickAnge there's an open PR that's gone stale #2664 (cc @rivToadd)

For now, you can manually override the annotations iirc by setting it false:

prometheus.io/scrape: "false"

[NickAnge]

Hey @jaronoff97 . Thank you, we will try it

[Toaddyan]

This was fixed in the v1alpha1 version of it, but now it seems the change is in the v1beta1 version of it. If you migrate forward to the v1beta1, this feature will work.