Set watch option

[jaronoff97]

Description: This is a performance improvement for us to only watch items we manage.

Link to tracking Issue(s):

Partially resolves #2808

Testing: Tested locally

Documentation: n/a

[jaronoff97]

I followed the docs from the operator-sdk but they're out of date after this release kubernetes-sigs/controller-runtime#2300. This is the best way going forward!

[swiatekm]

I believe the reason for e2e test failures is that in the course of adding sidecars or instrumentations, we sometimes ask the API Server for resources owning the Pod - ReplicaSets, Deployments, and the like. Those don't have the label, and hence we fail.

We probably need a more nuanced policy here. Something like:

• All resources that may own Pods
• All operator CRs (what if someone sets their own managed-by label on those?)
• All resources with the ownership label, as you've already done

Incidentally, are we doing the right thing by setting managed-by to the operator in defaulter webhooks? Intuitively, the operator is not managing these, but whoever created them instead. Is this standard practice?

[jaronoff97]

@swiatekm-sumo i see... i think this makes sense. I'll make some changes and we can test it out. I think we could even put this behind a featuregate given its just an option on the manager config, that would let us know pretty clearly if this is helpful for performance as well. My worry is that there's some unknown unknown consequence of this

[swiatekm]

I think you need ReplicaSet and CronJob here as well. Or maybe just Job, ReplicaSet and DaemonSet (the immediate Pod owners) are sufficient? Both are probably worth testing.

[swiatekm]

This may also be easier to test via envtest, as we're running into problems at the webhook level. In general, I think we should move our setup code out of main.go and actually use it for the controller tests. WDYT?