Investigate why e2e tests did not catch OOMKilled errors

[jcreixell]

Component(s)

auto-instrumentation

Describe the issue you're reporting

This is a follow up to #3479

Auto-instrumentation for some languages could lead to OOMKilled errors with default settings. This was fixed in #3473 but we need to understand why e2e tests did not catch this problem and modify them to ensure that they do.

[iblancasa]

@IshwarKanse, can you take a look into this?

[IshwarKanse]

I'll check on this next week.

[atoulme]

@IshwarKanse any update?

[IshwarKanse]

@atoulme In the e2e tests, we have asserts for container status to check that the containers in the pod are running. https://github.com/open-telemetry/opentelemetry-operator/blob/main/tests/e2e-instrumentation/instrumentation-python/01-assert.yaml#L68 https://github.com/open-telemetry/opentelemetry-operator/blob/main/tests/e2e-instrumentation/instrumentation-go/02-assert.yaml#L60 Also according to comment #3479 (comment) this could be caused due to disk speeds and specific to some envs. The issue is still under investigation and once we have a root cause, we can try to update our tests to detect it.