Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

auto-instrumentation: can't preserve ownership of initcontainer volume #2616

Closed
leofvo opened this issue Feb 12, 2024 · 6 comments
Closed

auto-instrumentation: can't preserve ownership of initcontainer volume #2616

leofvo opened this issue Feb 12, 2024 · 6 comments
Labels
bug Something isn't working needs triage

Comments

@leofvo
Copy link

leofvo commented Feb 12, 2024

Component(s)

instrumentation

What happened?

Description

I using python auto-instrumentation in a kubernetes cluster. I have installed the operator and the instrumentation CRD (all up to date).
I have deployed about 6 services, all using fastapi. And one of my service isn't sending metrics and traces to my collector.
The collector is the same, operator same, same namespace and the instrument CR too.
image

Steps to Reproduce

Expected Result

Init container booting and finished without any issues..

Actual Result

When my pods is starting, the init container launch and this kind of error message appears:
cp: can't preserve ownership of '/otel-auto-instrumentation-python/./zipp-3.17.0.dist-info/top_level.txt': Operation not permitted

I can't get traces or metrics exported to my collector...

Here is a screenshot of the initcontainer logs:

image

Kubernetes Version

v1.28.5-eks-5e0fdde

Operator version

v0.93.0

Collector version

v0.90.1

Environment information

││     app.kubernetes.io/component: controller-manager
││     app.kubernetes.io/instance: opentelemetry-operator
││     app.kubernetes.io/managed-by: Helm
││     app.kubernetes.io/name: opentelemetry-operator
││     app.kubernetes.io/version: 0.93.0
││     argocd.argoproj.io/instance: observability
││     helm.sh/chart: opentelemetry-operator-0.47.0

Log output

No response

Additional context

No response

@leofvo leofvo added bug Something isn't working needs triage labels Feb 12, 2024
@leofvo
Copy link
Author

leofvo commented Feb 13, 2024

Update:
I tried to upgrade the volume size limit, still having the issue.

It doesn't seem to be related to the available storage limit.

@illrill
Copy link

illrill commented Feb 15, 2024

Can you post the application's Deployment manifest?

I've faced the same issue with the Node.js instrumentation, which I've been able to pinpoint to container settings runAsNonRoot: true (this gets preserve ownership errors) vs runAsNonRoot: false (everything works).

@leofvo
Copy link
Author

leofvo commented Feb 19, 2024

I had runAsNonRoot: true, I removed it and now my traces are collected, but still having the error message in my initContainer logs.
Thanks for the help !

@smallc2009
Copy link

this issue is also related my one #2726 . it looks like it doesn't affect the collecting trace with these message, see #1013 , but still need a proper way to avoid.

@iblancasa
Copy link
Contributor

I would say #2695 should fix this issue

@atoulme
Copy link
Contributor

atoulme commented Mar 12, 2025

Closing as complete as #2695 is now merged. Please open a new issue if the problem is still present.

@atoulme atoulme closed this as completed Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage
Projects
None yet
Development

No branches or pull requests

5 participants