Open
Description
I just tried to setup an example project based on the bank-of-anthos demo GCP demo application. I'm trying to use the auto instrumentation for the python frontend service but I'm facing a Module not found error
Describe your environment
1.21.10-gke.2000
Bank-of-Anthos-GCP Github page (https://github.com/GoogleCloudPlatform/bank-of-anthos)
Python3.7 for the python services
Steps to reproduce
Describe exactly how to reproduce the error. Include a code sample if applicable.
Modify one of the python services k8s manifests with the following annotations to the PodTemplate part:
apiVersion: apps/v1
kind: Deployment
metadata:
name: frontend
spec:
selector:
matchLabels:
app: frontend
template:
metadata:
labels:
app: frontend
annotations:
sidecar.opentelemetry.io/inject: "true"
instrumentation.opentelemetry.io/inject-python: "true"
spec:
What is the expected behavior?
Instrumentation should not crash with the following error: ModuleNotFoundError: No module named 'opentelemetry.util._once'
What is the actual behavior?
Distribution distro configuration failed
Traceback (most recent call last):
File "/otel-auto-instrumentation/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 37, in _load_distros
distro = entry_point.load()()
File "/otel-auto-instrumentation/pkg_resources/__init__.py", line 2471, in load
return self.resolve()
File "/otel-auto-instrumentation/pkg_resources/__init__.py", line 2477, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/otel-auto-instrumentation/opentelemetry/distro/__init__.py", line 19, in <module>
from opentelemetry.sdk._configuration import _OTelSDKConfigurator
File "/env/lib/python3.7/site-packages/opentelemetry/sdk/__init__.py", line 19, in <module>
from . import metrics, trace, util
File "/env/lib/python3.7/site-packages/opentelemetry/sdk/trace/__init__.py", line 40, in <module>
from opentelemetry import trace as trace_api
File "/otel-auto-instrumentation/opentelemetry/trace/__init__.py", line 113, in <module>
from opentelemetry.util._once import Once
ModuleNotFoundError: No module named 'opentelemetry.util._once'
Failed to auto initialize opentelemetry
Traceback (most recent call last):
File "/otel-auto-instrumentation/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 121, in initialize
distro = _load_distros()
File "/otel-auto-instrumentation/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 52, in _load_distros
raise exc
File "/otel-auto-instrumentation/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 37, in _load_distros
distro = entry_point.load()()
File "/otel-auto-instrumentation/pkg_resources/__init__.py", line 2471, in load
return self.resolve()
File "/otel-auto-instrumentation/pkg_resources/__init__.py", line 2477, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/otel-auto-instrumentation/opentelemetry/distro/__init__.py", line 19, in <module>
from opentelemetry.sdk._configuration import _OTelSDKConfigurator
File "/env/lib/python3.7/site-packages/opentelemetry/sdk/__init__.py", line 19, in <module>
from . import metrics, trace, util
File "/env/lib/python3.7/site-packages/opentelemetry/sdk/trace/__init__.py", line 40, in <module>
from opentelemetry import trace as trace_api
File "/otel-auto-instrumentation/opentelemetry/trace/__init__.py", line 113, in <module>
from opentelemetry.util._once import Once
ModuleNotFoundError: No module named 'opentelemetry.util._once'
{"timestamp": "2022-05-16 11:20:14", "message": "info | Starting gunicorn 20.1.0", "severity": "INFO"}
{"timestamp": "2022-05-16 11:20:14", "message": "info | Listening at: http://0.0.0.0:8080 (7)", "severity": "INFO"}
{"timestamp": "2022-05-16 11:20:14", "message": "info | Using worker: gthread", "severity": "INFO"}
{"timestamp": "2022-05-16 11:20:14", "message": "info | Booting worker with pid: 10", "severity": "INFO"}
{"timestamp": "2022-05-16 11:20:16", "message": "exception | Exception in worker process", "severity": "ERROR"}
Traceback (most recent call last):
File "/env/lib/python3.7/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
worker.init_process()
File "/env/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
super().init_process()
File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 134, in init_process
self.load_wsgi()
File "/env/lib/python3.7/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
self.wsgi = self.app.wsgi()
File "/env/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
return self.load_wsgiapp()
File "/env/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
return util.import_app(self.app_uri)
File "/env/lib/python3.7/site-packages/gunicorn/util.py", line 359, in import_app
mod = importlib.import_module(module)
File "/opt/python3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/vmagent/app/frontend.py", line 31, in <module>
from opentelemetry import trace
File "/otel-auto-instrumentation/opentelemetry/trace/__init__.py", line 113, in <module>
from opentelemetry.util._once import Once
ModuleNotFoundError: No module named 'opentelemetry.util._once'
{"timestamp": "2022-05-16 11:20:16", "message": "info | Worker exiting (pid: 10)", "severity": "INFO"}
{"timestamp": "2022-05-16 11:20:16", "message": "info | Shutting down: Master", "severity": "INFO"}
{"timestamp": "2022-05-16 11:20:16", "message": "info | Reason: Worker failed to boot.", "severity": "INFO"}
Additional context
apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
name: my-instrumentation
spec:
exporter:
endpoint: http://otelcollector-collector:4317
propagators:
- tracecontext
- baggage
- b3
sampler:
type: parentbased_traceidratio
argument: "0.25"
java:
image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-java:latest
nodejs:
image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-nodejs:latest
python:
image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-python:latest