Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] OTLP ingest fails on initialisation when providing DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_MAX_RECV_MSG_SIZE_MIB #33977

Open
KiraShatskyi opened this issue Feb 12, 2025 · 1 comment
Labels

Comments

@KiraShatskyi
Copy link

When specifying DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_MAX_RECV_MSG_SIZE_MIB environment variable, the OTLP ingest fails and does not start.

The logs suggest a straightforward issue of failed string to int conversion:

February 12, 2025 at 12:06 (UTC+1:00) | 2025-02-12 11:06:14 UTC \| CORE \| ERROR \| (comp/otelcol/collector/impl-pipeline/pipeline.go:112 in func1) \| Error running the OTLP ingest pipeline: failed to get config: cannot unmarshal the configuration: decoding failed due to the following error(s): | datadog-agent
-- | -- | --
February 12, 2025 at 12:06 (UTC+1:00) | error decoding 'receivers': error reading configuration for "otlp": decoding failed due to the following error(s): | datadog-agent
February 12, 2025 at 12:06 (UTC+1:00) | error decoding '': decoding failed due to the following error(s): | datadog-agent
February 12, 2025 at 12:06 (UTC+1:00) | 'protocols.grpc.max_recv_msg_size_mib' expected type 'int', got unconvertible type 'string', value: '16'

Agent Environment
Agent version: 7.62.2
Relevant environment variables supplied to the agent:

DD_OTLP_ENABLED=true
DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT=0.0.0.0:4317
DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT=0.0.0.0:4318
DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_MAX_RECV_MSG_SIZE_MIB=16

Describe what happened:
OTLP ingest pipeline errors on init because of failed decoding of ENV variable.

Describe what you expected:
Agent working as expected without error.

Steps to reproduce the issue:

  1. Start the DD agent with the provided environment variables in Agent Environment section.
  2. Observe in logs the error of Error running the OTLP ingest as shows before.
  3. Observe that ingest pipeline did not start (pushing anything to that port fails).

Additional environment details (Operating System, Cloud provider, etc):
OS: Linux/X86_64
Cloud Provider: AWS ECS Fargate

@github-actions github-actions bot added the team/opentelemetry OpenTelemetry team label Feb 12, 2025
@liustanley
Copy link
Contributor

Thanks for reporting this, I was able to reproduce the issue and opened an internal ticket to investigate and fix this behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants