Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove suggestion to process internal telemetry through collector #5749

Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 15 additions & 63 deletions content/en/docs/collector/internal-telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,21 @@ service:
port: 8888
```

Alternatively, the Collector can push its internal metrics via to an OTLP
backend via the following configuration:

```yaml
service:
telemetry:
metrics:
readers:
- periodic:
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
```

You can adjust the verbosity of the Collector metrics output by setting the
`level` field to one of the following values:

Expand Down Expand Up @@ -151,69 +166,6 @@ Note that the `tracer_provider` section there corresponds to `traces` here.
[kitchen-sink-config]:
https://github.com/open-telemetry/opentelemetry-configuration/blob/main/examples/kitchen-sink.yaml

### Self-monitoring

The Collector can be configured to push its own telemetry to an
[OTLP receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver)
and send the data through configured pipelines. In the following example, the
Collector is configured to push metrics, traces, and logs every 10s using OTLP
gRPC to `localhost:14317`:

```yaml
receivers:
otlp/internal:
protocols:
grpc:
endpoint: localhost:14317
exporters:
debug:
service:
pipelines:
metrics:
receivers: [otlp/internal]
exporters: [debug]
traces:
receivers: [otlp/internal]
exporters: [debug]
telemetry:
metrics:
readers:
- periodic:
interval: 10000
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
traces:
processors:
- batch:
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
logs:
processors:
- batch:
exporter:
otlp:
protocol: grpc/protobuf
endpoint: http://localhost:14317
```

{{% alert title="Caution" color="warning" %}}

When self-monitoring, the Collector collects its own telemetry and sends it to
the desired backend for analysis. This can be a risky practice. If the Collector
is underperforming, its self-monitoring capability could be impacted. As a
result, the self-monitored telemetry might not reach the backend in time for
critical analysis.

Moreover, sending internal telemetry through the Collector's own pipelines can
create a continuous loop of spans, metric points, or logs, putting undue strain
on the Collector's performance. This setup should not be used in production.

{{% /alert %}}

## Types of internal telemetry

The OpenTelemetry Collector aims to be a model of observable service by clearly
Expand Down
Loading