feat(trace_buffer): adopt libdatadog's TraceBuffer and SharedRuntime

[Aaalibaba42]

What?

Replace the in-tree AsyncTraceExporter (buffering, flushing, shutdown orchestration) with
libdatadog's TraceBuffer + SharedRuntime, which is the same implementation that was upstreamed
from this repo. The DatadogExporter is now a thin wrapper that owns a TraceBuffer and a
SharedRuntime instead of managing its own background thread, batch queue, and condvar
synchronisation.

Why?

The trace buffer logic was moved to libdatadog so it can be shared across language tracers. Keeping
a duplicate copy here means diverging bug fixes, duplicated tests, and extra maintenance. Adopting
the libdatadog version lets us delete ~860 lines of exporter/mod.rs and benefit from upstream
improvements (e.g. runtime-aware shutdown) for free.

How?

1. Adopt TraceBuffer + SharedRuntime — deleted exporter/mod.rs entirely; rewired
   DatadogExporter in span_exporter.rs to create a SharedRuntime, build a TraceExporter
   via TraceExporterBuilder, and wrap both in a TraceBuffer. The MapperExporter implements
   Export<SpanData> for the OTel-to-DD conversion.

2. Fix tokio-inside-tokio panic — the buffer worker runs inside SharedRuntime's tokio
   scheduler, so the old synchronous send_trace_chunks triggered a nested block_on. Switched
   to send_trace_chunks_async and moved the OTel-to-DD conversion inside the async block so
   borrows stay within the future. Also added a pre-shutdown flush_and_wait to ensure the stats
   worker starts before the runtime tears down.

3. Add Drop impl for DatadogExporter — callers that forget to call shutdown() now get a
   best-effort teardown with a 1s timeout. The impl is idempotent: a prior explicit shutdown makes
   the Drop a no-op. Includes two regression tests exercising drop-inside-tokio and
   drop-after-explicit-shutdown.

Additional Notes

• libdatadog deps currently point to the jwiriath/tracebuffer-sharedruntime-rusttracer branch;
  these need to be updated to main (or pinned versions) once the libdatadog PR lands.
• The test-utils feature gate exposes a wait_ready hook on MapperExporter so snapshot tests
  can wait for agent info before asserting on stats-derived metrics.

[datadog-prod-us1-5]

 

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

Check Pull Request CI Status | ensure-ci-success     

🔄 Retry job. This looks flaky and may succeed on retry.

Job failed due to some checks still running, not retrying anymore.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 167d226 | Docs | Datadog PR Page | Give us feedback!}