feat(internal/log/telemetry): add SDK error-reporting API for Error Tracking#4997
feat(internal/log/telemetry): add SDK error-reporting API for Error Tracking#4997darccio wants to merge 12 commits into
Conversation
…racking Introduce a centralized, PII-safe pipeline that forwards SDK-originated errors from `internal/log.Error` to instrumentation telemetry (→ Error Tracking) without changing existing local log output. ## What changed **Auto-forward sink** (`internal/log/log.go`): - `SetErrorTelemetrySink(f)` installs a forwarding hook; called before the local aggregation lock so telemetry is notified on every non-rate-limited `Error` call. - `atomic.Pointer` ensures lock-free concurrent access. **Sink registration** (`internal/telemetry/log/forward.go`): - `init()` registers `forwardError` with `log.SetErrorTelemetrySink`. - Uses the raw format string as the constant telemetry message (never interpolated); type-switches args to attach only `error` values, each scrubbed through `NewSafeError`. - Always attaches a redacted stacktrace via `WithStacktrace()`. **Policy table** (`internal/telemetry/log/policy.go`): - Single source of truth for per-template actions: `report` (default), `downgrade` (→ warn), or `exclude`. - Pre-seeded with agent-connectivity and user-misconfiguration templates. **Explicit helpers** (`internal/telemetry/log/helpers.go`): - `ReportError(msg, err, opts...)` — for swallowed-error branches. - `ReportPanic(recovered, msg)` — for `recover()` sites. - Both enforce the constant-message contract and policy table. **`constantlogmsg` analyzer** (`internal/telemetry/log/analyzer/`): - `go/analysis` pass that rejects non-constant first arguments on `log.Error`, `log.Warn`, `ReportError`, and `ReportPanic`. - Standalone `cmd/main.go` runnable as `go vet -vettool`. - Makefile target `make lint/errlog`. **`MIGRATION-CONTRACT.md`**: canonical before/after for each of the five call-site patterns Prompt 2 (automated migration) will encounter. No existing log output is changed. No import cycle is introduced. All tests pass under `-race`.
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: e9d5965 | Docs | Datadog PR Page | Give us feedback! |
BenchmarksBenchmark execution time: 2026-07-10 15:51:12 Comparing candidate commit e9d5965 in PR branch Found 2 performance improvements and 5 performance regressions! Performance is the same for 318 metrics, 1 unstable metrics, 1 flaky benchmarks without significant changes.
|
|
We should remove these files and make the new analyzer includes the functionality https://github.com/DataDog/dd-trace-go/blob/main/rules/telemetry_rules.go and https://github.com/DataDog/dd-trace-go/blob/main/rules/logging_rules.go |
… telemetrylog.{Debug,Warn,Error}
Extends DefaultFuncs to check all three telemetry log functions (and their
Logger method counterparts) in addition to the existing ReportError/ReportPanic
helpers. A single FuncSpec per name covers both call styles because resolveFunc
maps both pkg-level and method calls to the same (pkgPath, name) key via the
type checker.
Adds a skipPkgs mechanism (DefaultSkipPkgs, New variadic param) so the analyzer
does not flag the internal/telemetry/log package's own delegation calls — e.g.
func Error(message string,...){ defaultLogger.Load().Error(message,...) }.
Adds TestAnalyzerSkip to verify that listed packages are silently ignored even
when they contain variable-message calls.
This brings constantlogmsg to feature parity with the telemetryLogConstantMessage
ruleguard rule in rules/telemetry_rules.go, making it possible to drop that rule
once this analyzer is wired into all CI paths.
- gofmt the policy table (was failing the golangci-lint gofmt/goimports checks) - wire the constantlogmsg analyzer into CI (static-checks.yml) and fix the violations it actually catches: a non-constant test call in log_test.go, and instrumentation.Logger's legitimate variable-message passthrough (added to DefaultSkipPkgs, same rationale as internal/telemetry/log itself) - seed the policy table with the two named exclusion categories that were missing but already firing at Error level: W3C/propagation parse failures (propagating_tags.go) and sampler-rule errors (samplingrules.go) - exclude the telemetry writer's own panic-encoding messages to avoid a self-referential report-about-the-reporter loop back into the pipeline that just failed - implement the missing Warn opt-in forwarding path (SetWarnTelemetrySink / forwardWarn / warnOptedIn) — Warn was documented as "off by default, opt-in per template" but had no hook at all - add benchmarks for the sink and helper hot paths (BenchmarkErrorWithTelemetrySink, BenchmarkForwardError(_Excluded), BenchmarkReportError, BenchmarkReportPanic) - fix MIGRATION-CONTRACT.md's canonical recover() pattern, which double-reported telemetry for a single panic (local log.Error auto-forwards the error arg, then ReportPanic reports it again) - update CONTRIBUTING.md and README.md for the new make lint/errlog target
Content has been copied over to Prompt 2 directly; no longer needed as a file in this repo.
…th go/analysis checks Kemal's review feedback on #4997: consolidate rules/telemetry_rules.go and rules/logging_rules.go (golangci-lint gocritic/ruleguard DSL rules) into the new constantlogmsg analyzer, so the SDK logging safety checks run as one standalone go vet tool (`make lint/errlog`) instead of split across two mechanisms. - telemetrysafety.go: replaces telemetryLogSmartSlogAny, telemetryLogStringErrorCall, and telemetryLogRawErrorUsage. A single LogValuer-implements check subsumes the first and third (a raw error never implements slog.LogValuer); the message distinguishes "wrap with NewSafeError" for error-typed values from the generic LogValuer guidance for everything else. - formatverbs.go: replaces internalLogFormatVerbs, stdLogFormatVerbs, and their err.Error()-suggestion counterparts. Covers internal/log's four functions (ruleguard's original scope) plus the standard "log" package in the same file allow-list as .golangci.yml's depguard config. - nolint.go: a narrow //nolint reader (same-line or line-above, matching golangci-lint's own convention). Needed because these checks move from a gocritic-based linter (nolint-aware) to a standalone go vet pass (which isn't) — without it, existing `//nolint:gocritic` exceptions this migration inherited would start failing. - cmd/main.go now runs all three analyzers via multichecker. Deliberately did not extend constantlogmsg's own scope to internal/log's Debug/Info (ruleguard checked all four): neither reaches telemetry, so the PII-to-Error-Tracking rationale doesn't apply, and doing so surfaced an unrelated passthrough test helper outside this PR's scope. Verified the new checks introduce no regressions: ran the combined tool against the whole repo before wiring in CI, found and fixed the 4 real "prefer err.Error()" hits it revealed (manual_api_ddtest.go, sql.go x3) which predated this PR, and confirmed telemetrysafety finds zero raw-error/non-LogValuer violations repo-wide (existing SafeError usage is already consistent everywhere). rules/telemetry_rules.go and rules/logging_rules.go removed, along with the gocritic ruleguard settings in .golangci.yml and the now-unused github.com/quasilyte/go-ruleguard/dsl direct dependency (go mod tidy).
- internal/telemetry/log/analyzer/nolint.go: use strings.SplitSeq instead of strings.Split in a range loop (modernize/stringsseq). - Add missing copyright headers to the 8 new analyzer testdata files. - internal/stacktrace/contribs_generated.go: drop github.com/quasilyte/go-ruleguard/dsl, matching what CI's generate check computed after the dependency was dropped from go.mod. - README.md, scripts/README.md: sync the lint/errlog description text with the Makefile (make docs output), matching what CI's check-docs computed. check-title was fixed separately by editing the PR title (comma in the scope broke the Conventional Commits regex).
The lint job's reviewdog/action-golangci-lint step installs its own Go toolchain and pins GOROOT to whatever it resolves (go1.26.4 in the failing run). The Setup Go and development tools step I'd added afterward restores a _tools/bin cache keyed by a *different* "stable" Go resolution (go1.26.5), which had been built by a separate job. GOROOT stayed pinned to 1.26.4 while the restored cache's tool binaries were built by 1.26.5, so `go run` failed compiling the standard library with: compile: version "go1.26.4" does not match go tool version "go1.26.5" Every other job in this file (copyright, check-docs, check-modules, etc.) avoids this by giving each check its own clean job with a single Setup Go step — nothing else touches Go beforehand. lint/errlog now follows the same pattern in its own `lint-errlog` job instead of sharing the `lint` job with the two golangci-lint steps.
This reverts commit d08c59f1c9ed81bef39fca4bd7ab55d2f4bee3ab. Keeping lint/errlog in the shared lint job per review feedback; the actual fix for the toolchain mismatch is dropping check-latest: true from the shared setup-go composite action (next commit).
The lint job's reviewdog/action-golangci-lint step installs its own Go toolchain first, resolving a loose "1.x" spec from whatever's already cached on the runner (go1.26.4). The shared setup-go composite action ran afterward requesting "stable" with check-latest: true, which bypasses the action's bundled version manifest and always queries upstream for the newest release — landing on go1.26.5, a patch newer than the manifest (and the runner's pre-cached Go) knew about. GOROOT ended up pinned to the first install while later steps picked up build artifacts expecting the second, so `go run` failed compiling the standard library with: compile: version "go1.26.4" does not match go tool version "go1.26.5" Dropping check-latest makes "stable" resolve through the same static manifest basis every other install on the runner already uses, so it lines up with whatever's already set up instead of racing ahead of it. This lets lint/errlog stay in the shared lint job instead of needing its own.
…einstein-959818 # Conflicts: # go.mod # go.sum # internal/stacktrace/contribs_generated.go
CI's setup-go with check-latest can leave GOROOT pointing at an older Go patch than the go binary on PATH, causing go/packages stdlib recompiles to fail with "compile: version X does not match go tool version Y". Clearing GOROOT lets go self-detect the correct root instead of trusting a stale env var.
Summary
internal/log.Errorso every non-rate-limited call is mirrored to instrumentation telemetry. The format string is the constant dedup key; variadicerrorargs are scrubbed viaNewSafeError; a redacted stack trace is always attached. No change to local log output.internal/telemetry/log/policy.go): single place to classify format templates asreport/downgrade(→ warn) /exclude. Pre-seeded with agent-connectivity and user-misconfiguration noise.internal/telemetry/log/helpers.go):ReportError(msg, err, opts...)for swallowed errors;ReportPanic(recovered, msg)forrecover()sites. Both enforce the constant-message contract.constantlogmsganalyzer (internal/telemetry/log/analyzer/):go vet-compatible pass that rejects non-constant message arguments. Standalone binary forgo vet -vettool;make lint/errlogtarget added.Test plan
go build ./...— no compile errorsgo vet ./internal/log/... ./internal/telemetry/log/...— cleango test ./internal/log/... ./internal/telemetry/log/... -count=1 -race— all green (17 new tests)make lint/errlog— no violations in existing codebaseinternal/logdoes not importinternal/telemetry/log)TestErrorLocalOutputUnchanged)Reviewer notes
init()inforward.go; it is active whenever any code importsinternal/telemetry/log. Existing uses oflog.Errorrequire no changes — the auto-forward handles them. Non-constant first arguments will be flagged by the new analyzer.policyTableinpolicy.gois intentionally sparse. Prompt 2 (migration) will populate it as call sites are classified.internalimport restriction) and aNew([]FuncSpec)constructor for test-scoped configuration.Related epic: APMLP-1503