Skip to content

feat(besreceiver): emit bazel.fetch span per Fetch BEP event - #70

Merged
chagui merged 1 commit into
mainfrom
chagui/besreceiver/issue-61-fetch
Apr 25, 2026
Merged

chagui merged 1 commit into
mainfrom
chagui/besreceiver/issue-61-fetch

Conversation

@chagui

@chagui chagui commented Apr 24, 2026 •

Copy link
Copy Markdown
Owner

Makes external dependency downloads (http_archive, http_file, module resolution, OCI, file:// mirrors) visible in the trace flamegraph as bazel.fetch spans parented to the root bazel.build. For repos with heavy external surface area, fetch latency is often the dominant cost of clean builds and was previously invisible.

Review-driven design choices

  • URL redaction by default. Userinfo (user:token@host) is unconditionally stripped — fetch URLs frequently carry credentials inline. The query string is also stripped by default because pre-signed S3/GCS URLs put credentials in X-Amz-* parameters; operators who need them (debugging custom mirrors, version-pin tracking) opt in via pii.include_fetch_query_string: true. Redaction lives in a new redactFetchURL helper and is applied to both the span attribute and the log body. Non-RFC-3986 references (oci://repo/image@sha256:…, file:// mirrors) pass through unmodified rather than being dropped.
  • Filter check hoisted above log emission. handleFetch now runs state.filter.LevelForTarget("").allowFetch() once, at the top of the handler, before any log or span work. Drop / build_only operators no longer leak the URL into logs while suppressing the span. Per-rule patterns don't apply to fetches (no owning target, so the filter falls through to default_level); this is documented explicitly in docs/attributes.md.
  • Zero-width span timing. The Fetch proto carries no timestamps; the original implementation used start = invocation createdAt, end = time.Now(), which painted every fetch as a build-stream-spanning bar. The span is now zero-width at event-processing time — correctly indicating "this fetch happened" without fabricating duration. Threading OrderedBuildEvent.Event.EventTime for richer timing is a follow-up if real per-event timestamps become useful.
  • SpanID disambiguation. SpanIDs now mix the URL, the FetchId.Downloader enum, and a per-invocation sequence counter (state.fetchSeq). Repeated fetches of the same URL — transient retries, multi-arch toolchain pulls, repository_cache evictions — produce distinct SpanIDs that downstream backends do not collapse.
  • bazel.fetch.downloader attribute. Surfaces the FetchId.Downloader enum (HTTP, GRPC, UNKNOWN) so retries via different transports are visible in queries.
  • fetch failed: <url> status message. The redacted URL is included in the failure status so the trace identifies which fetch broke without forcing operators to cross-reference span attrs.

Tests added

7 new on top of the existing 6: userinfo redaction (span + log), query-string default-strip, query-string opt-in, non-standard URL pass-through, downloader attribute, same-URL distinct SpanIDs, instant span timing, late post-flush emission, filter-suppresses-log-too.

Deferred to follow-up

  • OTel semconv url.full alignment (dual-emit vs rename) — needs schema discussion.
  • Threading OrderedBuildEvent.Event.EventTime through eventMsg for a real per-event timestamp.
  • Replacing nolint:gocyclo on eventTypeName with a map[reflect.Type]string dispatch — applies to every dispatch site, scoped out of this PR.

Closes #61.

@chagui chagui mentioned this pull request Apr 24, 2026
1 task
@chagui
chagui marked this pull request as ready for review April 24, 2026 17:30
@chagui
chagui force-pushed the chagui/besreceiver/issue-61-fetch branch 3 times, most recently from 179d3ca to 2777a07 Compare April 25, 2026 12:41
Fetch events carry external-resource download outcome (success +
URL), but were previously dropped. On clean builds dominated by
http_archive / module downloads, this hides the first-order cause of
slow builds. Each Fetch event now becomes a bazel.fetch span parented
to the root bazel.build span with bazel.fetch.url and
bazel.fetch.success attributes; failures set span status Error with
message "fetch failed".

The Fetch proto carries no timing, so the span spans invocation
createdAt → now() at event-processing time — bounds the fetch on the
flamegraph without claiming a precise duration.

Filter behaviour mirrors per-target spans: drop / build_only
default_level suppresses bazel.fetch via a new allowFetch() level
helper. Empty URL or pre-BuildStarted arrival is silently dropped.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@chagui
chagui force-pushed the chagui/besreceiver/issue-61-fetch branch from 2777a07 to f443647 Compare April 25, 2026 13:03
@chagui
chagui merged commit e9968d3 into main Apr 25, 2026
5 checks passed
@chagui
chagui deleted the chagui/besreceiver/issue-61-fetch branch April 25, 2026 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consume Fetch BEP events as bazel.fetch spans

1 participant