Repository navigation
feat(besreceiver): route Progress events to logs pipeline - #68
Merged
Merged
Conversation
chagui
force-pushed
the
chagui/besreceiver/issue-60-progress
branch
from
April 25, 2026 08:00
accd31d to
d903569
Compare
chagui
marked this pull request as ready for review
April 25, 2026 08:01
chagui
force-pushed
the
chagui/besreceiver/issue-60-progress
branch
from
April 25, 2026 08:14
d903569 to
b5af6af
Compare
Bazel Progress BEP events (streaming stdout/stderr) were previously
discarded. Operators couldn't see the output stream without parsing Bazel
logs out-of-band. Route them through the logs pipeline behind an opt-in
config so operators can correlate build output with traces.
- New progress.{enabled, max_chunk_size} config block; disabled by
default because payloads are large and may contain secrets echoed by
actions. max_chunk_size: 0 disables the per-record cap.
- One log record per non-empty stream: stdout -> INFO, stderr -> ERROR,
with bazel.progress.{stream,truncated,original_bytes} attributes and
the invocation's TraceID for correlation.
- Unknown invocation ids (Progress before BuildStarted) are silently
dropped, matching the pattern used by every other handler.
- Config validation rejects negative max_chunk_size.
Closes #60.
This was referenced Apr 25, 2026
chagui
force-pushed
the
chagui/besreceiver/issue-60-progress
branch
from
April 25, 2026 09:19
b5af6af to
22f904e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds opt-in consumption of Bazel
ProgressBEP events as OTel log records. Each non-empty stream produces one record atINFOseverity, stamped withbazel.progress.{stream,bytes,truncated,opaque_count}. When the body exceedsprogress.max_chunk_sizethe receiver clips at a UTF-8 rune boundary (never producing invalid encoding) andbazel.progress.truncated=true.max_chunk_size: 0disables the cap; config validation rejects negative values.Default is
progress.enabled: falseto avoid surprise log volume — Bazel builds with--build_event_publish_all_actionsproduce a lot of Progress. Operators opt in explicitly, and the bundled 64 KiB chunk size keeps the worst-case per-record payload bounded once they do.Review-driven design choices
INFOon both streams. Bazel routes normal-build chatter (Loading:,Analyzing:,Build completed successfully) through stderr — mapping stderr toERRORwould page on every green build. Operators who need elevation can stack atransformprocessordownstream.build_event_stream.proto:303-304: "stderr has been emitted before stdout if both are present"). Both records on a single Progress event share the sameopaque_countso consumers can pair them and reconstruct chunk order.truncateUTF8helper (also adopted byreconstructCommandLineto dedup the existing manual loop).bytesandtruncatedattributes. Both are stamped on every record so downstream queries can rely on attribute presence rather than guarding for absent keys. The previousoriginal_bytes-only-on-truncation contract is gone — replaced bybytes(always = original byte length).handleProgresschecksstate.flushed, matching the gate every other event handler uses; late Progress arriving afterBuildFinishedis dropped silently rather than appearing in a separate consumer batch.emitLog/emitProgressLogdeduplication. Both now share anewLogPayloadhelper for the resource/scope boilerplate, eliminating the duplicatedservice.name="bazel"literal and the duplicated record-construction logic.Memory bounding — explicit non-goal
This PR truncates the log record body, not the inbound gRPC payload. The full Progress message is still allocated by
proto.Unmarshalbefore the handler runs. Seedocs/adr/0002-large-payload-handling.mdand the concrete follow-ups #64/#65/#66 for the heap-bounding story.ACK back-pressure — known trade-off, follow-up filed
Routing Progress through the existing synchronous
ConsumeLogspath widens an existing back-pressure surface: a stalled logs pipeline now holds BES ACKs on the highest-volume BEP event class. The pattern pre-exists for every other handler, so fixing it correctly is project-wide. Filed as #72 (async/bounded logs path).Deferred to follow-ups
progress.stdout/progress.stderrtogglesCloses #60.