[Sampler.BottomFloor] Add fixed-memory bottom-k log sampler - #5108
Open
Yun-Ting wants to merge 2 commits into
Open
[Sampler.BottomFloor] Add fixed-memory bottom-k log sampler#5108Yun-Ting wants to merge 2 commits into
Yun-Ting wants to merge 2 commits into
Conversation
… quality Enroll the package in the verify-aot-compat job, correct the package description which still advertised span sampling after that scope was removed, and document how callsite identity affects sampling quality. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
infra
Pull request dashboard statusWaiting on the author · refreshed 2026-08-22 15:28 UTC Two things need attention:
Status above doesn't look right?
|
Member
|
If we were to add this, I think it needs more test coverage. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #
Design discussion issue #
Changes
Adds
OpenTelemetry.Sampler.BottomFloor, a fixed-memory, self-calibratingbottom-k sampler for log records.
Chatty services have no good options today: raising the log level loses the rare
error buried among a flood of routine records, fixed-rate head sampling loses it
too and reports nothing about what was dropped, and per-logger rate limiting
silently truncates whatever happens to be loudest. This component bounds export
volume while keeping rare callsites representable, and stamps every retained
record with the arithmetic needed to reconstruct the original counts.
How it works
Each arrival is assigned an exponential priority scaled by its callsite's
weight, and the reservoir retains the
k + 1smallest. When the window closes,the
kbest form the sample and the remaining entry becomes the inclusionthreshold. Per-callsite estimates then set the next window's weights, so
frequent callsites are pushed down and rare or newly seen ones keep a high
chance of inclusion. The only required setting is the per-window budget
k.Retained records carry
otel.logs.adjusted_count(the reciprocal of therecord's inclusion probability) and
otel.logs.cv2(an adequacy signal for theestimate), so downstream aggregation switches from
COUNT(*)toSUM(otel.logs.adjusted_count)and stays unbiased.What is included
BottomFloorSampler<TCallsite>— the algorithm, with no OpenTelemetrydependency, usable for non-log streams.
BottomFloorLogExporter— decorates aBaseExporter<LogRecord>and sampleseach export batch.
BottomFloorLogRecordProcessor— the ready-to-register form, combiningbatching and sampling. One export batch is one sampling window.
MaxLogsPerSpanPerWindow(default0), so aspan's logs stay retrievable together while a chatty span cannot dominate.
Notes for reviewers
Compression ratio under load is
Budget / maxExportBatchSize. A window nolarger than the budget keeps everything it holds, so the constructor rejects
that combination rather than silently no-op sampling. Quiet windows pass
through whole, which is the "bottom floor" behaviour.
Two dependencies on SDK internals, both isolated and both degrading safely:
LogRecord.Copy()for retention. Records are pooled and reclaimed as thebatch enumerator advances, so anything held past that visit must be copied.
If it cannot be bound, the exporter forwards every batch unsampled
rather than emitting recycled records.
BaseExporter<T>.ParentProviderforwarding, so a decorated exporter stillresolves its
Resource. UsesUnsafeAccessoron .NET 9+ and reflectionearlier; degrades to a no-op.
BaseExporter<T>exposes no supported seam for the second one: the setter isinternal, there is no virtual hook, and no constructor overload. The one
supported alternative — wrapping a
BaseProcessor<T>and subclassingCompositeProcessor<T>, which does forward to children — was rejected becauseit would give up the batching and flush timer inherited from
BatchExportProcessorand would not remove theLogRecord.Copy()dependencyanyway. Rationale is recorded in
DESIGN.md; an upstream API request isplanned.
README snippets are compiled and executed as tests (
ReadmeExampleTests),and a test asserts the README still matches the compiled source character for
character. Editing a snippet therefore requires editing both files.
Sampling accuracy is exercised end to end by
examples/bottom-floor, whichrecovers arrival counts to well under 1% at 10x compression.
Merge requirement checklist
CHANGELOG.mdfiles updated for non-trivial changes