don't propagate baggage when integration is disabled #6692

lucaspimentel · 2025-02-20T19:22:16Z

Summary of changes

When implementing baggage in #6157., we changed the logic in HttpMessageHandlerCommon so that it would propagate baggage even if the http client integration was disabled. This changes this behavior so that headers are only injected if the integration is enabled.

Bonus change: cleaned up the integration tests that were modified in this PR

added StringUtil.GetAllHeaders() so we only run the Regex once on the stdout instead of a separate regex for each header
use FluentAssertions
use modern C# constructs like var and using statements (I recommend ignoring whitespace when viewing the diff)

Reason for change

This behavior is more correct. Disabling the integration should mean the integration doesn't change any app behavior at all, including injecting http headers. There are separate settings to disable trace and baggage propagators, if needed.

Implementation details

Move the Inject() call into the if(enabled) block, where it used to be before #6157.

Test coverage

Added assertions to HttpMessageHandlerTests.TracingDisabled_DoesNotSubmitsTraces() to verify that none of the known propagation headers are injected when integration is disabled (not just baggage).

andrewlock · 2025-02-20T19:57:23Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6692) - mean (69ms)  : 66, 72
     .   : milestone, 69,
    master - mean (78ms)  : 73, 83
     .   : milestone, 78,

    section CallTarget+Inlining+NGEN
    This PR (6692) - mean (1,003ms)  : 980, 1025
     .   : milestone, 1003,
    master - mean (1,105ms)  : 1047, 1164
     .   : milestone, 1105,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6692) - mean (102ms)  : 100, 104
     .   : milestone, 102,
    master - mean (114ms)  : 107, 121
     .   : milestone, 114,

    section CallTarget+Inlining+NGEN
    This PR (6692) - mean (674ms)  : 654, 693
     .   : milestone, 674,
    master - mean (715ms)  : 678, 751
     .   : milestone, 715,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6692) - mean (89ms)  : 87, 90
     .   : milestone, 89,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6692) - mean (629ms)  : 608, 650
     .   : milestone, 629,
    master - mean (635ms)  : 617, 653
     .   : milestone, 635,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6692) - mean (191ms)  : 187, 194
     .   : milestone, 191,
    master - mean (191ms)  : 187, 196
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6692) - mean (1,106ms)  : 1079, 1133
     .   : milestone, 1106,
    master - mean (1,108ms)  : 1080, 1137
     .   : milestone, 1108,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6692) - mean (271ms)  : 267, 276
     .   : milestone, 271,
    master - mean (271ms)  : 267, 275
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (6692) - mean (863ms)  : 832, 894
     .   : milestone, 863,
    master - mean (866ms)  : 835, 898
     .   : milestone, 866,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6692) - mean (262ms)  : 258, 266
     .   : milestone, 262,
    master - mean (262ms)  : 259, 266
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (6692) - mean (845ms)  : 811, 879
     .   : milestone, 845,
    master - mean (847ms)  : 815, 879
     .   : milestone, 847,

andrewlock

LGTM

andrewlock · 2025-02-22T00:05:18Z

Benchmarks Report for tracer 🐌

Benchmarks for #6692 compared to master:

1 benchmarks are faster, with geometric mean 1.154
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.33μs	49.1ns	483ns	0.0131	0.00437	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.6μs	60.4ns	456ns	0.0207	0.0103	0	5.81 KB
master	`StartStopWithChild`	net472	16μs	48.8ns	189ns	1.06	0.341	0.103	6.2 KB
#6692	`StartStopWithChild`	net6.0	8.08μs	46.4ns	353ns	0.0162	0.00809	0	5.61 KB
#6692	`StartStopWithChild`	netcoreapp3.1	10.7μs	61.6ns	473ns	0.0159	0.00529	0	5.8 KB
#6692	`StartStopWithChild`	net472	16.2μs	74.8ns	290ns	1.04	0.302	0.103	6.2 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	491μs	408ns	1.53μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	648μs	665ns	2.57μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	846μs	872ns	3.38μs	0.422	3.3 KB
#6692	`WriteAndFlushEnrichedTraces`	net6.0	485μs	540ns	2.02μs	0	2.7 KB
#6692	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	656μs	661ns	2.56μs	0	2.7 KB
#6692	`WriteAndFlushEnrichedTraces`	net472	846μs	538ns	2.09μs	0.422	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	128μs	470ns	1.82μs	0.197	14.47 KB
master	`SendRequest`	netcoreapp3.1	147μs	472ns	1.83μs	0.22	17.28 KB
master	`SendRequest`	net472	0.00265ns	0.00115ns	0.00444ns	0	0 b
#6692	`SendRequest`	net6.0	132μs	588ns	2.28μs	0.199	14.47 KB
#6692	`SendRequest`	netcoreapp3.1	148μs	171ns	663ns	0.149	17.27 KB
#6692	`SendRequest`	net472	0.00543ns	0.00171ns	0.00664ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6692

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	41.56 KB	41.89 KB	338 B	0.81%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	563μs	3.06μs	16.5μs	0.571	0	0	41.51 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	647μs	3.67μs	24.6μs	0.319	0	0	41.56 KB
master	`WriteAndFlushEnrichedTraces`	net472	850μs	4.01μs	16.1μs	8.39	2.52	0.419	53.33 KB
#6692	`WriteAndFlushEnrichedTraces`	net6.0	562μs	2.97μs	16μs	0.584	0	0	41.51 KB
#6692	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	670μs	3.79μs	26.5μs	0.355	0	0	41.89 KB
#6692	`WriteAndFlushEnrichedTraces`	net472	843μs	2.69μs	10.1μs	8.69	2.48	0.414	53.26 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.27μs	1.92ns	7.44ns	0.0139	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.79μs	4.01ns	15ns	0.0136	0	1.02 KB
master	`ExecuteNonQuery`	net472	2μs	1.74ns	6.52ns	0.156	0.000999	987 B
#6692	`ExecuteNonQuery`	net6.0	1.35μs	2.31ns	8.94ns	0.0141	0	1.02 KB
#6692	`ExecuteNonQuery`	netcoreapp3.1	1.73μs	2.36ns	8.5ns	0.0138	0	1.02 KB
#6692	`ExecuteNonQuery`	net472	2.05μs	2.57ns	9.95ns	0.156	0.00102	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.32μs	1.08ns	4.16ns	0.0139	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.56μs	0.918ns	3.55ns	0.0133	976 B
master	`CallElasticsearch`	net472	2.62μs	2.18ns	8.45ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.26μs	0.695ns	2.6ns	0.0133	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.62μs	0.96ns	3.59ns	0.0137	1.02 KB
master	`CallElasticsearchAsync`	net472	2.49μs	1.35ns	5.07ns	0.167	1.05 KB
#6692	`CallElasticsearch`	net6.0	1.19μs	0.533ns	2ns	0.0137	976 B
#6692	`CallElasticsearch`	netcoreapp3.1	1.56μs	0.455ns	1.7ns	0.0134	976 B
#6692	`CallElasticsearch`	net472	2.52μs	1.17ns	4.51ns	0.158	995 B
#6692	`CallElasticsearchAsync`	net6.0	1.27μs	0.605ns	2.26ns	0.0135	952 B
#6692	`CallElasticsearchAsync`	netcoreapp3.1	1.6μs	0.681ns	2.55ns	0.0134	1.02 KB
#6692	`CallElasticsearchAsync`	net472	2.59μs	1.53ns	5.53ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.32μs	1.07ns	3.84ns	0.0134	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.61μs	0.544ns	2.03ns	0.0128	952 B
master	`ExecuteAsync`	net472	1.85μs	0.519ns	1.8ns	0.145	915 B
#6692	`ExecuteAsync`	net6.0	1.2μs	0.413ns	1.6ns	0.0132	952 B
#6692	`ExecuteAsync`	netcoreapp3.1	1.63μs	1.14ns	4.27ns	0.0123	952 B
#6692	`ExecuteAsync`	net472	1.89μs	0.656ns	2.54ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.41μs	2.45ns	9.18ns	0.0331	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.36μs	4.44ns	17.2ns	0.0375	2.85 KB
master	`SendAsync`	net472	7.49μs	2.48ns	9.6ns	0.494	3.12 KB
#6692	`SendAsync`	net6.0	4.24μs	1.21ns	4.68ns	0.0319	2.31 KB
#6692	`SendAsync`	netcoreapp3.1	5.23μs	2.97ns	11.5ns	0.0393	2.85 KB
#6692	`SendAsync`	net472	7.49μs	2.36ns	8.85ns	0.494	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.54μs	0.546ns	2.04ns	0.023	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.13μs	1.45ns	5.43ns	0.0225	1.64 KB
master	`EnrichedLog`	net472	2.47μs	2.63ns	9.85ns	0.25	1.57 KB
#6692	`EnrichedLog`	net6.0	1.53μs	0.901ns	3.49ns	0.023	1.64 KB
#6692	`EnrichedLog`	netcoreapp3.1	2.1μs	0.781ns	2.92ns	0.0223	1.64 KB
#6692	`EnrichedLog`	net472	2.58μs	1.44ns	5.39ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	114μs	352ns	1.27μs	0.0568	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	119μs	225ns	870ns	0.0601	0	4.28 KB
master	`EnrichedLog`	net472	149μs	232ns	898ns	0.668	0.223	4.46 KB
#6692	`EnrichedLog`	net6.0	111μs	122ns	439ns	0.0553	0	4.28 KB
#6692	`EnrichedLog`	netcoreapp3.1	119μs	164ns	634ns	0.0601	0	4.28 KB
#6692	`EnrichedLog`	net472	149μs	168ns	651ns	0.666	0.222	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.19μs	0.641ns	2.4ns	0.0303	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.09μs	1.09ns	4.1ns	0.0285	2.2 KB
master	`EnrichedLog`	net472	4.84μs	2.24ns	8.67ns	0.319	2.02 KB
#6692	`EnrichedLog`	net6.0	3.14μs	0.936ns	3.63ns	0.0315	2.2 KB
#6692	`EnrichedLog`	netcoreapp3.1	4.19μs	0.885ns	3.43ns	0.0293	2.2 KB
#6692	`EnrichedLog`	net472	4.92μs	1.55ns	6.02ns	0.321	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.39μs	0.615ns	2.3ns	0.0159	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.88μs	1.12ns	4.35ns	0.015	1.14 KB
master	`SendReceive`	net472	2.18μs	1.55ns	5.78ns	0.183	1.16 KB
#6692	`SendReceive`	net6.0	1.42μs	0.68ns	2.63ns	0.0156	1.14 KB
#6692	`SendReceive`	netcoreapp3.1	1.77μs	1.74ns	6.51ns	0.0151	1.14 KB
#6692	`SendReceive`	net472	2.04μs	1.25ns	4.69ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.85μs	1.04ns	3.76ns	0.0228	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.92μs	2.28ns	8.84ns	0.0218	1.65 KB
master	`EnrichedLog`	net472	4.26μs	2.22ns	8.32ns	0.323	2.04 KB
#6692	`EnrichedLog`	net6.0	2.7μs	0.512ns	1.85ns	0.0229	1.6 KB
#6692	`EnrichedLog`	netcoreapp3.1	3.79μs	2.06ns	7.99ns	0.0228	1.65 KB
#6692	`EnrichedLog`	net472	4.24μs	1.18ns	4.24ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6692

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0	1.154	625.82	542.09

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	410ns	0.696ns	2.69ns	0.00808	576 B
master	`StartFinishSpan`	netcoreapp3.1	546ns	1.64ns	6.36ns	0.00767	576 B
master	`StartFinishSpan`	net472	586ns	1.22ns	4.71ns	0.0917	578 B
master	`StartFinishScope`	net6.0	626ns	1.06ns	4.1ns	0.0099	696 B
master	`StartFinishScope`	netcoreapp3.1	708ns	1.46ns	5.65ns	0.00919	696 B
master	`StartFinishScope`	net472	822ns	1.84ns	7.14ns	0.104	658 B
#6692	`StartFinishSpan`	net6.0	455ns	0.302ns	1.17ns	0.00802	576 B
#6692	`StartFinishSpan`	netcoreapp3.1	571ns	0.444ns	1.72ns	0.00773	576 B
#6692	`StartFinishSpan`	net472	630ns	0.457ns	1.77ns	0.0915	578 B
#6692	`StartFinishScope`	net6.0	542ns	0.267ns	1ns	0.00986	696 B
#6692	`StartFinishScope`	netcoreapp3.1	693ns	0.364ns	1.41ns	0.00921	696 B
#6692	`StartFinishScope`	net472	863ns	1.22ns	4.74ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	587ns	0.866ns	3.36ns	0.00983	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	928ns	1.4ns	5.44ns	0.00902	696 B
master	`RunOnMethodBegin`	net472	998ns	1.56ns	6.05ns	0.104	658 B
#6692	`RunOnMethodBegin`	net6.0	642ns	0.783ns	3.03ns	0.00978	696 B
#6692	`RunOnMethodBegin`	netcoreapp3.1	925ns	0.886ns	3.43ns	0.00928	696 B
#6692	`RunOnMethodBegin`	net472	1.09μs	1.2ns	4.64ns	0.104	658 B

github-actions bot added the area:integrations label Feb 20, 2025

lucaspimentel added the area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) label Feb 20, 2025

lucaspimentel changed the title ~~don't propagate context when integration is disabled~~ don't propagate baggage when integration is disabled Feb 20, 2025

andrewlock approved these changes Feb 21, 2025

View reviewed changes

lucaspimentel marked this pull request as ready for review February 21, 2025 22:22

lucaspimentel requested review from a team as code owners February 21, 2025 22:22

lucaspimentel added 12 commits February 21, 2025 18:04

don't propagate context if http client span is not created

023f4ec

clean up

9be5935

clean up

0e0058f

convert to FluentAssertions

d6d7411

check for other headers

6ca5535

use modern C# syntax

d14c1c5

add StringUtil.GetAllHeaders()

59228a5

rewrite test to use StringUtil.GetAllHeaders()

0e9f467

use FluentAssertions and StringUtil.GetAllHeaders()

b22b148

use modern C# syntax

6dbece0

update comment

13da099

verify more headers are not injected

3383b41

lucaspimentel force-pushed the lpimentel/dont-propagate-when-integration-is-disabled branch from 43d2f6d to 3383b41 Compare February 21, 2025 23:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

don't propagate baggage when integration is disabled #6692

don't propagate baggage when integration is disabled #6692

lucaspimentel commented Feb 20, 2025 •

edited

Loading

andrewlock commented Feb 20, 2025 •

edited

Loading

andrewlock left a comment

andrewlock commented Feb 22, 2025

Raw results

Raw results

Raw results

More allocations ⚠️ in #6692

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6692

Raw results

Raw results

don't propagate baggage when integration is disabled #6692

Are you sure you want to change the base?

don't propagate baggage when integration is disabled #6692

Conversation

lucaspimentel commented Feb 20, 2025 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

andrewlock commented Feb 20, 2025 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock left a comment

Choose a reason for hiding this comment

andrewlock commented Feb 22, 2025

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

More allocations ⚠️ in #6692

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6692

Raw results

Raw results

lucaspimentel commented Feb 20, 2025 •

edited

Loading

andrewlock commented Feb 20, 2025 •

edited

Loading