Eliminate passive-health async frame from the default proxy pipeline by artl93 · Pull Request #3040 · dotnet/yarp

artl93 · 2026-07-13T10:32:08Z

Motivation

The parameterless MapReverseProxy() pipeline always passed through PassiveHealthCheckMiddleware.Invoke. Because that method awaited the forwarding pipeline, an asynchronously completed request allocated its state machine/continuation even when passive health was disabled (the default).

This removes that ubiquitous frame: 112 B/request in the complete routing/load-balancing pipeline measured below.

Change

The default pipeline records passive-health outcomes in ForwarderMiddleware, which already has an unavoidable async frame. Recording still happens:

after forwarding and after destination concurrency counters are decremented;
from the final reverse-proxy feature, so downstream cluster/destination reassignment is honored;
for normal success and non-success ForwarderError results, but not thrown exceptions, matching the previous post-await behavior.

The custom MapReverseProxy(configureApp) overload keeps public UsePassiveHealthChecks() middleware semantics and does not double-record.

BenchmarkDotNet

Exact binary comparison: baseline 49b23da6d8 versus implementation 4e8c0103f2; PR head 71d1ad10 adds tests only. Runtime SHA-256 guards fail each benchmark process if it loads the wrong product DLL.

Cross-platform BenchmarkDotNet 0.15.8 short apples-to-apples run on .NET 10.0.10 (1 launch, 1 warmup, 3 measurement iterations), with a reusable incomplete IValueTaskSource forwarder:

Scenario	Baseline B/op	Candidate B/op	Delta
Minimal; 8/64 destinations, round-robin/P2C; healthy filtering; passive health enabled	1736	1624	-112
100/1000 routes	1768	1656	-112
Session affinity miss	2016	1904	-112
Session affinity hit	2672	2560	-112
Synchronous control	1480	1480	0
Custom pipeline without passive middleware	1624	1624	0

The exact allocation table reproduced on Linux x64, Windows x64, and macOS arm64: workflow and raw artifacts. Product DLL SHA-256: baseline 0c813f18...bd9f4, candidate d1e6a57f...ac190.

BDN_JOB=Dry  dotnet run -c Release --no-build -- --filter '*' --noOverwrite
BDN_JOB=Short dotnet run -c Release --no-build -- --filter '*' --apples --noOverwrite

YARP is not configured in EgorBot's supported polling repositories; its current source names dotnet/runtime and EgorBot/Benchmarks, not dotnet/yarp (configuration). The cross-platform matrix above is the campaign-policy substitute, not an EgorBot result.

Validation

./.dotnet/dotnet test test/ReverseProxy.Tests/Yarp.ReverseProxy.Tests.csproj \
  --no-restore --filter-class '*ReverseProxyIEndpointRouteBuilderExtensionsTests' --no-progress
./build.sh -test

The focused full-pipeline theories pass 20/20 across net8/net9. The repository command passes the build and Application, ReverseProxy unit/functional, and Kubernetes suites on supported net8/net9 targets (0 warnings, 0 errors).

The new TestServer coverage exercises both MapReverseProxy overloads and proves exact-once post-forwarding recording for asynchronous success and normal ForwarderError, no recording for asynchronous throws, propagated request cancellation, or missing destinations, unchanged exceptions/status codes, and no forwarder invocation on the no-destination path. Existing focused tests cover disabled/integrated gating, counter cleanup, and cluster/destination reassignment.

A fresh exact-SHA real-Kestrel control used direct and proxy H1/H2, 32 connections, 5 s warmup + 12 s measurement, and a 100-byte response. All eight runs completed with zero errors, but machine conditions also moved the direct controls, so it is only a no-regression/allocation-direction check. Five earlier alternating paired rounds gave direct-adjusted allocation deltas of -108.6 +/- 102.7 B/request (H1) and -150.8 +/- 135.9 B/request (H2).

Risks / limits

The first experiment (731432ef7f) made an unsafe entry-time passive-health decision. The final implementation (4e8c0103f2) deliberately records after forwarding in the existing frame, preserving middleware ordering and reassignment semantics.
This optimizes only the built-in parameterless pipeline. Custom pipelines retain the public middleware behavior.
Full-pipeline fault coverage uses TestServer with a controlled IHttpForwarder; real Kestrel controls transport integration but do not inject every fault shape.
Allocation removal is deterministic; short-run timing and Kestrel working-set/latency data are noisy. No throughput, latency, or working-set improvement is claimed.

AI disclosure

GitHub Copilot assisted with profiling, implementation, tests, benchmark automation, and drafting. I reviewed the code, benchmark identity checks, results, and semantic coverage.

… when disabled PassiveHealthCheckMiddleware.Invoke was an `async` method that always awaited `_next`, so it heap-allocated an async state machine (~112 B) on every proxied request through the default MapReverseProxy pipeline -- even though passive health checks are disabled by default and there is no post-processing to do. Short-circuit to `return _next(context)` when passive health is not engaged for the request's cluster (read at entry, mirroring SessionAffinityMiddleware and LimitsMiddleware). The outcome recording still re-reads the cluster/destination after `_next`, preserving ReassignProxyRequest semantics for the documented "reassign first" pattern and for enabled clusters. Measured (Apple M-series arm64, .NET 9.0.2): - In-process pipeline A/B: -112 B/req on every default-pipeline shape (1/100/1000 routes, 8/64 destinations, round-robin/p2c, affinity hit/miss, healthy filter); passive-health-enabled path unchanged; synchronous path unchanged. - E2E (real Kestrel proxy -> backend): -112 B/req (HTTP/1.1) / -106 B/req (HTTP/2) with no throughput, latency, or working-set regression. Adds differential tests locking in reassignment/ordering semantics and a durable benchmark (perf-hunt/) that reproduces the A/B. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f1144d71-c639-4408-84cf-97c6a8f8355c

Fold default-pipeline passive health recording into ForwarderMiddleware's existing async frame instead of using an entry-time fast path. This retains final proxy feature and custom pipeline ordering semantics, including downstream transforms and request reassignment, while preserving the 112 B/request allocation reduction. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f1144d71-c639-4408-84cf-97c6a8f8355c

Cover the default pipeline's final recording point for successful and failed forwarding, thrown exceptions, disabled recording, and downstream cluster reassignment. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f1144d71-c639-4408-84cf-97c6a8f8355c

artl93 · 2026-07-13T11:10:01Z

Exact-SHA real-Kestrel raw results

All 40 H1/H2 direct/proxy measurements completed with errors=0. Revision order alternated by round. The body contains the statistical summary and caveats.

E2E command log SHA-256: 3019329eb0f991d1d54d9adc2e7436c6fdf7c6ee462dca07406966e4c5ad7980
E2E structured-results SHA-256: 8730381c3a3890f21717b317df6cb480141f7847c67421e0ec86373ea845ae67
Canonical E2E CSV SHA-256: 75686212ac6e097b3d28b36202d6c711ff57454f55f66508f12d31f196809e2b

All 40 structured RESULT lines

RESULT label=publication-r1-baseline-direct-h1 target=direct http=1 conns=32 dur=12.1s requests=237221 errors=0 rps=19618 p50us=898 p90us=999 p99us=18281 p999us=34460 maxus=61630 alloc_per_req_B=1591.9 ws_MB=314.4 gc0=1 gc1=1 gc2=1
RESULT label=publication-r1-final-direct-h1 target=direct http=1 conns=32 dur=12.0s requests=222454 errors=0 rps=18523 p50us=918 p90us=1091 p99us=18741 p999us=33074 maxus=72329 alloc_per_req_B=1606.9 ws_MB=318.2 gc0=1 gc1=1 gc2=1
RESULT label=publication-r1-baseline-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=121063 errors=0 rps=10079 p50us=1852 p90us=2010 p99us=31471 p999us=48764 maxus=66514 alloc_per_req_B=4575.1 ws_MB=345.8 gc0=2 gc1=1 gc2=1
RESULT label=publication-r1-final-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=125151 errors=0 rps=10395 p50us=1766 p90us=1978 p99us=31880 p999us=52243 maxus=71781 alloc_per_req_B=4448.7 ws_MB=344.3 gc0=2 gc1=1 gc2=1
RESULT label=publication-r1-baseline-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=601446 errors=0 rps=50026 p50us=320 p90us=511 p99us=10619 p999us=43950 maxus=85967 alloc_per_req_B=2081.6 ws_MB=326.0 gc0=6 gc1=2 gc2=1
RESULT label=publication-r1-final-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=628696 errors=0 rps=52350 p50us=300 p90us=463 p99us=10887 p999us=47185 maxus=69068 alloc_per_req_B=2078.1 ws_MB=326.5 gc0=6 gc1=2 gc2=1
RESULT label=publication-r1-baseline-proxy-h2 target=proxy http=2 conns=32 dur=12.1s requests=189221 errors=0 rps=15675 p50us=1083 p90us=1442 p99us=39540 p999us=65014 maxus=74768 alloc_per_req_B=6034.8 ws_MB=353.2 gc0=5 gc1=2 gc2=1
RESULT label=publication-r1-final-proxy-h2 target=proxy http=2 conns=32 dur=12.0s requests=169345 errors=0 rps=14101 p50us=1124 p90us=1551 p99us=43526 p999us=74675 maxus=109919 alloc_per_req_B=5970.3 ws_MB=352.8 gc0=5 gc1=2 gc2=1
RESULT label=publication-r2-final-direct-h1 target=direct http=1 conns=32 dur=12.1s requests=226858 errors=0 rps=18751 p50us=906 p90us=1014 p99us=19843 p999us=50613 maxus=90962 alloc_per_req_B=1602.2 ws_MB=314.7 gc0=1 gc1=1 gc2=1
RESULT label=publication-r2-baseline-direct-h1 target=direct http=1 conns=32 dur=12.0s requests=257461 errors=0 rps=21438 p50us=887 p90us=987 p99us=15821 p999us=31560 maxus=67859 alloc_per_req_B=1574.4 ws_MB=317.6 gc0=1 gc1=1 gc2=1
RESULT label=publication-r2-final-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=130996 errors=0 rps=10907 p50us=1782 p90us=1971 p99us=25729 p999us=50503 maxus=66622 alloc_per_req_B=4430.0 ws_MB=344.8 gc0=2 gc1=1 gc2=1
RESULT label=publication-r2-baseline-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=134353 errors=0 rps=11187 p50us=1751 p90us=1957 p99us=24871 p999us=45713 maxus=71332 alloc_per_req_B=4531.7 ws_MB=345.6 gc0=3 gc1=1 gc2=1
RESULT label=publication-r2-final-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=665991 errors=0 rps=55453 p50us=296 p90us=469 p99us=10068 p999us=35069 maxus=86133 alloc_per_req_B=2072.8 ws_MB=326.0 gc0=7 gc1=1 gc2=1
RESULT label=publication-r2-baseline-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=654232 errors=0 rps=54482 p50us=300 p90us=475 p99us=9320 p999us=41715 maxus=73764 alloc_per_req_B=2074.3 ws_MB=326.9 gc0=7 gc1=2 gc2=1
RESULT label=publication-r2-final-proxy-h2 target=proxy http=2 conns=32 dur=12.0s requests=205255 errors=0 rps=17058 p50us=1111 p90us=1473 p99us=30572 p999us=62102 maxus=93719 alloc_per_req_B=5904.0 ws_MB=352.7 gc0=6 gc1=2 gc2=1
RESULT label=publication-r2-baseline-proxy-h2 target=proxy http=2 conns=32 dur=12.0s requests=204318 errors=0 rps=16969 p50us=1099 p90us=1430 p99us=31634 p999us=65594 maxus=102583 alloc_per_req_B=6014.1 ws_MB=352.5 gc0=6 gc1=2 gc2=1
RESULT label=publication-r3-baseline-direct-h1 target=direct http=1 conns=32 dur=12.1s requests=205585 errors=0 rps=17056 p50us=907 p90us=1646 p99us=21787 p999us=48579 maxus=70494 alloc_per_req_B=1626.5 ws_MB=316.5 gc0=1 gc1=1 gc2=1
RESULT label=publication-r3-final-direct-h1 target=direct http=1 conns=32 dur=12.0s requests=177747 errors=0 rps=14788 p50us=919 p90us=3364 p99us=25826 p999us=50953 maxus=84934 alloc_per_req_B=1666.9 ws_MB=315.1 gc0=1 gc1=1 gc2=1
RESULT label=publication-r3-baseline-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=114986 errors=0 rps=9552 p50us=1838 p90us=2033 p99us=36930 p999us=65705 maxus=116296 alloc_per_req_B=4598.4 ws_MB=345.8 gc0=2 gc1=1 gc2=1
RESULT label=publication-r3-final-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=111652 errors=0 rps=9295 p50us=1861 p90us=2372 p99us=33512 p999us=51445 maxus=73405 alloc_per_req_B=4500.2 ws_MB=344.6 gc0=2 gc1=1 gc2=1
RESULT label=publication-r3-baseline-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=653762 errors=0 rps=54438 p50us=295 p90us=485 p99us=10448 p999us=44297 maxus=66776 alloc_per_req_B=2074.4 ws_MB=328.2 gc0=7 gc1=1 gc2=1
RESULT label=publication-r3-final-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=615747 errors=0 rps=51254 p50us=324 p90us=515 p99us=10165 p999us=47653 maxus=84143 alloc_per_req_B=2079.3 ws_MB=327.3 gc0=6 gc1=1 gc2=1
RESULT label=publication-r3-baseline-proxy-h2 target=proxy http=2 conns=32 dur=12.1s requests=188292 errors=0 rps=15618 p50us=1134 p90us=1452 p99us=39067 p999us=66184 maxus=83466 alloc_per_req_B=6037.0 ws_MB=352.1 gc0=5 gc1=2 gc2=1
RESULT label=publication-r3-final-proxy-h2 target=proxy http=2 conns=32 dur=12.0s requests=185435 errors=0 rps=15441 p50us=1139 p90us=1480 p99us=39317 p999us=66367 maxus=128688 alloc_per_req_B=5929.4 ws_MB=352.6 gc0=5 gc1=2 gc2=1
RESULT label=publication-r4-final-direct-h1 target=direct http=1 conns=32 dur=12.0s requests=253016 errors=0 rps=21067 p50us=867 p90us=973 p99us=17367 p999us=36062 maxus=65775 alloc_per_req_B=1578.0 ws_MB=315.5 gc0=1 gc1=1 gc2=1
RESULT label=publication-r4-baseline-direct-h1 target=direct http=1 conns=32 dur=12.0s requests=245565 errors=0 rps=20394 p50us=848 p90us=1006 p99us=19010 p999us=43676 maxus=104467 alloc_per_req_B=1584.4 ws_MB=317.4 gc0=1 gc1=1 gc2=1
RESULT label=publication-r4-final-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=89918 errors=0 rps=7479 p50us=1857 p90us=7154 p99us=42650 p999us=61872 maxus=84196 alloc_per_req_B=4627.4 ws_MB=342.2 gc0=2 gc1=1 gc2=1
RESULT label=publication-r4-baseline-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=115767 errors=0 rps=9640 p50us=1722 p90us=2952 p99us=37527 p999us=57375 maxus=69488 alloc_per_req_B=4597.0 ws_MB=345.3 gc0=2 gc1=1 gc2=1
RESULT label=publication-r4-final-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=649363 errors=0 rps=54067 p50us=217 p90us=411 p99us=11930 p999us=38189 maxus=58950 alloc_per_req_B=2074.0 ws_MB=327.6 gc0=7 gc1=1 gc2=1
RESULT label=publication-r4-baseline-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=573651 errors=0 rps=47763 p50us=222 p90us=441 p99us=13879 p999us=42714 maxus=92664 alloc_per_req_B=2084.8 ws_MB=327.0 gc0=6 gc1=1 gc2=1
RESULT label=publication-r4-final-proxy-h2 target=proxy http=2 conns=32 dur=12.0s requests=346718 errors=0 rps=28872 p50us=769 p90us=1290 p99us=7730 p999us=51535 maxus=101830 alloc_per_req_B=5791.2 ws_MB=353.6 gc0=10 gc1=2 gc2=1
RESULT label=publication-r4-baseline-proxy-h2 target=proxy http=2 conns=32 dur=12.0s requests=139195 errors=0 rps=11588 p50us=1001 p90us=1506 p99us=50713 p999us=67140 maxus=68313 alloc_per_req_B=6143.1 ws_MB=352.8 gc0=4 gc1=2 gc2=1
RESULT label=publication-r5-baseline-direct-h1 target=direct http=1 conns=32 dur=12.0s requests=132632 errors=0 rps=11034 p50us=922 p90us=8629 p99us=27130 p999us=51641 maxus=69304 alloc_per_req_B=1768.6 ws_MB=316.0 gc0=0 gc1=0 gc2=0
RESULT label=publication-r5-final-direct-h1 target=direct http=1 conns=32 dur=12.0s requests=129448 errors=0 rps=10778 p50us=932 p90us=9023 p99us=29270 p999us=51345 maxus=72028 alloc_per_req_B=1778.6 ws_MB=310.5 gc0=0 gc1=0 gc2=0
RESULT label=publication-r5-baseline-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=56502 errors=0 rps=4704 p50us=2346 p90us=21070 p99us=48503 p999us=70387 maxus=85033 alloc_per_req_B=5077.1 ws_MB=342.4 gc0=1 gc1=1 gc2=1
RESULT label=publication-r5-final-proxy-h1 target=proxy http=1 conns=32 dur=12.0s requests=59663 errors=0 rps=4957 p50us=2052 p90us=20575 p99us=51570 p999us=68114 maxus=78864 alloc_per_req_B=4916.8 ws_MB=339.3 gc0=1 gc1=1 gc2=1
RESULT label=publication-r5-baseline-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=225285 errors=0 rps=18711 p50us=250 p90us=2425 p99us=34166 p999us=52496 maxus=71189 alloc_per_req_B=2227.8 ws_MB=324.9 gc0=2 gc1=1 gc2=1
RESULT label=publication-r5-final-direct-h2 target=direct http=2 conns=32 dur=12.0s requests=239488 errors=0 rps=19930 p50us=238 p90us=2325 p99us=29947 p999us=57297 maxus=83473 alloc_per_req_B=2213.8 ws_MB=324.4 gc0=2 gc1=1 gc2=1
RESULT label=publication-r5-baseline-proxy-h2 target=proxy http=2 conns=32 dur=12.0s requests=150781 errors=0 rps=12555 p50us=604 p90us=2434 p99us=47368 p999us=69204 maxus=88954 alloc_per_req_B=6101.3 ws_MB=313.3 gc0=4 gc1=2 gc2=1
RESULT label=publication-r5-final-proxy-h2 target=proxy http=2 conns=32 dur=12.1s requests=169062 errors=0 rps=14000 p50us=638 p90us=1405 p99us=48116 p999us=67462 maxus=77456 alloc_per_req_B=5956.5 ws_MB=352.5 gc0=5 gc1=2 gc2=1

The complete local evidence manifest contains 122 hashed generated files. The benchmark harness archive is SHA-256 9d25d6a8a6a5975f15b19e943e56b45ffe3587c6831de93b42cc0902cb896fb3.

artl93 · 2026-07-20T01:41:00Z

Still draft — please treat this as an evidence/semantic review request, not a ready-for-merge signal. The body now links the valid Linux/Windows/macOS exact-binary allocation artifacts; ./build.sh -test is green; Kestrel timing remains explicitly non-claim. I especially want skeptical review of recording order, downstream cluster/destination reassignment, and exception/cancellation equivalence.

Exercise both MapReverseProxy overloads through the full TestServer pipeline for asynchronous success, forwarding errors, thrown exceptions, request cancellation, and missing destinations. Verify passive-health recording order and exact-once behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f1144d71-c639-4408-84cf-97c6a8f8355c

Art Leonard and others added 3 commits July 12, 2026 06:41

artl93 changed the title ~~WIP: DO NOT REVIEW Eliminate passive-health async frame from the default proxy pipeline~~ Eliminate passive-health async frame from the default proxy pipeline Jul 20, 2026

artl93 requested a review from MihaZupan July 20, 2026 01:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Eliminate passive-health async frame from the default proxy pipeline - #3040

Eliminate passive-health async frame from the default proxy pipeline#3040
artl93 wants to merge 4 commits into
dotnet:mainfrom
artl93:artl93-yarp-headline-perf-hunt

artl93 commented Jul 13, 2026 •

edited

Loading

Uh oh!

artl93 commented Jul 13, 2026

Uh oh!

artl93 commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

artl93 commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Change

BenchmarkDotNet

Validation

Risks / limits

AI disclosure

Uh oh!

artl93 commented Jul 13, 2026

Exact-SHA real-Kestrel raw results

Uh oh!

artl93 commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

artl93 commented Jul 13, 2026 •

edited

Loading