fix(otlp-http): limit response body size to prevent memory exhaustion by SATVIKsynopsis · Pull Request #3501 · open-telemetry/opentelemetry-rust

SATVIKsynopsis · 2026-05-07T13:03:39Z

Changes

I traced all HTTP response handling to export_http_once in opentelemetry-oltp/src/exporter/http/mod.rs and since all three signal exporters funnel through this one function, fixing it here covered everything. I also added a MAX_RESPONSE_BODY_BYTES (10 mb) cap on 2xx response bodies and stopped reading non-2xx bodies entirely. I also added a test with a mock client that returns an oversized body to verify the cap works.

Merge requirement checklist

CONTRIBUTING guidelines followed
Unit tests added/updated (if applicable)
Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
Changes in public API reviewed (if applicable)

linux-foundation-easycla · 2026-05-07T13:03:46Z

The committers listed above are authorized under a signed CLA.

✅ login: SATVIKsynopsis / name: SATVIKsynopsis (223f668, 3725278, 43b2fd2, 658b6d2, d426c90, d863d9d, ec69053, ef6bae3)

codecov · 2026-05-07T13:06:40Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.8%. Comparing base (5a07ce1) to head (ec69053).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #3501     +/-   ##
=======================================
- Coverage   84.1%   82.8%   -1.3%     
=======================================
  Files        126     130      +4     
  Lines      26720   27289    +569     
=======================================
+ Hits       22490   22622    +132     
- Misses      4230    4667    +437

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

cijothomas · 2026-05-07T14:13:50Z

 #[cfg(feature = "experimental-http-retry")]
 use crate::retry_classification::http::classify_http_error;

+///  This represents the maximum bytes read from an oltp hhtp response body.


Suggested change

/// This represents the maximum bytes read from an oltp hhtp response body.

/// This represents the maximum bytes read from an OTLP Http response body.

cijothomas · 2026-05-07T14:15:50Z


+///  This represents the maximum bytes read from an oltp hhtp response body.
+/// It guards against memory exhaustion from a malicious endpoint.
+const MAX_RESPONSE_BODY_BYTES: usize = 10 * 1024 * 1024; 


wondering what is a good limit to put here! 10MB feel high (just out of my intuition!)
Any prior arts to follow?

I initially though to use a small figure like 5 mb but then thought for easier flow 10 mb could be used. any suggestions you have on this matter?

i am thinking 4-5 mb could be good to use.

.NET picked 4 MB. Let's pick that!
@lalitb do you know if there is a good max limit to pick?

4 MB would be nice to go with

lalitb · 2026-05-07T14:56:44Z

        // Return the response, consuming the body to save a copy
-        Ok(response.into_body())
+        let body = response.into_body();
+        if body.len() > MAX_RESPONSE_BODY_BYTES {


This fix does not prevent memory exhaustion yet. By the time this code checks body.len(), the HTTP client has already read the full response into Response<Bytes>. So a malicious endpoint can still force the large allocation; this only skips parsing after the memory is already used.

A better fix is to apply the limit while reading the response body in the HTTP client implementations, like the reqwest and hyper clients in opentelemetry-http. They should read the body in chunks and stop once it crosses the max size. Then OtlpHttpClient can safely consume the already-limited Bytes.

Thanks. That makes sense. I will fix it according to the suggestion

SATVIKsynopsis · 2026-05-07T20:35:34Z

The current fix checks the size after the full body is already buffered. To truly prevent the allocation, the body needs to be read in chunks and aborted early as suggested earlier as well. I have an approach using bytes_stream() for reqwest async, Read::take() for reqwest blocking and body.frame() for hyper. should i go ahead with this or is there a preferred streaming pattern in this codebase i should follow?

lalitb · 2026-05-08T20:56:13Z

The current fix checks the size after the full body is already buffered. To truly prevent the allocation, the body needs to be read in chunks and aborted early as suggested earlier as well. I have an approach using bytes_stream() for reqwest async, Read::take() for reqwest blocking and body.frame() for hyper. should i go ahead with this or is there a preferred streaming pattern in this codebase i should follow?

Yes, that approach sounds right. The key requirement is that the limit is enforced while the body is being read, before it becomes Response<Bytes>. Using bytes_stream() for reqwest async, Read::take() for reqwest blocking, and body.frame() for hyper sounds like the right direction here. Please make sure each path aborts as soon as the accumulated bytes exceed MAX_RESPONSE_BODY_BYTES, so OtlpHttpClient only receives already-bounded Bytes.

Can we try to accommodate this in next release - #3508 - @cijothomas if it can be fixed soon

SATVIKsynopsis · 2026-05-09T10:06:14Z

Updated the fix to use chunked reading in all three HTTP client implementations as in reqwest async, reqwest blocking and hyper so the limit is enforced while the body is being read before it becomes Response<Bytes>. adding unit tests for the body limit in opentelemetry-http directly would require a mock HTTP server. happy to add one if you can suggest a preferred test crate for this codebase.

lalitb · 2026-05-11T05:02:52Z

+            let mut body_bytes = bytes::BytesMut::new();
+            let mut body = response.into_body();
+            while let Some(frame) = body.frame().await {
+                if let Ok(frame) = frame {


I think we should not ignore the Err case here. Before this change, collect().await? would return the body read error to the caller. With if let Ok(frame) = frame, a mid-body read failure can be silently skipped and we may return a truncated body as if it succeeded.

lalitb · 2026-05-11T05:06:28Z

            let headers = std::mem::take(response.headers_mut());
+            let mut body_bytes = bytes::BytesMut::new();
+            while let Some(chunk) = response.chunk().await? {
+                body_bytes.extend_from_slice(&chunk);


nit - can we do check before extending -

body_bytes.len() + chunk.len() > MAX_RESPONSE_BODY_BYTES

SATVIKsynopsis · 2026-05-11T06:35:43Z

I have added the Err case in hyper client and resolved the nitpick.

SATVIKsynopsis · 2026-05-16T10:48:15Z

Hi, just checking in. let me know if any further changes are needed or if there's anything blocking the review. Happy to address any feedback.

SATVIKsynopsis added 2 commits May 7, 2026 18:04

oltp http change

43b2fd2

samll fix

223f668

SATVIKsynopsis requested a review from a team as a code owner May 7, 2026 13:03

SATVIKsynopsis changed the title ~~OLTP HTTP tracing for non-2xx bodies~~ fix(otlp-http): limit response body size to prevent memory exhaustion May 7, 2026

test for within limit

3725278

cijothomas reviewed May 7, 2026

View reviewed changes

SATVIKsynopsis added 2 commits May 7, 2026 19:47

name change in comment

658b6d2

4 mb constant size and linting fix

ef6bae3

lalitb reviewed May 7, 2026

View reviewed changes

chunks reading for max response added

d426c90

lint fix

d863d9d

lalitb reviewed May 11, 2026

View reviewed changes

err case added and nitpick solved

ec69053

	/// This represents the maximum bytes read from an oltp hhtp response body.
	/// This represents the maximum bytes read from an OTLP Http response body.

Conversation

SATVIKsynopsis commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Merge requirement checklist

Uh oh!

linux-foundation-easycla Bot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lalitb May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SATVIKsynopsis commented May 7, 2026

Uh oh!

lalitb commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SATVIKsynopsis commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SATVIKsynopsis commented May 11, 2026

Uh oh!

SATVIKsynopsis commented May 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SATVIKsynopsis commented May 7, 2026 •

edited

Loading

linux-foundation-easycla Bot commented May 7, 2026 •

edited

Loading

codecov Bot commented May 7, 2026 •

edited

Loading

lalitb May 7, 2026 •

edited

Loading

lalitb commented May 8, 2026 •

edited

Loading

SATVIKsynopsis commented May 9, 2026 •

edited

Loading