Skip to content

use write data API in bidirectional streaming#3809

Merged
sbiscigl merged 1 commit intomainfrom
write-api-streaming
Apr 30, 2026
Merged

use write data API in bidirectional streaming#3809
sbiscigl merged 1 commit intomainfrom
write-api-streaming

Conversation

@sbiscigl
Copy link
Copy Markdown
Collaborator

@sbiscigl sbiscigl commented Apr 27, 2026

Issue #, if available:

#3257

Description of changes:

This changes the implementation of bidirectional streaming when using the CRT HTTP client to use a push-based API rather than a polling-based API. Previously, bidirectional streaming used a ConcurrentStreamBuf that the HTTP client polled in a hot loop waiting for data/ Now, when built with -DUSE_CRT_HTTP_CLIENT=ON, the SDK uses HttpWriteDataStreamBuf which pushes data directly to the HTTP/2 connection the moment it's written. This is fully backwards compatible. No code changes required. The same streaming code automatically benefits:

// HttpWriteDataStreamBuf is a std::streambuf that writes to an HTTP/2 connection
class HttpWriteDataStreamBuf : public std::streambuf {
  // When the buffer fills, push it to the wire immediately
  bool SendBuffer(bool endStream) {
    m_stream->WriteData(data, onComplete, endStream);  // CRT push API
  }
};

// AudioStream uses it as its underlying streambuf
class AudioStream : public EventEncoderStream {
  AudioStream(shared_ptr<HttpWriteDataStreamBuf> buf)
    : EventEncoderStream(buf) {}  // writes go directly to wire

  void WriteAudioEvent(const AudioEvent& event) {
    // encode + sign event, write to streambuf → WriteData → wire
    WriteEvent(encode(event));
  }
}

auto OnStreamReady = [](AudioStream& stream) {
  stream.WriteAudioEvent(audioEvent);  // now pushes directly to wire
  stream.Close();
};

client.StartStreamTranscriptionAsync(request, OnStreamReady, OnResponseCallback, nullptr);

This has been verified against the current transcribe example where CPU usage dropped from 100% to background noise levels. This applies to all bidirectional streaming operations across all services (Transcribe, Bedrock, QBusiness, Polly, ConnectHealth, etc.) and is only active when built with -DUSE_CRT_HTTP_CLIENT=ON. The legacy path (libcurl/winhttp) is unchanged.

Additionally this is an ongoing effort where the CRT HTTP client will eventually become the default client of the SDK. This fix will not be ported to libcurl or winhttp.

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@sbiscigl sbiscigl force-pushed the write-api-streaming branch from baa395f to f5c3bd7 Compare April 27, 2026 20:44
@sbiscigl sbiscigl marked this pull request as ready for review April 27, 2026 20:44
@sbiscigl sbiscigl force-pushed the write-api-streaming branch 3 times, most recently from 7ec51c5 to a657579 Compare April 28, 2026 15:27
Comment thread src/aws-cpp-sdk-core/source/utils/event/EventEncoderStream.cpp Outdated
@sbiscigl sbiscigl force-pushed the write-api-streaming branch 2 times, most recently from 122992e to 281d322 Compare April 29, 2026 15:43
Comment thread src/aws-cpp-sdk-core/source/utils/event/EventEncoderStream.cpp Outdated
@sbiscigl sbiscigl force-pushed the write-api-streaming branch from 281d322 to cfcf9e0 Compare April 29, 2026 20:13
@sbiscigl sbiscigl force-pushed the write-api-streaming branch 5 times, most recently from d5a5baf to c418a10 Compare April 30, 2026 18:29
@sbiscigl sbiscigl force-pushed the write-api-streaming branch from c418a10 to e7bd49a Compare April 30, 2026 20:00
@sbiscigl sbiscigl merged commit 37faa13 into main Apr 30, 2026
3 of 6 checks passed
@sbiscigl sbiscigl deleted the write-api-streaming branch April 30, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants