Skip to content

[Enhancement]Extend SimulcastEncoderAdapter to support adaptive strming#75

Open
ipavlidakis wants to merge 1 commit intodevelopfrom
iliaspavlidakis/ios-1481-webrtcextend-simulcastencoderadapter-to-support-adaptive
Open

[Enhancement]Extend SimulcastEncoderAdapter to support adaptive strming#75
ipavlidakis wants to merge 1 commit intodevelopfrom
iliaspavlidakis/ios-1481-webrtcextend-simulcastencoderadapter-to-support-adaptive

Conversation

@ipavlidakis
Copy link
Contributor

Summary

This fixes a runtime SimulcastEncoderAdapter edge case where SEA has multiple
encoder contexts, but only one spatial layer is actually active.

In that state, SEA was returning aggregated simulcast EncoderInfo instead of
the active encoder's runtime adaptation hints. The result is that a
single-active-layer call can behave differently from true single-layer
publishing, especially for quality scaling and other adaptation decisions.

Problem

SimulcastEncoderAdapter::GetEncoderInfo() only had a passthrough behavior
when stream_contexts_.size() == 1.

In practice, SEA can still have multiple stream contexts after init while
runtime bitrate allocation pauses all but one layer.

When that happened, SEA continued to report aggregated simulcast info and
explicitly disabled scaling_settings.

That means downstream components do not see the active encoder's real runtime
capabilities and hints, including:

  • quality scaling thresholds
  • min_pixels_per_frame
  • is_qp_trusted
  • resolution_bitrate_limits
  • min_qp
  • supports_native_handle
  • preferred_pixel_formats

Root Cause

The bug is in SimulcastEncoderAdapter::GetEncoderInfo().

The old logic treated "single stream context" as equivalent to "single active
layer", but those are not the same thing at runtime.

Once SEA enters multi-encoder mode, the number of stream contexts may stay
greater than one even if only one layer is currently unpaused.

Fix

In multi-encoder mode, SEA still aggregates the simulcast-specific fields as it
did before.

If exactly one stream is unpaused, SEA now forwards the active encoder's
runtime-sensitive fields:

  • scaling_settings
  • supports_native_handle
  • has_trusted_rate_controller
  • is_hardware_accelerated
  • is_qp_trusted
  • resolution_bitrate_limits
  • min_qp
  • preferred_pixel_formats

SEA still preserves its aggregated simulcast fields:

  • implementation_name
  • fps_allocation
  • requested_resolution_alignment
  • apply_alignment_to_all_simulcast_layers

If more layers become active again later in the call, SEA goes back to the
existing aggregated behavior without needing re-init.

Why This Is Safe

This is a narrow behavior change.

It does not change:

  • init-time encoder selection
  • SEA bypass mode
  • the pre-init GetEncoderInfo() path
  • bitrate allocation / pause-unpause logic
  • true multi-layer simulcast aggregation

It only changes the runtime case where SEA is effectively behaving like
single-layer publishing but was still reporting multi-layer aggregated info.

Tests

Added regression coverage in simulcast_encoder_adapter_unittest.cc for:

  • forwarding active encoder info when only one spatial layer is unpaused
  • restoring aggregated SEA behavior when a second layer becomes active again

The tests verify forwarding of:

  • scaling thresholds
  • min_pixels_per_frame
  • native-handle support
  • QP trust
  • resolution bitrate limits
  • min_qp

The tests also verify that fps_allocation still preserves SEA's spatial-slot
layout.

Validation

Validated by compiling the two edited translation units with the generated
build commands:

  • media/engine/simulcast_encoder_adapter.cc
  • media/engine/simulcast_encoder_adapter_unittest.cc

I did not complete:

  • a full rtc_media_unittests run
  • the iOS H.264 runtime repro validation in this PR

Reviewer Notes

Please pay special attention to:

  • whether forwarding supports_native_handle in the single-unpaused case is
    the right runtime behavior
  • whether there are any consumers of aggregated is_hardware_accelerated or
    has_trusted_rate_controller that should remain aggregated even in the
    single-unpaused state
  • whether we want a follow-up integration test around VideoStreamEncoder
    re-reading EncoderInfo after runtime pause/unpause transitions

@ipavlidakis ipavlidakis self-assigned this Mar 13, 2026
@ipavlidakis ipavlidakis added the enhancement New feature or request label Mar 13, 2026
@ipavlidakis ipavlidakis changed the base branch from main to develop March 13, 2026 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant