Skip to content

sender: make capture-time RTP rewrite opt-in - #174

Open
hanguyen-nuro wants to merge 1 commit into
mainfrom
nuro/capture-ts-opt-in
Open

sender: make capture-time RTP rewrite opt-in#174
hanguyen-nuro wants to merge 1 commit into
mainfrom
nuro/capture-ts-opt-in

Conversation

@hanguyen-nuro

Copy link
Copy Markdown
Contributor

Problem

captureTimestampInterceptor overwrites every frame's RTP timestamp with the supplied capture time (captureUs*9/100). That is unsafe for any track whose sending is gated outside the RTP stack: while such a track is idle it emits no packets, but the capture clock keeps advancing, so when it starts sending it enters mid-session with a base derived from wall time and receivers never render it. Tracks that send continuously from session start are unaffected, which is why this went unnoticed.

Reproduction

14 outbound camera tracks, where only the first 5 are allocated non-zero bitrate at startup and the remaining 9 are switched on later by an operator request. Every late-started track stayed black despite being funded, encoding and sending; the 5 continuous ones were fine throughout.

Isolated by building the same consumer against a matrix of variants (one variable at a time, each run validated by logging the funding transition and the per-track encode counters):

variant late-started tracks render?
as-is no
FrameBuffer capacity 2 instead of 1 no
previous release (no rewrite) yes
rewrite disabled, everything else identical yes
rewrite enabled but all tracks funded from t=0 yes

So the rewrite is the trigger, independent of the frame-buffer change.

Change

Gate the overwrite behind sender.CaptureTimestampRewrite(), default off. Capture times passed to SetCaptureTSUs are still recorded, so callers that use them only for telemetry are unaffected.

Note there is no way to both preserve the packetizer's timeline and carry an absolute capture instant in the same field. Callers whose tracks may start or stop should carry capture time out-of-band (e.g. a data channel keyed by the RTP timestamp), which also survives an SFU that strips header extensions on egress.

Tests

  • EncodesCaptureTime now opts in explicitly (behavior change is intentional).
  • Added DisabledByDefault — packetizer timestamp preserved, capture time still recorded.
  • Added RewriteToggle — the gate is honored per frame.

Full sender suite passes; gofmt and go vet clean.

I have not established the receiver-side reason a wall-clock-derived base is rejected for a mid-session start, so this PR gates the hazard rather than claiming to explain it.

captureTimestampInterceptor overwrote every frame's RTP timestamp with the
supplied capture time (captureUs*9/100). That is unsafe for any track whose
sending is gated outside the RTP stack: while such a track is idle it emits no
packets, but the capture clock keeps advancing, so when it starts sending it
enters mid-session with a base derived from wall time and receivers never
render it. Tracks that send continuously from session start are unaffected,
which is why this went unnoticed.

Observed with 14 outbound camera tracks where only the first few are allocated
non-zero bitrate at startup and the rest are switched on later by the operator:
every late-started track stayed black while funded, encoding and sending.
Disabling the rewrite restores all of them, with no other change.

Gate the overwrite behind sender.CaptureTimestampRewrite(), default off.
Capture times passed to SetCaptureTSUs are still recorded, so callers using
them only for telemetry keep working.

Note there is no way to both preserve the packetizer's timeline and carry an
absolute capture instant in the same field; callers whose tracks may start or
stop should carry capture time out-of-band (e.g. a data channel keyed by the
RTP timestamp), which also survives an SFU that strips header extensions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant