Skip to content

chore: drop no-op _framerate=15 patch on avatar video track#31

Merged
aliev merged 1 commit into
mainfrom
chore/remove-noop-avatar-fps-patch
May 15, 2026
Merged

chore: drop no-op _framerate=15 patch on avatar video track#31
aliev merged 1 commit into
mainfrom
chore/remove-noop-avatar-fps-patch

Conversation

@aliev

@aliev aliev commented May 15, 2026

Copy link
Copy Markdown
Member

The avatar._sync.video_track._framerate = 15 line was reasoned to halve VP8 encode rate by slowing next_timestamp() pacing on aiortc's VideoStreamTrack. In practice it does nothing for output rate.

Agent._prepare_rtc() wraps our track in a VideoForwarder with fps=30 hardcoded:

forwarder = VideoForwarder(
    self._video_track,
    max_buffer=30,
    fps=30,  # drives consumer poll rate
    ...
)

The forwarder pulls from our track at 30 Hz. With _framerate=15 the underlying track simply returns the same frame twice in a row; the forwarder still emits 30 fps and the encoder still runs at 30 fps, just on duplicate input. CPU savings are effectively zero, and the absence of visible 15 fps lag in earlier load tests confirmed nothing about the output was actually capped.

AVSynchronizer.fps doesn't help either — it controls _pending release timing, not consumer poll rate, so the forwarder's 30 fps poll dominates.

What stays

tts=inworld.TTS(..., sample_rate=24000) is unchanged — that one is verified working (live memory drop measured after deploy when Inworld stopped emitting 16kHz that needed in-loop resampling to 24kHz).

Followup

If we want to actually cap the outbound encode rate, the lever is the VideoForwarder's fps argument, not the underlying track. That's deeper in framework territory and not worth doing for this demo.

The `avatar._sync.video_track._framerate = 15` line was reasoned to
halve VP8 encode rate by slowing `next_timestamp()` pacing on aiortc's
`VideoStreamTrack`. In practice it does nothing for output rate.

`Agent._prepare_rtc()` wraps our track in a `VideoForwarder` with
`fps=30` hardcoded:

    forwarder = VideoForwarder(
        self._video_track,
        max_buffer=30,
        fps=30,  # ← drives consumer poll rate
        ...
    )

The forwarder pulls from our track at 30 Hz. With `_framerate=15` the
underlying track simply returns the *same* frame twice in a row; the
forwarder still emits 30 fps and the encoder still runs at 30 fps,
just on duplicate input. CPU savings are effectively zero, and the
absence of visible 15 fps lag confirmed nothing about the output was
actually capped.

`AVSynchronizer.fps` doesn't help either — it controls `_pending`
release timing, not consumer poll rate, so the forwarder's 30 fps
poll dominates.

`sample_rate=24000` on `inworld.TTS(...)` stays — that one *is*
verified working (live memory drop measured after deploy).
@vercel

vercel Bot commented May 15, 2026

Copy link
Copy Markdown

@aliev must be a member of the GetStreamio team on Vercel to deploy.
- Click here to add @aliev to the team.
- If you initiated this build, request access.

Learn more about collaboration on Vercel and other options here.

@aliev aliev marked this pull request as ready for review May 15, 2026 10:17
@aliev aliev merged commit 3b71997 into main May 15, 2026
2 of 3 checks passed
@aliev aliev deleted the chore/remove-noop-avatar-fps-patch branch May 15, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant