Describe the bug
AudioEnergyFilter gates the frames the Deepgram plugin sends to the websocket. Once the 1 second cooldown elapses with no audio above its RMS threshold, the plugin stops sending audio entirely (KeepAlive still goes out, so the socket looks healthy).
The problem is that Deepgram's endpointing can only fire by observing silence in the audio stream. If we stop sending before it has, the utterance is never endpointed. No final transcript arrives, and the stream stays stuck until the caller speaks again, at which point the pending final is flushed.
That's bad on its own, but it also stalls the agent. runEOUDetection bails when there's no accumulated transcript:
// agents/src/voice/audio_recognition.ts
if (this.stt && !this.audioTranscript && this.turnDetectionMode !== 'manual') {
this.logger.debug('skipping EOU detection');
The Python plugin doesn't have this problem, it sends every frame unconditionally:
for frame in frames:
self._audio_duration_collector.push(frame.duration)
await ws.send_bytes(frame.data.tobytes())
In agents-js only the deepgram and sarvam plugins use AudioEnergyFilter, and the
cooldown isn't configurable from STTOptions.
Relevant log output
Instrumented the plugin to log when the energy filter starts and stops forwarding frames.
A healthy turn, Deepgram finalises before the audio is cut:
20.25 interim transcript "make an appointment"
20.87 speech_final received
21.11 energy filter stops forwarding audio
A stalled turn from the same call, audio cut 0.20s after the last interim:
31.38 interim transcript "It just says"
31.58 energy filter stops forwarding audio
... nothing from Deepgram for the entire gap ...
42.35 energy filter resumes forwarding (10.8s with no audio sent)
42.54 interim transcript "It just says,"
Across that call Deepgram needed 0.5-1.2s after the last interim to send speech_final. Every
turn where the filter cut before that stalled; every turn where it cut after was fine.
Describe your environment
OS: Amazon Linux in ECS
Node: 24.14.0
@livekit/agents: 1.4.6
@livekit/agents-plugin-deepgram: 1.4.6
@livekit/agents-plugin-silero: 1.4.6
Have validated bug still exists in 1.6.2 as well.
Minimal reproducible example
The easiest way to trigger it is bump up the endpointing to a large number,
To trigger it you want audio quiet enough to fall under the filter's rms > 0.004 threshold near the end of an utterance, which is easy with phone audio and someone trailing off, and hard with a close mic.
Additional information
No response
Describe the bug
AudioEnergyFiltergates the frames the Deepgram plugin sends to the websocket. Once the 1 second cooldown elapses with no audio above its RMS threshold, the plugin stops sending audio entirely (KeepAlive still goes out, so the socket looks healthy).The problem is that Deepgram's
endpointingcan only fire by observing silence in the audio stream. If we stop sending before it has, the utterance is never endpointed. No final transcript arrives, and the stream stays stuck until the caller speaks again, at which point the pending final is flushed.That's bad on its own, but it also stalls the agent.
runEOUDetectionbails when there's no accumulated transcript:The Python plugin doesn't have this problem, it sends every frame unconditionally:
In agents-js only the
deepgramandsarvamplugins useAudioEnergyFilter, and thecooldown isn't configurable from
STTOptions.Relevant log output
Instrumented the plugin to log when the energy filter starts and stops forwarding frames.
A healthy turn, Deepgram finalises before the audio is cut:
A stalled turn from the same call, audio cut 0.20s after the last interim:
Across that call Deepgram needed 0.5-1.2s after the last interim to send
speech_final. Everyturn where the filter cut before that stalled; every turn where it cut after was fine.
Describe your environment
OS: Amazon Linux in ECS
Node: 24.14.0
@livekit/agents: 1.4.6
@livekit/agents-plugin-deepgram: 1.4.6
@livekit/agents-plugin-silero: 1.4.6
Have validated bug still exists in 1.6.2 as well.
Minimal reproducible example
The easiest way to trigger it is bump up the endpointing to a large number,
To trigger it you want audio quiet enough to fall under the filter's
rms > 0.004threshold near the end of an utterance, which is easy with phone audio and someone trailing off, and hard with a close mic.Additional information
No response