Skip to content

Conversation

@simllll
Copy link
Contributor

@simllll simllll commented Nov 13, 2025

Description

Implement elevenlabs STT support, based on python's elevenlabs and deepgram pattern for relatime in ts.

todo:

  • update the root README support matrix.

Changes Made

  • added new STT class
  • support for 3 elevenlabs models
  • added small README

Pre-Review Checklist

  • Build passes: All builds (lint, typecheck, tests) pass locally
  • AI-generated code reviewed: Removed unnecessary comments and ensured code quality
  • Changes explained: All changes are properly documented and justified above
  • Scope appropriate: All changes relate to the PR title, or explanations provided for why they're included
  • Video demo: A small video demo showing changes works as expected and did not break any existing functionality using Agent Playground (if applicable)

Testing

Manually on my local machine, tested elevenlabs realtime scribe v2 mostly for now.

  • Automated tests added/updated (if applicable)
  • All tests pass
  • Make sure both restaurant_agent.ts and realtime_agent.ts work properly (for major changes)

Additional Notes


Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.

@changeset-bot
Copy link

changeset-bot bot commented Nov 13, 2025

⚠️ No Changeset found

Latest commit: 5620c70

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@simllll simllll force-pushed the feat/elevenlabs-stt branch from ed81240 to 43b0dd1 Compare November 13, 2025 10:58
@simllll
Copy link
Contributor Author

simllll commented Nov 13, 2025

It works quite fine, but I have one weird issue, the elevenlabs WS is closing after quite a short time of inactivity. I tried to find a way to add a keepalive, but couldn't find anything that works (there is no keepalive message type and also i cannot just send an empty string). I guess it is related to some kind of timeout/missing keepalive, but as there is no "real" official api documentation right now for this endpoint (except for this doc https://elevenlabs.io/docs/cookbooks/speech-to-text/streaming), I guess we have to either live with it or find another trick ;-) let's see what python implemetation does livekit/agents#3909

[12:09:09.431] ERROR (330998): STT: WebSocket closed unexpectedly with code 1000: 
[12:09:09.431] WARN (330998): STT: failed to connect to ElevenLabs, retrying in 0 seconds: Error: WebSocket closed (1/32)
[12:09:09.664] INFO (330998): STT: ElevenLabs session started with ID: 9d36f5415c1b4f0cb238e0e7b8facdb6

[12:09:24.701] ERROR (330998): STT: WebSocket closed unexpectedly with code 1000: 
[12:09:24.701] WARN (330998): STT: failed to connect to ElevenLabs, retrying in 0 seconds: Error: WebSocket closed (1/32)
[12:09:24.905] INFO (330998): STT: ElevenLabs session started with ID: 849a9786018d4a1494d61c84cf2e8ea0

code 1000 seems to be not even really an error.
image
maybe we need to set some init parameters to keep it on

unfortunately it doens't help the disconnect error.

[16:30:14.475] ERROR (390804): STT: WebSocket closed unexpectedly with code 1000:
[16:30:14.475] WARN (390804): STT: failed to connect to ElevenLabs, retrying in 0 seconds: Error: WebSocket closed (1/32)
@simllll simllll mentioned this pull request Nov 19, 2025
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant