-
Notifications
You must be signed in to change notification settings - Fork 195
elevenlabs stt support including scribe v2 realtime #829
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
also merge README
ed81240 to
43b0dd1
Compare
|
It works quite fine, but I have one weird issue, the elevenlabs WS is closing after quite a short time of inactivity. I tried to find a way to add a keepalive, but couldn't find anything that works (there is no keepalive message type and also i cannot just send an empty string). I guess it is related to some kind of timeout/missing keepalive, but as there is no "real" official api documentation right now for this endpoint (except for this doc https://elevenlabs.io/docs/cookbooks/speech-to-text/streaming), I guess we have to either live with it or find another trick ;-) let's see what python implemetation does livekit/agents#3909 code 1000 seems to be not even really an error. |
unfortunately it doens't help the disconnect error. [16:30:14.475] ERROR (390804): STT: WebSocket closed unexpectedly with code 1000: [16:30:14.475] WARN (390804): STT: failed to connect to ElevenLabs, retrying in 0 seconds: Error: WebSocket closed (1/32)

Description
Implement elevenlabs STT support, based on python's elevenlabs and deepgram pattern for relatime in ts.
todo:
Changes Made
Pre-Review Checklist
Testing
Manually on my local machine, tested elevenlabs realtime scribe v2 mostly for now.
restaurant_agent.tsandrealtime_agent.tswork properly (for major changes)Additional Notes
Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.