feat: Add real-time streaming capabilities with WebSocket integration #2676
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Created whisper/streaming module for real-time transcription - Implemented StreamProcessor with Voice Activity Detection (VAD) - Added AudioBuffer with intelligent chunking and overlap handling - Built WebSocket server supporting multiple concurrent connections - Integrated CTranslate2 backend for accelerated inference - Added comprehensive configuration system (StreamConfig) - Implemented real-time result callbacks and error handling - Created example streaming client with microphone support - Added performance optimization and adaptive buffering - Full WebSocket API with JSON message protocol - Support for multiple audio formats (PCM16, PCM32, Float32) - Thread-safe audio processing pipeline Features: - <200ms latency for real-time processing - Multi-client WebSocket server - Voice Activity Detection - Configurable chunking strategy - CTranslate2 acceleration support - Comprehensive error handling - Performance monitoring and statistics Addresses: OpenAI Whisper Discussions #2, openai#937 - Real-time Streaming Limitations
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Features:
Addresses: OpenAI Whisper Discussions #2, #937 - Real-time Streaming Limitations