Speech-to-Speech
ActiveDescription
Build local voice agents with open-source models. An end-to-end speech-to-speech pipeline from HuggingFace for fully local voice AI agent deployment.
Key Features
- Fully open modular speech-to-speech pipeline: VAD → STT → LLM → TTS
- Multiple STT backends: Whisper, Parakeet TDT (sub-100ms), Lightning Whisper MLX
- Flexible LLM integration: HuggingFace Transformers, mlx-lm, OpenAI API
- Multiple TTS engines: ChatTTS, Pocket TTS (voice cloning), Kokoro-82M, Qwen3-TTS
- Four deployment modes: Realtime WebSocket, Server/Client, WebSocket, and Local
- Optimized Apple Silicon support with MLX acceleration for all pipeline stages
Use Cases
💡 Building local voice AI agents without cloud dependencies
💡 Real-time conversational AI with low-latency speech processing
💡 Custom voice agent development with swappable model components
💡 Privacy-sensitive voice applications with fully offline deployment
💡 Prototyping speech-to-speech systems with rapid model experimentation
Strengths & Limitations
✅ Strengths
- • Actively maintained, recent updates
- • High community interest (13.0k stars)
- • Permissive open-source license (Apache-2.0)
- • Established track record (2 years in production)
Categories
Quick Start
Install: pip install speech-to-speech. Run: speech-to-speech (defaults to Parakeet TDT + OpenAI LLM + Qwen3-TTS). For local Mac: speech-to-speech --local_mac_optimal_settings. Optional backends: pip install "speech-to-speech[kokoro]". Set OPENAI_API_KEY for cloud LLM.