voiceAgent is a local, real-time voice assistant for Windows built around one constraint: it has to be fast enough that talking to it feels like a conversation, and it has to stop when you talk over it.
Audio from the mic goes through Silero VAD and an echo gate into a state machine. Parakeet TDT 0.6B transcribes each utterance (a CUDA + int8 CPU hybrid), Smart Turn v3.2 decides whether a pause is a real end of turn, and a LangGraph "brain" streams a reply with tools. Kokoro splits the streaming text into sentences and starts speaking on the first one.
Barge-in happens in two stages: 96 ms of speech ducks the TTS to 25%, 320 ms of sustained speech cuts it and cancels the model stream. Every stage is benchmarked and the numbers are in the repo's results/REPORT.md.
Hard parts
- Holding mid-sentence pauses without adding latency at real turn ends (Smart Turn endpointing)
- Stopping the agent interrupting itself: echo-gated, two-stage barge-in
- Keeping first audio under half a second by streaming TTS sentence by sentence
