vocal_helper.eot module

vocal_helper.eot

Semantic end-of-turn (EOT) detection stage — inspired by LiveKit’s turn-detector model (April 2026 release, see livekit/turn-detector on Hugging Face) which fine-tuned a Qwen2.5-0.5B distilled from Qwen2.5-7B to score, in ~10 ms / inference, whether a partial transcript looks like a completed speaker turn.

LiveKit reports a 39 % reduction in false-positive interruptions when the semantic EOT signal is fused with Silero VAD. Their claim is that the latency cost of turn-detection is the largest hidden contributor to perceived voice-agent lag.

Why this matters for vocal-helper

Our SileroVADStage emits a VoicedSegment after min_silence_ms of trailing silence. That’s a rigid threshold : a speaker who takes a 350 ms breath mid-sentence gets cut into two segments, the ASR sees two fragments, the diarizer sees two embeddings (sometimes assigning different speakers !), and the LLM analyst gets a worse signal. Empirically the AMI dev-slice contains ~ 15 % of utterances under 400 ms — they are mostly back-channels and breaths, not closed turns.

The SemanticEOTStage sits between SileroVADStage and OnlineDiarStage. For every incoming VoicedSegment it :

  1. Runs a fast STT pass (whisper.cpp turbo, same model the downstream WhisperStage uses — kept in a thread pool to avoid stalling the loop).

  2. Asks a small classifier LLM (qwen2.5:3b by default — close enough in capability to LiveKit’s distilled 0.5B target, already available via Ollama on the user’s machine) whether the partial transcript is a complete thought.

  3. If complete → emit the segment immediately.

  4. If incomplete → buffer it, wait for the next segment, then merge and re-evaluate. After max_merge_s seconds of accumulation we force an emit regardless.

The stage is opt-in — disabled by default to keep the zero-dependency path (Silero alone) ; users wire it in via PipelineConfig.eot.

Author

Warith HARCHAOUI — https://linkedin.com/in/warith-harchaoui

class vocal_helper.eot.SemanticEOTStage(*, eot_model='qwen2.5:3b', stt_model='large-v3-turbo-q5_0', max_merge_s=4.0, min_incomplete_ms=800, host=None)[source]

Bases: object

Producer/consumer EOT gating stage.

Parameters:

  • eot_model (str) – Ollama model used as the EOT classifier. Default qwen2.5:3b — small enough to run at ~ 50 ms / classification on Apple Silicon while broadly equivalent in capability to the LiveKit turn-detector’s 0.5B target.

  • stt_model (str) – pywhispercpp model used for the partial transcript pass. Default large-v3-turbo-q5_0 — same as the downstream WhisperStage. We could cache one instance shared by both stages in a future revision.

  • max_merge_s (float) – Maximum total duration of a merged-on-incomplete chain. After this we force-emit regardless of the classifier’s verdict.

  • min_incomplete_ms (int) – Segments shorter than this are presumed back-channels (acks / breaths) and gated by the classifier ; longer segments are emitted directly without an LLM call (cheap heuristic).

  • host (str, optional) – Ollama host URL. Defaults to the OLLAMA_HOST env var or http://localhost:11434.

async run(inbox, outbox)[source]

Consume :class:`VoicedSegment`s, gate them by semantic EOT.

Parameters:

  • inbox (Queue)

  • outbox (Queue)

Return type:

None