vocal_helper.llm module
vocal_helper.llm
Optional LLM analyst stage. Consumes Utterance events and
maintains a rolling summary of the conversation up to
recent_window_s (default 60 s) before now.
Algorithm
Keep a deque of recent utterances with timestamps.
After every new
Utterance: - Move utterances whoset1is older thannow − recent_window_sfrom the recent buffer into the summarisation queue.If the summarisation queue grew by
flush_every_n(default 5) new utterances, ask the LLM to fold them into the runningsummaryfield.Emit a
SummarySnapshotwith the current(summary, recent)pair.
The model is never hard-coded here. The stage receives a resolved
engine descriptor (from best_engine_ai_helper.ensure on the
package’s llm.brief.yaml) and routes every request through
best_engine_ai_helper.llm.chat — which dispatches to Ollama or vLLM
per the descriptor. chat is synchronous, so the call is offloaded to
a worker thread to keep the event loop responsive.
If the LLM is unreachable, the stage logs a warning and emits a
SummarySnapshot with the previous summary unchanged,
so downstream consumers never miss an event.
- class vocal_helper.llm.GemmaAnalystStage(*, engine, recent_window_s=60.0, flush_every_n=5, flush_every_s=60.0, prompt_template='You are a meeting note-taker. Update the running summary below by integrating the new utterances. Keep it concise (≤ 6 bullet points), preserve speaker attributions, and drop low-signal small talk. Output only the updated summary, nothing else.\n\nCurrent summary:\n{summary}\n\nNew utterances (older → newer):\n{new_block}\n')[source]
Bases:
objectProducer/consumer LLM analyst with a rolling summary.
- Parameters:
engine (dict) – Resolved engine descriptor from
best_engine_ai_helper.ensure(<vocal_helper package dir>)— it names the backend (Ollama / vLLM), the base URL, and the text model to serve. No default model is baked in here.recent_window_s (float) – How many seconds of verbatim transcript to keep before folding into the summary. Default 60 s.
flush_every_n (int) – Update the summary every
flush_every_nnew utterances that crossed the recent window. Default 5.prompt_template (str) – Override the canonical summarisation prompt. Two placeholders :
{summary}(current digest) and{new_block}(newly evicted utterances). Default keeps a ≤ 6-bullet meeting digest.flush_every_s (float | None)