best_engine_ai_helper.llm module

llm — pluggable local-model backend for best-engine-ai-helper.

Provides two public functions, chat and embed, that route requests to the backend selected by the SPREZZATURE_LLM_BACKEND environment variable. All skill scripts call only these two functions; the transport details (Ollama JSON API vs OpenAI-compatible REST vs LangChain) are invisible to callers.

Supported backends

ollama

Default. POSTs to {SPREZZATURE_LLM_BASE_URL}/api/generate. Works offline once the model is pulled.

openai

Any OpenAI-compatible server: vLLM, llama.cpp, LM Studio, Text Generation Inference. POSTs to {SPREZZATURE_LLM_BASE_URL}/v1/chat/completions.

langchain

Thin wrapper over ChatOllama or ChatOpenAI from LangChain. Only useful if you need LangChain retrievers or agent abstractions.

Environment variables

SPREZZATURE_LLM_BACKEND

ollama | openai | langchain. Defaults to ollama.

SPREZZATURE_LLM_BASE_URL

Base URL of the server. Defaults to http://localhost:11434.

BEST_LLM_TEXT (legacy alias: SPREZZATURE_LLM_TEXT)

Model tag for text-only prompts. When unset, falls back to the selection persisted by pull in ~/.best-engine-ai-helper/config.json, then to the qwen3:8b default. Resolved by config.text_model().

BEST_LLM_VISION (legacy alias: SPREZZATURE_LLM_VISION)

Model tag for prompts that include images. Same precedence as the text model; resolved by config.vision_model().

SPREZZATURE_LLM_API_KEY

API key for servers that require one. Empty string by default (most local servers do not require authentication).

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.llm.chat(prompt, *, system=None, images=None, json_schema=None, model=None, temperature=0.2)[source]

Send a prompt to the configured local model and return the response.

The backend is selected by SPREZZATURE_LLM_BACKEND. All three backends (ollama, openai, langchain) accept the same arguments so callers are backend-agnostic.

Parameters:

  • prompt (str) – User-facing prompt text.

  • system (str or None) – System-level instructions sent before the user prompt. Use for persona, output format constraints, or house style rules.

  • images (list[bytes] or None) – Raw image bytes (PNG or JPEG). When provided, the vision model is used unless model is specified explicitly. The Ollama backend encodes images as base64; the OpenAI backend uses data URI content parts.

  • json_schema (dict or None) – When provided, the response is constrained to this JSON Schema. The Ollama backend passes it as format (grammar-constrained structured output) and the OpenAI backend as a json_schema response format, so the returned JSON matches the schema’s shape – not merely valid JSON of some arbitrary shape.

  • model (str or None) – Override the model tag. Defaults to SPREZZATURE_LLM_VISION when images are present, or SPREZZATURE_LLM_TEXT otherwise.

  • temperature (float) – Sampling temperature. Lower values are more deterministic. Defaults to 0.2 because structured extraction tasks benefit from low variance.

Returns:

When json_schema is provided and the model returns valid JSON, the result is parsed and returned as a dict. Otherwise a plain string.

Return type:

str or dict

Raises:

  • RuntimeError – If the HTTP request fails, the backend is unreachable, or the response is malformed.

  • ValueError – If SPREZZATURE_LLM_BACKEND is set to an unrecognised value.

Examples

>>> # Text prompt (no model running needed for this docstring to parse)
>>> # result = chat("Summarise this paper in one sentence.")
>>> # Vision prompt
>>> # with open("chart.png", "rb") as f:
>>> #     result = chat("Describe the chart.", images=[f.read()])
best_engine_ai_helper.llm.embed(text)[source]

Return an embedding vector for the given text.

Only the Ollama backend is supported for embeddings. The OpenAI-compatible embedding endpoint (/v1/embeddings) is not yet implemented because the retrieval use case is not yet in scope.

Parameters:

text (str) – Input text to embed.

Returns:

Dense embedding vector from the Ollama /api/embeddings endpoint.

Return type:

list[float]

Raises:

Examples

>>> # vec = embed("hello world")  # requires Ollama running
>>> # len(vec) > 0
>>> True
True