best_engine_ai_helper.engine module

engine — resolve a repo’s LLM/VLM usage brief into a concrete serving engine.

The suite’s model-selection contract has two YAML files per consumer repo:

  1. an input brief (committed, hardware-independent) describing what the repo needs from an LLM/VLM — the kinds, memory headroom, comfort floor, and a free-text task; and

  2. an output engine file (gitignored, machine-specific) this module writes: the backend chosen for the current machine plus the concrete model per kind, sized realistically for that backend.

Backend rule (resolve(..., backend="auto")): vLLM only when a real discrete GPU (NVIDIA/AMD) is detected; otherwise Ollama — so macOS, CPU-only Linux, and Intel-iGPU machines all use Ollama, and only a CUDA/ROCm box gets vLLM. This keeps picks realistic (vLLM on plain CPU is the weak path). When vLLM gains an Apple-Silicon runtime, widening this rule is the only change needed.

No DEFAULT_MODEL constant lives in any consumer: the model is always read from the resolved engine file. ensure() is the missing-file policy — a missing engine file is auto-resolved from the brief; a missing brief is a hard error, because the brief is committed and its absence is a real bug.

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.engine.default_backend()[source]

Return the backend for the current machine.

vLLM only when a real discrete GPU (NVIDIA/AMD) is detected; Ollama everywhere else (macOS, CPU-only Linux, Intel iGPU). Endgame: when vLLM runs well on Mac and CPU-only Linux too, replace this whole body with return "vllm" and the suite is fully on vLLM.

Return type:

str

best_engine_ai_helper.engine.ensure(directory='.', *, brief='llm.brief.yaml', engine='llm.engine.yaml', backend='auto', endpoint=None, write=True)[source]

Return the engine descriptor for a repo, resolving it on first use.

Missing-file policy (the suite contract):

  • the engine file exists -> load and return it (fast path, no detection);

  • it is missing but the brief exists -> resolve from the brief, write the engine file (unless write=False), and return it;

  • both are missing -> raise. A committed brief is mandatory; its absence is a real bug, not a machine that has not run detection yet.

Parameters:

  • directory (str | Path) – Repo directory holding the two contract files.

  • brief (str) – Filenames within directory (default to the suite canonical names).

  • engine (str) – Filenames within directory (default to the suite canonical names).

  • backend (see resolve().)

  • endpoint (see resolve().)

  • write (bool) – Persist a freshly resolved engine file. False resolves in-memory only.

Return type:

dict[str, Any]

best_engine_ai_helper.engine.load_brief(brief)[source]

Return the brief as a dict, whether given inline or as a YAML path.

Parameters:

brief (str | Path | dict[str, Any])

Return type:

dict[str, Any]

best_engine_ai_helper.engine.load_engine(path)[source]

Read an engine descriptor written by write_engine().

Parameters:

path (str | Path)

Return type:

dict[str, Any]

best_engine_ai_helper.engine.model_for(engine, kind)[source]

Return (backend, base_url, model) for kind from an engine descriptor.

Raises KeyError if the descriptor has no entry for kind (e.g. asking for a VLM from an engine resolved for an llm-only brief).

Parameters:
Return type:

tuple[str, str, str]

best_engine_ai_helper.engine.resolve(brief, *, backend='auto', endpoint=None, catalog=None, hw=None, compute=None)[source]

Resolve a usage brief into a concrete engine descriptor.

The brief’s mode selects local vs cloud (default local):

  • local (default) -> a hardware-specific descriptor: the backend chosen for this machine (Ollama/vLLM) plus the model per kind.

  • cloud -> a provider descriptor (provider, model, optional base_url/api_key_env) plus a local fallback resolved from the SAME brief, so a failed paid call degrades to the always-available local model (paid -> local, the safe direction). See _resolve_cloud().

Parameters:

  • brief (str | Path | dict) – The input brief (path to llm.brief.yaml or an already-loaded dict). Keys: mode (local/cloud, default local), kind (llm/vlm/both), headroom, min_tps, structured_output, task (free text).

  • backend ({'auto', 'ollama', 'vllm'}) – auto picks per default_backend(); an explicit value forces it.

  • endpoint (str or None) – Override the server base URL (defaults to the local endpoint for the backend).

  • catalog (optional) – Injectable for tests; default to the live catalog and detected hardware.

  • hw (optional) – Injectable for tests; default to the live catalog and detected hardware.

  • compute (optional) – Injectable for tests; default to the live catalog and detected hardware.

Returns:

The engine descriptor (see write_engine() for the on-disk shape).

Return type:

dict

best_engine_ai_helper.engine.write_engine(engine, path)[source]

Write an engine descriptor to path as YAML with a do-not-commit header.

The file is machine-specific (it encodes the chosen backend and models for this hardware), so it belongs in .gitignore, not in version control.

Parameters:
Return type:

Path