best_engine_ai_helper.engine module

engine — resolve a repo’s LLM/VLM usage brief into a concrete serving engine.

The suite’s model-selection contract has two YAML files per consumer repo:

  1. an input brief (committed, hardware-independent) describing what the repo needs from an LLM/VLM — the kinds, memory headroom, comfort floor, and a free-text task; and

  2. an output engine file (gitignored, machine-specific) this module writes: the backend chosen for the current machine plus the concrete model per kind, sized realistically for that backend.

Backend rule (resolve(..., backend="auto")): vLLM only on Linux with a real discrete GPU (NVIDIA/AMD); Ollama everywhere else — macOS (vLLM has no native macOS runtime), Windows (no native vLLM wheel, WSL-only), CPU-only machines (vLLM’s CPU-only path is its own weak spot, notably on Linux), and Intel iGPUs all use Ollama; only a Linux CUDA/ROCm box gets vLLM, chiefly for the request-batching/concurrency handling it offers under real production load, which Ollama does not match. This keeps picks realistic. When vLLM gains a first-class runtime elsewhere, widening this rule is the only change needed.

No DEFAULT_MODEL constant lives in any consumer: the model is always read from the resolved engine file. ensure() is the missing-file policy — a missing engine file is auto-resolved from the brief; a missing brief is a hard error, because the brief is committed and its absence is a real bug.

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.engine.default_backend()[source]

Return the backend for the current machine.

A three-branch decision tree, matching what each backend actually supports well rather than what it merely runs on:

  • macOS -> Ollama. vLLM ships no native macOS runtime.

  • CPU-only (Linux or macOS) -> Ollama. vLLM’s CPU-only path — notably on Linux — is its own reported weak spot; Ollama’s CPU path is the well-trodden one.

  • discrete GPU (NVIDIA/AMD) on Linux — the real production case -> vLLM, chiefly for the request-batching/concurrency handling it offers under real load, which Ollama does not match.

Windows falls back to Ollama too (vLLM has no native Windows wheel, only a WSL path), so in practice: vLLM only on Linux with a real discrete GPU; Ollama everywhere else. Endgame: when vLLM gains a first-class runtime on Mac, Windows, or CPU-only Linux, replace this whole body with return "vllm" and the suite is fully on vLLM.

Return type:

str

best_engine_ai_helper.engine.ensure(directory='.', *, brief='llm.brief.yaml', engine='llm.engine.yaml', backend='auto', endpoint=None, write=True)[source]

Return the engine descriptor for a repo, resolving it on first use.

Missing-file policy (the suite contract):

  • the engine file exists -> load and return it (fast path, no detection);

  • it is missing but the brief exists -> resolve from the brief, write the engine file (unless write=False), and return it;

  • both are missing -> raise. A committed brief is mandatory; its absence is a real bug, not a machine that has not run detection yet.

Parameters:

  • directory (str | Path) – Repo directory holding the two contract files.

  • brief (str) – Filenames within directory (default to the suite canonical names).

  • engine (str) – Filenames within directory (default to the suite canonical names).

  • backend (see resolve().)

  • endpoint (see resolve().)

  • write (bool) – Persist a freshly resolved engine file. False resolves in-memory only.

Return type:

dict[str, Any]

best_engine_ai_helper.engine.load_brief(brief)[source]

Return the brief as a dict, whether given inline or as a YAML path.

Parameters:

brief (str | Path | dict[str, Any])

Return type:

dict[str, Any]

best_engine_ai_helper.engine.load_engine(path)[source]

Read an engine descriptor written by write_engine().

Parameters:

path (str | Path)

Return type:

dict[str, Any]

best_engine_ai_helper.engine.model_for(engine, kind)[source]

Return (backend, base_url, model) for kind from an engine descriptor.

Raises KeyError if the descriptor has no entry for kind (e.g. asking for a VLM from an engine resolved for an llm-only brief).

Parameters:
Return type:

tuple[str, str, str]

best_engine_ai_helper.engine.resolve(brief, *, backend='auto', endpoint=None, catalog=None, hw=None, compute=None, cloud_catalog=None)[source]

Resolve a usage brief into a concrete engine descriptor.

The brief’s mode selects local vs cloud (default local):

  • local (default) -> a hardware-specific descriptor: the backend chosen for this machine (Ollama/vLLM) plus the model per kind.

  • cloud -> a provider descriptor (provider, model, optional base_url/api_key_env) plus a local fallback resolved from the SAME brief, so a failed paid call degrades to the always-available local model (paid -> local, the safe direction). model is OPTIONAL: when omitted, the best paid model is auto-picked from the bundled pricing.yaml catalog on a quality-vs-price trade-off (see _resolve_cloud() and best_engine_ai_helper.cloud_catalog).

Parameters:

  • brief (str | Path | dict) – The input brief (path to llm.brief.yaml or an already-loaded dict). Keys: mode (local/cloud, default local), kind (llm/vlm/both), headroom, min_tps, structured_output, task (free text); cloud-only: provider, model (both optional -> auto-pick), vlm_model, base_url, api_key_env, quality_vs_cost (0..1, default cloud_catalog.DEFAULT_QUALITY_VS_COST, only used when auto-picking).

  • backend ({'auto', 'ollama', 'vllm'}) – auto picks per default_backend(); an explicit value forces it. Local mode only.

  • endpoint (str or None) – Override the server base URL (defaults to the local endpoint for the backend, or the provider’s default API root for cloud mode).

  • catalog (optional) – Injectable for tests; default to the live local catalog and detected hardware.

  • hw (optional) – Injectable for tests; default to the live local catalog and detected hardware.

  • compute (optional) – Injectable for tests; default to the live local catalog and detected hardware.

  • cloud_catalog (optional) – Injectable for tests; defaults to the live pricing.yaml catalog. Cloud mode only, and only consulted when the brief omits model.

Returns:

The engine descriptor (see write_engine() for the on-disk shape).

Return type:

dict

best_engine_ai_helper.engine.write_engine(engine, path)[source]

Write an engine descriptor to path as YAML with a do-not-commit header.

The file is machine-specific (it encodes the chosen backend and models for this hardware), so it belongs in .gitignore, not in version control.

Parameters:
Return type:

Path