best_engine_ai_helper package

Subpackages

Submodules

Module contents

best_engine_ai_helper — pick the best local LLM/VLM for the current hardware.

Public API (importable without invoking the CLI):

from best_engine_ai_helper.detect import platform_name, chip_vendor, available_memory from best_engine_ai_helper.catalog import load_catalog, estimate_ram from best_engine_ai_helper.hardware import load_hardware, lookup_chip from best_engine_ai_helper.score import select, rank, effective_budget from best_engine_ai_helper import text_model, vision_model # cheap tag resolvers

The CLI entry point is best-engine-ai-helper (see cli.py). Importing this package does not trigger any CLI parsing or subprocess calls.

Downstream suite packages that just need “which model tag do I use?” should call text_model() / vision_model() — they resolve an env override, then the selection persisted by pull, then a safe default, without ever probing hardware, so they are cheap and deterministic (CI-safe).

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.available_memory()[source]

Detect available memory for model inference.

Reads three pools from os_helper’s hardware facts:

  1. Apple Silicon unified memory (macOS with Apple chip)

  2. NVIDIA/AMD VRAM (summed across all visible GPUs)

  3. System RAM (always populated)

Returns:

A dict with the following keys:

unified_gbfloat or None

Apple Silicon unified memory pool, in GB.

vram_gbfloat or None

Discrete GPU VRAM (sum of all visible GPUs), in GB.

ram_gbfloat

Total system RAM in GB. Always a positive float.

Return type:

dict[str, float | None]

Examples

>>> mem = available_memory()
>>> set(mem.keys()) == {'unified_gb', 'vram_gb', 'ram_gb'}
True
>>> mem['ram_gb'] > 0
True
best_engine_ai_helper.chip_name()[source]

Return the Apple Silicon chip name (e.g. "Apple M2 Max"), or None.

Delegates to os_helper.apple_chip_name(); None on non-macOS platforms or when the chip line is absent (old Intel Macs).

Return type:

str | None

best_engine_ai_helper.chip_vendor()[source]

Identify the primary compute vendor for model inference.

Returns:

One of: ‘apple’, ‘nvidia’, ‘amd’, ‘intel’, ‘cpu’. Delegates to os_helper.gpu_vendor().

Return type:

str

Examples

>>> chip_vendor() in ('apple', 'nvidia', 'amd', 'intel', 'cpu')
True
best_engine_ai_helper.compute_profile()[source]

Describe the machine’s inference accelerator and memory bandwidth.

Returns a dict with:

  • accelerator: "gpu-metal" (Apple Silicon), "gpu-cuda" (NVIDIA), "gpu-rocm" (AMD), or "cpu" (no discrete accelerator detected).

  • chip: the chip / GPU name when known, else None.

  • bandwidth_gbs: memory bandwidth in GB/s when the chip/GPU matches a known model in _APPLE_BANDWIDTH_GBS / _NVIDIA_BANDWIDTH_GBS / _AMD_BANDWIDTH_GBS, else None. This is the ceiling on decode throughput; token generation reads the whole active model from memory once per token, so tokens/s scales with it.

Bandwidth is tabulated for Apple Silicon (per-chip) and for discrete NVIDIA/AMD GPUs (per-board, matched on the model name os_helper reports). An unrecognised GPU model — a new SKU not yet in the table, or a multi-GPU box where the name string is ambiguous — degrades to bandwidth_gbs: None rather than a fabricated number; callers treat that as “throughput not estimated”, never as zero.

Return type:

dict[str, Any]

best_engine_ai_helper.default_backend()[source]

Return the backend for the current machine.

vLLM only when a real discrete GPU (NVIDIA/AMD) is detected; Ollama everywhere else (macOS, CPU-only Linux, Intel iGPU). Endgame: when vLLM runs well on Mac and CPU-only Linux too, replace this whole body with return "vllm" and the suite is fully on vLLM.

Return type:

str

best_engine_ai_helper.effective_budget(hw, headroom=0.5, load=None)[source]

Compute the memory budget (GB) a model may occupy at run time.

The budget is the accelerator’s usable memory pool, scaled by an extra headroom margin left for the operating system, your own application, and KV-cache growth as context fills. On Apple Silicon the usable pool is not the whole unified memory: Metal caps GPU allocations at about 66% of the pool at or below 36 GB and about 75% above it, beyond which inference spills to CPU. Compare a catalog entry’s ram_gb (already a peak-inference estimate, weights plus a moderate KV cache) against this budget.

Parameters:

  • hw (dict[str, float | None]) – Output of detect.available_memory(). Expected keys: unified_gb, vram_gb, ram_gb.

  • headroom (float) – Extra safety fraction applied on top of the accelerator cap, reserving room for the OS, the caller’s workload, and KV growth. Defaults to MAX_HEADROOM (0.5) and is clamped down to it — a larger value is never honoured, to keep picks realistic.

  • load (dict or None) – Live server state from detect.server_load() (available_ram_gb, cpu_percent, disk_free_gb, …). When given, the theoretical accelerator budget is additionally capped at what is ACTUALLY free right now (another process, or an already-running engine, holds memory the static hardware totals in hw know nothing about), and further derated when the CPU is already saturated or the disk is nearly full. None (the default) reproduces the pre-existing, load-blind behaviour exactly.

Returns:

Effective memory budget in GB.

Return type:

float

Examples

>>> effective_budget({'unified_gb': 96.0, 'vram_gb': None, 'ram_gb': 96.0})
36.0
>>> effective_budget({'unified_gb': None, 'vram_gb': 24.0, 'ram_gb': 64.0})
11.04
>>> # headroom above the 0.5 ceiling is clamped, not honoured
>>> effective_budget({'unified_gb': 96.0, 'vram_gb': None, 'ram_gb': 96.0}, headroom=0.85)
36.0
>>> # a busy machine gets a smaller budget than an idle one with the same hardware
>>> hw = {'unified_gb': 96.0, 'vram_gb': None, 'ram_gb': 96.0}
>>> effective_budget(hw, load={'available_ram_gb': 10.0})
10.0
best_engine_ai_helper.ensure(directory='.', *, brief='llm.brief.yaml', engine='llm.engine.yaml', backend='auto', endpoint=None, write=True)[source]

Return the engine descriptor for a repo, resolving it on first use.

Missing-file policy (the suite contract):

  • the engine file exists -> load and return it (fast path, no detection);

  • it is missing but the brief exists -> resolve from the brief, write the engine file (unless write=False), and return it;

  • both are missing -> raise. A committed brief is mandatory; its absence is a real bug, not a machine that has not run detection yet.

Parameters:

  • directory (str | Path) – Repo directory holding the two contract files.

  • brief (str) – Filenames within directory (default to the suite canonical names).

  • engine (str) – Filenames within directory (default to the suite canonical names).

  • backend (see resolve().)

  • endpoint (see resolve().)

  • write (bool) – Persist a freshly resolved engine file. False resolves in-memory only.

Return type:

dict[str, Any]

best_engine_ai_helper.estimate_ram(disk_gb, quant)[source]

Estimate peak inference RAM from on-disk model size.

The estimate covers the model weights plus KV cache at the default context length (4K tokens). For models with very large context windows (256K+), actual RAM may exceed this estimate significantly; treat it as a lower bound.

Parameters:

  • disk_gb (float) – On-disk footprint of the model in GB.

  • quant (str) – Quantization identifier, e.g. ‘Q4_K_M’, ‘Q8_0’, ‘FP16’, ‘Q2_K’.

Returns:

Estimated peak RAM in GB.

Return type:

float

Examples

>>> estimate_ram(6.1, 'Q4_K_M')
6.832
>>> estimate_ram(10.0, 'FP16')
10.5
best_engine_ai_helper.estimated_tokens_per_second(entry, bandwidth_gbs, backend='ollama')[source]

Estimate local decode throughput (tokens/s) for a model on this machine.

Token generation is memory-bandwidth bound: each new token requires reading the model’s active weights from memory once, so the ceiling is bandwidth / model_bytes. The estimate derates that ceiling by a backend-specific decode efficiency (see _DECODE_EFFICIENCY_BY_BACKEND) to reflect KV-cache reads, kernel overhead, and sampling. It describes steady-state generation, not the compute-bound prefill of a long prompt.

Parameters:

  • entry (dict[str, Any]) – Catalog entry; its model_footprint_gb() is the active-model size.

  • bandwidth_gbs (float or None) – Memory bandwidth in GB/s from detect.compute_profile(). When None (unknown hardware, e.g. an unrecognised discrete-GPU model) the estimate is not computable and None is returned.

  • backend ({'ollama', 'vllm'}) – Serving backend. Affects both the size proxy (heavier FP16 weights under vLLM decode more slowly than the Q4 Ollama figure) AND the decode efficiency (vLLM’s PagedAttention + CUDA-graph decode tracks closer to the bandwidth ceiling than llama.cpp’s — confirmed to differ on identical Ubuntu + discrete-GPU hardware, not just a cross-platform artifact).

Returns:

Estimated tokens/s, rounded to one decimal, or None when bandwidth or model size is unavailable.

Return type:

float or None

best_engine_ai_helper.family_brief(family_id)[source]

Return the representative brief for a family (ready for engine.resolve()).

Parameters:

family_id (str) – The family id.

Returns:

The brief block with mode: local.

Return type:

dict[str, Any]

Examples

>>> family_brief("F2")["kind"]
'llm'
best_engine_ai_helper.get_family(family_id)[source]

Return one family by id, with a helpful error when it is unknown.

Parameters:

family_id (str) – The family id ("F1", "F2" or "F3").

Returns:

The family dict.

Return type:

dict[str, Any]

Raises:

KeyError – If no family carries that id.

Examples

>>> get_family("F3")["name"]
'embeddings'
best_engine_ai_helper.get_usage(name)[source]

Return one profile by name, with a helpful error when it is unknown.

Parameters:

name (str) – The profile name (e.g. "text2sql").

Returns:

The profile dict.

Return type:

dict[str, Any]

Raises:

KeyError – If no profile carries that name; the message suggests close matches.

Examples

>>> get_usage("text2sql")["family"]
'F1'
best_engine_ai_helper.list_families()[source]

Enumerate families for discovery: id, name, members, summary.

Returns:

One compact row per family, in catalog order.

Return type:

list[dict[str, Any]]

Examples

>>> [r["id"] for r in list_families()]
['F1', 'F2', 'F3']
best_engine_ai_helper.list_usages()[source]

Enumerate profiles for discovery: name, family, status, summary.

Returns:

One compact row per profile, in catalog order.

Return type:

list[dict[str, Any]]

Examples

>>> rows = list_usages()
>>> {"name", "family", "status", "summary"} <= set(rows[0])
True
best_engine_ai_helper.load_catalog(catalog_path=None)[source]

Load the bundled seed catalog merged with the user’s local cache.

Cache entries whose id matches a seed entry overwrite the seed entry. New cache entries (no matching seed id) are appended. The seed is never modified on disk.

Parameters:

catalog_path (Path or None) – Path to the seed models.yaml. Defaults to the bundled file next to pyproject.toml. Pass an explicit path in tests to use a fixture.

Returns:

Merged model entries. Each entry is guaranteed to have at minimum: id, kind, ram_gb, benchmarks.

Return type:

list[dict[str, Any]]

Raises:

FileNotFoundError – If catalog_path is given explicitly and does not exist.

Examples

>>> entries = load_catalog()
>>> len(entries) > 0
True
>>> all('id' in e for e in entries)
True
best_engine_ai_helper.load_config()[source]

Return the persisted selection dict, or an empty dict if absent/unreadable.

Reads ~/.best-engine-ai-helper/config.json (written by pull.write_env()). A missing file is the normal “never ran detection” case, so it maps to {} rather than an error; a corrupt file is treated the same way so a bad write can never break a downstream caller.

Returns:

The parsed config, or {} when the file does not exist or cannot be parsed as a JSON object.

Return type:

dict

best_engine_ai_helper.load_engine(path)[source]

Read an engine descriptor written by write_engine().

Parameters:

path (str | Path)

Return type:

dict[str, Any]

best_engine_ai_helper.load_families(usages_path=None)[source]

Load the bundled families merged with the user overlay.

Parameters:

usages_path (Path or None) – Path to the seed usages.yaml. Defaults to the bundled file.

Returns:

Family dicts, each with at least id, brief and members.

Return type:

list[dict[str, Any]]

Examples

>>> [f["id"] for f in load_families()]
['F1', 'F2', 'F3']
best_engine_ai_helper.load_hardware(hardware_path=None)[source]

Load the bundled hardware chip table merged with the user’s local cache.

Cache entries whose (chip, memory_gb) pair matches a seed entry overwrite it. New entries are appended. The seed file is never modified.

Parameters:

hardware_path (Path or None) – Path to the seed hardware.yaml. Defaults to the bundled file. Pass an explicit path in tests to use a fixture.

Returns:

Merged hardware entries. Each entry has at minimum: chip, vendor, memory_gb, ollama_usable_gb.

Return type:

list[dict[str, Any]]

Examples

>>> entries = load_hardware()
>>> len(entries) > 0
True
>>> all('chip' in e for e in entries)
True
best_engine_ai_helper.load_usages(usages_path=None)[source]

Load the bundled usage profiles merged with the user overlay.

Parameters:

usages_path (Path or None) – Path to the seed usages.yaml. Defaults to the bundled file; pass an explicit path in tests.

Returns:

Profile dicts, each with at least name, brief and family.

Return type:

list[dict[str, Any]]

Examples

>>> names = [p["name"] for p in load_usages()]
>>> "text2sql" in names and "embeddings" in names
True
best_engine_ai_helper.lookup_chip(chip_name, hardware)[source]

Find a hardware entry by a case-insensitive substring match on the chip name.

When multiple entries share the same chip name (for example, an Apple M2 Max at 32 GB and at 96 GB), this returns the first match in the list order. The caller should supply the most specific chip string available to avoid ambiguity.

Parameters:

  • chip_name (str) – Chip name or substring to search for, e.g. 'Apple M2 Max'.

  • hardware (list[dict[str, Any]]) – Hardware entries as returned by load_hardware().

Returns:

The first matching entry, or None if no entry contains chip_name as a case-insensitive substring.

Return type:

dict[str, Any] or None

Examples

>>> hw = load_hardware()
>>> entry = lookup_chip('Apple M2 Max', hw)
>>> entry is not None
True
>>> entry['vendor']
'apple'
best_engine_ai_helper.model_footprint_gb(entry, backend='ollama')[source]

Estimate a model’s peak inference memory (GB) on a given serving backend.

The catalog’s ram_gb is an Ollama Q4 estimate. vLLM instead loads the full FP16/BF16 HuggingFace weights (~2 bytes/param) plus KV cache and runtime buffers, so the same model is markedly heavier there. Sizing a vLLM pick against ram_gb would over-promise and pick a model that will not actually fit — the “unrealistic recommendation” this guards against.

Parameters:

  • entry (dict[str, Any]) – Catalog entry; uses ram_gb (Ollama) or size_b (vLLM).

  • backend ({'ollama', 'vllm'}) – Serving backend the footprint is estimated for. Unknown values are treated as 'ollama' (the conservative catalog figure).

Returns:

Estimated peak memory in GB.

Return type:

float

best_engine_ai_helper.model_for(engine, kind)[source]

Return (backend, base_url, model) for kind from an engine descriptor.

Raises KeyError if the descriptor has no entry for kind (e.g. asking for a VLM from an engine resolved for an llm-only brief).

Parameters:
Return type:

tuple[str, str, str]

best_engine_ai_helper.parse_task(task)[source]

Turn a vague task phrase into the model kinds and benchmark axis it implies.

Returns a dict with kinds (subset of ["llm", "vlm"] in pull order), application (the benchmark axis for the text model), matched (the keywords that fired, for the report’s justification), and language (best-effort ISO 639-1 code from _detect_language(), or None). A task that mentions nothing visual still gets an LLM on the generalist axis; any vision keyword adds a VLM.

Parameters:

task (str or None) – Free-text task description. None, a blank/whitespace-only string, or text with no detectable language (symbols/digits only) falls back to a generic text-assistant profile, but logs a loud warning first: a recommendation with no clean task description carries no useful label for activity/cost monitoring or for the report’s justification, so a caller skipping it should see that reflected back.

Return type:

dict[str, Any]

Examples

>>> parse_task("write product descriptions and check photo quality")["kinds"]
['llm', 'vlm']
>>> parse_task(None)["application"]  # logs a WARNING, still resolves
'generalist'
best_engine_ai_helper.platform_name()[source]

Return the current OS as a short lowercase string.

Returns:

One of: ‘darwin’, ‘linux’, ‘windows’. Delegates to os_helper.platform_name().

Return type:

str

Examples

>>> platform_name() in ('darwin', 'linux', 'windows')
True
best_engine_ai_helper.rank(hw, catalog, kind, headroom=0.5, application=None, backend='ollama', load=None)[source]

Return all candidates sorted by benchmark score, annotated with fit status.

Each entry in the result gets a _fits key (bool) indicating whether it fits within the effective budget. The top entry is identical to what select() would return.

Parameters:

  • hw (dict[str, float | None]) – Output of detect.available_memory().

  • catalog (list[dict[str, Any]]) – Merged model entries from catalog.load_catalog().

  • kind ({'llm', 'vlm'}) – The inference kind to filter and rank by.

  • headroom (float) – Extra safety on top of the accelerator cap. Clamped to MAX_HEADROOM (0.5); default 0.5.

  • application (str or None) – Optional use-case keyword ("code", "math", "ocr", "vision", "chat", "generalist"). Drives which benchmark column is used for ranking.

  • backend ({'ollama', 'vllm'}) – Serving backend, so _fits reflects the footprint that actually loads (FP16 for vLLM, Q4 ram_gb for Ollama).

  • load (dict or None) – Live server state from detect.server_load(), forwarded to effective_budget(). None reproduces the load-blind behaviour.

Returns:

Candidates sorted descending by benchmark score, each with _fits.

Return type:

list[dict[str, Any]]

Examples

>>> catalog = [{'id': 'v', 'kind': 'vlm', 'ram_gb': 9.0,
...             'benchmarks': {'vision': 80}}]
>>> rank({'unified_gb': 96.0, 'vram_gb': None, 'ram_gb': 96.0}, catalog, 'vlm')[0]['_fits']
True
best_engine_ai_helper.recommend_engines(hw, catalog, task=None, *, headroom=0.5, compute=None, min_tps=15.0, backend='ollama', kinds=None, load=None)

Recommend the best engine per needed kind for this hardware and task.

Parameters:

  • hw (dict) – Memory description from detect.available_memory().

  • catalog (list of dict) – Benchmark catalog from catalog.load_catalog().

  • task (str or None) – Free-text or keyword task. None means a generalist text assistant.

  • headroom (float) – Memory safety margin passed to score.effective_budget() (clamped to score.MAX_HEADROOM).

  • compute (dict or None) – Compute profile from detect.compute_profile() (accelerator + bandwidth). None disables the throughput estimate.

  • min_tps (float) – Comfort throughput floor; a fitting model below it is only picked when no comfortable one exists (and the choice is warned about).

  • backend ({'ollama', 'vllm'}) – Serving backend, so memory fit and throughput reflect what actually loads. Threaded to score.rank() and _candidate_row().

  • kinds (list of str or None) – Explicit kinds to resolve (["llm"], ["vlm"] or both), overriding the kinds inferred from task. The task text still selects the benchmark axis. Used when the caller already knows what it needs (e.g. a brief that declares kind: both).

  • load (dict or None) – Live server state from detect.server_load() (current free RAM, CPU/GPU/disk usage, already-running engines). Forwarded to score.effective_budget()/score.rank() so the recommendation reflects what else is happening on this machine right now, not only its theoretical capacity. None (the default) reproduces the load-blind behaviour exactly; also included as server_load in the returned report when given, for activity monitoring.

Returns:

A JSON-ready report: the parsed task, hardware, memory budget, and for each needed kind the chosen model plus the full ranked candidate table, with per-model fit and estimated throughput.

Return type:

dict

best_engine_ai_helper.resolve(brief, *, backend='auto', endpoint=None, catalog=None, hw=None, compute=None)[source]

Resolve a usage brief into a concrete engine descriptor.

The brief’s mode selects local vs cloud (default local):

  • local (default) -> a hardware-specific descriptor: the backend chosen for this machine (Ollama/vLLM) plus the model per kind.

  • cloud -> a provider descriptor (provider, model, optional base_url/api_key_env) plus a local fallback resolved from the SAME brief, so a failed paid call degrades to the always-available local model (paid -> local, the safe direction). See _resolve_cloud().

Parameters:

  • brief (str | Path | dict) – The input brief (path to llm.brief.yaml or an already-loaded dict). Keys: mode (local/cloud, default local), kind (llm/vlm/both), headroom, min_tps, structured_output, task (free text).

  • backend ({'auto', 'ollama', 'vllm'}) – auto picks per default_backend(); an explicit value forces it.

  • endpoint (str or None) – Override the server base URL (defaults to the local endpoint for the backend).

  • catalog (optional) – Injectable for tests; default to the live catalog and detected hardware.

  • hw (optional) – Injectable for tests; default to the live catalog and detected hardware.

  • compute (optional) – Injectable for tests; default to the live catalog and detected hardware.

Returns:

The engine descriptor (see write_engine() for the on-disk shape).

Return type:

dict

best_engine_ai_helper.resolve_family(family_id, *, backend='auto', endpoint=None, catalog=None, hw=None, compute=None)[source]

Resolve a whole family into one machine-specific engine descriptor.

Resolving a family yields a single model for the group (the shared pick), whereas resolve_usage() yields the possibly-specialised model for one profile. The result is machine-specific — persist it gitignored, never commit.

Parameters:
Returns:

The engine descriptor, annotated with the family’s metadata.

Return type:

dict[str, Any]

Examples

>>> hw = {"unified_gb": 96.0, "vram_gb": None, "ram_gb": 96.0}
>>> cat = [{"id": "emb", "kind": "embed", "ram_gb": 1.2,
...         "benchmarks": {"mteb": 66}}]
>>> eng = resolve_family("F3", catalog=cat, hw=hw,
...                      compute={"chip": "M2", "accelerator": "apple"})
>>> eng["embed"]["model"], eng["family"]
('emb', 'F3')
best_engine_ai_helper.resolve_usage(name, *, backend='auto', endpoint=None, catalog=None, hw=None, compute=None)[source]

Resolve a usage profile into a machine-specific engine descriptor.

This is the “give me the model for profile name” entry point. It reads only the profile’s needs, then lets best-engine choose the concrete model for this machine. The returned descriptor is machine-specific: persist it to a gitignored file (see engine.write_engine()), never commit it.

Parameters:

  • name (str) – The profile name ("text2sql", "rag-answer", …).

  • backend ({'auto', 'ollama', 'vllm'}) – Serving backend; auto picks per engine.default_backend().

  • endpoint (str or None) – Override the server base URL.

  • catalog (optional) – Injectable for tests; default to the live catalog and detected hardware.

  • hw (optional) – Injectable for tests; default to the live catalog and detected hardware.

  • compute (optional) – Injectable for tests; default to the live catalog and detected hardware.

Returns:

The engine descriptor, annotated with the profile’s metadata.

Return type:

dict[str, Any]

Examples

>>> hw = {"unified_gb": 96.0, "vram_gb": None, "ram_gb": 96.0}
>>> compute = {"accelerator": "apple", "chip": "M2", "bandwidth_gbs": None}
>>> cat = [{"id": "coder", "kind": "llm", "size_b": 7, "ram_gb": 5.0,
...         "benchmarks": {"general": 66, "code": 85},
...         "structured_output": True, "vllm_id": "org/Coder"}]
>>> eng = resolve_usage("text2sql", backend="ollama", catalog=cat,
...                     hw=hw, compute=compute)
>>> eng["llm"]["model"], eng["usage"], eng["status"]
('coder', 'text2sql', 'stable')
best_engine_ai_helper.resolved_models()[source]

Return both resolved tags in one call, reading the config file at most once.

Returns:

{"text": <tag>, "vision": <tag>} — the same values text_model() and vision_model() would return.

Return type:

dict

best_engine_ai_helper.select(hw, catalog, kind, headroom=0.5, application=None, backend='ollama', load=None)[source]

Pick the best-scoring model that fits in available memory.

Candidates for a ‘vlm’ selection include both VLMs and LLMs with vision capability (kind == ‘vlm’). Candidates for a ‘llm’ selection include text-only LLMs and VLMs (since a VLM handles text-only prompts equally).

Selection order: 1. Filter: keep entries whose ram_gb fits within the effective budget. 2. Rank: structured-output capability first (a model that cannot honour

Ollama structured output is never chosen over one that can), then benchmark score (application-specific if given, else vision for VLM or general for LLM). This matches rank(), so rank(...)[0] and select(...) agree.

  1. Last resort: if nothing fits, return the smallest model in the catalog rather than raising; the caller decides whether to warn the user.

Parameters:

  • hw (dict[str, float | None]) – Output of detect.available_memory().

  • catalog (list[dict[str, Any]]) – Merged model entries from catalog.load_catalog().

  • kind ({'llm', 'vlm'}) – The type of model to select.

  • headroom (float) – Extra safety on top of the accelerator cap. Clamped to MAX_HEADROOM (0.5); default 0.5.

  • application (str or None) – Optional use-case keyword ("code", "math", "ocr", "vision", "chat", "generalist"). Selects the benchmark axis used for scoring. None uses the default kind-based rule.

  • backend ({'ollama', 'vllm'}) – Serving backend, so memory fit is checked against the footprint that actually loads (FP16 for vLLM, Q4 ram_gb for Ollama).

  • load (dict or None) – Live server state from detect.server_load(), forwarded to effective_budget(). None reproduces the load-blind behaviour.

Returns:

The selected catalog entry.

Return type:

dict[str, Any]

Raises:

ValueError – If the catalog is empty.

Examples

>>> hw = {'unified_gb': 96.0, 'vram_gb': None, 'ram_gb': 96.0}
>>> catalog = [{'id': 'v', 'kind': 'vlm', 'ram_gb': 9.0,
...             'benchmarks': {'vision': 80}}]
>>> select(hw, catalog, kind='vlm')['id']
'v'
best_engine_ai_helper.text_model()[source]

Return the model tag to use for text-only prompts (lint, summaries, …).

Precedence: BEST_LLM_TEXT env (or legacy SPREZZATURE_LLM_TEXT) -> persisted config.json -> DEFAULT_TEXT_MODEL. Never probes hardware, never raises.

Return type:

str

best_engine_ai_helper.to_markdown(report)[source]

Render a recommend() report as a Markdown document.

Parameters:

report (dict[str, Any])

Return type:

str

best_engine_ai_helper.usage_brief(name)[source]

Return the resolvable brief for a profile (ready for engine.resolve()).

Parameters:

name (str) – The profile name.

Returns:

The brief block with mode: local.

Return type:

dict[str, Any]

Examples

>>> usage_brief("classification")["kind"]
'llm'
best_engine_ai_helper.vision_model()[source]

Return the model tag to use for prompts that include images (alt-text, OCR).

Precedence: BEST_LLM_VISION env (or legacy SPREZZATURE_LLM_VISION) -> persisted config.json -> DEFAULT_VISION_MODEL. Never probes hardware, never raises.

Return type:

str

best_engine_ai_helper.write_engine(engine, path)[source]

Write an engine descriptor to path as YAML with a do-not-commit header.

The file is machine-specific (it encodes the chosen backend and models for this hardware), so it belongs in .gitignore, not in version control.

Parameters:
Return type:

Path

best_engine_ai_helper.write_report(report, stem)[source]

Write both a JSON and a Markdown rendering; return (md_path, json_path).

Parameters:
Return type:

tuple[Path, Path]