best_engine_ai_helper.recommend module

recommend — the end-to-end “which engine” algorithm.

Takes three inputs, at least one of them allowed to be vague:

  1. a hardware description (memory pool + compute profile),

  2. the benchmark catalog (models with public scores), and

  3. a task, which may be a free-text phrase (“retail product descriptions and image-quality checks”) rather than a fixed keyword,

and returns the best local engine to pull for each kind the task needs (LLM for text, VLM for images), justified by three factors it makes explicit:

  • task fit — the model’s benchmark on the axis the task maps to,

  • memory fit — whether it fits the accelerator’s usable budget (so it runs on the GPU, not spilled to CPU), and

  • throughput — an estimate of decode tokens/s from memory bandwidth.

The result is a plain dict (ready for json.dumps) and, via to_markdown(), a human-readable report. Nothing here calls a model or the network; it is pure ranking over the catalog and the detected hardware.

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.recommend.parse_task(task)[source]

Turn a vague task phrase into the model kinds and benchmark axis it implies.

Returns a dict with kinds (subset of ["llm", "vlm"] in pull order), application (the benchmark axis for the text model), matched (the keywords that fired, for the report’s justification), and language (best-effort ISO 639-1 code from _detect_language(), or None). A task that mentions nothing visual still gets an LLM on the generalist axis; any vision keyword adds a VLM.

Parameters:

task (str or None) – Free-text task description. None, a blank/whitespace-only string, or text with no detectable language (symbols/digits only) falls back to a generic text-assistant profile, but logs a loud warning first: a recommendation with no clean task description carries no useful label for activity/cost monitoring or for the report’s justification, so a caller skipping it should see that reflected back.

Return type:

dict[str, Any]

Examples

>>> parse_task("write product descriptions and check photo quality")["kinds"]
['llm', 'vlm']
>>> parse_task(None)["application"]  # logs a WARNING, still resolves
'generalist'
best_engine_ai_helper.recommend.recommend(hw, catalog, task=None, *, headroom=0.5, compute=None, min_tps=15.0, backend='ollama', kinds=None, load=None)[source]

Recommend the best engine per needed kind for this hardware and task.

Parameters:

  • hw (dict) – Memory description from detect.available_memory().

  • catalog (list of dict) – Benchmark catalog from catalog.load_catalog().

  • task (str or None) – Free-text or keyword task. None means a generalist text assistant.

  • headroom (float) – Memory safety margin passed to score.effective_budget() (clamped to score.MAX_HEADROOM).

  • compute (dict or None) – Compute profile from detect.compute_profile() (accelerator + bandwidth). None disables the throughput estimate.

  • min_tps (float) – Comfort throughput floor; a fitting model below it is only picked when no comfortable one exists (and the choice is warned about).

  • backend ({'ollama', 'vllm'}) – Serving backend, so memory fit and throughput reflect what actually loads. Threaded to score.rank() and _candidate_row().

  • kinds (list of str or None) – Explicit kinds to resolve (["llm"], ["vlm"] or both), overriding the kinds inferred from task. The task text still selects the benchmark axis. Used when the caller already knows what it needs (e.g. a brief that declares kind: both).

  • load (dict or None) – Live server state from detect.server_load() (current free RAM, CPU/GPU/disk usage, already-running engines). Forwarded to score.effective_budget()/score.rank() so the recommendation reflects what else is happening on this machine right now, not only its theoretical capacity. None (the default) reproduces the load-blind behaviour exactly; also included as server_load in the returned report when given, for activity monitoring.

Returns:

A JSON-ready report: the parsed task, hardware, memory budget, and for each needed kind the chosen model plus the full ranked candidate table, with per-model fit and estimated throughput.

Return type:

dict

best_engine_ai_helper.recommend.to_markdown(report)[source]

Render a recommend() report as a Markdown document.

Parameters:

report (dict[str, Any])

Return type:

str

best_engine_ai_helper.recommend.write_report(report, stem)[source]

Write both a JSON and a Markdown rendering; return (md_path, json_path).

Parameters:
Return type:

tuple[Path, Path]