best_engine_ai_helper.recommend module

recommend — the end-to-end “which engine” algorithm.

Takes three inputs, at least one of them allowed to be vague:

  1. a hardware description (memory pool + compute profile),

  2. the benchmark catalog (models with public scores), and

  3. a task, which may be a free-text phrase (“retail product descriptions and image-quality checks”) rather than a fixed keyword,

and returns the best local engine to pull for each kind the task needs (LLM for text, VLM for images), justified by three factors it makes explicit:

  • task fit — the model’s benchmark on the axis the task maps to,

  • memory fit — whether it fits the accelerator’s usable budget (so it runs on the GPU, not spilled to CPU), and

  • throughput — an estimate of decode tokens/s from memory bandwidth.

The result is a plain dict (ready for json.dumps) and, via to_markdown(), a human-readable report. Nothing here calls a model or the network; it is pure ranking over the catalog and the detected hardware.

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.recommend.parse_task(task)[source]

Turn a vague task phrase into the model kinds and benchmark axis it implies.

Returns a dict with kinds (subset of ["llm", "vlm"] in pull order), application (the benchmark axis for the text model), and matched (the keywords that fired, for the report’s justification). A task that mentions nothing visual still gets an LLM on the generalist axis; any vision keyword adds a VLM.

Parameters:

task (str | None)

Return type:

dict[str, Any]

best_engine_ai_helper.recommend.recommend(hw, catalog, task=None, *, headroom=0.85, compute=None)[source]

Recommend the best engine per needed kind for this hardware and task.

Parameters:

  • hw (dict) – Memory description from detect.available_memory().

  • catalog (list of dict) – Benchmark catalog from catalog.load_catalog().

  • task (str or None) – Free-text or keyword task. None means a generalist text assistant.

  • headroom (float) – Memory safety margin passed to score.effective_budget().

  • compute (dict or None) – Compute profile from detect.compute_profile() (accelerator + bandwidth). None disables the throughput estimate.

Returns:

A JSON-ready report: the parsed task, hardware, memory budget, and for each needed kind the chosen model plus the full ranked candidate table, with per-model fit and estimated throughput.

Return type:

dict

best_engine_ai_helper.recommend.to_markdown(report)[source]

Render a recommend() report as a Markdown document.

Parameters:

report (dict[str, Any])

Return type:

str

best_engine_ai_helper.recommend.write_report(report, stem)[source]

Write both a JSON and a Markdown rendering; return (md_path, json_path).

Parameters:
Return type:

tuple[Path, Path]