best_engine_ai_helper.recommend module

recommend — the end-to-end “which engine” algorithm.

Takes three inputs, at least one of them allowed to be vague:

  1. a hardware description (memory pool + compute profile),

  2. the benchmark catalog (models with public scores), and

  3. a task, which may be a free-text phrase (“retail product descriptions and image-quality checks”) rather than a fixed keyword,

and returns the best local engine to pull for each kind the task needs (LLM for text, VLM for images), justified by three factors it makes explicit:

  • task fit — the model’s benchmark on the axis the task maps to,

  • memory fit — whether it fits the accelerator’s usable budget (so it runs on the GPU, not spilled to CPU), and

  • throughput — an estimate of decode tokens/s from memory bandwidth.

The result is a plain dict (ready for json.dumps) and, via to_markdown(), a human-readable report. Nothing here calls a model or the network; it is pure ranking over the catalog and the detected hardware.

cloud_recommend() is the paid counterpart: same task-to-axis mapping (parse_task()), but ranks best_engine_ai_helper.cloud_catalog’s paid-model catalog on quality vs. price instead of local hardware fit — no hardware probe, no API key, nothing reaches the network; it only tells you which model would be picked. to_markdown() renders it as an extra section when a report carries a "cloud" key (see to_markdown_cloud()).

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.recommend.cloud_recommend(task=None, *, kinds=None, quality_vs_cost=None, provider=None, catalog=None)[source]

Recommend the best PAID model per needed kind for a task (reference only).

The cloud counterpart of recommend(): same parse_task() mapping from a free-text task to the kinds needed and the benchmark axis, but ranks best_engine_ai_helper.cloud_catalog’s paid-model catalog on quality vs. price instead of local memory/throughput fit. No hardware probe, no API key, no network call — every paid model in the catalog is reachable from any machine, so there is nothing to detect; this only tells you which one would be picked.

Parameters:

  • task (str or None) – Free-text or keyword task, same parsing as recommend().

  • kinds (list of {'llm', 'vlm'} or None) – Explicit kinds to resolve, overriding the kinds inferred from task.

  • quality_vs_cost (float or None) – 0..1 weight on quality vs. price (clamped into range by cloud_catalog). None uses cloud_catalog.DEFAULT_QUALITY_VS_COST.

  • provider (str or None) – Restrict candidates to one provider (e.g. "openai"). None ranks across every provider in the catalog.

  • catalog (list[dict[str, Any]] or None) – Injectable for tests; defaults to cloud_catalog.load_cloud_catalog().

Returns:

A JSON-ready report: the parsed task, the quality/price weight, and for each needed kind the chosen paid model plus the full ranked candidate table (mirrors recommend()’s shape under "recommendations").

Return type:

dict

Examples

>>> catalog = [{"id": "m", "provider": "p", "kind": "llm",
...             "structured_output": True, "benchmarks": {"general": 80},
...             "input_per_1m": 1.0, "output_per_1m": 2.0}]
>>> rep = cloud_recommend("write a summary", catalog=catalog)
>>> rep["recommendations"]["llm"]["chosen"]["id"]
'm'
best_engine_ai_helper.recommend.parse_task(task)[source]

Turn a vague task phrase into the model kinds and benchmark axis it implies.

Returns a dict with kinds (subset of ["llm", "vlm"] in pull order), application (the benchmark axis for the text model), matched (the keywords that fired, for the report’s justification), and language (best-effort ISO 639-1 code from _detect_language(), or None). A task that mentions nothing visual still gets an LLM on the generalist axis; any vision keyword adds a VLM.

Parameters:

task (str or None) – Free-text task description. None, a blank/whitespace-only string, or text with no detectable language (symbols/digits only) falls back to a generic text-assistant profile, but logs a loud warning first: a recommendation with no clean task description carries no useful label for activity/cost monitoring or for the report’s justification, so a caller skipping it should see that reflected back.

Return type:

dict[str, Any]

Examples

>>> parse_task("write product descriptions and check photo quality")["kinds"]
['llm', 'vlm']
>>> parse_task(None)["application"]  # logs a WARNING, still resolves
'generalist'
best_engine_ai_helper.recommend.recommend(hw, catalog, task=None, *, headroom=0.5, compute=None, min_tps=15.0, backend='ollama', kinds=None, load=None)[source]

Recommend the best engine per needed kind for this hardware and task.

Parameters:

  • hw (dict) – Memory description from detect.available_memory().

  • catalog (list of dict) – Benchmark catalog from catalog.load_catalog().

  • task (str or None) – Free-text or keyword task. None means a generalist text assistant.

  • headroom (float) – Memory safety margin passed to score.effective_budget() (clamped to score.MAX_HEADROOM).

  • compute (dict or None) – Compute profile from detect.compute_profile() (accelerator + bandwidth). None disables the throughput estimate.

  • min_tps (float) – Comfort throughput floor; a fitting model below it is only picked when no comfortable one exists (and the choice is warned about).

  • backend ({'ollama', 'vllm'}) – Serving backend, so memory fit and throughput reflect what actually loads. Threaded to score.rank() and _candidate_row().

  • kinds (list of str or None) – Explicit kinds to resolve (["llm"], ["vlm"] or both), overriding the kinds inferred from task. The task text still selects the benchmark axis. Used when the caller already knows what it needs (e.g. a brief that declares kind: both).

  • load (dict or None) – Live server state from detect.server_load() (current free RAM, CPU/GPU/disk usage, already-running engines). Forwarded to score.effective_budget()/score.rank() so the recommendation reflects what else is happening on this machine right now, not only its theoretical capacity. None (the default) reproduces the load-blind behaviour exactly; also included as server_load in the returned report when given, for activity monitoring.

Returns:

A JSON-ready report: the parsed task, hardware, memory budget, and for each needed kind the chosen model plus the full ranked candidate table, with per-model fit and estimated throughput.

Return type:

dict

best_engine_ai_helper.recommend.to_markdown(report)[source]

Render a recommend() report as a Markdown document.

Parameters:

report (dict[str, Any])

Return type:

str

best_engine_ai_helper.recommend.to_markdown_cloud(report)[source]

Render a cloud_recommend() report as a Markdown section.

Parameters:

report (dict[str, Any]) – Output of cloud_recommend().

Returns:

A Markdown document, structurally parallel to to_markdown()’s per-kind sections but scored on quality-vs-price instead of memory fit.

Return type:

str

best_engine_ai_helper.recommend.write_report(report, stem)[source]

Write both a JSON and a Markdown rendering; return (md_path, json_path).

Parameters:
Return type:

tuple[Path, Path]