best_engine_ai_helper.sources.apxml module

sources.apxml — open-weight model specs from the ApXML LLM directory.

ApXML (https://apxml.com/models) publishes a directory of open-weight LLMs and VLMs with the facts this project selects on: parameter count, modality, architecture (dense vs MoE), context length, licence, the HuggingFace weights URL, and — most valuable here — ApXML’s own peak inference VRAM estimate at Q4/Q8/FP16. Those VRAM figures share the semantics of a catalog entry’s ram_gb (weights plus a moderate KV cache), so they feed the fit decision directly.

The pages are server-rendered React (Next.js). The model data is not exposed as a REST endpoint; it is streamed inside self.__next_f.push([1, "<chunk>"]) script calls. We reconstruct the payload from those chunks and brace-match the model object out of it — no headless browser required.

One deliberate gap: the numeric benchmark scores (LiveBench, Aider, MMLU-Pro, GPQA, …) are fetched by a client-side call and are absent from the static HTML, so this adapter never synthesises benchmark axes. It contributes spec and memory-fit metadata; leaderboard sources contribute the scores.

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.sources.apxml.fetch_open_weight_models(session=None, timeout=30.0, limit=None)[source]

Fetch and normalize every open-weight model in the ApXML directory.

Network-bound: one request for the directory plus one per model. Pages that fail to fetch or parse are skipped rather than aborting the whole refresh, so a single dead link never empties the feed.

Parameters:

session (requests.Session or None) – Optional shared session for connection reuse.
timeout (float) – Per-request timeout in seconds.
limit (int or None) – Stop after this many models (useful for smoke tests); None fetches all.

Returns:

Normalized spec dicts as returned by parse_model_page().

Return type:

list[dict[str, Any]]

Examples

>>> models = fetch_open_weight_models(limit=1)
>>> models[0]['kind'] in ('llm', 'vlm')
True

best_engine_ai_helper.sources.apxml.parse_directory_slugs(html)[source]

Extract the open-weight model slugs listed on a directory page.

Parameters:: html (str) – Raw HTML of the ApXML models directory.
Returns:: Unique model slugs in first-seen order, minus non-model links.
Return type:: list[str]

Examples

>>> parse_directory_slugs('<a href="/models/qwen3-8b">')
['qwen3-8b']

best_engine_ai_helper.sources.apxml.parse_model_page(html)[source]

Parse one ApXML model detail page into a normalized spec dict.

Parameters:: html (str) – Raw HTML of an ApXML /models/<slug> page.
Returns:: Normalized fields (see below), or None when the page holds no model object. kind is "vlm" for multimodal models, else "llm". ram_gb mirrors the Q4 VRAM estimate — the quant this project pulls by default — so it drops straight into the fit check.
Return type:: dict[str, Any] or None

Examples

>>> spec = parse_model_page(open('fixture.html').read())
>>> spec['kind'] in ('llm', 'vlm')
True