best_engine_ai_helper.detect module

detect — hardware detection for local model selection.

Probes the current machine for available memory (unified, VRAM, RAM) and identifies the GPU or CPU vendor. All public functions return simple scalars or dicts so callers need no knowledge of OS internals.

Probe order matters: Apple Silicon detection runs first because macOS can also report nvidia-smi output in certain VM/eGPU setups. After Apple, NVIDIA, then AMD, then a CPU-only fallback using half of available RAM as a conservative estimate of what a model loader can actually use.

Author

Warith Harchaoui <warith.harchaoui@deraison.ai>

best_engine_ai_helper.detect.available_memory()

Detect available memory for model inference.

Probes in priority order: 1. Apple Silicon unified memory (macOS with Apple chip) 2. NVIDIA VRAM via nvidia-smi 3. AMD VRAM via rocm-smi 4. System RAM via psutil (always populated)

Returns:

A dict with the following keys:

unified_gbfloat or None

Apple Silicon unified memory pool, in GB.

vram_gbfloat or None

Discrete GPU VRAM (sum of all visible GPUs), in GB.

ram_gbfloat

Total system RAM in GB. Always a positive float.

Return type:

dict[str, float | None]

Examples

>>> mem = available_memory()
>>> set(mem.keys()) == {'unified_gb', 'vram_gb', 'ram_gb'}
True
>>> mem['ram_gb'] > 0
True

best_engine_ai_helper.detect.chip_name()

Return the Apple Silicon chip name (e.g. "Apple M2 Max"), or None.

Read from system_profiler on macOS; None on other platforms or when the chip line is absent.

Return type:

str | None

best_engine_ai_helper.detect.chip_vendor()

Identify the primary compute vendor for model inference.

Checks in order: Apple Silicon, NVIDIA (nvidia-smi), AMD (rocm-smi), then falls back to ‘cpu’.

Returns:

One of: ‘apple’, ‘nvidia’, ‘amd’, ‘intel’, ‘cpu’.

Return type:

str

Examples

>>> chip_vendor() in ('apple', 'nvidia', 'amd', 'intel', 'cpu')
True

best_engine_ai_helper.detect.compute_profile()

Describe the machine’s inference accelerator and memory bandwidth.

Returns a dict with:

• accelerator: "gpu-metal" (Apple Silicon), "gpu-cuda" (NVIDIA), "gpu-rocm" (AMD), or "cpu" (no discrete accelerator detected).

• chip: the chip / GPU name when known, else None.

• bandwidth_gbs: memory bandwidth in GB/s when known, else None. This is the ceiling on decode throughput; token generation reads the whole active model from memory once per token, so tokens/s scales with it.

Bandwidth is only tabulated for Apple Silicon here (published specs); discrete-GPU bandwidth is left None because VRAM size, not bandwidth, is the binding constraint the catalog already models, and the figure varies by exact board. Callers treat a None bandwidth as “throughput not estimated”.

Return type:

dict[str, Any]

best_engine_ai_helper.detect.platform_name()

Return the current OS as a short lowercase string.

Returns:

One of: ‘darwin’, ‘linux’, ‘windows’.

Return type:

str

Examples

>>> platform_name() in ('darwin', 'linux', 'windows')
True