ann_router.policy module

The selection policy — pure, tunable decision math (no side effects).

This is the score.py analogue from best-engine-ai-helper: it holds the named, versioned thresholds and the branch predicates that turn a Criteria into an ordered, justified shortlist of backends. It imports nothing heavy and touches no engine, so it is fully unit -testable on its own — the router layer (router.py) is what consults availability and instantiates the winner.

The decision tree reproduced here is the suite’s own documented policy (and the one already shipped, in two-branch form, inside the roitelet prototype):

  1. n < EXACT_MAX_N (scaled by latency_budget_ms, see effective_exact_max_n()) -> exact (approximation is pointless)

  2. else frequent updates + target_recall < HIGH_RECALL -> turbovec (O(1) add/remove)

  3. else very large volume + GPU/batch -> faiss (IVF+PQ, scales)

  4. else persistence + metadata filters -> qdrant / pgvector

  5. else read-only + tight memory -> annoy (frozen, mmap, lean)

  6. else stable in-memory corpus (the default) -> hnsw (best recall/latency)

ScaNN was dropped: no Apple-Silicon wheel exists, and the project has definitively abandoned it as a supported backend (see CHANGELOG.md).

Consumes: ann_router.spec. Produces: rank_backends(), plus the THRESHOLDS constants.

Author: Warith Harchaoui <warith.harchaoui@deraison.ai>

class ann_router.policy.Rule(backend, eligible, reason)[source]

Bases: object

One branch of the decision tree: a backend, a guard, and its rationale.

Parameters:

  • backend (str) – The backend this rule selects when eligible.

  • eligible (Callable[[Criteria, dict], bool]) – Predicate deciding whether the rule fires for given criteria/thresholds.

  • reason (Callable[[Criteria, dict], str]) – Produces the human-readable justification when the rule fires.

backend: str
eligible: Callable[[Criteria, dict], bool]
reason: Callable[[Criteria, dict], str]
ann_router.policy.effective_exact_max_n(c, t)[source]

Scale EXACT_MAX_N by how loose/tight the caller’s latency budget is.

EXACT_MAX_N is calibrated at a reference budget (LATENCY_REFERENCE_MS); a brute-force scan is one O(n * dim) matmul, so its cost is ~linear in n for fixed dim and a budget k times looser (or tighter) than the reference affords k times the corpus size. This is the one place Criteria.latency_budget_ms is load-bearing in the decision tree — every rule below compares against this scaled value, never the raw threshold, so the tree stays gapless.

Parameters:

  • c (Criteria) – The problem description (reads latency_budget_ms).

  • t (dict) – The merged thresholds (reads EXACT_MAX_N, LATENCY_REFERENCE_MS).

Returns:

The latency-adjusted exact/ANN crossover.

Return type:

float

Examples

>>> t = {"EXACT_MAX_N": 10_000, "LATENCY_REFERENCE_MS": 10.0}
>>> effective_exact_max_n(Criteria(n_vectors=1, dim=8, latency_budget_ms=10.0), t)
10000.0
>>> effective_exact_max_n(Criteria(n_vectors=1, dim=8, latency_budget_ms=1.0), t)
1000.0
ann_router.policy.rank_backends(c, thresholds=None)[source]

Return the ordered, justified backend shortlist for the criteria.

This is the pure heart of the router: it applies every rule in priority order and returns one row per eligible rule, each carrying the backend name and its rationale. Availability and the final pick are decided in ann_router.router.route(), keeping this function side-effect-free.

Parameters:

  • c (Criteria) – The measured problem description.

  • thresholds (dict, optional) – Overrides for THRESHOLDS (tunable policy). Missing keys fall back to the module defaults.

Returns:

[{"backend": str, "reason": str}, ...] in priority order — the first element is the policy’s preferred choice before availability is applied.

Return type:

list of dict

Examples

>>> rank_backends(Criteria(n_vectors=500, dim=128))[0]["backend"]
'exact'
>>> # dynamic corpus at the house default target_recall=0.95: turbovec's
>>> # calibrated benchmarks undershoot that recall, so HNSW wins instead.
>>> rank_backends(Criteria(n_vectors=500_000, dim=768, dynamic=True))[0]["backend"]
'hnsw'
>>> # same dynamic corpus, recall relaxed below HIGH_RECALL: turbovec wins.
>>> rank_backends(Criteria(n_vectors=500_000, dim=768, dynamic=True,
...                        target_recall=0.85))[0]["backend"]
'turbovec'
>>> rank_backends(Criteria(n_vectors=200_000, dim=768,
...                        metadata_filtering=True))[0]["backend"]
'qdrant'
ann_router.policy.raw_memory_gb(c)[source]

Estimate the RAM a full-precision (float32) flat index would need.

Parameters:

c (Criteria) – The problem description.

Returns:

Approximate gibibytes for n_vectors * dim float32 values.

Return type:

float

Examples

>>> round(raw_memory_gb(Criteria(n_vectors=1_000_000, dim=768)), 2)
2.86