ann_router.spec module

Typed problem criteria that drive ANN-backend selection.

This module holds the input side of the router: a single, explicit Criteria dataclass describing the vector-search problem to solve (corpus size, dimensionality, recall target, latency/memory budgets, update pattern, metadata-filtering need, hardware, persistence), plus the BackendChoice result the router returns.

The design goal — mirrored from the best-engine-ai-helper sibling — is that the choice is measured and discussable: every field here is a lever that can flip the decision, and the router explains which levers mattered.

Consumes: nothing (pure data). Produces: Criteria and BackendChoice values consumed by ann_router.policy and ann_router.router.

Author: Warith Harchaoui <warith.harchaoui@deraison.ai>

class ann_router.spec.BackendChoice(backend, rationale, config=<factory>, considered=<factory>, criteria=<factory>)

Bases: object

The router’s decision: which backend, why, and how to configure it.

Parameters:

• backend (str) – The selected backend name (one of BackendName).

• rationale (str) – Human-readable justification naming the criteria that drove the pick — the “discussable” part of the router’s contract.

• config (dict) – Backend-specific build parameters the router recommends (e.g. HNSW M/ef_construction, FAISS nlist/m).

• considered (list of dict, optional) – The full ranked shortlist (each entry: name, eligible, reason, available) so callers can audit or override the decision.

• criteria (dict, optional) – The echoed input criteria for provenance.

Examples

>>> choice = BackendChoice(backend="exact", rationale="tiny corpus", config={})
>>> choice.backend
'exact'

backend: Literal['exact', 'turbovec', 'hnsw', 'faiss', 'annoy', 'qdrant', 'pgvector']

config: dict[str, Any]

considered: list[dict[str, Any]]

criteria: dict[str, Any]

rationale: str

to_dict()

Return a JSON-serialisable view of the choice.

Returns:

Ready to json.dumps for the CLI/API/MCP surfaces.

Return type:

dict

Examples

>>> BackendChoice("exact", "tiny", {}).to_dict()["backend"]
'exact'

class ann_router.spec.Criteria(n_vectors, dim, target_recall=0.95, latency_budget_ms=10.0, memory_budget_gb=None, dynamic=False, metadata_filtering=False, hardware='cpu', persistence=False, batch_queries=False, metric='cosine', extra=<factory>)

Bases: object

Measured description of an approximate-nearest-neighbour problem.

Every attribute is a decision lever. The two mandatory ones — n_vectors and dim — set the scale; the rest refine the choice and each carries a house default that matches the most common single-machine RAG workload.

Parameters:

• n_vectors (int) – Number of vectors in (or expected in) the corpus. Below policy.EXACT_MAX_N an exact brute-force scan is already instant and perfectly accurate, so approximation is pointless.

• dim (int) – Embedding dimensionality (e.g. 384, 768, 1536, 2048).

• target_recall (float, optional) – Desired recall@k against the exact ground truth, in (0, 1]. Defaults to 0.95. High values push toward exact/HNSW and away from aggressively quantised indexes.

• latency_budget_ms (float, optional) – Per-query latency budget in milliseconds. Defaults to 10.0 (an interactive budget). Drives the exact→ANN crossover.

• memory_budget_gb (float or None, optional) – Soft cap on index RAM in gibibytes. None (default) means “not a constraint”. A tight budget favours quantised (turbovec/FAISS-PQ) or memory-mapped (Annoy) backends.

• dynamic (bool, optional) – True when the corpus receives frequent adds/removes and must not be rebuilt. Defaults to False (static corpus). Frequent updates favour turbovec (O(1) add/remove); graph indexes like HNSW handle deletes poorly (tombstones only).

• metadata_filtering (bool, optional) – True when queries must be filtered by structured metadata (payload/where clauses). Defaults to False. Favours Qdrant/pgvector.

• hardware ({"cpu", "gpu", "apple_silicon"}, optional) – The accelerator available. Defaults to "cpu". Use ann_router.detect.detect_hardware() to fill this in automatically.

• persistence (bool, optional) – True when the index must survive process restarts / live in a database. Defaults to False. Favours Qdrant/pgvector, or a save/load round-trip for the in-memory engines.

• batch_queries (bool, optional) – True when queries arrive in large batches (throughput regime rather than interactive). Defaults to False. Combined with gpu this favours FAISS.

• metric ({"cosine", "l2", "ip"}, optional) – Distance metric the vectors are embedded for. Defaults to "cosine".

• extra (dict, optional) – Free-form escape hatch for backend-specific hints (e.g. an existing DB DSN). Never required by the core policy.

Examples

>>> c = Criteria(n_vectors=5_000, dim=768)
>>> c.dim
768
>>> c.target_recall            # house default
0.95
>>> Criteria(n_vectors=2_000_000, dim=1536, dynamic=True).dynamic
True

Notes

The dataclass is intentionally plain (no validation on construction beyond validate(), which the router calls) so it round-trips cleanly through JSON for the CLI / API / MCP surfaces via to_dict() / from_dict().

batch_queries: bool = False

dim: int

dynamic: bool = False

extra: dict[str, Any]

classmethod from_dict(data)

Rebuild a Criteria from a (possibly partial) mapping.

Parameters:

data (dict) – Keys matching the dataclass fields. n_vectors and dim are required; unknown keys are ignored so the CLI/API can pass through loosely.

Returns:

The reconstructed criteria.

Return type:

Criteria

Examples

>>> Criteria.from_dict({"n_vectors": 100, "dim": 8, "dynamic": True}).dynamic
True

hardware: Literal['cpu', 'gpu', 'apple_silicon'] = 'cpu'

latency_budget_ms: float = 10.0

memory_budget_gb: float | None = None

metadata_filtering: bool = False

metric: Literal['cosine', 'l2', 'ip'] = 'cosine'

n_vectors: int

persistence: bool = False

target_recall: float = 0.95

to_dict()

Return a JSON-serialisable view of the criteria.

Returns:

One key per attribute; safe to json.dumps.

Return type:

dict

Examples

>>> Criteria(n_vectors=100, dim=8).to_dict()["n_vectors"]
100

validate()

Assert the criteria are internally consistent.

Raises:

ValueError – If a numeric field is out of its valid range.

Return type:

None

Examples

>>> Criteria(n_vectors=100, dim=8).validate() is None
True
>>> try:
...     Criteria(n_vectors=-1, dim=8).validate()
... except ValueError as exc:
...     print("rejected")
rejected