ann_router package

Subpackages

ann_router.backends package
- Submodules
- Module contents

Submodules

Module contents

ann-router — pick the right ANN vector-search backend from measured criteria.

ann-router is a brique in Warith Harchaoui’s AI Helpers suite. Like its sibling best-engine-ai-helper (which picks the best local LLM for a machine), it is a router: you describe the vector-search problem — corpus size, dimensionality, recall target, latency/memory budgets, update pattern, metadata-filtering need, hardware, persistence — and it selects, justifies, and can instantiate the appropriate engine among:

exact (brute force) · turbovec · HNSW (hnswlib) · FAISS (IVF/PQ) · Annoy · Qdrant · pgvector

instead of marrying a single library.

Importing this package is cheap and dependency-free: no engine’s optional dependency is imported at import time, so import ann_router works even with only numpy installed. A backend whose dependency is absent reports itself unavailable and the router routes around it, explaining the fallback.

Quick start

>>> import numpy as np
>>> import ann_router as ar
>>> rng = np.random.default_rng(0)
>>> vecs = rng.standard_normal((500, 64)).astype(np.float32)
>>> choice = ar.route(ar.Criteria(n_vectors=500, dim=64))
>>> choice.backend
'exact'
>>> index, choice = ar.auto_index(vecs, ar.Criteria(n_vectors=500, dim=64))
>>> ids, dists = index.search(vecs[:2], k=5)
>>> ids.shape
(2, 5)

Author: Warith Harchaoui <warith.harchaoui@deraison.ai>

class ann_router.ANNIndex(dim, metric='cosine', **kwargs)[source]

Bases: ABC

Abstract base class for every ANN backend adapter.

Subclasses wrap one engine behind a uniform surface so the router — and downstream code — can build, query, mutate, and persist an index without knowing which engine is underneath. Not every backend supports every operation; those raise NotSupported and advertise the limitation via capabilities().

Parameters:

dim (int) – Embedding dimensionality the index is built for.
metric ({"cosine", "l2", "ip"}, optional) – Distance metric. Defaults to "cosine".
**kwargs – Backend-specific build parameters (e.g. HNSW M).

Notes

Concrete constructors must not import their heavy dependency at module import time — only inside methods (or a lazily-called checker) — so that import ann_router stays cheap and dependency-free.

abstractmethod add(vectors)[source]

Append vectors, assigning them the next contiguous ids.

Parameters:: vectors (numpy.ndarray) – Shape (m, dim).
Raises:: NotSupported – If the backend is frozen after build (e.g. Annoy).
Return type:: None

abstractmethod add_with_ids(vectors, ids)[source]

Append vectors with explicit ids.

Parameters:

vectors (numpy.ndarray) – Shape (m, dim).
ids (numpy.ndarray) – Shape (m,) integer ids.

Raises:

NotSupported – If the backend cannot map external ids or is frozen.

Return type:

None

abstractmethod build(vectors, ids=None)[source]

Build the index from an initial batch of vectors.

Parameters:

vectors (numpy.ndarray) – Shape (n, dim), dtype float32 (coerced if needed).
ids (numpy.ndarray, optional) – Shape (n,) integer ids. Defaults to range(n).

Returns:

self, so calls can be chained.

Return type:

ANNIndex

abstractmethod classmethod capabilities()[source]

Return the static capability descriptor for this backend.

Returns:: What the backend can do — readable without importing its dependency.
Return type:: Capabilities

abstractmethod classmethod is_available()[source]

Return True if the backend’s dependency is importable.

Returns:: True when the engine can actually be used on this machine.
Return type:: bool

abstractmethod load(path)[source]

Load a previously save()d index from path.

Parameters:: path (str) – Source path produced by save().
Returns:: self, populated from disk.
Return type:: ANNIndex

abstractmethod remove(ids)[source]

Delete vectors by id.

Parameters:: ids (numpy.ndarray) – Shape (m,) integer ids to drop.
Raises:: NotSupported – If the backend cannot delete (e.g. Annoy) or only tombstones.
Return type:: None

abstractmethod save(path)[source]

Persist the index to path.

Parameters:: path (str) – Destination file (or directory) path.
Raises:: NotSupported – If the backend cannot serialise itself.
Return type:: None

abstractmethod search(queries, k)[source]

Return the k nearest neighbours of each query.

Parameters:

queries (numpy.ndarray) – Shape (q, dim).
k (int) – Number of neighbours per query.

Returns:

ids (numpy.ndarray) – Shape (q, k) integer neighbour ids (-1 pads missing slots).
distances (numpy.ndarray) – Shape (q, k) distances under the index metric.

Return type:

tuple[ndarray, ndarray]

class ann_router.BackendChoice(backend, rationale, config=<factory>, considered=<factory>, criteria=<factory>)[source]

Bases: object

The router’s decision: which backend, why, and how to configure it.

Parameters:

backend (str) – The selected backend name (one of BackendName).
rationale (str) – Human-readable justification naming the criteria that drove the pick — the “discussable” part of the router’s contract.
config (dict) – Backend-specific build parameters the router recommends (e.g. HNSW M/ef_construction, FAISS nlist/m).
considered (list of dict, optional) – The full ranked shortlist (each entry: name, eligible, reason, available) so callers can audit or override the decision.
criteria (dict, optional) – The echoed input criteria for provenance.

Examples

>>> choice = BackendChoice(backend="exact", rationale="tiny corpus", config={})
>>> choice.backend
'exact'

backend: Literal['exact', 'turbovec', 'hnsw', 'faiss', 'annoy', 'qdrant', 'pgvector']

config: dict[str, Any]

considered: list[dict[str, Any]]

criteria: dict[str, Any]

rationale: str

to_dict()[source]

Return a JSON-serialisable view of the choice.

Returns:: Ready to json.dumps for the CLI/API/MCP surfaces.
Return type:: dict

Examples

>>> BackendChoice("exact", "tiny", {}).to_dict()["backend"]
'exact'

exception ann_router.BackendUnavailable[source]

Bases: ImportError

Raised when a backend’s optional dependency is not installed.

Importing ann_router must never fail because faiss or turbovec are absent, so backends defer their heavy imports. When a caller actually tries to use an uninstalled backend, this actionable error names the pip install extra that fixes it.

Examples

>>> raise BackendUnavailable("faiss not installed. Run: pip install 'ann-router[faiss]'")
Traceback (most recent call last):
...
ann_router.base.BackendUnavailable: faiss not installed. Run: pip install 'ann-router[faiss]'

class ann_router.Capabilities(name, supports_add, supports_remove, supports_filter, persistent, needs_gpu, approximate, metrics, pip_extra)[source]

Bases: object

Static description of what a backend can and cannot do.

The router reads this without importing the backend’s dependency, so it can rank engines even when their libraries are absent. Each flag maps to a routing consequence documented inline.

Parameters:

name (str) – The backend identifier (matches ann_router.spec.BackendName).
supports_add (bool) – Can accept vectors after the initial build (streaming inserts).
supports_remove (bool) – Can delete vectors by id without a full rebuild.
supports_filter (bool) – Can restrict a search by structured metadata / payload.
persistent (bool) – Naturally survives process restarts (a database) as opposed to needing an explicit save/load round-trip.
needs_gpu (bool) – Requires a GPU to be worthwhile (or at all).
approximate (bool) – Returns approximate neighbours (vs. exact ground truth).
metrics (tuple of str) – The distance metrics the backend supports.
pip_extra (str) – The optional-dependency extra that installs it (empty for exact).

Examples

>>> cap = Capabilities(name="exact", supports_add=True, supports_remove=True,
...                     supports_filter=False, persistent=False, needs_gpu=False,
...                     approximate=False, metrics=("cosine", "l2", "ip"), pip_extra="")
>>> cap.approximate
False

approximate: bool

metrics: tuple[str, ...]

name: str

needs_gpu: bool

persistent: bool

pip_extra: str

supports_add: bool

supports_filter: bool

supports_remove: bool

to_dict()[source]

Return a JSON-serialisable view of the descriptor.

Returns:: One key per flag.
Return type:: dict

Examples

>>> from ann_router.backends.exact import ExactIndex
>>> ExactIndex.capabilities().to_dict()["name"]
'exact'

class ann_router.Criteria(n_vectors, dim, target_recall=0.95, latency_budget_ms=10.0, memory_budget_gb=None, dynamic=False, metadata_filtering=False, hardware='cpu', persistence=False, batch_queries=False, metric='cosine', extra=<factory>)[source]

Bases: object

Measured description of an approximate-nearest-neighbour problem.

Every attribute is a decision lever. The two mandatory ones — n_vectors and dim — set the scale; the rest refine the choice and each carries a house default that matches the most common single-machine RAG workload.

Parameters:

n_vectors (int) – Number of vectors in (or expected in) the corpus. Below policy.EXACT_MAX_N an exact brute-force scan is already instant and perfectly accurate, so approximation is pointless.
dim (int) – Embedding dimensionality (e.g. 384, 768, 1536, 2048).
target_recall (float, optional) – Desired recall@k against the exact ground truth, in (0, 1]. Defaults to 0.95. High values push toward exact/HNSW and away from aggressively quantised indexes.
latency_budget_ms (float, optional) – Per-query latency budget in milliseconds. Defaults to 10.0 (an interactive budget). Drives the exact→ANN crossover.
memory_budget_gb (float or None, optional) – Soft cap on index RAM in gibibytes. None (default) means “not a constraint”. A tight budget favours quantised (turbovec/FAISS-PQ) or memory-mapped (Annoy) backends.
dynamic (bool, optional) – True when the corpus receives frequent adds/removes and must not be rebuilt. Defaults to False (static corpus). Frequent updates favour turbovec (O(1) add/remove); graph indexes like HNSW handle deletes poorly (tombstones only).
metadata_filtering (bool, optional) – True when queries must be filtered by structured metadata (payload/where clauses). Defaults to False. Favours Qdrant/pgvector.
hardware ({"cpu", "gpu", "apple_silicon"}, optional) – The accelerator available. Defaults to "cpu". Use ann_router.detect.detect_hardware() to fill this in automatically.
persistence (bool, optional) – True when the index must survive process restarts / live in a database. Defaults to False. Favours Qdrant/pgvector, or a save/load round-trip for the in-memory engines.
batch_queries (bool, optional) – True when queries arrive in large batches (throughput regime rather than interactive). Defaults to False. Combined with gpu this favours FAISS.
metric ({"cosine", "l2", "ip"}, optional) – Distance metric the vectors are embedded for. Defaults to "cosine".
extra (dict, optional) – Free-form escape hatch for backend-specific hints (e.g. an existing DB DSN). Never required by the core policy.

Examples

>>> c = Criteria(n_vectors=5_000, dim=768)
>>> c.dim
768
>>> c.target_recall            # house default
0.95
>>> Criteria(n_vectors=2_000_000, dim=1536, dynamic=True).dynamic
True

Notes

The dataclass is intentionally plain (no validation on construction beyond validate(), which the router calls) so it round-trips cleanly through JSON for the CLI / API / MCP surfaces via to_dict() / from_dict().

batch_queries: bool = False

dim: int

dynamic: bool = False

extra: dict[str, Any]

classmethod from_dict(data)[source]

Rebuild a Criteria from a (possibly partial) mapping.

Parameters:: data (dict) – Keys matching the dataclass fields. n_vectors and dim are required; unknown keys are ignored so the CLI/API can pass through loosely.
Returns:: The reconstructed criteria.
Return type:: Criteria

Examples

>>> Criteria.from_dict({"n_vectors": 100, "dim": 8, "dynamic": True}).dynamic
True

hardware: Literal['cpu', 'gpu', 'apple_silicon'] = 'cpu'

latency_budget_ms: float = 10.0

memory_budget_gb: float | None = None

metadata_filtering: bool = False

metric: Literal['cosine', 'l2', 'ip'] = 'cosine'

n_vectors: int

persistence: bool = False

target_recall: float = 0.95

to_dict()[source]

Return a JSON-serialisable view of the criteria.

Returns:: One key per attribute; safe to json.dumps.
Return type:: dict

Examples

>>> Criteria(n_vectors=100, dim=8).to_dict()["n_vectors"]
100

validate()[source]

Assert the criteria are internally consistent.

Raises:: ValueError – If a numeric field is out of its valid range.
Return type:: None

Examples

>>> Criteria(n_vectors=100, dim=8).validate() is None
True
>>> try:
...     Criteria(n_vectors=-1, dim=8).validate()
... except ValueError as exc:
...     print("rejected")
rejected

exception ann_router.NotSupported[source]

Bases: RuntimeError

Raised when a backend genuinely cannot perform a requested operation.

This is distinct from “not yet implemented”: it flags a fundamental limitation of the engine (e.g. Annoy is frozen after build and cannot add or remove). The router uses the capability descriptor to avoid routing a dynamic workload to such a backend, but the guard here is the last line of defence for callers who instantiate a backend directly.

Examples

>>> raise NotSupported("annoy: remove() unsupported (frozen index)")
Traceback (most recent call last):
...
ann_router.base.NotSupported: annoy: remove() unsupported (frozen index)

ann_router.all_capabilities()[source]

Return the capability descriptor of every registered backend.

Returns:: {name: Capabilities} — readable even for uninstalled backends.
Return type:: dict

Examples

>>> all_capabilities()["annoy"].supports_remove
False

ann_router.auto_index(vectors, criteria, ids=None, thresholds=None)[source]

Route the criteria, instantiate the winning backend, and build the index.

Parameters:

vectors (numpy.ndarray) – Corpus of shape (n, dim).
criteria (Criteria) – The problem description. If its n_vectors/dim disagree with vectors the array wins (the criteria are advisory for routing).
ids (numpy.ndarray, optional) – Explicit ids of shape (n,); defaults to range(n).
thresholds (dict, optional) – Policy threshold overrides.

Returns:

index (ANNIndex) – A built, queryable index of the chosen backend.
choice (BackendChoice) – The routing decision (so the caller can inspect/log the rationale).

Return type:

tuple[ANNIndex, BackendChoice]

Examples

>>> rng = np.random.default_rng(0)
>>> vecs = rng.standard_normal((500, 32)).astype(np.float32)
>>> idx, choice = auto_index(vecs, Criteria(n_vectors=500, dim=32))
>>> choice.backend
'exact'
>>> ids, dists = idx.search(vecs[:1], k=5)
>>> ids.shape
(1, 5)

ann_router.available_backends()[source]

Return the names of backends whose dependency is importable here.

Returns:: Sorted-by-preference subset of BACKENDS that can actually run.
Return type:: list of str

Examples

>>> "exact" in available_backends()   # numpy is always present
True

ann_router.backend_catalog()[source]

Return the prose backend catalog from backends.yaml.

Returns:: One entry per backend: name, summary, when, pip_extra.
Return type:: list of dict

Examples

>>> {b["name"] for b in backend_catalog()} >= {"exact", "hnsw", "faiss"}
True

ann_router.detect_hardware()[source]

Classify the local accelerator into the router’s three-way taxonomy.

Returns:: "gpu" when a CUDA GPU is found (it dominates the batch regime), else "apple_silicon" on M-series Macs, else "cpu".
Return type:: {“gpu”, “apple_silicon”, “cpu”}

Examples

>>> detect_hardware() in {"gpu", "apple_silicon", "cpu"}
True

ann_router.get_backend(name)[source]

Return the adapter class registered under name.

Parameters:: name (str) – A backend identifier (see ann_router.spec.BackendName).
Returns:: The adapter class (not an instance).
Return type:: type[ANNIndex]
Raises:: KeyError – If name is not a known backend.

Examples

>>> get_backend("exact").__name__
'ExactIndex'

ann_router.hardware_profiles()[source]

Return the accelerator profiles from hardware.yaml.

Returns:: One entry per hardware class: hardware, summary, unlocks.
Return type:: list of dict

Examples

>>> sorted(p["hardware"] for p in hardware_profiles())
['apple_silicon', 'cpu', 'gpu']

ann_router.hardware_report()[source]

Return a small JSON-ready dict describing the host for the CLI/API.

Returns:: os, machine, cpu_count, workers and the classified hardware label.
Return type:: dict

Examples

>>> report = hardware_report()
>>> report["hardware"] in {"gpu", "apple_silicon", "cpu"}
True

ann_router.policy_thresholds(path=None)[source]

Return the policy thresholds, optionally overridden from a YAML file.

Resolution order: the in-code ann_router.policy.THRESHOLDS defaults, overlaid with the packaged policy.yaml, overlaid with an external file (path argument or the ANN_ROUTER_POLICY env var) if present.

Parameters:: path (str, optional) – Path to an external policy.yaml to overlay. Falls back to the ANN_ROUTER_POLICY environment variable, then to no override.
Returns:: The merged {THRESHOLD_NAME: value} mapping, ready to pass to ann_router.policy.rank_backends().
Return type:: dict

Examples

>>> t = policy_thresholds()
>>> t["EXACT_MAX_N"]
1000

ann_router.rank_backends(c, thresholds=None)[source]

Return the ordered, justified backend shortlist for the criteria.

This is the pure heart of the router: it applies every rule in priority order and returns one row per eligible rule, each carrying the backend name and its rationale. Availability and the final pick are decided in ann_router.router.route(), keeping this function side-effect-free.

Parameters:

c (Criteria) – The measured problem description.
thresholds (dict, optional) – Overrides for THRESHOLDS (tunable policy). Missing keys fall back to the module defaults.

Returns:

[{"backend": str, "reason": str}, ...] in priority order — the first element is the policy’s preferred choice before availability is applied.

Return type:

list of dict

Examples

>>> rank_backends(Criteria(n_vectors=500, dim=128))[0]["backend"]
'exact'
>>> # dynamic corpus at the house default target_recall=0.95: turbovec's
>>> # calibrated benchmarks undershoot that recall, so HNSW wins instead.
>>> rank_backends(Criteria(n_vectors=500_000, dim=768, dynamic=True))[0]["backend"]
'hnsw'
>>> # same dynamic corpus, recall relaxed below HIGH_RECALL: turbovec wins.
>>> rank_backends(Criteria(n_vectors=500_000, dim=768, dynamic=True,
...                        target_recall=0.85))[0]["backend"]
'turbovec'
>>> rank_backends(Criteria(n_vectors=200_000, dim=768,
...                        metadata_filtering=True))[0]["backend"]
'qdrant'

ann_router.route(c, thresholds=None)[source]

Select an available backend for the criteria and justify the choice.

Parameters:

c (Criteria) – The measured problem description.
thresholds (dict, optional) – Overrides for the policy thresholds (tunable). See ann_router.policy.THRESHOLDS.

Returns:

The chosen backend, its rationale, recommended config, and the full considered shortlist (each entry flagged eligible/available/chosen).

Return type:

BackendChoice

Examples

>>> route(Criteria(n_vectors=500, dim=64)).backend
'exact'
>>> choice = route(Criteria(n_vectors=500_000, dim=768, metadata_filtering=True))
>>> choice.backend in {"qdrant", "pgvector", "hnsw", "exact", "turbovec"}
True

ann_router.to_markdown(choice)[source]

Render a routing decision as a human-readable Markdown report.

Parameters:: choice (BackendChoice) – A decision produced by route().
Returns:: Markdown with the pick, the rationale, and the considered table.
Return type:: str

Examples

>>> md = to_markdown(route(Criteria(n_vectors=500, dim=64)))
>>> md.splitlines()[0]
'# ann-router decision: `exact`'