turbovec

A Rust vector index with Python bindings, implementing Google Research's TurboQuant quantizer (arXiv:2504.19874) and scoring it with hand-written SIMD kernels. MIT, on PyPI and crates.io. 16,771★ / 1,444 forks / 16 open issues, created 2026-03-26, last pushed 2026-08-21 — figures read 2026-09-10 from the GitHub API. By a distance the largest project this spoke holds.

Its headline is a memory claim: a 10M-document corpus at 31 GB in float32 fits in 4 GB, and searches faster than FAISS while doing it.

Why this spoke and not a database one

The compression is the product; the kernels are the reason it wins, and they are this spoke's subject at full scale:

• ARM — NEON SDOT and SMMLA dot-product instructions scoring a vector-major layout directly.
• x86 — AVX-512 VNNI for the 4-bit dot product, and vpermb driving the LUT scan that carries the short 2-bit accumulate loop. AVX2 and scalar fallbacks below that.
• Filtering happens inside the kernel at 32-vector block granularity: blocks with no allowed slot are short-circuited before any LUT lookup, so a selective allowlist skips the SIMD cost instead of paying it and discarding the result.

That is data-parallel work done with real SIMD rather than within a register, on the exact question this spoke's open questions have been asking: where does hand-written low-level code still beat the alternative?

The measurements

All cells: 100K vectors, 1K queries, k=64, median of 5 runs, against FAISS IndexPQFastScan with sub-quantizer counts matched to the same bit rate.

It wins every measured config on both architectures — and the gap between 4-bit and 2-bit is the interesting part: 3.4–3.5× where the dot-product instructions (SDOT/SMMLA, VNNI) do the work, and 20–26% where the 2-bit path falls back to a LUT scan. The advantage tracks which instruction the width can reach, not the compression ratio.

Mutation is where the difference is structural, not incremental. A single add() lands at 6.3–19.7 µs (7.6–13.9× faster than FAISS's); remove(id) is an O(1) swap-and-pop at 0.44–1.22 µs against FAISS's remove_ids at 0.19–1.02 seconds, because FAISS repacks its stored codes on every call. Six orders of magnitude, and it is a data-structure decision rather than a kernel one.

Recall is roughly a wash: calibrated TurboQuant beats FAISS IndexPQ at R@1 on three of four OpenAI cells by 0.9–2.9 points, trails by 0.7 on one, and both reach 1.0 by k=8. On GloVe d=200 — low dimension, where the method's asymptotic assumption is loosest — FAISS keeps an edge at deeper k.

Why T2 rather than T3

The author is benchmarking his own project against the incumbent, which is the shape this spoke usually tiers T3 (valyu-billion-scale-vector-index). Three things separate it: the benchmark scripts and the raw result JSONs are published, the hardware is named for every number, and the README flags an artifact in its own harness — round-trip timings that come in below the isolated dirty write, attributed to a standalone fsync inflating the separate step rather than claimed as a win. It also argues against itself on baseline choice, deliberately comparing to a stronger FAISS configuration than the TurboQuant paper used. Reproducible in principle, unreplicated in fact.

How the quantizer works

Data-oblivious, with no training phase — which is what makes online ingest possible at all. Normalize each vector to a direction and store the norm; apply one shared random rotation, after which every coordinate follows a Beta distribution converging to N(0, 1/d) regardless of the input data; then quantize with a Lloyd-Max codebook computed from that known distribution rather than from the corpus. TQ+ adds two scalars per coordinate (shift and scale) fitted once from ~1024 sampled rows, recovering up to +2.2pp R@1 where finite dimension makes the asymptotic assumption drift.

The bit-level point: 2-bit means four buckets per coordinate, 4-bit sixteen, and the codebook is mathematics rather than k-means over the data — so the expensive part of a PQ index disappears and what remains is scoring, which is where the SIMD lives.

Connections

swar · population-count · hamming-distance · valyu-billion-scale-vector-index · uops-info · cortex-a76-optimization-guide · bit-manipulation · synthesis

• cross-spoke: RAG and embedding retrieval as an application belong to ../research-wiki and ../agentic-tooling-wiki (the LangChain / LlamaIndex / Haystack / Agno drop-ins); the kernel is here