GitHub - RyanCodrai/turbovec: A vector index built on TurboQuant, written in Rust with Python bindings

Add vectors, they're indexed — no train step, no parameter tuning, no rebuilds as the corpus grows. Fast SIMD search. Hand-written kernels — NEON SDOT/SMMLA on ARM, AVX-512 VNNI and vpermb on x86, with AVX2 and scalar fallbacks — beat FAISS IndexPQFastScan in every measured config, averaging 3.4× at 4-bit and 23% at 2-bit across the eight cells of each width, on both architectures. Incremental saves. sync(path) persists just what changed since the last sync — one fsync per call, crash-safe at any byte, and a removal or a small append costs milliseconds however large the index. write/load stay for whole-file snapshots. Filter at search time. Pass an id allowlist (or a slot bitmask) to search() and the kernel honours it directly. You always get up to k results from the allowed set — no over-fetching, no recall hit on selective filters. Pure local. No managed service, no data leaving your machine or VPC. Pair with any open-source embedding model for a fully air-gapped RAG stack. Building RAG where privacy, memory, or latency matters? You're in the right place.

Need stable ids that survive deletes? Use IdMapIndex: import numpy as np from turbovec import IdMapIndex index = IdMapIndex(dim=1536, bit_width=4) index.add_with_ids(vectors, np.array([1001, 1002, 1003], dtype=np.uint64)) scores, ids = index.search(query, k=10) # ids are your uint64 external ids index.remove(1002) # O(1) by id index.write("my_index.tvim") loaded = IdMapIndex.load("my_index.tvim") index.sync("my_index.tvim") # durable incremental save, ids included Hybrid retrieval (filtered search) Restrict results to a candidate set produced by another system (SQL, BM25, ACL, time window, …): import numpy as np from turbovec import IdMapIndex idx = IdMapIndex(dim=1536, bit_width=4) idx.add_with_ids(vectors, ids) # Stage 1: external system narrows to candidate ids. allowed = np.array(db.execute("SELECT id FROM docs WHERE tenant=?", (t,)).fetchall(), dtype=np.uint64) # Stage 2: dense rerank within the candidate set. scores, ids = idx.search(query, k=10, allowlist=allowed) Filtering happens inside the SIMD kernel at 32-vector block granularity: blocks with no allowed slots are short-circuited before any LUT lookup or scoring work, and individual non-allowed slots inside scored blocks are dropped at heap-insert. Selective allowlists (small fraction of the index allowed) therefore avoid most of the SIMD cost rather than paying it and discarding the result afterwards. The output length is min(k, n_allowed), where n_allowed counts distinct allowed vectors — when fewer vectors are allowed than k you get exactly that many results rather than padded fallbacks. See docs/api.md for the full reference. Framework integrations Drop-in replacements for the in-tree reference vector / document stores in each framework. Same public surface, same persistence semantics, same retriever and pipeline wiring — swap the import and keep your pipeline. LangChain — pip install turbovec[langchain] · replaces langchain_core.vectorstores.InMemoryVectorStore LlamaIndex — pip install turbovec[llama-index] · replaces llama_index.core.vector_stores.SimpleVectorStore Haystack — pip install turbovec[haystack] · replaces haystack.document_stores.in_memory.InMemoryDocumentStore Agno — pip install turbovec[agno] · replaces agno.vectordb.lancedb.LanceDb Rust cargo add turbovec use turbovec::TurboQuantIndex; let mut index = TurboQuantIndex::new(1536, 4).unwrap(); index.add(&vectors); let results = index.search(&queries, 10); index.write("index.tv").unwrap(); let loaded = TurboQuantIndex::load("index.tv").unwrap(); For stable external ids that survive deletes: use turbovec::IdMapIndex; let mut index = IdMapIndex::new(1536, 4).unwrap(); index.add_with_ids(&vectors, &[1001, 1002, 1003]).unwrap(); let (scores, ids) = index.search(&queries, 10); index.remove(1002); index.write("index.tvim").unwrap(); let loaded = IdMapIndex::load("index.tvim").unwrap(); Recall TurboQuant vs FAISS IndexPQ (LUT256, nbits=8) — the paper's Section 4.4 baseline. 100K vectors, k=64. FAISS PQ sub-quantizer counts sized to match TurboQuant's bit rate (m=d/4 at 2-bit, m=d/2 at 4-bit). The charts plot calibrated TurboQuant (TQ+). Across OpenAI d=1536 and d=3072, TQ+ beats FAISS at R@1 on three of four cells (by 0.9–2.9 points; d=1536 4-bit trails by 0.7), and both reach 1.0 by k=8 (≥0.997 already at k≤4). GloVe d=200 is the harder regime — at low dim the asymptotic Beta assumption is looser. TQ+ lands ahead of FAISS at R@1 at both bit widths (+1.9 at 4-bit, +0.8 at 2-bit), with FAISS keeping a slim edge at 2-bit from k≈8. Uncalibrated numbers are in the JSONs (tq_recalls). A note on baselines. We compare against FAISS IndexPQ (LUT256, nbits=8, float32 LUT) because it's the default production-grade PQ most users would reach for. This is a stronger baseline than the custom u8-LUT PQ in the TurboQuant paper — FAISS uses a higher-precision LUT at scoring time and k-means++ for codebook training.

Insertion measures per-vector add() latency on a warm, populated index (built untimed) at n=1 — a single-vector add() — and n=100 — a 100-vector batch, showing how far batching amortizes the per-call overhead — against add() into the trained, populated FAISS IndexPQFastScan (training untimed). A single add() lands in 6.3–19.7 µs depending on the cell (7.6–13.9× faster than a FAISS single add), and a 100-vector batch amortizes TurboQuant to 4.6–16.3 µs/vector (4.6–15.1× faster than the same batch into FAISS). Removal measures per-op remove-by-id latency at n=1 (the steady per-op rate over 1000 removes) and n=100 (the first 100 removes on a fresh index): IdMapIndex.remove(id) — O(1) swap-and-pop plus the id-map bookkeeping — lands at 0.44–1.22 µs and 0.59–1.37 µs per op across the cells. The FAISS column is the same user-visible operation, remove_ids on an IndexIDMap over IndexPQFastScan, which repacks the stored codes on every call: 0.19–1.02 s per single remove at 100K, with cost doubling alongside code size — which is why the removal charts use a log-scale axis. Charts show the single-threaded cells (RAYON_NUM_THREADS=1); the _mt cells are measured too and match at n=1, since a single add is serial.