Your embeddings are 32x bigger than they need to be. Maybe.
Every OpenAI embedding you store in Postgres is 6 KB. A hundred million documents is 600 GB of vectors before you build a single index, and an HNSW index wants to live in RAM. That is how a RAG prototype turns into a database bill nobody planned for.
The fix sounds simple: store fewer bits per number. pgvector already has halfvec (16 bits) and binary_quantize (1 bit). VectorChord ships RaBitQ. TurboQuant from Google Research is the new name everyone mentions. Each one promises a smaller index with "minimal recall loss".
I spent most of this year building TurboQuant and RaBitQ into Postgres as a Rust extension. The most useful thing I learned is this: whether compression is safe depends on your embedding model, not on the codec. Here is what that looks like in numbers, the mistake that cost me months, and how to check your own data in three minutes.
The same trick, three very different results
I took pgvector's recommended setup for binary quantization (search 1-bit codes, then rerank the best 100 with the full vectors) and ran it on three public datasets. Recall@10 means: of the 10 results exact search would return, how many did we get back.
| Dataset | Recall@10, binary + rerank |
|---|---|
| OpenAI ada-002 embeddings (DBpedia) | 0.992 |
| GloVe word vectors | 0.348 |
| SIFT image descriptors | 0.055 |
On OpenAI embeddings it is almost free: a 32x smaller search for 1% of the results. On SIFT it returns the wrong answer 95% of the time. SIFT values are all positive, so one sign bit per number says almost nothing.
Your embeddings are not DBpedia. A vendor benchmark on someone else's vectors tells you very little about yours, and nothing in your monitoring will flag that 20% of your search results quietly got worse. You find out when users stop trusting the answers.
What each option costs, on real embeddings
On 100,000 OpenAI ada-002 embeddings, exhaustive search over the codes (so this is the compression alone, before any index), averaged over three random seeds:
| Option | Bytes per vector | Smaller by | Recall@10 |
|---|---:|---:|---:|
| float32 | 6,144 | 1x | 1.000 |
| halfvec | 3,072 | 2x | 0.999 |
| RaBitQ 4-bit | 776 | 8x | 0.972 |
| TurboQuant 4-bit | 772 | 8x | 0.953 |
| RaBitQ 1-bit | 200 | 31x | 0.834 |
| binary_quantize | 192 | 32x | 0.707 |
| RaBitQ 1-bit, rerank top 100 | 200 searched | 31x | 0.999 |
| TurboQuant 3-bit, rerank with 8-bit | 580 searched | 11x | 0.996 |
Three practical takeaways:
- halfvec is free. If your index fits in memory at half size, stop here.
- At one bit, the codec matters. RaBitQ keeps 13 more points of recall than a plain sign bit for 8 extra bytes per row, because it centres the data and corrects its own estimate.
- Search small, rerank precise. The 100 best candidates from tiny codes almost always contain the true top 10. Reranking them gets you back to 0.99+ while the part you search stays 10 to 30 times smaller.
In Postgres, today
You don't need to write an extension any more. Same 100,000 embeddings, real indexes in Postgres 18, each at the first setting that reaches 0.95 recall:
| Index | Size | Recall@10 | p50 latency |
|---|---:|---:|---:|
| pgvector HNSW, float32 | 781 MiB | 0.959 | 2.34 ms |
| pgvector HNSW, halfvec | 391 MiB | 0.957 | 2.31 ms |
| pgvector HNSW, binary + rerank | 48 MiB | 0.963 | 2.83 ms |
| VectorChord, default | 812 MiB | 0.981 | 2.58 ms |
| VectorChord, rerank_in_table | 31 MiB | 0.957 | 1.69 ms |
Two things I would not have guessed:
- VectorChord's default index is bigger than plain HNSW. It keeps a second copy of every vector inside the index for reranking. One option, rerank_in_table = true, stores only the compressed codes: 25x smaller, and the fastest index in this test. VectorChord warns that this mode gets slow when the table has to come from disk; here the table fit in memory, so test it at your real size.
- An HNSW graph costs more than the codes in it. The binary index uses about 500 bytes per row for 192 bytes of codes. The rest is neighbour lists. It is still 16x smaller than float32.
The mistake that cost me months
TurboQuant comes in two flavours. One spends all its bits on the codes. The other spends one bit on a correction that makes inner products mathematically unbiased. Unbiased sounded strictly better, so that is what I shipped in my extension.
It isn't, for search. Same total bits per number, 100,000 GloVe vectors:
| Bits | With the correction bit | All bits on the codes |
|---|---:|---:|
| 5 | 0.806 | 0.925 |
| 4 | 0.667 | 0.867 |
| 3 | 0.478 | 0.753 |
Up to 28 points of recall lost. Search only needs the right order of results for one query, and the uncorrected codes shrink every score by the same factor, which never changes the order. The correction removes that harmless shrink and adds noise, and noise is exactly what reorders neighbours.
A second detail was worth up to 11 more points: after decoding, rescale each vector back to unit length. The decoded lengths wobble by about 1% from row to row, and that wobble leaks straight into the ranking.
Both bugs looked like "compression is just lossy". Neither was. When your compressed recall is far below the paper, suspect the estimator before you blame the bits.
Which one should you use?
- Under a few million vectors: keep float32 or halfvec. Memory is cheap at that size.
- Memory is the bill, recall must stay high: a 1-bit index with a rerank from the table. pgvector's binary + rerank or VectorChord with rerank_in_table, whichever wins on your data.
- Constant inserts, data that drifts: TurboQuant. Nothing to train, a vector is compressed the moment it arrives.
- Very large, slow-changing collections: RaBitQ with IVF (VectorChord), with an error bound that tells you how many candidates to rerank.
- Changing embedding models: measure again. Your old compression may not survive the new model.
Check your own data in three minutes
I turned all of this into shrinkvec, an open-source tool. Point it at a table and it samples your vectors, computes the exact answers, measures every option above, builds the real indexes in a scratch schema and prints the SQL for the smallest one that keeps your recall:
pip install 'shrinkvec[pg]'
shrinkvec run --dsn "$DATABASE_URL" --table documents --column embedding --target 0.95
Or try it with nothing installed, on real OpenAI embeddings with Postgres, pgvector and VectorChord inside one container:
docker run --rm -p 8080:8080 ghcr.io/ahmad-codex/shrinkvec:demo
Put --min-recall 0.95 in CI and a model or codec change that hurts search fails the build instead of reaching users.
Every number in this post comes from it. The TurboQuant and RaBitQ implementations are checked against the error figures their papers publish and against the reference libraries (turbovec and the official rabitqlib) on the same vectors. The method and every raw result are in the repository, and one script regenerates all of them.
Compression doesn't replace a good index. It decides what fits in memory, and that is usually what decides the bill. Measure it on your data before you ship it.
---
References: TurboQuant (Zandieh et al., ICLR 2026) · RaBitQ (Gao and Long, SIGMOD 2024) · Extended RaBitQ (Gao et al., SIGMOD 2025) · pgvector · VectorChord · turbovec · shrinkvec
Ahmad Shawar builds LLM and vector search systems in .NET, TypeScript and Rust. GitHub · LinkedIn
TurboVec GitHub Explained: Low-Bit SIMD Search in Rust and Python
turbovec by ryancodrai: ryancodrai/turbovec
Python/API Documentation: github.com/ryancodrai/turbovec/blob/main/docs/api…
How It Works Guide: github.com/ryancodrai/turbovec/blob/main/docs/how…
TurboVec is an in-process vector search index built around a Rust core and Python bindings. This overview explains why local AI retrieval runs into physical-memory limits, then traces the design trade-off: TurboQuant uses normalization, deterministic orthogonal rotation, Lloyd-Max scalar quantization, and bit-packing to keep high-dimensional vectors compact. The original floating-point coordinates are not preserved, so the memory savings come with lossy approximate inner-product search.
You’ll see why the architecture favors exhaustive flat scans over graph navigation, how SIMD paths on x86 and ARM shape dimension requirements, why dimensions must be positive multiples of eight and capped at 16,384, and why L2 distance is not the native scoring mode. The operational overview covers float32 array boundaries, ID maps, TQ+ calibration, top-five search results, allowlist filtering, and incremental sync. This is a conceptual architecture overview, not an installation walkthrough or hands-on benchmark.
TimeStamps:
0:00 Local retrieval meets the memory bottleneck
0:22 TurboVec’s compressed flat-search architecture
1:14 Normalize, rotate, and spread vector energy
1:52 Lloyd-Max quantization and irreversible bit-packing
2:39 SIMD requirements and inner-product limits
3:56 Python FFI boundaries and ID-map setup
4:37 TQ+ calibration before ingestion
5:13 Queries, allowlists, and incremental sync
6:48 Linear scans and the memory-efficiency trade-off
7:17 Operational boundaries and local retrieval fit
🧠 Local retrieval is bounded by vector memory footprint
🌀 TurboQuant rotates, quantizes, and packs embeddings
⚙️ SIMD scanning enforces dimension and metric constraints
💾 ID maps, filters, and incremental sync shape operations
TurboVec fits best when local memory matters and approximate inner-product retrieval is acceptable. Expect exhaustive linear scans, lossy representations, strict float32 and dimension rules, and a calibration decision before bulk ingestion. Compare that compact in-process model with graph or remote vector systems when exact embeddings or different latency characteristics matter.
#TurboVec #VectorSearch #Rust
[de-DE] YNAS ynp_b1b43b8ef8d2229c Cinematic
Privater automatisierter Staging-Upload. Paketidentität: ynp_b1b43b8ef8d2229c; Ausgabe: cinematic.
[es-US] YNAS ynp_b1b43b8ef8d2229c cinemático
Carga de entorno de prueba de automatización privada. Identidad del paquete: ynp_b1b43b8ef8d2229c; salida: cinemática.
[fr-FR] YNAS ynp_b1b43b8ef8d2229c cinématique
Téléchargement de l'environnement de test d'automatisation privé. Identifiant du package : ynp_b1b43b8ef8d2229c ; sortie : cinématique.
[hi] YNAS ynp_b1b43b8ef8d2229c सिनेमैटिक
निजी स्वचालन स्टेजिंग अपलोड। पैकेज पहचान: ynp_b1b43b8ef8d2229c; आउटपुट: सिनेमाई।
[id] YNAS ynp_b1b43b8ef8d2229c sinematik
Unggahan staging otomatisasi pribadi. Identitas paket: ynp_b1b43b8ef8d2229c; output: sinematik.
[it] YNAS ynp_b1b43b8ef8d2229c cinematic
Caricamento di automazione privata. Identità del pacchetto: ynp_b1b43b8ef8d2229c; output: cinematico.
[pt-BR] YNAS ynp_b1b43b8ef8d2229c cinematográfico
Upload de teste de automação privada. Identidade do pacote: ynp_b1b43b8ef8d2229c; saída: cinemática.
1 like · 90 views
Mentions
Your embeddings are 32x bigger than they need to be. Maybe. Every OpenAI embedding you store in Postgres is 6 KB. A hundred million documents is 600 GB of vectors before you build a single index, and an HNSW index wants to live in RAM. That is how a RAG prototype turns into a database bill nobody planned for. The fix sounds simple: store fewer bits per number. pgvector already has halfvec (16 bits) and binary_quantize (1 bit). VectorChord ships RaBitQ. TurboQuant from Google Research is the new name everyone mentions. Each one promises a smaller index with "minimal recall loss". I spent most of this year building TurboQuant and RaBitQ into Postgres as a Rust extension. The most useful thing I learned is this: whether compression is safe depends on your embedding model, not on the codec. Here is what that looks like in numbers, the mistake that cost me months, and how to check your own data in three minutes. The same trick, three very different results I took pgvector's recommended setup for binary quantization (search 1-bit codes, then rerank the best 100 with the full vectors) and ran it on three public datasets. Recall@10 means: of the 10 results exact search would return, how many did we get back. | Dataset | Recall@10, binary + rerank | |---|---| | OpenAI ada-002 embeddings (DBpedia) | 0.992 | | GloVe word vectors | 0.348 | | SIFT image descriptors | 0.055 | On OpenAI embeddings it is almost free: a 32x smaller search for 1% of the results. On SIFT it returns the wrong answer 95% of the time. SIFT values are all positive, so one sign bit per number says almost nothing. Your embeddings are not DBpedia. A vendor benchmark on someone else's vectors tells you very little about yours, and nothing in your monitoring will flag that 20% of your search results quietly got worse. You find out when users stop trusting the answers. What each option costs, on real embeddings On 100,000 OpenAI ada-002 embeddings, exhaustive search over the codes (so this is the compression alone, before any index), averaged over three random seeds: | Option | Bytes per vector | Smaller by | Recall@10 | |---|---:|---:|---:| | float32 | 6,144 | 1x | 1.000 | | halfvec | 3,072 | 2x | 0.999 | | RaBitQ 4-bit | 776 | 8x | 0.972 | | TurboQuant 4-bit | 772 | 8x | 0.953 | | RaBitQ 1-bit | 200 | 31x | 0.834 | | binary_quantize | 192 | 32x | 0.707 | | RaBitQ 1-bit, rerank top 100 | 200 searched | 31x | 0.999 | | TurboQuant 3-bit, rerank with 8-bit | 580 searched | 11x | 0.996 | Three practical takeaways: - halfvec is free. If your index fits in memory at half size, stop here. - At one bit, the codec matters. RaBitQ keeps 13 more points of recall than a plain sign bit for 8 extra bytes per row, because it centres the data and corrects its own estimate. - Search small, rerank precise. The 100 best candidates from tiny codes almost always contain the true top 10. Reranking them gets you back to 0.99+ while the part you search stays 10 to 30 times smaller. In Postgres, today You don't need to write an extension any more. Same 100,000 embeddings, real indexes in Postgres 18, each at the first setting that reaches 0.95 recall: | Index | Size | Recall@10 | p50 latency | |---|---:|---:|---:| | pgvector HNSW, float32 | 781 MiB | 0.959 | 2.34 ms | | pgvector HNSW, halfvec | 391 MiB | 0.957 | 2.31 ms | | pgvector HNSW, binary + rerank | 48 MiB | 0.963 | 2.83 ms | | VectorChord, default | 812 MiB | 0.981 | 2.58 ms | | VectorChord, rerank_in_table | 31 MiB | 0.957 | 1.69 ms | Two things I would not have guessed: - VectorChord's default index is bigger than plain HNSW. It keeps a second copy of every vector inside the index for reranking. One option, rerank_in_table = true, stores only the compressed codes: 25x smaller, and the fastest index in this test. VectorChord warns that this mode gets slow when the table has to come from disk; here the table fit in memory, so test it at your real size. - An HNSW graph costs more than the codes in it. The binary index uses about 500 bytes per row for 192 bytes of codes. The rest is neighbour lists. It is still 16x smaller than float32. The mistake that cost me months TurboQuant comes in two flavours. One spends all its bits on the codes. The other spends one bit on a correction that makes inner products mathematically unbiased. Unbiased sounded strictly better, so that is what I shipped in my extension. It isn't, for search. Same total bits per number, 100,000 GloVe vectors: | Bits | With the correction bit | All bits on the codes | |---|---:|---:| | 5 | 0.806 | 0.925 | | 4 | 0.667 | 0.867 | | 3 | 0.478 | 0.753 | Up to 28 points of recall lost. Search only needs the right order of results for one query, and the uncorrected codes shrink every score by the same factor, which never changes the order. The correction removes that harmless shrink and adds noise, and noise is exactly what reorders neighbours. A second detail was worth up to 11 more points: after decoding, rescale each vector back to unit length. The decoded lengths wobble by about 1% from row to row, and that wobble leaks straight into the ranking. Both bugs looked like "compression is just lossy". Neither was. When your compressed recall is far below the paper, suspect the estimator before you blame the bits. Which one should you use? - Under a few million vectors: keep float32 or halfvec. Memory is cheap at that size. - Memory is the bill, recall must stay high: a 1-bit index with a rerank from the table. pgvector's binary + rerank or VectorChord with rerank_in_table, whichever wins on your data. - Constant inserts, data that drifts: TurboQuant. Nothing to train, a vector is compressed the moment it arrives. - Very large, slow-changing collections: RaBitQ with IVF (VectorChord), with an error bound that tells you how many candidates to rerank. - Changing embedding models: measure again. Your old compression may not survive the new model. Check your own data in three minutes I turned all of this into shrinkvec, an open-source tool. Point it at a table and it samples your vectors, computes the exact answers, measures every option above, builds the real indexes in a scratch schema and prints the SQL for the smallest one that keeps your recall: pip install 'shrinkvec[pg]' shrinkvec run --dsn "$DATABASE_URL" --table documents --column embedding --target 0.95 Or try it with nothing installed, on real OpenAI embeddings with Postgres, pgvector and VectorChord inside one container: docker run --rm -p 8080:8080 ghcr.io/ahmad-codex/shrinkvec:demo Put --min-recall 0.95 in CI and a model or codec change that hurts search fails the build instead of reaching users. Every number in this post comes from it. The TurboQuant and RaBitQ implementations are checked against the error figures their papers publish and against the reference libraries (turbovec and the official rabitqlib) on the same vectors. The method and every raw result are in the repository, and one script regenerates all of them. Compression doesn't replace a good index. It decides what fits in memory, and that is usually what decides the bill. Measure it on your data before you ship it. --- References: TurboQuant (Zandieh et al., ICLR 2026) · RaBitQ (Gao and Long, SIGMOD 2024) · Extended RaBitQ (Gao et al., SIGMOD 2025) · pgvector · VectorChord · turbovec · shrinkvec Ahmad Shawar builds LLM and vector search systems in .NET, TypeScript and Rust. GitHub · LinkedIn
TurboVec GitHub Explained: Low-Bit SIMD Search in Rust and Python turbovec by ryancodrai: ryancodrai/turbovec Python/API Documentation: github.com/ryancodrai/turbovec/blob/main/docs/api… How It Works Guide: github.com/ryancodrai/turbovec/blob/main/docs/how… TurboVec is an in-process vector search index built around a Rust core and Python bindings. This overview explains why local AI retrieval runs into physical-memory limits, then traces the design trade-off: TurboQuant uses normalization, deterministic orthogonal rotation, Lloyd-Max scalar quantization, and bit-packing to keep high-dimensional vectors compact. The original floating-point coordinates are not preserved, so the memory savings come with lossy approximate inner-product search. You’ll see why the architecture favors exhaustive flat scans over graph navigation, how SIMD paths on x86 and ARM shape dimension requirements, why dimensions must be positive multiples of eight and capped at 16,384, and why L2 distance is not the native scoring mode. The operational overview covers float32 array boundaries, ID maps, TQ+ calibration, top-five search results, allowlist filtering, and incremental sync. This is a conceptual architecture overview, not an installation walkthrough or hands-on benchmark. TimeStamps: 0:00 Local retrieval meets the memory bottleneck 0:22 TurboVec’s compressed flat-search architecture 1:14 Normalize, rotate, and spread vector energy 1:52 Lloyd-Max quantization and irreversible bit-packing 2:39 SIMD requirements and inner-product limits 3:56 Python FFI boundaries and ID-map setup 4:37 TQ+ calibration before ingestion 5:13 Queries, allowlists, and incremental sync 6:48 Linear scans and the memory-efficiency trade-off 7:17 Operational boundaries and local retrieval fit 🧠 Local retrieval is bounded by vector memory footprint 🌀 TurboQuant rotates, quantizes, and packs embeddings ⚙️ SIMD scanning enforces dimension and metric constraints 💾 ID maps, filters, and incremental sync shape operations TurboVec fits best when local memory matters and approximate inner-product retrieval is acceptable. Expect exhaustive linear scans, lossy representations, strict float32 and dimension rules, and a calibration decision before bulk ingestion. Compare that compact in-process model with graph or remote vector systems when exact embeddings or different latency characteristics matter. #TurboVec #VectorSearch #Rust [de-DE] YNAS ynp_b1b43b8ef8d2229c Cinematic Privater automatisierter Staging-Upload. Paketidentität: ynp_b1b43b8ef8d2229c; Ausgabe: cinematic. [es-US] YNAS ynp_b1b43b8ef8d2229c cinemático Carga de entorno de prueba de automatización privada. Identidad del paquete: ynp_b1b43b8ef8d2229c; salida: cinemática. [fr-FR] YNAS ynp_b1b43b8ef8d2229c cinématique Téléchargement de l'environnement de test d'automatisation privé. Identifiant du package : ynp_b1b43b8ef8d2229c ; sortie : cinématique. [hi] YNAS ynp_b1b43b8ef8d2229c सिनेमैटिक निजी स्वचालन स्टेजिंग अपलोड। पैकेज पहचान: ynp_b1b43b8ef8d2229c; आउटपुट: सिनेमाई। [id] YNAS ynp_b1b43b8ef8d2229c sinematik Unggahan staging otomatisasi pribadi. Identitas paket: ynp_b1b43b8ef8d2229c; output: sinematik. [it] YNAS ynp_b1b43b8ef8d2229c cinematic Caricamento di automazione privata. Identità del pacchetto: ynp_b1b43b8ef8d2229c; output: cinematico. [pt-BR] YNAS ynp_b1b43b8ef8d2229c cinematográfico Upload de teste de automação privada. Identidade do pacote: ynp_b1b43b8ef8d2229c; saída: cinemática.
@DivyanshBh24521 @DivyanshBh24521 check this in the benchmarks folder of turbovec, github.com/RyanCodrai/turbovec/tree/main/benchmar…
RyanCodrai / turbovec: A vector index built on TurboQuant, written in Rust with Python bindings ★17324 RyanCodrai/turbovec