timescale

PRISM vector search: clusters, compressed scans and reranking A visual explanation of the PRISM search pipeline and its recall tradeoffs, based on timescale/pg_vectorsearch . The diagrams are illustrative, not benchmark results. The project is alpha. A friend and I are building Armature, which generated this video: app.armature.ai . CodeCast explains codebases: app.armature.ai reddit.com

pg_textsearch ⭐ 4k BM25 full-text search for Postgres with WAND query optimization, for developers adding ranked search to Postgres. 🔗 timescale/pg_textsearch #opensource t.co

Postgres Full-Text Search vs Elasticsearch vs Meilisearch: BM25 Benchmarked (2026) Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com. Postgres full-text search can now rank results as well as Elasticsearch, as long as you use a BM25 extension instead of the built-in ts_rank. I loaded three public BEIR retrieval datasets (5,183 to 171,332 documents) into five setups on one box: built-in tsvector, ParadeDB's pg_search, Timescale's pg_textsearch, Elasticsearch 9.5.5 and Meilisearch 1.55.0. Then I scored every engine with the datasets' own relevance judgments. The short version: both Postgres BM25 extensions landed within about 0.05 nDCG@10 of Elasticsearch on every dataset, answered faster in my setup, and made new rows searchable in 2-4 ms instead of about 700 ms. The built-in tsvector search returned nothing at all for 274 of 300 natural-language queries with its default AND matching. Meilisearch, which is built for search-as-you-type, ranked long documents worst. TL;DR: Postgres full-text search vs Elasticsearch vs Meilisearch If your data already lives in PostgreSQL and you need ranked keyword search, try a BM25 extension (pg_textsearch or pg_search) before adding a search cluster. Keep Elasticsearch for multi-node scale, heavy aggregations, log search and complex analyzers. Use Meilisearch for instant, typo-tolerant search over short fields like product names, not for ranking long documents. | Engine (my setup) | nDCG@10 SciFact | nDCG@10 FiQA | nDCG@10 TREC-COVID | p50 latency, 171k docs | New row searchable after | | --- | --- | --- | --- | --- | --- | | Postgres tsvector, AND (websearch_to_tsquery) | 0.074 | 0.055 | 0.285 | 1.86 ms | 2.3 ms | | Postgres tsvector, OR + ts_rank | 0.565 | 0.147 | 0.561 | 238.83 ms | 2.3 ms | | pg_search 0.26.2 (ParadeDB) | 0.668 | 0.253 | 0.687 | 8.32 ms | 3.7 ms | | pg_textsearch 1.5.1 (Timescale) | 0.688 | 0.249 | 0.633 | 2.19 ms | 1.7 ms | | Elasticsearch 9.5.5 | 0.691 | 0.254 | 0.688 | 12.29 ms | 705.8 ms | | Meilisearch 1.55.0 (stop words, frequency) | 0.501 | 0.135 | 0.238 | 5.50 ms | 61.9 ms | nDCG@10 runs from 0 to 1, and higher means the right documents sit higher in the top 10. "New row searchable after" is the median of 10 trials on the FiQA index. How I tested Everything ran on one Linux VM with 8 vCPUs and 15 GB of RAM, one engine at a time, with the same documents and queries. - Datasets (from BEIR): SciFact (5,183 scientific abstracts, 300 test queries), FiQA-2018 (57,638 finance forum posts, 648 queries) and TREC-COVID (171,332 papers, 50 queries). Each comes with human relevance judgments, so I could score ranking quality instead of guessing. - Metrics: nDCG@10 and recall@100, computed from the top 100 results per query. These are the two numbers BEIR reports. - Latency: 20 warm-up queries, then every query 3 times asking for the top 10. I report p50 from those runs. Postgres was queried through psycopg with prepared statements over a Unix socket. Elasticsearch and Meilisearch were queried over local HTTP with Python requests, so their times include HTTP and JSON overhead. That is how an app talks to them too, but keep it in mind. - PostgreSQL 18 with shared_buffers = 1GB, work_mem = 64MB and maintenance_work_mem = 512MB. Elasticsearch ran as a single node with 1 shard and 0 replicas. Meilisearch ran with default settings, then with an English stop-word list. - Freshness: insert one document containing a random token, then poll search every 5 ms until it comes back. 10 trials per engine. Each document's title and body were indexed with English stemming in every engine. Queries are the raw natural-language questions from the datasets, which is what a search box sends. As a sanity check, my Elasticsearch scores (0.691, 0.254, 0.688) are close to the BM25 baselines in the BEIR paper (0.665, 0.236, 0.656), so the setup isn't broken. !Bar chart of nDCG@10 on SciFact, FiQA and TREC-COVID for Postgres tsvector, pg_search, pg_textsearch, Elasticsearch and Meilisearch Built-in Postgres full-text search: tsvector, GIN and ts_rank Postgres ships with full-text search: to_tsvector turns text into stemmed lexemes, a GIN index makes @@ matches fast, and ts_rank sorts the matches. This is the setup most tutorials show: -- Built-in Postgres full-text search ALTER TABLE docs ADD COLUMN tsv tsvector GENERATED ALWAYS AS ( setweight(to_tsvector('english', coalesce(title, '')), 'A') || setweight(to_tsvector('english', coalesce(body, '')), 'B') ) STORED; CREATE INDEX docs_tsv_gin ON docs USING gin (tsv); SELECT id FROM docs, websearch_to_tsquery('english', $1) q WHERE tsv @@ q ORDER BY ts_rank(tsv, q) DESC LIMIT 10; It works for short, keyword-style input. With real questions it fell apart: websearch_to_tsquery joins every word with AND, so a document must contain all of them. On SciFact, 274 of 300 queries returned zero rows. On FiQA it was 393 of 648. Switching the AND to OR fixes recall but exposes two other problems: -- OR the terms instead of AND SELECT id FROM docs, cast(replace(plainto_tsquery('english', $1)::text, ' & ', ' | ') AS tsquery) q WHERE tsv @@ q ORDER BY ts_rank(tsv, q) DESC LIMIT 10; 1. Ranking quality. ts_rank counts how often the terms appear in a document. It has no inverse document frequency, so a common word counts as much as a rare one, and long documents win. nDCG@10 was 0.565 on SciFact against 0.691 for Elasticsearch, and 0.147 against 0.254 on FiQA. 2. Speed. An OR query matches a large share of the table, and Postgres has to compute ts_rank for every match before it can sort. On the 171k-document TREC-COVID set the median query took 238.83 ms, with p95 at 394.13 ms. Built-in full-text search is still useful for filters ("documents that contain these words") and for small tables. For a ranked search box, it is the weakest option I tested. pg_search and pg_textsearch: BM25 inside Postgres BM25 is the ranking function Elasticsearch, OpenSearch and Lucene use by default. It weighs rare terms more than common ones and normalizes for document length. Two Postgres extensions now add it with their own index type: - pg_search from ParadeDB builds an index on Tantivy, a Rust search library modeled on Lucene. - pg_textsearch from Timescale adds a bm25 index type and a <@> scoring operator on top of Postgres's own text configurations. -- pg_search 0.26.2 CREATE EXTENSION pg_search; -- the exact statement from my benchmark; id is the key column CREATE INDEX docs_search ON docs USING paradedb ( id, (title::pdb.unicode_words('stemmer=english', 'stopwords_language=english')), (body::pdb.unicode_words('stemmer=english', 'stopwords_language=english')) ); SELECT id, pdb.score(id) AS score FROM docs WHERE title ||| $1 OR body ||| $1 -- ||| matches any of the terms ORDER BY pdb.score(id) DESC LIMIT 10; -- pg_textsearch 1.5.1 CREATE EXTENSION pg_textsearch; CREATE INDEX docs_bm25 ON docs USING bm25 ((title || ' ' || body)) WITH (text_config = 'english'); -- <@> returns a negative BM25 score, so ascending order puts the best match first SELECT id FROM docs ORDER BY (title || ' ' || body) <@> to_bm25query($1, 'docs_bm25') LIMIT 10; Results against Elasticsearch, same queries: | nDCG@10 / recall@100 | SciFact | FiQA | TREC-COVID | | --- | --- | --- | --- | | pg_search | 0.668 / 0.901 | 0.253 / 0.552 | 0.687 / 0.113 | | pg_textsearch | 0.688 / 0.918 | 0.249 / 0.549 | 0.633 / 0.124 | | Elasticsearch | 0.691 / 0.919 | 0.254 / 0.549 | 0.688 / 0.117 | On SciFact and FiQA the three are within 0.023 of each other, which is noise at this query count. On TREC-COVID, pg_textsearch scored lower on nDCG@10 (0.633) but had the highest recall@100. It indexes title and body as one string, while pg_search and Elasticsearch scored the two fields separately. Recall@100 is low for every engine on TREC-COVID because each of its 50 queries has between 111 and 1,266 judged-relevant papers, far more than 100. Latency was where the extensions stood out in my setup: | p50 / p95 latency, top 10 | SciFact (5k) | FiQA (58k) | TREC-COVID (171k) | | --- | --- | --- | --- | | pg_textsearch | 0.31 / 0.62 ms | 0.60 / 1.51 ms | 2.19 / 5.27 ms | | pg_search | 1.03 / 1.55 ms | 3.74 / 5.58 ms | 8.32 / 19.02 ms | | Elasticsearch | 2.27 / 3.10 ms | 1.85 / 2.44 ms | 12.29 / 19.73 ms | | Meilisearch (stop words) | 2.23 / 3.71 ms | 3.51 / 7.01 ms | 5.14 / 7.84 ms | Part of Elasticsearch's gap is the HTTP round trip, so read this as "fast enough that you will not notice", not as a verdict on Elasticsearch's engine. A note on Elasticsearch tuning: with tie_breaker: 0 on the multi_match query, TREC-COVID dropped to 0.552 nDCG@10 but p50 fell to 2.72 ms. With tie_breaker: 0.5 (the table above) it scored 0.688 at 12.29 ms. One gotcha: pick one BM25 extension per database Both extensions register an index access method named bm25 (I created the pg_search index with its paradedb name, but the extension still claims bm25), so I could not install them in the same database. I ran them in two databases on the same server. Pick one per database. Typos: fuzzy matching can hurt more than it helps I made a typo version of SciFact by swapping two letters in the longest word (5+ letters) of each query, then ran it again. | nDCG@10 on SciFact | Clean queries | With a typo | | --- | --- | --- | | pg_textsearch | 0.688 | 0.654 | | pg_search | 0.668 | 0.634 | | pg_search with pdb.fuzzy(1) | 0.532 | 0.487 | | Postgres tsvector OR | 0.565 | 0.520 | | Elasticsearch, fuzziness: AUTO vs off (typo run) | | 0.660 vs 0.652 | One typo in a 12-word question cost every engine only a few points, because the other words still match. Turning on fuzzy matching did not pay for itself here: pg_search with pdb.fuzzy(1) lost to plain pg_search even on the typo queries, and on FiQA it collapsed to 0.039 nDCG@10 at 69.78 ms p50. Fuzzy terms match many unrelated words and drown the ranking. Elasticsearch's fuzziness: AUTO gained less than 0.01. Save fuzzy matching for short inputs like names and product titles, where a typo can be the whole query. Meilisearch: great for instant search, wrong tool for long documents Meilisearch scored lowest everywhere: 0.378 nDCG@10 on SciFact and 0.063 on FiQA with defaults. That is not a bug. Meilisearch doesn't use BM25. It sorts results through ranking-rule buckets (words matched, typos, proximity, attribute, exactness) designed for search-as-you-type on short fields, and its default matchingStrategy: last drops query words from the end until something matches. Two settings helped: // Meilisearch settings that helped on long documents { "stopWords": ["a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", "such", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", "with"] } …plus "matchingStrategy": "frequency" in the search request, which drops the most common words first. With both, SciFact rose from 0.378 to 0.501 and FiQA from 0.063 to 0.135. Still well behind BM25. If your search box is for product names, city names or a command palette, Meilisearch's typo tolerance and speed are what you want. If users type questions against articles, tickets or docs, it is the wrong tool. It was also the heaviest on disk. For the 171k TREC-COVID documents, the Meilisearch database grew to 1,891 MB and took 37.88 s to index, against 61.5 MB for the pg_textsearch index. Index build time and size TREC-COVID, 171,332 documents, 196.4 MB of raw table: | | Build time | Index size | | --- | --- | --- | | tsvector generated column + GIN | 26.31 s + 2.77 s | 65.7 MB GIN (plus the stored tsvector column) | | pg_search | 3.92 s | 82.6 MB | | pg_textsearch | 10.03 s | 61.5 MB | | Elasticsearch (bulk load + refresh) | 14.64 s | 98.1 MB store | | Meilisearch | 37.88 s | 1,891 MB database | The stored tsvector column nearly doubled the table, from 196.4 MB to 386.5 MB. The BM25 indexes don't need an extra column. Freshness: when does a new row show up in search? This is the difference users notice. A Postgres index is updated inside the same transaction as the insert, so the row is searchable the moment the commit returns: | Insert, then poll until found (median of 10, FiQA) | Time | | --- | --- | | pg_textsearch | 1.7 ms | | tsvector + GIN | 2.3 ms | | pg_search | 3.7 ms | | Meilisearch (async task queue) | 61.9 ms | | Elasticsearch (default 1 s refresh) | 705.8 ms | Elasticsearch makes new documents visible on its periodic refresh, 1 second by default. Meilisearch queues every write as an async task. Neither is a problem by itself, but with a separate search engine you also need a sync pipeline (outbox table, queue or CDC) that keeps it in step with the database, plus a plan for when it falls behind. With a Postgres extension there is nothing to sync, and a search filter can join your real tables and respect row-level security in the same query. Using Postgres BM25 search from Node.js There is no special client. It's a normal parameterized query, here with pg: // search.ts import pg from "pg"; const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL }); export async function search(q: string, limit = 10) { const { rows } = await pool.query( `SELECT id, title, pdb.score(id) AS score FROM docs WHERE title ||| $1 OR body ||| $1 ORDER BY pdb.score(id) DESC LIMIT $2`, [q, limit], ); return rows; } With Prisma or Drizzle, use their raw-SQL escape hatches ($queryRaw or the sql template) and keep the user input as a bound parameter. Never interpolate the query string into SQL. I compared the two ORMs in Drizzle vs Prisma in 2026. Because the search index lives next to your data, you can also combine it with vectors in the same database. pgvector installed fine alongside pg_search in my test database. Hybrid search (BM25 + embeddings, merged with reciprocal rank fusion) is a few lines of SQL from there. The vector side is covered in pgvector vs Pinecone vs Qdrant. Which should you use? - Already on Postgres, need a search box: start with pg_textsearch or pg_search. In my test they matched Elasticsearch's ranking, answered in single-digit milliseconds and needed no second system. pg_textsearch was the lightest and fastest. pg_search has the richer query language (field-level matching, fuzzy, phrase and more). Check that your host lets you install the extension. Managed Postgres providers support different extension lists. - Built-in tsvector: fine for "contains these words" filters and small tables. Don't use ts_rank for a natural-language search box. - Elasticsearch (or OpenSearch): when search is its own product. That means multiple nodes, billions of documents, aggregations and facets at scale, log analytics, many languages and custom analyzers. Budget for the sync pipeline and the cluster. - Meilisearch: instant, typo-tolerant search over short text, such as product catalogs, docs titles or app search. Not ranked retrieval over long text. If you are building a SaaS on Postgres, as I do with NestJS and Next.js (see the secure NestJS + Prisma API starter), a BM25 extension removes a whole service from the architecture. For more Postgres 18 numbers, see UUIDv7 vs UUIDv4 vs bigint primary keys. Limitations - One machine, one run per configuration. Small nDCG differences (under about 0.02) are within noise, especially on TREC-COVID's 50 queries. - Default or near-default settings. A tuned Elasticsearch analyzer, BM25 k1/b values or Meilisearch ranking rules could change the order. - Read-only query load from one client. I didn't test concurrent writes, replication, or very large corpora (millions of documents). - Versions: PostgreSQL 18, pg_search 0.26.2, pg_textsearch 1.5.1, Elasticsearch 9.5.5, Meilisearch 1.55.0. These extensions move fast, so re-check on newer releases. FAQ Is Postgres full-text search as good as Elasticsearch? With a BM25 extension, it was in my test. pg_textsearch and pg_search scored within about 0.05 nDCG@10 of Elasticsearch 9.5.5 on SciFact, FiQA and TREC-COVID. Built-in tsvector with ts_rank was clearly worse, because ts_rank has no inverse document frequency. Why does Postgres full-text search return no results for my query? websearch_to_tsquery and plainto_tsquery join every word with AND. In my test 274 of 300 SciFact queries returned zero rows. OR the terms or use a BM25 extension. Can I install pg_search and pg_textsearch together? Not in the same database in my test: both register an index access method named bm25. I ran them in two databases on the same server. --- Originally published on fabwaseem.com.

Most SSRF bugs aren’t a missing check. The check looks at the URL string while the connection goes somewhere else: a hex IP, an IPv4-mapped IPv6 address, or a hostname that resolves to 10.0.0.5 the second time. ressrf checks where it matters: after DNS & before connect. timescale/ressrf

Couple of weeks ago I showed what goes into RSigma and what comes out. Here's what happens in between, step by step: Steps 4 to 6 are optional. It's all one open-source Rust binary. Repo: timescale/rsigma Docs: rsigma.io

Introducing pg_vectorsearch, an innovative index that outperforms pgvector's IVFFlat and HNSW for faster vector searches. Dive into enhanced efficiency and streamline your data retrieval processes. Discover how this advancement can elevate your projects! timescale/pg_vectorsearch

pg_vectorsearch vs pgvector: The 42-Hour Vector Index Just Became a 100-Minute One If you have ever kicked off a pgvector HNSW index build on a large embedding table and watched the hours tick by, the announcement Tiger Data made on October 13 reads like a targeted attack on your pain. Their new open-source extension, pg_vectorsearch, introduces an index called PRISM. On the same 128 GiB machine where a pgvector HNSW build over 100 million embeddings never finished at all, PRISM finished in 1 hour 40 minutes. On a machine 4x more expensive, HNSW needed 42 hours to do what PRISM did in 100 minutes. That is the headline. The useful question is what sits underneath it, because "should I switch my vector index" is a decision with real migration cost, and the answer in October 2026 is more conditional than the launch post makes it sound. Here is the breakdown: what PRISM actually is, what the numbers do and do not prove, and a checklist for deciding whether your workload should touch it yet. One disclosure up front: pg_vectorsearch shipped as an alpha one day before this article, so nobody outside Tiger Data has production experience with it, including me. Everything below comes from Tiger Data's announcement, their public benchmark tooling, and independent writeups. I will mark clearly which numbers are vendor-reported. What was actually announced Tiger Data, the company behind TimescaleDB, released pg_vectorsearch as an open-source PostgreSQL extension under the PostgreSQL license. It adds a new index access method called PRISM, short for Partitioned Routing Index for Similarity Matching. Three facts matter more than the name: - It is a native Postgres index. Not a sidecar service, not a separate database. You create an index the way you create a GiST or HNSW index, and your existing tooling, backups, and replication keep working. - It indexes your existing pgvector columns. No table rewrite, no data copy. A table already using pgvector's vector or halfvec types can get a PRISM index, and Tiger Data says your pgvector queries run unchanged. - It requires PostgreSQL 18 or later. That alone is a blocker for a large share of production deployments still on 15 through 17. The positioning detail that got less attention: Tiger Data describes pg_vectorsearch as the successor to pgvectorscale, their earlier DiskANN-based extension, which is heading into maintenance mode. If you adopted pgvectorscale last year on the strength of its Pinecone-comparison benchmarks, the roadmap just moved under you. That is worth knowing before your next infrastructure planning cycle. How PRISM works, without the paper HNSW, the index inside pgvector that most teams use, builds a multi-layer graph. Every vector is a node, and a search hops from node to node, narrowing down toward the nearest neighbors. The problem at scale is not compute. It is memory. Graph hops are unpredictable in advance, so the CPU spends most of its time stalled waiting on RAM, and the index itself has to stay resident. For 100 million vectors, Tiger Data measured the HNSW index at roughly 180 GB. PRISM takes the older IVF idea and hardens it. It groups similar vectors into k-means partitions, then stores each partition as compressed RaBitQ codes at roughly one bit per dimension, packed into whole database pages. A query finds the nearest few partitions and scans them sequentially. Sequential scans keep the CPU prefetcher busy and let SIMD instructions score many compressed vectors per instruction. Instead of a random-access graph walk, you get streaming passes over tightly packed pages. The result shows up in the storage number: 25 GB for the PRISM index versus 180 GB for HNSW on the same 100 million vectors. A 7x smaller index is what buys the cheap build and the small-machine query performance. The numbers, with their labels attached All figures below are from Tiger Data's launch benchmarks on 100 million Cohere embeddings, measured with their public benchmark tool, rekall, at recall@10 against exact brute-force results. They are vendor-reported, and every HNSW comparison used pgvector's halfvec type. - Build time at 100M vectors: HNSW could not finish building on a 128 GiB machine. On a 512 GiB machine, the build took 42 hours. PRISM built in 1 hour 40 minutes on the 128 GiB machine. - Query latency at 100M vectors: PRISM served 95% recall at a 6 ms median on the 128 GiB machine, versus HNSW's 8 ms on the machine with 4x the memory. - Index size: 25 GB for PRISM versus about 180 GB for HNSW. - Small scale holds up too: at 1 million vectors, PRISM served 99% recall in 15 ms on a 2 GiB machine. HNSW on a 4 GiB machine, twice the memory, took a reported 392 ms, and its build on that small box took about 21.7 hours while thrashing. Independent writeups of the announcement flagged the gaps fairly: the post does not state the embedding dimensionality of the test set, gives no throughput numbers, and contains no direct comparison against IVFFlat. Multi-client throughput testing is listed as still pending. None of that makes the numbers wrong, but it means the comparison you actually care about, PRISM versus whatever your workload does today on your hardware, does not exist yet. Run the rekall tool on your own data before believing any of this for your case. The alpha fine print is not small Tiger Data is unusually blunt about the state of the release, and the release notes deserve to be quoted in spirit: alpha quality, meant for testing, not production. Specifically: - Heavy churn needs a manual rebalance. If your table gets constant inserts, partition quality degrades until you intervene by hand. - Deletes do not reuse space yet. A delete-heavy workload will bloat the index with no built-in recovery. - Filtered search is on the GA roadmap, not in the alpha. If your RAG pipeline filters by tenant ID, date, or access control before similarity ranking, the alpha cannot do that efficiently. - Indexes should be recreated between prereleases. Whatever you build today is disposable. That last cluster of caveats points at the same conclusion: PRISM is for evaluation and read-mostly workloads right now, not for your production embedding table with daily churn. Should you switch? A decision checklist The interesting part of this launch is not "PRISM beats HNSW." It is that the threshold where teams felt forced to leave Postgres for a dedicated vector database just moved again. Save this checklist: - Under ~1 million vectors: stay on plain pgvector with HNSW. It works, it is boring, and PRISM's advantages are smallest exactly where your problem is easiest. This is most RAG projects, most internal search tools, most prototypes. - 1M to ~50M vectors, existing HNSW builds completing in acceptable time: stay put, but benchmark PRISM in staging. If your HNSW index no longer fits comfortably in memory and your bill is driven by RAM, PRISM's 7x smaller footprint is a direct cost argument. - 50M+ vectors, or HNSW builds that take days or fail: this is the audience the announcement was written for. Test PRISM now, on Postgres 18, in a non-production environment, with the rekall benchmark on your own embeddings. - Heavy insert/update churn or frequent deletes: wait for GA. The alpha's rebalance and space-reuse gaps hit exactly this pattern. - Filtering-heavy queries (tenant scoping, metadata filters): wait for GA. Filtered streaming search is on the roadmap, not shipped. - Currently on a dedicated vector DB (Pinecone, Qdrant, Weaviate) mainly for scale: this announcement is your quarterly reminder to re-run the stay-on-Postgres math. The direction of travel for two years has been Postgres absorbing the workloads that justified those services. - On Postgres 17 or earlier: the extension requires Postgres 18, so your first milestone is an upgrade, not an index change. The bigger story: the vector database market keeps shrinking Step back from the benchmark and look at the sequence. In 2023, running RAG meant standing up a dedicated vector database. Then pgvector made small-scale vector search a one-line extension install. Then pgvectorscale pushed Postgres into the tens of millions of vectors. Now PRISM is claiming the 100 million range on hardware a fraction of the cost, with no second system to operate, secure, back up, or keep consistent with your relational data. For a certain class of workload, the "we need a vector database" conversation is ending the same way the "we need a cache server" and "we need a search cluster" conversations ended for many teams: with a Postgres feature. Every team that chose a dedicated vector DB for scale reasons rather than feature reasons should keep a running estimate of what it would cost to move back, because the answer changes every quarter, and lately it changes in Postgres's favor. What I would actually do If I were planning a vector workload today: start every new project on plain pgvector regardless of expected size, because the exit cost from Postgres is low and the entry cost of a dedicated service is permanent. Revisit at the first sign of memory pressure, and at that point evaluate pgvectorscale's StreamingDiskANN for current production needs and PRISM for the 100M+ future, with the understanding that one is production-hardened and the other is a 48-hour-old alpha. And before any switch, generate your own numbers with rekall, because the only benchmark that matters is the one run on your embeddings, your hardware, and your recall target. I write about databases, AI infrastructure, and backend engineering every week. Subscribe, it is free, and it helps more than you know. Have you hit the ceiling of HNSW on a real workload? I am curious how long your largest index build took and whether it pushed you to a dedicated vector database or you stayed on Postgres. Tell me in the comments. Sources: Tiger Data's October 13 announcement (TimescaleDB on X), the pgvectorscale repository for the maintenance-mode note, and independent coverage of the launch benchmarks.

THE TUPLES ARE LEAKIN 🫤 pgvectorscale#276

pgvector vs Pinecone vs Qdrant: Benchmarked for RAG in 2026 I loaded the same 100,000 OpenAI embeddings into pgvector and Qdrant and measured recall, latency, index build and filtered search. Pinecone is covered from its docs only, since I didn't run it. The original, with any updates, lives on fabwaseem.com. pgvector is a PostgreSQL extension that stores embeddings in a vector column and searches them with HNSW or IVFFlat indexes, next to your normal tables. Qdrant is an open-source vector database you run yourself or use as a managed cloud. Pinecone is a fully managed, serverless vector database you only reach through an API. For a RAG app, the real question is whether you need a separate vector database at all. I loaded the same 100,000 OpenAI embeddings (1,536 dimensions) into pgvector 0.8.7 on PostgreSQL 18.6 and into Qdrant 1.19.2 on one machine. Then I measured recall, latency, build time and filtered search. Pinecone is covered from its official docs and limits only, because I didn't run it and I won't invent numbers for it. TL;DR: pgvector vs Pinecone vs Qdrant, which should I use? Start with pgvector if you already run Postgres and have up to a few million vectors. One database, SQL joins, transactions and backups you already have. Choose Qdrant when vector search is the core workload and you need the best recall per millisecond or heavy filtering. Choose Pinecone when you want zero infrastructure and accept a managed service with usage pricing. | | pgvector | Qdrant | Pinecone | | --- | --- | --- | --- | | What it is | Postgres extension | Vector database (open source) | Managed vector database (closed) | | Hosting | Anywhere Postgres runs | Self-host or Qdrant Cloud | Pinecone's cloud only | | Index | HNSW, IVFFlat | HNSW per segment | Managed (serverless, object storage) | | Max indexed dims | 2,000 vector, 4,000 halfvec | 1,536 tested here | 20,000 dense | | Filtering | SQL WHERE (post-filter, iterative scans) | Payload indexes + filter-aware HNSW | Metadata filters (40 KB per record) | | Consistency | Transactional (ACID) | Upserts readable after wait=true | Eventually consistent | | Recall@10 at ~2.6 ms, my test | 0.978 (ef_search 100) | 0.985 (hnsw_ef 40) | not tested | | Cost to start | Your Postgres | Free binary / free cloud tier | Free Starter; Standard $50/month minimum | Do I need a vector database for RAG? Usually not at first. If your app already uses PostgreSQL, pgvector adds vector search without a second system to deploy, secure, back up and keep in sync. In my test it answered top-10 queries over 100,000 OpenAI embeddings in 1.5-4 ms, with 95-99% recall. A dedicated database starts to pay off at very large scale, under heavy filtering, or when search load needs to scale separately. The hidden cost of a separate vector store is synchronisation. Every document insert, update and delete has to reach two systems, and a failed write leaves them disagreeing. With pgvector the embedding sits in the same row and the same transaction as the document it describes. What is pgvector? pgvector is an open-source PostgreSQL extension that adds vector, halfvec, bit and sparsevec types, distance operators like <=> (cosine) and <-> (L2), and approximate HNSW and IVFFlat indexes. Per the pgvector README, it "performs exact nearest neighbor search, which provides perfect recall" by default. An index trades a little recall for speed. CREATE EXTENSION vector; CREATE TABLE documents ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, tenant_id int NOT NULL, content text NOT NULL, embedding vector(1536) NOT NULL ); CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops); -- top 5 chunks for a question, inside one tenant SELECT id, content FROM documents WHERE tenant_id = $1 ORDER BY embedding <=> $2 LIMIT 5; Index limits matter for model choice. HNSW can index a vector of "up to 2,000 dimensions" and a halfvec of "up to 4,000 dimensions". A 3,072-dimension model such as text-embedding-3-large needs a halfvec index or a shorter output size. What are Qdrant and Pinecone? Qdrant is an open-source vector search engine written in Rust. It stores points (vectors plus a JSON payload), builds an HNSW index per segment, and adds payload indexes so filters run inside the graph search. You can run a single binary, Docker or Qdrant Cloud. Pinecone is a managed, serverless vector database. Per its architecture docs, "all your vector data is stored in highly efficient, distributed object storage." Pinecone "separates write and read operations into different paths, with each scaling independently based on demand." There is no self-hosted version. How I benchmarked pgvector and Qdrant 1. Data. The first 100,000 rows of the public DBpedia OpenAI embeddings dataset: 1,536-dimension text-embedding-ada-002 vectors of Wikipedia entities, a typical RAG shape. 1,000 held-out rows from the same dataset served as queries. 2. Ground truth. Exact cosine top-10 for every query, computed with NumPy. Recall@10 is the share of those 10 that each engine returned. 3. Machine. One 8-vCPU, 15 GB Linux box, everything on localhost, no Docker. 4. pgvector. PostgreSQL 18.6 with pgvector 0.8.7, shared_buffers = 2GB, maintenance_work_mem = 2GB. HNSW with defaults (m = 16, ef_construction = 64), loaded with binary COPY through psycopg 3. 5. Qdrant. The Qdrant 1.19.2 release binary, cosine distance, m = 16 and ef_construct = 64 to match pgvector, default optimizer settings (it made 4 segments). Loaded with the official Python client over gRPC from a single process. 6. Queries. One client, sequential, 50 warm-up queries, then 1,000 timed queries. Latency includes the Python client round trip on both sides. 7. Filters. Every vector got a random category from 0 to 9, about 10% of rows each. Filtered queries asked for the top 10 inside one category. Is pgvector fast enough? Recall and latency results Yes, at this scale both engines are fast. pgvector reached 0.948 recall@10 at 1.45 ms median with default settings, and 0.988 at 4.0 ms with hnsw.ef_search = 200. Qdrant reached 0.985 recall at 2.6 ms median and 0.999 at 5.4 ms. At roughly equal recall (about 98.5-98.8%), Qdrant was about 1.5 times faster: 2.6 ms against 4.0 ms. !pgvector vs Qdrant recall@10 vs median latency on 100,000 OpenAI embeddings: Qdrant 0.985 at 2.6 ms, pgvector 0.978 at 2.6 ms and 0.988 at 4.0 ms | Engine and setting | Recall@10 | p50 latency | p95 latency | Queries/s (1 client) | | --- | --- | --- | --- | --- | | pgvector, ef_search = 40 (default) | 0.948 | 1.45 ms | 2.16 ms | 683 | | pgvector, ef_search = 100 | 0.978 | 2.60 ms | 3.70 ms | 389 | | pgvector, ef_search = 200 | 0.988 | 4.02 ms | 5.82 ms | 249 | | pgvector halfvec index, ef_search = 100 | 0.973 | 2.33 ms | 3.30 ms | 433 | | Qdrant, hnsw_ef = 40 | 0.985 | 2.63 ms | 3.56 ms | 368 | | Qdrant, hnsw_ef = 100 | 0.997 | 3.73 ms | 4.66 ms | 255 | | Qdrant, hnsw_ef = 200 | 0.999 | 5.37 ms | 6.99 ms | 176 | | pgvector exact (no index) | 1.000 | 380 ms | 445 ms | 3 | | Qdrant exact (no index) | 1.000 | 24.7 ms | 32.6 ms | 39 | Why does Qdrant get higher recall at the same ef? It splits the collection into segments (4 here) and searches each one with that ef, so it explores more candidates in total. Compare the engines at equal recall, not equal settings. How fast do they load and index? Qdrant indexed faster: 30.8 seconds from the first upload to a fully indexed collection, with the upload itself taking 20.2 s from one Python process. pgvector loaded the rows with COPY in 3.9 s, then needed 69.1 s to build the HNSW index with the default 2 parallel workers. | | pgvector | Qdrant | | --- | --- | --- | | Load 100k vectors | 3.9 s (COPY) | 20.2 s (Python client, gRPC) | | Build HNSW index | 69.1 s | included, indexed at 30.8 s | | Table / data size | 798 MB | — | | HNSW index size | 781 MB (vector), 391 MB (halfvec) | — | | Server memory after test | Postgres buffers + OS cache | 1,293 MB RSS | Use the halfvec trick in pgvector: indexing embedding::halfvec(1536) halved the index from 781 MB to 391 MB and cost about half a point of recall (0.973 against 0.978 at ef_search = 100). Query the index with the same cast: CREATE INDEX ON documents USING hnsw ((embedding::halfvec(1536)) halfvec_cosine_ops); SELECT id FROM documents ORDER BY embedding::halfvec(1536) <=> $1::halfvec(1536) LIMIT 10; How does filtered vector search compare? Qdrant handled filters best. With a payload index on category, filtered top-10 queries returned 0.993 recall at 2.1 ms median. pgvector gave three very different results depending on the plan Postgres picked, so you need to know how its filtering works. | Filtered query (10% of rows match) | Rows returned (of 10) | Recall@10 | p50 | | --- | --- | --- | --- | | pgvector, planner's choice (B-tree on category, exact sort) | 10 | 1.000 | 44.2 ms | | pgvector, HNSW, iterative_scan = off | 3.92 | 0.389 | 1.77 ms | | pgvector, HNSW, iterative_scan = strict_order | 10 | 0.915 | 3.62 ms | | pgvector, HNSW, iterative_scan = relaxed_order | 10 | 0.953 | 3.37 ms | | Qdrant, payload index | 10 | 0.993 | 2.11 ms | Why does pgvector return fewer rows with a filter? Because HNSW filters after the index scan. The README explains: "If a condition matches 10% of rows, with HNSW and the default hnsw.ef_search of 40, only 4 rows will match on average." I measured 3.92. Since pgvector 0.8.0, iterative index scans keep walking the graph until enough rows match. -- once per session or per transaction (SET LOCAL) SET hnsw.iterative_scan = relaxed_order; SELECT id, content FROM documents WHERE tenant_id = $1 ORDER BY embedding <=> $2 LIMIT 10; Postgres often picks the exact plan by itself for a selective filter. In my test it used the category B-tree and sorted about 10,000 rows exactly: perfect recall, but 44 ms instead of about 3 ms. That is fine for one tenant with a few thousand chunks. For large tenants, check EXPLAIN and turn on iterative scans. If one filter dominates, consider a partial index or partitioning by tenant. What does Pinecone offer that pgvector and Qdrant don't? Fully managed, serverless scaling with nothing to run. Reads and writes scale separately, storage lives in object storage, and you never tune ef or m. The trade-offs are documented: - Eventual consistency. The known limitations page says "Pinecone is eventually consistent, so there can be a slight delay before upserted records are available to query." - Operation limits. Per the operation limits: dense vectors up to 20,000 dimensions, 40 KB of filterable metadata per record, 1,000 records or 2 MB per upsert batch, and top_k up to 10,000. - Plans. The object limits give the free Starter plan 2 GB of serverless storage and 5 indexes per project, all in AWS us-east-1. Pinecone pricing lists the Standard plan at a $50/month minimum. - Network hop. Every query leaves your VPC. Expect your cloud region's round-trip time on top of search time. My localhost latencies are not comparable to Pinecone and I did not measure it. When should I choose each one? Decide by what you already run and what dominates your workload. The pgvector default is strong for product teams. Move when you measure a reason, not before. !Decision guide: choose pgvector if you already run Postgres, Qdrant when search is the product, Pinecone for zero infrastructure 1. Choose pgvector when your data lives in Postgres, you need joins, row-level security or transactions with your embeddings, and you have up to a few million vectors per index. Add pgvectorscale (StreamingDiskANN) if the index outgrows RAM. 2. Choose Qdrant when search is the product: tens of millions of vectors, many filters, multi-vector or hybrid search, or quantization to cut memory. Its quantization guide says scalar quantization achieves "4x compression with minimal accuracy loss." 3. Choose Pinecone when nobody on the team should run a database, traffic is spiky, and a usage bill plus eventual consistency are acceptable. How do I move from pgvector to Qdrant later? Keep embeddings and IDs portable from day one. Use the same primary key in both systems, store the embedding model name and dimensions with each row, and write a re-sync job that streams rows by ID. Qdrant accepts integer or UUID point IDs, so a UUIDv7 primary key maps across directly. Then run both side by side, compare recall on a fixed set of real queries, and switch reads before you stop writing to Postgres. Common mistakes with vector search in production - Indexing 3,072-dimension vectors as vector. HNSW on vector stops at 2,000 dimensions. Use halfvec or a smaller output dimension. - Forgetting iterative scans. Filtered HNSW queries in pgvector can silently return fewer than LIMIT rows. - Comparing engines at equal settings. ef means different work in different engines. Compare at equal recall. - Measuring recall on random vectors. Use real embeddings from your model. Random data hides graph quality. - Creating Qdrant payload indexes after loading. The indexing docs say payload indexes should be created before ingesting data, so filter-aware HNSW edges get built. - Ignoring re-embedding cost. Switching embedding models means re-embedding everything, whichever store you use. If you want a working starting point, my open-source Next.js AI starter kit, Forgekit, ships RAG on pgvector. For self-hosting Postgres or Qdrant next to your app, see self-hosting Next.js with Coolify. Key takeaways - With 100,000 OpenAI embeddings, pgvector 0.8.7 hit 0.948 recall@10 at 1.45 ms with defaults and 0.988 at 4.0 ms with ef_search 200. - Qdrant 1.19.2 hit 0.985 recall at 2.6 ms, about 1.5x faster than pgvector at roughly equal recall. - pgvector's HNSW filters after scanning: a 10% filter returned only 3.92 of 10 rows until iterative scans were enabled. - Qdrant's payload index gave the best filtered search in my test: 0.993 recall at 2.1 ms. - A halfvec HNSW index halved pgvector's index size (781 MB to 391 MB) for about half a point of recall. - Pinecone is serverless and eventually consistent, supports 20,000-dimension vectors, and its Standard plan has a $50/month minimum. - Start with pgvector if you already run Postgres; move to Qdrant or Pinecone when measurements show a need. FAQ Is pgvector as good as Pinecone or Qdrant? For most RAG apps up to a few million vectors, yes. In my test of 100,000 OpenAI embeddings, pgvector reached 0.988 recall@10 at 4.0 ms median, and Qdrant reached 0.985 at 2.6 ms. Qdrant was about 1.5 times faster at equal recall and better at filtered search, but pgvector keeps embeddings in the same transactional database as your data. Do I need a vector database for RAG? Not usually at the start. If your app already uses PostgreSQL, the pgvector extension adds vector columns and HNSW indexes without a second system to deploy, back up and keep in sync. Consider a dedicated vector database when you reach tens of millions of vectors, need heavy filtering at low latency, or need to scale search separately. Why does pgvector return fewer results than my LIMIT? Because pgvector's HNSW index applies WHERE filters after scanning the index. With a filter matching 10 percent of rows and the default ef_search of 40, about 4 rows match on average; I measured 3.92. Set hnsw.iterative_scan to relaxed_order or strict_order, available since pgvector 0.8.0, so the scan continues until enough rows match. How many dimensions can pgvector index? pgvector's HNSW and IVFFlat indexes support the vector type up to 2,000 dimensions, halfvec up to 4,000 and bit up to 64,000. For a 3,072-dimension model such as text-embedding-3-large, index the column as halfvec or request a smaller output dimension. Pinecone supports dense vectors up to 20,000 dimensions. Is Qdrant faster than pgvector? In my single-machine test, yes, at equal recall. Qdrant 1.19.2 returned 0.985 recall@10 at 2.6 ms median against pgvector's 0.988 at 4.0 ms, and indexed 100,000 vectors in about 31 seconds against pgvector's 69-second HNSW build. With default settings, pgvector was faster per query but at lower recall, 0.948 at 1.45 ms. How much does Pinecone cost? Pinecone has a free Starter plan with 2 GB of serverless storage and up to 5 indexes per project, all in AWS us-east-1. Its Standard plan for production has a $50 per month minimum, after which you pay as you go for usage, according to Pinecone's pricing page. Qdrant's open-source build and pgvector are free to self-host. What is halfvec in pgvector? halfvec is pgvector's half-precision vector type, storing each dimension in 2 bytes instead of 4. Indexing an expression like embedding::halfvec(1536) halves the HNSW index size; in my test it shrank from 781 MB to 391 MB with recall dropping from 0.978 to 0.973. It also raises the indexable limit to 4,000 dimensions. Is Pinecone eventually consistent? Yes. Pinecone's documentation states that it is eventually consistent, so there can be a slight delay before upserted records can be queried, and suggests checking index stats to confirm counts. pgvector writes are transactional and visible on commit, and Qdrant upserts sent with wait=true are applied before the call returns. --- Originally published on fabwaseem.com.

pg_vectorsearch vs pgvector HNSW: 42 Hours vs 49 Minutes. Here's the Catch On a 64 vCPU, 512 GiB machine, pgvector took 42 hours to build an HNSW index over 100 million vectors. On the same box, Timescale's new pg_vectorsearch extension built its index in 49 minutes and answered queries in half the time. Those are the numbers in the launch post from Tiger Data (formerly Timescale), published October 13, 2026. If you run RAG on Postgres, that headline is hard to ignore. I went through the post, the repo, the tuning docs, the benchmark tool's config files and the Hacker News thread to see how far the pg_vectorsearch vs pgvector claim holds. Short answer: the query-side win is real and big. The build-time comparison is tilted, and the extension is still alpha, so production is off the table for now. What pg_vectorsearch actually is The extension adds a new index access method called PRISM (Partitioned Routing Index for Similarity Matching). The README calls it "conceptually similar to pgvector's IVFFlat", which is the clearest way to think about it. It's a partition index: - a hierarchical centroid tree routes each query to a few clusters - vectors in those clusters are stored as RaBitQ codes, about one bit per dimension - candidates get filtered on the compressed codes, then reranked with full-precision vectors from the heap That last part explains most of the memory difference. HNSW has to keep a big graph in memory to be fast, while PRISM only keeps a small tree there and scans compact posting lists for the rest. It also reads pgvector's types. vec32 and vec16 are binary-compatible with vector and halfvec, so you can put a PRISM index on an existing table: CREATE EXTENSION pg_vectorsearch; CREATE INDEX documents_embedding_prism ON documents USING prism (embedding vec32_cosine_ops); SET prism.nprobe = 40; -- 0 = auto, roughly 0.95 recall The pg_vectorsearch vs pgvector HNSW benchmark numbers All figures are p50 latency at recall@10 against brute force, from the launch post. HNSW ran on pgvector's 16-bit halfvec. 100M Cohere embeddings | | PRISM, r8i.4xlarge (128 GiB) | PRISM, r8i.16xlarge (512 GiB) | HNSW, r8i.16xlarge (512 GiB) | |---|---|---|---| | Build time | 1 h 40 min | 49 min | 42 h | | Index size | 25 GB | 25 GB | 180 GB | | p50 @ 95% recall | 6 ms | 4 ms | 8 ms | | p50 @ 99% recall | 21 ms | 19 ms | not reached | HNSW didn't finish building on the 128 GiB box. When they moved the index built on the big machine over to it, queries took over a second at 95% recall and never reached 99%. 1M vectors | | PRISM, c8g.medium (2 GiB) | PRISM, c8g.large (4 GiB) | HNSW, c8g.large (4 GiB) | |---|---|---|---| | Build time | ~3 min | ~3.1 min | ~21.7 h | | Index size | 280 MB | 280 MB | 2.05 GB | | p50 @ 95% recall | 9 ms | 3 ms | 122 ms | | p50 @ 99% recall | 15 ms | 9 ms | 392 ms | HNSW wasn't really tried on the 2 GiB box; the post says they "couldn't realistically try" it there, and the table marks it "did not finish". The headline "6 ms at 100M on 4x less memory" is the 128 GiB PRISM number against the 512 GiB HNSW number. I think that's fair: both hit 95% recall, and one machine costs a quarter of the other. Where the comparison is tilted Start with the HNSW build settings. Tiger Data's benchmark tool, rekall, ships an HNSW config (hnsw-m16.toml) with m = 16 and ef_construction = 200. pgvector's default ef_construction is 64. Building with 200 buys a better graph and costs a lot more time. Reasonable if you're chasing 99% recall, but it inflates the build-time gap. The post doesn't say which config produced the published table, so I'm reading the repo, not the run. The post also doesn't publish maintenance_work_mem, and pgvector's README is blunt about why that matters. Once the graph stops fitting in maintenance_work_mem, you get hnsw graph no longer fits into maintenance_work_mem after 100000 tuples and the build "will take significantly more time". A 2.05 GB HNSW index on a 4 GiB machine is going to hit that wall, and a 21.7 h build for 1M vectors is what hitting it looks like. I wouldn't call that normal HNSW performance. Same story for the 122 ms queries: the index doesn't fit in RAM next to the heap, so you're reading from disk. The numbers aren't dishonest, though. "HNSW falls over when it doesn't fit in memory" is exactly the point Tiger Data is making. But if your HNSW index already fits in RAM with headroom, the 1M table doesn't describe your setup. Then there's what the post leaves out. No QPS, no p99, no concurrency (multi-client throughput testing is listed as GA work), no filtered search, no IVFFlat numbers even though the README compares PRISM to IVFFlat, and no published embedding dimension. Filtered search is where people actually suffer with pgvector, and PRISM's answer today is basic: per the tuning docs, the executor applies your WHERE clause after the index has emitted its top-k, and the scan inflates k based on estimated selectivity. Filters that are anti-correlated with vector distance can still return too few rows. The docs suggest a partial index on the filter. Streaming search with WHERE filters is on the GA roadmap. On HN, esafak asked the obvious thing: "Shouldn't you lead with your benchmarks if you're fast? qps and precision/recall, please." jascha_eng replied that the benchmarks are in the post and the suite is open source. Both are right. They're still vendor numbers, and I haven't seen anyone outside Tiger Data publish a rerun yet. pgvectorscale's fate worries me more than the benchmarks pgvectorscale, Tiger Data's 2024 DiskANN-style index, "will enter maintenance mode," per the launch post. On HN, Tiger Data CTO mfreed said parts of pgvectorscale's graph could become unreachable and degrade recall, and when they couldn't find a fix they decided to start over with a partition index. So the vendor that told you two years ago that its graph index was the answer is now telling you graph indexes are the problem. They might be right. They also showed you how fast a Postgres vector extension can go from flagship to maintenance mode. On top of that, the practical blockers: - It's alpha. The README says "meant for testing, not production" and warns that you may have to drop and recreate the extension between prereleases. - It needs PostgreSQL 18+. pgvector supports 13+. - The hyperscalers don't offer it yet. On HN, devy pointed out it isn't on Azure's or the other big clouds' extension allow-lists, so on those you wait or self-host PG 18. When another commenter raised the same problem, jascha_eng's answer was that you can move to Tiger Cloud. - Index maintenance is manual and marked experimental. Posting lists grow on insert and "never split on their own." CALL prism_rebalance('items_idx'); splits them in place, but only on flat, single-level indexes built with centroid_fastscan = off. The default CREATE INDEX doesn't produce that shape, so for a default-built index the fix is a full REINDEX, which rewrites the whole index. A rebalance also leaves replaced lists on disk until the next REINDEX. - Don't build on an empty table. Index sizing uses the row count at CREATE INDEX, so load first and index second. - Be careful with high dimensions. The tuning docs say the packed SIMD fastscan layout always fits up to ~1900 dims at 8 KB pages, and auto falls back to a slower layout past that. On HN, devy said "PRISM caps at 1968 dimensions"; I couldn't find a hard cap in the docs. If you use 2048-dim embeddings (VoyageAI offers them), test before you plan on it. What to do with pgvector today If HNSW builds are killing you, fix the build before you change extensions. From the pgvector docs: SET maintenance_work_mem = '8GB'; -- size it so the graph fits SET max_parallel_maintenance_workers = 7; -- plus leader CREATE INDEX ON items USING hnsw ((embedding::halfvec(768)) halfvec_cosine_ops); Use halfvec (it halves the index and is what Tiger Data benchmarked against). Leave ef_construction at 64 unless you've measured that you need more. If filtered queries return too few rows, turn on hnsw.iterative_scan (pgvector 0.8.0+; current release is v0.8.7). So when does switching pay off? My take: pg_vectorsearch is the most interesting thing to happen to Postgres vector search since HNSW landed in pgvector. VectorChord took the same route of clustering plus quantized vectors, and its own benchmark says its QPS "could be twice that of pgvector" at equivalent recall on GIST 1M (960 dims). That's a vendor number too. Still, when two separate teams build partition indexes and both beat HNSW, I'm inclined to believe the direction. Try it now, on a replica or a staging box, if: - you're past ~10M vectors and your HNSW index doesn't fit in RAM, or you're paying for a huge instance just so it does - you need 99% recall and HNSW can't reach it at your size - you're on PG 18, self-hosted, and comfortable rebuilding indexes Stay on pgvector HNSW if: - your index fits in memory with headroom (most RAG apps under a few million chunks) - you're on RDS, Cloud SQL, Azure or another managed Postgres that doesn't list it - you rely on selective WHERE tenant_id = ... filters and can't add a partial index per tenant - you have heavy update churn and no time to schedule rebalances Revisit at GA, when filtered streaming search and automatic, incremental rebalancing are supposed to ship. Until then, 42 hours vs 49 minutes is a great slide and a bad reason to migrate production. What do you run vector search on right now: pgvector, VectorChord, pgvectorscale, or a dedicated vector DB like Qdrant? And if you've rerun rekall on your own hardware, did the numbers hold up? Tell me in the comments. Sources - pg_vectorsearch repo and README - pg_vectorsearch tuning docs - pg_vectorsearch maintenance docs - Tiger Data launch post with benchmarks - rekall benchmark tool - pgvector README - VectorChord vs pgvector benchmark - Hacker News discussion