timescale

A proxy login. An MFA reset. An admin role grant. A new identity provider. Each one alone is defensible. All four from the same user, in that order, within 30 minutes, is an account takeover. RSigma correlates them in memory with a single Sigma rule. timescale/rsigma

You Probably Don't Need a Dedicated Vector Database (Yet) Hot take for this Wednesday: for most RAG apps, adding a dedicated vector database on day one is premature infrastructure. If you already run Postgres, pgvector is probably enough. If you already run Elasticsearch or OpenSearch, its vector search is probably enough. Reach for a separate vector store when you have measured a scale, latency, or filtering problem your current database can't handle, not because a "RAG stack" diagram had a box labeled "vector DB." Embeddings are just another column. The hard part of RAG is usually chunking, retrieval quality, and evals. Where the floats live is rarely the bottleneck. What "just use Postgres" actually gets you in 2026 This isn't the pgvector of 2023. From the project's own README and changelog: - Two ANN index types. HNSW ("better query performance than IVFFlat (in terms of speed-recall tradeoff), but has slower build times and uses more memory") and IVFFlat (faster builds, less memory, lower speed-recall). - Smaller vectors when you need them. vector indexes up to 2,000 dimensions, halfvec up to 4,000, bit up to 64,000, and sparsevec up to 1,000 non-zero elements. The README's scaling advice: use halfvec for tables and binary quantization (with re-ranking) for indexes. - Filtering that doesn't silently starve you. The README is upfront: with approximate indexes, filtering is applied after the index scan, so if a condition matches 10% of rows, HNSW with the default hnsw.ef_search of 40 returns only about 4 rows on average. Since 0.8.0 (Oct 2024), iterative index scans keep scanning until enough rows match. You can also use partial indexes, or list partitioning per tenant. - Hybrid search. Combine with Postgres full-text search and merge with Reciprocal Rank Fusion or a cross-encoder. The pgvector-python repo has examples of both. - Everything else Postgres does. In the README's words: "ACID compliance, point-in-time recovery, JOINs, and all of the other great features of Postgres." - Scaling paths. Vertically, with read replicas, or sharded with Citus or PgDog. - Active maintenance. The latest release is 0.8.7 (Oct 1, 2026), and 0.9.0 is in progress with async I/O for HNSW. It's also available on the managed Postgres you probably already pay for. Amazon RDS for PostgreSQL, for example, announced pgvector 0.8.0 support in November 2024. (I covered the hands-on setup in an earlier post: pgvector: build RAG retrieval inside the Postgres you already run. This one is about the decision.) The hidden cost of a second database is sync The most underrated cost of a separate vector store isn't the bill. It's that you now have two sources of truth. Your documents, permissions, tenant IDs, and "is this deleted?" flags live in your main database. Their embeddings live somewhere else. Every create, update, delete, and permission change now has to reach both systems. If it doesn't, you get: - Ghost results: a deleted or unpublished document still gets retrieved. - Permission leaks: an ACL changed in Postgres, but the copy in the vector store still says the old thing. - Stale answers: the price, policy, or doc changed, and the embedding still reflects the old version. Inside one Postgres, the row and its embedding commit together in one transaction, and your WHERE tenant_id = $1 AND deleted_at IS NULL is the same filter your app already trusts. This isn't hypothetical. Instacart wrote about exactly this. They had moved full-text search from Elasticsearch to Postgres, then added semantic search through a separate FAISS-based ANN service. Their list of problems with that setup: - FAISS "had limitations on filtering documents based on their attributes at retrieval time," which forced overfetching and then post-filtering. As a result, "some relevant documents" were never fetched. - "Maintaining two separate services for recall came with developmental and operational overhead. Storing data in two separate systems and keeping them in-sync led to inconsistencies." They moved semantic retrieval into Postgres with pgvector and used real-time availability as a pre-filter. pgvector was "marginally slower than FAISS for larger retailers," but "it had better recall performance." In a production A/B test, searches with zero results dropped by 6%. That's a catalog of billions of items, with billions of writes per day. To be clear, this was a search-engine migration at Instacart scale, not a RAG chatbot. But the lesson carries over: keeping vectors next to the data they describe removed an entire class of bugs. "But the benchmarks!" You'll find benchmarks for every side. Here's one, and it should be read as the vendor claim it is. Timescale's pgvectorscale extension (a StreamingDiskANN index that complements pgvector) reports that on 50 million 768-dimension Cohere embeddings, Postgres with pgvector + pgvectorscale got 28x lower p95 latency and 16x higher query throughput than Pinecone's storage-optimized (s1) index at 99% recall, at 75% less cost when self-hosted on EC2. That's Timescale benchmarking its own product against one Pinecone index type. I wouldn't quote "28x" in a design review. What it does show is that "Postgres can't do vectors at tens of millions of rows" is no longer a safe assumption, so you should test it on your own data instead of assuming. The strongest case against me (and where it's right) The best counterargument I've read is Alex Jacobs' "The Case Against pgvector" (Oct 2025). It's worth reading in full. His points, fairly summarized: - Index builds are heavy. He reports that building an HNSW index on a few million vectors "can consume 10+ GB of RAM or more," on your production database, potentially for hours. (pgvector's own docs warn about maintenance_work_mem and the HNSW graph not fitting in memory.) - Continuous inserts have a cost. IVFFlat clusters drift and need rebuilds. HNSW inserts update the graph and add load to your transactional database. - Filtered search pushes query-planner problems onto you. Pre-filter vs post-filter is "the difference between queries that take 50ms and queries that take 5 seconds." - Hybrid search is DIY. You write the RRF yourself. - pgvectorscale isn't on RDS. - His conclusion: for many teams, a managed vector DB is simpler and maybe cheaper. Where he's right: if your corpus is in the millions and churning constantly, your Postgres is already busy with OLTP, and nobody on the team enjoys reading EXPLAIN, a managed vector store can genuinely be less work. Where I'd push back: - Filtering got better. The Hacker News reply from Discourse's Rafael dos Santos Silva points out that the pre/post-filtering problem "was fixed in version 0.8.0 via Iterative Scans." He also says Discourse runs pgvector "in thousands of databases." - "Hybrid is DIY" is about 30 lines of SQL, and the pgvector-python repo has an RRF example. (My estimate, not a measured number.) - A dedicated DB doesn't remove the sync problem. It creates it. Now the pipeline that keeps two stores consistent is yours to own. - Most RAG corpora aren't "millions of vectors churning constantly." An internal docs bot, a support knowledge base, or a product's help center usually sits in the thousands to low millions of chunks. That's my opinion from the projects I've seen, not a survey. When a dedicated vector database actually wins I'm not anti-vector-DB. I'm anti-default. I'd reach for one when at least one of these is true and measured: 1. Scale beyond one comfortable node. Hundreds of millions to billions of vectors, where index memory and build times stop fitting your Postgres instance, and sharding Postgres yourself is a bigger project than adopting a purpose-built system. For context, turbopuffer's published limits page says it has seen 1T+ documents globally and 250M+ namespaces. That's the kind of scale these systems are designed for (vendor-reported). 2. Vector QPS that would starve your OLTP. If vector reads would compete with checkout or login traffic, isolate them. A read replica is the first step. A separate system is the next. 3. Massive multitenancy. Millions of tiny per-user indexes (one per customer, workspace, or agent memory) are a natural fit for namespace-per-tenant designs on object storage. 4. High-churn, real-time ingestion where index maintenance on your primary database is measurably hurting p95. 5. Your source of truth isn't relational anyway. If your content lives in object storage or a lakehouse and Postgres would only be a vector cache, the "one database" argument goes away. Notice what's not on the list: "we're building RAG," "the tutorial used Pinecone," or "we might need it later." My decision checklist Before adding a vector database to the architecture diagram: 1. Start where your data lives. Postgres → pgvector. Elasticsearch/OpenSearch → their kNN (Elastic's docs say putting the filter inside the knn clause makes it a pre-filter, while filters outside it post-filter and "can return fewer than k results"). 2. Build an eval set first. 50 to 200 real queries with expected documents. Measure recall@10 and p95 latency on your corpus. 3. Use the cheap levers before new infrastructure: HNSW over IVFFlat, tune ef_search, enable iterative scans for filtered queries, halfvec or binary quantization, partial indexes or partitioning per tenant, a read replica. 4. Write down the trigger. For example: "We'll evaluate a dedicated store when p95 retrieval goes over X ms at Y QPS, or the index no longer fits in RAM." Pick the numbers yourself. Mine would be different from yours. 5. If you do split, design the sync on day one. Use an outbox or CDC pipeline, idempotent upserts, delete propagation, and a reconciliation job that diffs IDs between stores. Filter permissions at query time, not just at ingestion. The bottom line A vector database is a scaling decision, not a RAG requirement. For most teams, the right first vector store is the database that already holds the documents, the permissions, and the backups. Measure first. When the numbers say you've outgrown it, migrate on purpose. What's in your RAG stack right now: pgvector, your search engine, or a dedicated vector DB? And what pushed you there? References - pgvector README (indexes, limits, filtering, iterative scans, hybrid search, scaling): pgvector/pgvector - pgvector CHANGELOG (0.8.0 iterative scans; 0.8.7 released 2026-10-01): github.com/pgvector/pgvector/blob/master/CHANGELO… - pgvector-python hybrid search RRF example: github.com/pgvector/pgvector-python/blob/master/e… - AWS, "Amazon RDS for PostgreSQL supports pgvector 0.8.0" (Nov 2024): aws.amazon.com - Instacart, "How Instacart Built a Modern Search Infrastructure on Postgres" (May 2025): company.instacart.com - Timescale pgvectorscale README (vendor benchmark vs Pinecone s1): timescale/pgvectorscale - Alex Jacobs, "The Case Against pgvector" (Oct 2025): alex-jacobs.com - Hacker News reply by a Discourse developer on pgvector in production: news.ycombinator.com - Elastic docs, "Filter approximate kNN results": elastic.co - turbopuffer (published limits, vendor-reported): turbopuffer.com

PRISM vector search: clusters, compressed scans and reranking A visual explanation of the PRISM search pipeline and its recall tradeoffs, based on timescale/pg_vectorsearch . The diagrams are illustrative, not benchmark results. The project is alpha. A friend and I are building Armature, which generated this video: app.armature.ai . CodeCast explains codebases: app.armature.ai reddit.com

pg_textsearch ⭐ 4k BM25 full-text search for Postgres with WAND query optimization, for developers adding ranked search to Postgres. 🔗 timescale/pg_textsearch #opensource t.co

Postgres Full-Text Search vs Elasticsearch vs Meilisearch: BM25 Benchmarked (2026) Cross-posted from my blog. The original, with any updates, lives on fabwaseem.com. Postgres full-text search can now rank results as well as Elasticsearch, as long as you use a BM25 extension instead of the built-in ts_rank. I loaded three public BEIR retrieval datasets (5,183 to 171,332 documents) into five setups on one box: built-in tsvector, ParadeDB's pg_search, Timescale's pg_textsearch, Elasticsearch 9.5.5 and Meilisearch 1.55.0. Then I scored every engine with the datasets' own relevance judgments. The short version: both Postgres BM25 extensions landed within about 0.05 nDCG@10 of Elasticsearch on every dataset, answered faster in my setup, and made new rows searchable in 2-4 ms instead of about 700 ms. The built-in tsvector search returned nothing at all for 274 of 300 natural-language queries with its default AND matching. Meilisearch, which is built for search-as-you-type, ranked long documents worst. TL;DR: Postgres full-text search vs Elasticsearch vs Meilisearch If your data already lives in PostgreSQL and you need ranked keyword search, try a BM25 extension (pg_textsearch or pg_search) before adding a search cluster. Keep Elasticsearch for multi-node scale, heavy aggregations, log search and complex analyzers. Use Meilisearch for instant, typo-tolerant search over short fields like product names, not for ranking long documents. | Engine (my setup) | nDCG@10 SciFact | nDCG@10 FiQA | nDCG@10 TREC-COVID | p50 latency, 171k docs | New row searchable after | | --- | --- | --- | --- | --- | --- | | Postgres tsvector, AND (websearch_to_tsquery) | 0.074 | 0.055 | 0.285 | 1.86 ms | 2.3 ms | | Postgres tsvector, OR + ts_rank | 0.565 | 0.147 | 0.561 | 238.83 ms | 2.3 ms | | pg_search 0.26.2 (ParadeDB) | 0.668 | 0.253 | 0.687 | 8.32 ms | 3.7 ms | | pg_textsearch 1.5.1 (Timescale) | 0.688 | 0.249 | 0.633 | 2.19 ms | 1.7 ms | | Elasticsearch 9.5.5 | 0.691 | 0.254 | 0.688 | 12.29 ms | 705.8 ms | | Meilisearch 1.55.0 (stop words, frequency) | 0.501 | 0.135 | 0.238 | 5.50 ms | 61.9 ms | nDCG@10 runs from 0 to 1, and higher means the right documents sit higher in the top 10. "New row searchable after" is the median of 10 trials on the FiQA index. How I tested Everything ran on one Linux VM with 8 vCPUs and 15 GB of RAM, one engine at a time, with the same documents and queries. - Datasets (from BEIR): SciFact (5,183 scientific abstracts, 300 test queries), FiQA-2018 (57,638 finance forum posts, 648 queries) and TREC-COVID (171,332 papers, 50 queries). Each comes with human relevance judgments, so I could score ranking quality instead of guessing. - Metrics: nDCG@10 and recall@100, computed from the top 100 results per query. These are the two numbers BEIR reports. - Latency: 20 warm-up queries, then every query 3 times asking for the top 10. I report p50 from those runs. Postgres was queried through psycopg with prepared statements over a Unix socket. Elasticsearch and Meilisearch were queried over local HTTP with Python requests, so their times include HTTP and JSON overhead. That is how an app talks to them too, but keep it in mind. - PostgreSQL 18 with shared_buffers = 1GB, work_mem = 64MB and maintenance_work_mem = 512MB. Elasticsearch ran as a single node with 1 shard and 0 replicas. Meilisearch ran with default settings, then with an English stop-word list. - Freshness: insert one document containing a random token, then poll search every 5 ms until it comes back. 10 trials per engine. Each document's title and body were indexed with English stemming in every engine. Queries are the raw natural-language questions from the datasets, which is what a search box sends. As a sanity check, my Elasticsearch scores (0.691, 0.254, 0.688) are close to the BM25 baselines in the BEIR paper (0.665, 0.236, 0.656), so the setup isn't broken. !Bar chart of nDCG@10 on SciFact, FiQA and TREC-COVID for Postgres tsvector, pg_search, pg_textsearch, Elasticsearch and Meilisearch Built-in Postgres full-text search: tsvector, GIN and ts_rank Postgres ships with full-text search: to_tsvector turns text into stemmed lexemes, a GIN index makes @@ matches fast, and ts_rank sorts the matches. This is the setup most tutorials show: -- Built-in Postgres full-text search ALTER TABLE docs ADD COLUMN tsv tsvector GENERATED ALWAYS AS ( setweight(to_tsvector('english', coalesce(title, '')), 'A') || setweight(to_tsvector('english', coalesce(body, '')), 'B') ) STORED; CREATE INDEX docs_tsv_gin ON docs USING gin (tsv); SELECT id FROM docs, websearch_to_tsquery('english', $1) q WHERE tsv @@ q ORDER BY ts_rank(tsv, q) DESC LIMIT 10; It works for short, keyword-style input. With real questions it fell apart: websearch_to_tsquery joins every word with AND, so a document must contain all of them. On SciFact, 274 of 300 queries returned zero rows. On FiQA it was 393 of 648. Switching the AND to OR fixes recall but exposes two other problems: -- OR the terms instead of AND SELECT id FROM docs, cast(replace(plainto_tsquery('english', $1)::text, ' & ', ' | ') AS tsquery) q WHERE tsv @@ q ORDER BY ts_rank(tsv, q) DESC LIMIT 10; 1. Ranking quality. ts_rank counts how often the terms appear in a document. It has no inverse document frequency, so a common word counts as much as a rare one, and long documents win. nDCG@10 was 0.565 on SciFact against 0.691 for Elasticsearch, and 0.147 against 0.254 on FiQA. 2. Speed. An OR query matches a large share of the table, and Postgres has to compute ts_rank for every match before it can sort. On the 171k-document TREC-COVID set the median query took 238.83 ms, with p95 at 394.13 ms. Built-in full-text search is still useful for filters ("documents that contain these words") and for small tables. For a ranked search box, it is the weakest option I tested. pg_search and pg_textsearch: BM25 inside Postgres BM25 is the ranking function Elasticsearch, OpenSearch and Lucene use by default. It weighs rare terms more than common ones and normalizes for document length. Two Postgres extensions now add it with their own index type: - pg_search from ParadeDB builds an index on Tantivy, a Rust search library modeled on Lucene. - pg_textsearch from Timescale adds a bm25 index type and a <@> scoring operator on top of Postgres's own text configurations. -- pg_search 0.26.2 CREATE EXTENSION pg_search; -- the exact statement from my benchmark; id is the key column CREATE INDEX docs_search ON docs USING paradedb ( id, (title::pdb.unicode_words('stemmer=english', 'stopwords_language=english')), (body::pdb.unicode_words('stemmer=english', 'stopwords_language=english')) ); SELECT id, pdb.score(id) AS score FROM docs WHERE title ||| $1 OR body ||| $1 -- ||| matches any of the terms ORDER BY pdb.score(id) DESC LIMIT 10; -- pg_textsearch 1.5.1 CREATE EXTENSION pg_textsearch; CREATE INDEX docs_bm25 ON docs USING bm25 ((title || ' ' || body)) WITH (text_config = 'english'); -- <@> returns a negative BM25 score, so ascending order puts the best match first SELECT id FROM docs ORDER BY (title || ' ' || body) <@> to_bm25query($1, 'docs_bm25') LIMIT 10; Results against Elasticsearch, same queries: | nDCG@10 / recall@100 | SciFact | FiQA | TREC-COVID | | --- | --- | --- | --- | | pg_search | 0.668 / 0.901 | 0.253 / 0.552 | 0.687 / 0.113 | | pg_textsearch | 0.688 / 0.918 | 0.249 / 0.549 | 0.633 / 0.124 | | Elasticsearch | 0.691 / 0.919 | 0.254 / 0.549 | 0.688 / 0.117 | On SciFact and FiQA the three are within 0.023 of each other, which is noise at this query count. On TREC-COVID, pg_textsearch scored lower on nDCG@10 (0.633) but had the highest recall@100. It indexes title and body as one string, while pg_search and Elasticsearch scored the two fields separately. Recall@100 is low for every engine on TREC-COVID because each of its 50 queries has between 111 and 1,266 judged-relevant papers, far more than 100. Latency was where the extensions stood out in my setup: | p50 / p95 latency, top 10 | SciFact (5k) | FiQA (58k) | TREC-COVID (171k) | | --- | --- | --- | --- | | pg_textsearch | 0.31 / 0.62 ms | 0.60 / 1.51 ms | 2.19 / 5.27 ms | | pg_search | 1.03 / 1.55 ms | 3.74 / 5.58 ms | 8.32 / 19.02 ms | | Elasticsearch | 2.27 / 3.10 ms | 1.85 / 2.44 ms | 12.29 / 19.73 ms | | Meilisearch (stop words) | 2.23 / 3.71 ms | 3.51 / 7.01 ms | 5.14 / 7.84 ms | Part of Elasticsearch's gap is the HTTP round trip, so read this as "fast enough that you will not notice", not as a verdict on Elasticsearch's engine. A note on Elasticsearch tuning: with tie_breaker: 0 on the multi_match query, TREC-COVID dropped to 0.552 nDCG@10 but p50 fell to 2.72 ms. With tie_breaker: 0.5 (the table above) it scored 0.688 at 12.29 ms. One gotcha: pick one BM25 extension per database Both extensions register an index access method named bm25 (I created the pg_search index with its paradedb name, but the extension still claims bm25), so I could not install them in the same database. I ran them in two databases on the same server. Pick one per database. Typos: fuzzy matching can hurt more than it helps I made a typo version of SciFact by swapping two letters in the longest word (5+ letters) of each query, then ran it again. | nDCG@10 on SciFact | Clean queries | With a typo | | --- | --- | --- | | pg_textsearch | 0.688 | 0.654 | | pg_search | 0.668 | 0.634 | | pg_search with pdb.fuzzy(1) | 0.532 | 0.487 | | Postgres tsvector OR | 0.565 | 0.520 | | Elasticsearch, fuzziness: AUTO vs off (typo run) | | 0.660 vs 0.652 | One typo in a 12-word question cost every engine only a few points, because the other words still match. Turning on fuzzy matching did not pay for itself here: pg_search with pdb.fuzzy(1) lost to plain pg_search even on the typo queries, and on FiQA it collapsed to 0.039 nDCG@10 at 69.78 ms p50. Fuzzy terms match many unrelated words and drown the ranking. Elasticsearch's fuzziness: AUTO gained less than 0.01. Save fuzzy matching for short inputs like names and product titles, where a typo can be the whole query. Meilisearch: great for instant search, wrong tool for long documents Meilisearch scored lowest everywhere: 0.378 nDCG@10 on SciFact and 0.063 on FiQA with defaults. That is not a bug. Meilisearch doesn't use BM25. It sorts results through ranking-rule buckets (words matched, typos, proximity, attribute, exactness) designed for search-as-you-type on short fields, and its default matchingStrategy: last drops query words from the end until something matches. Two settings helped: // Meilisearch settings that helped on long documents { "stopWords": ["a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", "such", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", "with"] } …plus "matchingStrategy": "frequency" in the search request, which drops the most common words first. With both, SciFact rose from 0.378 to 0.501 and FiQA from 0.063 to 0.135. Still well behind BM25. If your search box is for product names, city names or a command palette, Meilisearch's typo tolerance and speed are what you want. If users type questions against articles, tickets or docs, it is the wrong tool. It was also the heaviest on disk. For the 171k TREC-COVID documents, the Meilisearch database grew to 1,891 MB and took 37.88 s to index, against 61.5 MB for the pg_textsearch index. Index build time and size TREC-COVID, 171,332 documents, 196.4 MB of raw table: | | Build time | Index size | | --- | --- | --- | | tsvector generated column + GIN | 26.31 s + 2.77 s | 65.7 MB GIN (plus the stored tsvector column) | | pg_search | 3.92 s | 82.6 MB | | pg_textsearch | 10.03 s | 61.5 MB | | Elasticsearch (bulk load + refresh) | 14.64 s | 98.1 MB store | | Meilisearch | 37.88 s | 1,891 MB database | The stored tsvector column nearly doubled the table, from 196.4 MB to 386.5 MB. The BM25 indexes don't need an extra column. Freshness: when does a new row show up in search? This is the difference users notice. A Postgres index is updated inside the same transaction as the insert, so the row is searchable the moment the commit returns: | Insert, then poll until found (median of 10, FiQA) | Time | | --- | --- | | pg_textsearch | 1.7 ms | | tsvector + GIN | 2.3 ms | | pg_search | 3.7 ms | | Meilisearch (async task queue) | 61.9 ms | | Elasticsearch (default 1 s refresh) | 705.8 ms | Elasticsearch makes new documents visible on its periodic refresh, 1 second by default. Meilisearch queues every write as an async task. Neither is a problem by itself, but with a separate search engine you also need a sync pipeline (outbox table, queue or CDC) that keeps it in step with the database, plus a plan for when it falls behind. With a Postgres extension there is nothing to sync, and a search filter can join your real tables and respect row-level security in the same query. Using Postgres BM25 search from Node.js There is no special client. It's a normal parameterized query, here with pg: // search.ts import pg from "pg"; const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL }); export async function search(q: string, limit = 10) { const { rows } = await pool.query( `SELECT id, title, pdb.score(id) AS score FROM docs WHERE title ||| $1 OR body ||| $1 ORDER BY pdb.score(id) DESC LIMIT $2`, [q, limit], ); return rows; } With Prisma or Drizzle, use their raw-SQL escape hatches ($queryRaw or the sql template) and keep the user input as a bound parameter. Never interpolate the query string into SQL. I compared the two ORMs in Drizzle vs Prisma in 2026. Because the search index lives next to your data, you can also combine it with vectors in the same database. pgvector installed fine alongside pg_search in my test database. Hybrid search (BM25 + embeddings, merged with reciprocal rank fusion) is a few lines of SQL from there. The vector side is covered in pgvector vs Pinecone vs Qdrant. Which should you use? - Already on Postgres, need a search box: start with pg_textsearch or pg_search. In my test they matched Elasticsearch's ranking, answered in single-digit milliseconds and needed no second system. pg_textsearch was the lightest and fastest. pg_search has the richer query language (field-level matching, fuzzy, phrase and more). Check that your host lets you install the extension. Managed Postgres providers support different extension lists. - Built-in tsvector: fine for "contains these words" filters and small tables. Don't use ts_rank for a natural-language search box. - Elasticsearch (or OpenSearch): when search is its own product. That means multiple nodes, billions of documents, aggregations and facets at scale, log analytics, many languages and custom analyzers. Budget for the sync pipeline and the cluster. - Meilisearch: instant, typo-tolerant search over short text, such as product catalogs, docs titles or app search. Not ranked retrieval over long text. If you are building a SaaS on Postgres, as I do with NestJS and Next.js (see the secure NestJS + Prisma API starter), a BM25 extension removes a whole service from the architecture. For more Postgres 18 numbers, see UUIDv7 vs UUIDv4 vs bigint primary keys. Limitations - One machine, one run per configuration. Small nDCG differences (under about 0.02) are within noise, especially on TREC-COVID's 50 queries. - Default or near-default settings. A tuned Elasticsearch analyzer, BM25 k1/b values or Meilisearch ranking rules could change the order. - Read-only query load from one client. I didn't test concurrent writes, replication, or very large corpora (millions of documents). - Versions: PostgreSQL 18, pg_search 0.26.2, pg_textsearch 1.5.1, Elasticsearch 9.5.5, Meilisearch 1.55.0. These extensions move fast, so re-check on newer releases. FAQ Is Postgres full-text search as good as Elasticsearch? With a BM25 extension, it was in my test. pg_textsearch and pg_search scored within about 0.05 nDCG@10 of Elasticsearch 9.5.5 on SciFact, FiQA and TREC-COVID. Built-in tsvector with ts_rank was clearly worse, because ts_rank has no inverse document frequency. Why does Postgres full-text search return no results for my query? websearch_to_tsquery and plainto_tsquery join every word with AND. In my test 274 of 300 SciFact queries returned zero rows. OR the terms or use a BM25 extension. Can I install pg_search and pg_textsearch together? Not in the same database in my test: both register an index access method named bm25. I ran them in two databases on the same server. --- Originally published on fabwaseem.com.

Most SSRF bugs aren’t a missing check. The check looks at the URL string while the connection goes somewhere else: a hex IP, an IPv4-mapped IPv6 address, or a hostname that resolves to 10.0.0.5 the second time. ressrf checks where it matters: after DNS & before connect. timescale/ressrf

Couple of weeks ago I showed what goes into RSigma and what comes out. Here's what happens in between, step by step: Steps 4 to 6 are optional. It's all one open-source Rust binary. Repo: timescale/rsigma Docs: rsigma.io

pg_vectorsearch vs pgvector: The 42-Hour Vector Index Just Became a 100-Minute One If you have ever kicked off a pgvector HNSW index build on a large embedding table and watched the hours tick by, the announcement Tiger Data made on October 13 reads like a targeted attack on your pain. Their new open-source extension, pg_vectorsearch, introduces an index called PRISM. On the same 128 GiB machine where a pgvector HNSW build over 100 million embeddings never finished at all, PRISM finished in 1 hour 40 minutes. On a machine 4x more expensive, HNSW needed 42 hours to do what PRISM did in 100 minutes. That is the headline. The useful question is what sits underneath it, because "should I switch my vector index" is a decision with real migration cost, and the answer in October 2026 is more conditional than the launch post makes it sound. Here is the breakdown: what PRISM actually is, what the numbers do and do not prove, and a checklist for deciding whether your workload should touch it yet. One disclosure up front: pg_vectorsearch shipped as an alpha one day before this article, so nobody outside Tiger Data has production experience with it, including me. Everything below comes from Tiger Data's announcement, their public benchmark tooling, and independent writeups. I will mark clearly which numbers are vendor-reported. What was actually announced Tiger Data, the company behind TimescaleDB, released pg_vectorsearch as an open-source PostgreSQL extension under the PostgreSQL license. It adds a new index access method called PRISM, short for Partitioned Routing Index for Similarity Matching. Three facts matter more than the name: - It is a native Postgres index. Not a sidecar service, not a separate database. You create an index the way you create a GiST or HNSW index, and your existing tooling, backups, and replication keep working. - It indexes your existing pgvector columns. No table rewrite, no data copy. A table already using pgvector's vector or halfvec types can get a PRISM index, and Tiger Data says your pgvector queries run unchanged. - It requires PostgreSQL 18 or later. That alone is a blocker for a large share of production deployments still on 15 through 17. The positioning detail that got less attention: Tiger Data describes pg_vectorsearch as the successor to pgvectorscale, their earlier DiskANN-based extension, which is heading into maintenance mode. If you adopted pgvectorscale last year on the strength of its Pinecone-comparison benchmarks, the roadmap just moved under you. That is worth knowing before your next infrastructure planning cycle. How PRISM works, without the paper HNSW, the index inside pgvector that most teams use, builds a multi-layer graph. Every vector is a node, and a search hops from node to node, narrowing down toward the nearest neighbors. The problem at scale is not compute. It is memory. Graph hops are unpredictable in advance, so the CPU spends most of its time stalled waiting on RAM, and the index itself has to stay resident. For 100 million vectors, Tiger Data measured the HNSW index at roughly 180 GB. PRISM takes the older IVF idea and hardens it. It groups similar vectors into k-means partitions, then stores each partition as compressed RaBitQ codes at roughly one bit per dimension, packed into whole database pages. A query finds the nearest few partitions and scans them sequentially. Sequential scans keep the CPU prefetcher busy and let SIMD instructions score many compressed vectors per instruction. Instead of a random-access graph walk, you get streaming passes over tightly packed pages. The result shows up in the storage number: 25 GB for the PRISM index versus 180 GB for HNSW on the same 100 million vectors. A 7x smaller index is what buys the cheap build and the small-machine query performance. The numbers, with their labels attached All figures below are from Tiger Data's launch benchmarks on 100 million Cohere embeddings, measured with their public benchmark tool, rekall, at recall@10 against exact brute-force results. They are vendor-reported, and every HNSW comparison used pgvector's halfvec type. - Build time at 100M vectors: HNSW could not finish building on a 128 GiB machine. On a 512 GiB machine, the build took 42 hours. PRISM built in 1 hour 40 minutes on the 128 GiB machine. - Query latency at 100M vectors: PRISM served 95% recall at a 6 ms median on the 128 GiB machine, versus HNSW's 8 ms on the machine with 4x the memory. - Index size: 25 GB for PRISM versus about 180 GB for HNSW. - Small scale holds up too: at 1 million vectors, PRISM served 99% recall in 15 ms on a 2 GiB machine. HNSW on a 4 GiB machine, twice the memory, took a reported 392 ms, and its build on that small box took about 21.7 hours while thrashing. Independent writeups of the announcement flagged the gaps fairly: the post does not state the embedding dimensionality of the test set, gives no throughput numbers, and contains no direct comparison against IVFFlat. Multi-client throughput testing is listed as still pending. None of that makes the numbers wrong, but it means the comparison you actually care about, PRISM versus whatever your workload does today on your hardware, does not exist yet. Run the rekall tool on your own data before believing any of this for your case. The alpha fine print is not small Tiger Data is unusually blunt about the state of the release, and the release notes deserve to be quoted in spirit: alpha quality, meant for testing, not production. Specifically: - Heavy churn needs a manual rebalance. If your table gets constant inserts, partition quality degrades until you intervene by hand. - Deletes do not reuse space yet. A delete-heavy workload will bloat the index with no built-in recovery. - Filtered search is on the GA roadmap, not in the alpha. If your RAG pipeline filters by tenant ID, date, or access control before similarity ranking, the alpha cannot do that efficiently. - Indexes should be recreated between prereleases. Whatever you build today is disposable. That last cluster of caveats points at the same conclusion: PRISM is for evaluation and read-mostly workloads right now, not for your production embedding table with daily churn. Should you switch? A decision checklist The interesting part of this launch is not "PRISM beats HNSW." It is that the threshold where teams felt forced to leave Postgres for a dedicated vector database just moved again. Save this checklist: - Under ~1 million vectors: stay on plain pgvector with HNSW. It works, it is boring, and PRISM's advantages are smallest exactly where your problem is easiest. This is most RAG projects, most internal search tools, most prototypes. - 1M to ~50M vectors, existing HNSW builds completing in acceptable time: stay put, but benchmark PRISM in staging. If your HNSW index no longer fits comfortably in memory and your bill is driven by RAM, PRISM's 7x smaller footprint is a direct cost argument. - 50M+ vectors, or HNSW builds that take days or fail: this is the audience the announcement was written for. Test PRISM now, on Postgres 18, in a non-production environment, with the rekall benchmark on your own embeddings. - Heavy insert/update churn or frequent deletes: wait for GA. The alpha's rebalance and space-reuse gaps hit exactly this pattern. - Filtering-heavy queries (tenant scoping, metadata filters): wait for GA. Filtered streaming search is on the roadmap, not shipped. - Currently on a dedicated vector DB (Pinecone, Qdrant, Weaviate) mainly for scale: this announcement is your quarterly reminder to re-run the stay-on-Postgres math. The direction of travel for two years has been Postgres absorbing the workloads that justified those services. - On Postgres 17 or earlier: the extension requires Postgres 18, so your first milestone is an upgrade, not an index change. The bigger story: the vector database market keeps shrinking Step back from the benchmark and look at the sequence. In 2023, running RAG meant standing up a dedicated vector database. Then pgvector made small-scale vector search a one-line extension install. Then pgvectorscale pushed Postgres into the tens of millions of vectors. Now PRISM is claiming the 100 million range on hardware a fraction of the cost, with no second system to operate, secure, back up, or keep consistent with your relational data. For a certain class of workload, the "we need a vector database" conversation is ending the same way the "we need a cache server" and "we need a search cluster" conversations ended for many teams: with a Postgres feature. Every team that chose a dedicated vector DB for scale reasons rather than feature reasons should keep a running estimate of what it would cost to move back, because the answer changes every quarter, and lately it changes in Postgres's favor. What I would actually do If I were planning a vector workload today: start every new project on plain pgvector regardless of expected size, because the exit cost from Postgres is low and the entry cost of a dedicated service is permanent. Revisit at the first sign of memory pressure, and at that point evaluate pgvectorscale's StreamingDiskANN for current production needs and PRISM for the 100M+ future, with the understanding that one is production-hardened and the other is a 48-hour-old alpha. And before any switch, generate your own numbers with rekall, because the only benchmark that matters is the one run on your embeddings, your hardware, and your recall target. I write about databases, AI infrastructure, and backend engineering every week. Subscribe, it is free, and it helps more than you know. Have you hit the ceiling of HNSW on a real workload? I am curious how long your largest index build took and whether it pushed you to a dedicated vector database or you stayed on Postgres. Tell me in the comments. Sources: Tiger Data's October 13 announcement (TimescaleDB on X), the pgvectorscale repository for the maintenance-mode note, and independent coverage of the launch benchmarks.

THE TUPLES ARE LEAKIN 🫤 pgvectorscale#276

pgvector vs Pinecone vs Qdrant: Benchmarked for RAG in 2026 I loaded the same 100,000 OpenAI embeddings into pgvector and Qdrant and measured recall, latency, index build and filtered search. Pinecone is covered from its docs only, since I didn't run it. The original, with any updates, lives on fabwaseem.com. pgvector is a PostgreSQL extension that stores embeddings in a vector column and searches them with HNSW or IVFFlat indexes, next to your normal tables. Qdrant is an open-source vector database you run yourself or use as a managed cloud. Pinecone is a fully managed, serverless vector database you only reach through an API. For a RAG app, the real question is whether you need a separate vector database at all. I loaded the same 100,000 OpenAI embeddings (1,536 dimensions) into pgvector 0.8.7 on PostgreSQL 18.6 and into Qdrant 1.19.2 on one machine. Then I measured recall, latency, build time and filtered search. Pinecone is covered from its official docs and limits only, because I didn't run it and I won't invent numbers for it. TL;DR: pgvector vs Pinecone vs Qdrant, which should I use? Start with pgvector if you already run Postgres and have up to a few million vectors. One database, SQL joins, transactions and backups you already have. Choose Qdrant when vector search is the core workload and you need the best recall per millisecond or heavy filtering. Choose Pinecone when you want zero infrastructure and accept a managed service with usage pricing. | | pgvector | Qdrant | Pinecone | | --- | --- | --- | --- | | What it is | Postgres extension | Vector database (open source) | Managed vector database (closed) | | Hosting | Anywhere Postgres runs | Self-host or Qdrant Cloud | Pinecone's cloud only | | Index | HNSW, IVFFlat | HNSW per segment | Managed (serverless, object storage) | | Max indexed dims | 2,000 vector, 4,000 halfvec | 1,536 tested here | 20,000 dense | | Filtering | SQL WHERE (post-filter, iterative scans) | Payload indexes + filter-aware HNSW | Metadata filters (40 KB per record) | | Consistency | Transactional (ACID) | Upserts readable after wait=true | Eventually consistent | | Recall@10 at ~2.6 ms, my test | 0.978 (ef_search 100) | 0.985 (hnsw_ef 40) | not tested | | Cost to start | Your Postgres | Free binary / free cloud tier | Free Starter; Standard $50/month minimum | Do I need a vector database for RAG? Usually not at first. If your app already uses PostgreSQL, pgvector adds vector search without a second system to deploy, secure, back up and keep in sync. In my test it answered top-10 queries over 100,000 OpenAI embeddings in 1.5-4 ms, with 95-99% recall. A dedicated database starts to pay off at very large scale, under heavy filtering, or when search load needs to scale separately. The hidden cost of a separate vector store is synchronisation. Every document insert, update and delete has to reach two systems, and a failed write leaves them disagreeing. With pgvector the embedding sits in the same row and the same transaction as the document it describes. What is pgvector? pgvector is an open-source PostgreSQL extension that adds vector, halfvec, bit and sparsevec types, distance operators like <=> (cosine) and <-> (L2), and approximate HNSW and IVFFlat indexes. Per the pgvector README, it "performs exact nearest neighbor search, which provides perfect recall" by default. An index trades a little recall for speed. CREATE EXTENSION vector; CREATE TABLE documents ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, tenant_id int NOT NULL, content text NOT NULL, embedding vector(1536) NOT NULL ); CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops); -- top 5 chunks for a question, inside one tenant SELECT id, content FROM documents WHERE tenant_id = $1 ORDER BY embedding <=> $2 LIMIT 5; Index limits matter for model choice. HNSW can index a vector of "up to 2,000 dimensions" and a halfvec of "up to 4,000 dimensions". A 3,072-dimension model such as text-embedding-3-large needs a halfvec index or a shorter output size. What are Qdrant and Pinecone? Qdrant is an open-source vector search engine written in Rust. It stores points (vectors plus a JSON payload), builds an HNSW index per segment, and adds payload indexes so filters run inside the graph search. You can run a single binary, Docker or Qdrant Cloud. Pinecone is a managed, serverless vector database. Per its architecture docs, "all your vector data is stored in highly efficient, distributed object storage." Pinecone "separates write and read operations into different paths, with each scaling independently based on demand." There is no self-hosted version. How I benchmarked pgvector and Qdrant 1. Data. The first 100,000 rows of the public DBpedia OpenAI embeddings dataset: 1,536-dimension text-embedding-ada-002 vectors of Wikipedia entities, a typical RAG shape. 1,000 held-out rows from the same dataset served as queries. 2. Ground truth. Exact cosine top-10 for every query, computed with NumPy. Recall@10 is the share of those 10 that each engine returned. 3. Machine. One 8-vCPU, 15 GB Linux box, everything on localhost, no Docker. 4. pgvector. PostgreSQL 18.6 with pgvector 0.8.7, shared_buffers = 2GB, maintenance_work_mem = 2GB. HNSW with defaults (m = 16, ef_construction = 64), loaded with binary COPY through psycopg 3. 5. Qdrant. The Qdrant 1.19.2 release binary, cosine distance, m = 16 and ef_construct = 64 to match pgvector, default optimizer settings (it made 4 segments). Loaded with the official Python client over gRPC from a single process. 6. Queries. One client, sequential, 50 warm-up queries, then 1,000 timed queries. Latency includes the Python client round trip on both sides. 7. Filters. Every vector got a random category from 0 to 9, about 10% of rows each. Filtered queries asked for the top 10 inside one category. Is pgvector fast enough? Recall and latency results Yes, at this scale both engines are fast. pgvector reached 0.948 recall@10 at 1.45 ms median with default settings, and 0.988 at 4.0 ms with hnsw.ef_search = 200. Qdrant reached 0.985 recall at 2.6 ms median and 0.999 at 5.4 ms. At roughly equal recall (about 98.5-98.8%), Qdrant was about 1.5 times faster: 2.6 ms against 4.0 ms. !pgvector vs Qdrant recall@10 vs median latency on 100,000 OpenAI embeddings: Qdrant 0.985 at 2.6 ms, pgvector 0.978 at 2.6 ms and 0.988 at 4.0 ms | Engine and setting | Recall@10 | p50 latency | p95 latency | Queries/s (1 client) | | --- | --- | --- | --- | --- | | pgvector, ef_search = 40 (default) | 0.948 | 1.45 ms | 2.16 ms | 683 | | pgvector, ef_search = 100 | 0.978 | 2.60 ms | 3.70 ms | 389 | | pgvector, ef_search = 200 | 0.988 | 4.02 ms | 5.82 ms | 249 | | pgvector halfvec index, ef_search = 100 | 0.973 | 2.33 ms | 3.30 ms | 433 | | Qdrant, hnsw_ef = 40 | 0.985 | 2.63 ms | 3.56 ms | 368 | | Qdrant, hnsw_ef = 100 | 0.997 | 3.73 ms | 4.66 ms | 255 | | Qdrant, hnsw_ef = 200 | 0.999 | 5.37 ms | 6.99 ms | 176 | | pgvector exact (no index) | 1.000 | 380 ms | 445 ms | 3 | | Qdrant exact (no index) | 1.000 | 24.7 ms | 32.6 ms | 39 | Why does Qdrant get higher recall at the same ef? It splits the collection into segments (4 here) and searches each one with that ef, so it explores more candidates in total. Compare the engines at equal recall, not equal settings. How fast do they load and index? Qdrant indexed faster: 30.8 seconds from the first upload to a fully indexed collection, with the upload itself taking 20.2 s from one Python process. pgvector loaded the rows with COPY in 3.9 s, then needed 69.1 s to build the HNSW index with the default 2 parallel workers. | | pgvector | Qdrant | | --- | --- | --- | | Load 100k vectors | 3.9 s (COPY) | 20.2 s (Python client, gRPC) | | Build HNSW index | 69.1 s | included, indexed at 30.8 s | | Table / data size | 798 MB | — | | HNSW index size | 781 MB (vector), 391 MB (halfvec) | — | | Server memory after test | Postgres buffers + OS cache | 1,293 MB RSS | Use the halfvec trick in pgvector: indexing embedding::halfvec(1536) halved the index from 781 MB to 391 MB and cost about half a point of recall (0.973 against 0.978 at ef_search = 100). Query the index with the same cast: CREATE INDEX ON documents USING hnsw ((embedding::halfvec(1536)) halfvec_cosine_ops); SELECT id FROM documents ORDER BY embedding::halfvec(1536) <=> $1::halfvec(1536) LIMIT 10; How does filtered vector search compare? Qdrant handled filters best. With a payload index on category, filtered top-10 queries returned 0.993 recall at 2.1 ms median. pgvector gave three very different results depending on the plan Postgres picked, so you need to know how its filtering works. | Filtered query (10% of rows match) | Rows returned (of 10) | Recall@10 | p50 | | --- | --- | --- | --- | | pgvector, planner's choice (B-tree on category, exact sort) | 10 | 1.000 | 44.2 ms | | pgvector, HNSW, iterative_scan = off | 3.92 | 0.389 | 1.77 ms | | pgvector, HNSW, iterative_scan = strict_order | 10 | 0.915 | 3.62 ms | | pgvector, HNSW, iterative_scan = relaxed_order | 10 | 0.953 | 3.37 ms | | Qdrant, payload index | 10 | 0.993 | 2.11 ms | Why does pgvector return fewer rows with a filter? Because HNSW filters after the index scan. The README explains: "If a condition matches 10% of rows, with HNSW and the default hnsw.ef_search of 40, only 4 rows will match on average." I measured 3.92. Since pgvector 0.8.0, iterative index scans keep walking the graph until enough rows match. -- once per session or per transaction (SET LOCAL) SET hnsw.iterative_scan = relaxed_order; SELECT id, content FROM documents WHERE tenant_id = $1 ORDER BY embedding <=> $2 LIMIT 10; Postgres often picks the exact plan by itself for a selective filter. In my test it used the category B-tree and sorted about 10,000 rows exactly: perfect recall, but 44 ms instead of about 3 ms. That is fine for one tenant with a few thousand chunks. For large tenants, check EXPLAIN and turn on iterative scans. If one filter dominates, consider a partial index or partitioning by tenant. What does Pinecone offer that pgvector and Qdrant don't? Fully managed, serverless scaling with nothing to run. Reads and writes scale separately, storage lives in object storage, and you never tune ef or m. The trade-offs are documented: - Eventual consistency. The known limitations page says "Pinecone is eventually consistent, so there can be a slight delay before upserted records are available to query." - Operation limits. Per the operation limits: dense vectors up to 20,000 dimensions, 40 KB of filterable metadata per record, 1,000 records or 2 MB per upsert batch, and top_k up to 10,000. - Plans. The object limits give the free Starter plan 2 GB of serverless storage and 5 indexes per project, all in AWS us-east-1. Pinecone pricing lists the Standard plan at a $50/month minimum. - Network hop. Every query leaves your VPC. Expect your cloud region's round-trip time on top of search time. My localhost latencies are not comparable to Pinecone and I did not measure it. When should I choose each one? Decide by what you already run and what dominates your workload. The pgvector default is strong for product teams. Move when you measure a reason, not before. !Decision guide: choose pgvector if you already run Postgres, Qdrant when search is the product, Pinecone for zero infrastructure 1. Choose pgvector when your data lives in Postgres, you need joins, row-level security or transactions with your embeddings, and you have up to a few million vectors per index. Add pgvectorscale (StreamingDiskANN) if the index outgrows RAM. 2. Choose Qdrant when search is the product: tens of millions of vectors, many filters, multi-vector or hybrid search, or quantization to cut memory. Its quantization guide says scalar quantization achieves "4x compression with minimal accuracy loss." 3. Choose Pinecone when nobody on the team should run a database, traffic is spiky, and a usage bill plus eventual consistency are acceptable. How do I move from pgvector to Qdrant later? Keep embeddings and IDs portable from day one. Use the same primary key in both systems, store the embedding model name and dimensions with each row, and write a re-sync job that streams rows by ID. Qdrant accepts integer or UUID point IDs, so a UUIDv7 primary key maps across directly. Then run both side by side, compare recall on a fixed set of real queries, and switch reads before you stop writing to Postgres. Common mistakes with vector search in production - Indexing 3,072-dimension vectors as vector. HNSW on vector stops at 2,000 dimensions. Use halfvec or a smaller output dimension. - Forgetting iterative scans. Filtered HNSW queries in pgvector can silently return fewer than LIMIT rows. - Comparing engines at equal settings. ef means different work in different engines. Compare at equal recall. - Measuring recall on random vectors. Use real embeddings from your model. Random data hides graph quality. - Creating Qdrant payload indexes after loading. The indexing docs say payload indexes should be created before ingesting data, so filter-aware HNSW edges get built. - Ignoring re-embedding cost. Switching embedding models means re-embedding everything, whichever store you use. If you want a working starting point, my open-source Next.js AI starter kit, Forgekit, ships RAG on pgvector. For self-hosting Postgres or Qdrant next to your app, see self-hosting Next.js with Coolify. Key takeaways - With 100,000 OpenAI embeddings, pgvector 0.8.7 hit 0.948 recall@10 at 1.45 ms with defaults and 0.988 at 4.0 ms with ef_search 200. - Qdrant 1.19.2 hit 0.985 recall at 2.6 ms, about 1.5x faster than pgvector at roughly equal recall. - pgvector's HNSW filters after scanning: a 10% filter returned only 3.92 of 10 rows until iterative scans were enabled. - Qdrant's payload index gave the best filtered search in my test: 0.993 recall at 2.1 ms. - A halfvec HNSW index halved pgvector's index size (781 MB to 391 MB) for about half a point of recall. - Pinecone is serverless and eventually consistent, supports 20,000-dimension vectors, and its Standard plan has a $50/month minimum. - Start with pgvector if you already run Postgres; move to Qdrant or Pinecone when measurements show a need. FAQ Is pgvector as good as Pinecone or Qdrant? For most RAG apps up to a few million vectors, yes. In my test of 100,000 OpenAI embeddings, pgvector reached 0.988 recall@10 at 4.0 ms median, and Qdrant reached 0.985 at 2.6 ms. Qdrant was about 1.5 times faster at equal recall and better at filtered search, but pgvector keeps embeddings in the same transactional database as your data. Do I need a vector database for RAG? Not usually at the start. If your app already uses PostgreSQL, the pgvector extension adds vector columns and HNSW indexes without a second system to deploy, back up and keep in sync. Consider a dedicated vector database when you reach tens of millions of vectors, need heavy filtering at low latency, or need to scale search separately. Why does pgvector return fewer results than my LIMIT? Because pgvector's HNSW index applies WHERE filters after scanning the index. With a filter matching 10 percent of rows and the default ef_search of 40, about 4 rows match on average; I measured 3.92. Set hnsw.iterative_scan to relaxed_order or strict_order, available since pgvector 0.8.0, so the scan continues until enough rows match. How many dimensions can pgvector index? pgvector's HNSW and IVFFlat indexes support the vector type up to 2,000 dimensions, halfvec up to 4,000 and bit up to 64,000. For a 3,072-dimension model such as text-embedding-3-large, index the column as halfvec or request a smaller output dimension. Pinecone supports dense vectors up to 20,000 dimensions. Is Qdrant faster than pgvector? In my single-machine test, yes, at equal recall. Qdrant 1.19.2 returned 0.985 recall@10 at 2.6 ms median against pgvector's 0.988 at 4.0 ms, and indexed 100,000 vectors in about 31 seconds against pgvector's 69-second HNSW build. With default settings, pgvector was faster per query but at lower recall, 0.948 at 1.45 ms. How much does Pinecone cost? Pinecone has a free Starter plan with 2 GB of serverless storage and up to 5 indexes per project, all in AWS us-east-1. Its Standard plan for production has a $50 per month minimum, after which you pay as you go for usage, according to Pinecone's pricing page. Qdrant's open-source build and pgvector are free to self-host. What is halfvec in pgvector? halfvec is pgvector's half-precision vector type, storing each dimension in 2 bytes instead of 4. Indexing an expression like embedding::halfvec(1536) halves the HNSW index size; in my test it shrank from 781 MB to 391 MB with recall dropping from 0.978 to 0.973. It also raises the indexable limit to 4,000 dimensions. Is Pinecone eventually consistent? Yes. Pinecone's documentation states that it is eventually consistent, so there can be a slight delay before upserted records can be queried, and suggests checking index stats to confirm counts. pgvector writes are transactional and visible on commit, and Qdrant upserts sent with wait=true are applied before the call returns. --- Originally published on fabwaseem.com.