Your Vector Database Thinks "Not Covered" Means "Covered"
TL;DR: An embedding's similarity score measures how much two texts share in topic and wording, and a negation is a one-word edit. I wrote 40 invented policy facts and tested six open embedding models. On every model, "Which plan doesn't cover X?" scored closer to "Which plan covers X?" than a plain rewording of the same question did, for 77.5–100% of the items. At a common semantic-cache cut-off (similarity 0.90, RedisVL's default), one model would serve the cached answer to 97.5% of the opposite questions and to 7.5% of the genuine rewordings. In retrieval, models found the "covered" chunk almost every time and the "not covered" chunk between 27.5% and 92.5% of the time. Adding BM25 through rank fusion made five of the six worse. Dropping English stop words, which removes "not", took BM25 to a coin flip. Re-ranking the top ten with a small cross-encoder took every model to 97.5% or better on the negated questions. Run the same test on your own model before you trust a similarity threshold.
A benefits assistant had a semantic cache in front of it, so repeated questions skipped the LLM. On Monday someone asked "Which plan covers dental implants?" and got the right answer: the Gold plan. On Tuesday someone else asked "Which plan doesn't cover dental implants?"
The cache embedded the new question, found Monday's at similarity 0.97, and returned "The Gold plan." That's the opposite of the right answer, delivered in a fraction of a second with no LLM call to blame. Someone asking "What plan pays for dental implants?", which means exactly the same as Monday's question, would have scored 0.85 and missed the cache.
The assistant is invented. The two numbers aren't: they're what all-MiniLM-L6-v2, the most downloaded sentence-similarity model on Hugging Face (about 200 million downloads a month in October 2026), gives for those three questions. The rest of this article is what happened when I ran the same comparison across six models, three tests and 40 facts.
What the score is actually measuring
Embedding models are trained to put texts that go together (a question and its answer, a title and its abstract, two captions of one picture) close together. Texts that go together share a topic and, usually, a lot of words. So the model learns a geometry where shared topic and shared wording count for most of the distance, and nothing in training forces a single "not" to move a sentence to the other side of the space.
From the vector's point of view, "Which plan doesn't cover dental implants?" is the original question with one word added. "What plan pays for dental implants?" swaps two words and adds a third. The negation is the smaller edit, so it lands closer.
This isn't new. NevIR (Weller, Lawrie and Van Durme, EACL 2024) built document pairs that differ only by negation and found most retrieval models, including state-of-the-art ones, "performing the same or worse than a random ranking", with cross-encoders doing best. Cao (ECAI 2025) tested general-purpose text embeddings and reported "a significant lack of negation awareness in these models". A June 2026 preprint on agent memory (Yadav, arXiv 2606.26511) reported that cosine similarity separated contradicted facts from duplicates with an AUROC of 0.59, close to chance, and gave the same reason: contradictions came out more similar to the original than duplicates did "because a value-flip is a smaller edit than a genuine rephrasing of the same fact." That result is small (98 labelled pairs, one embedding model), so I wanted to see how far it generalised. An August 2026 audit of similarity thresholds in agent frameworks (Frias, arXiv 2608.10216) is the closest to what follows: a production drift guard "caught 0 of 56 meaning-breaking mutations", including "withhold the study drug" → "administer the study drug", which "came in at cosine 0.9608".
Six models, 40 facts, three tests
I wrote 40 short policy facts across twelve domains: dental and health plans, home, travel and car insurance, refunds, software tiers and licences, staff benefits, airline fares, warranties, bank accounts, gym memberships and drug formularies. Each fact has three forms:
- the fact: "The Gold plan covers dental implants."
- a rewording with the same meaning: "Dental implants are paid for under the Gold plan."
- a contradiction: "The Gold plan does not cover dental implants."
The contradictions use several kinds of negation on purpose: "does not", "isn't", "excludes", "no", "never", "unpaid", "non-refundable". Each fact also has three questions: one asking which entity covers it, a rewording of that question, and the opposite question ("Which plan doesn't cover dental implants?"). Most opposite questions carry a negation word, but a few flip the meaning without one: "Which tier limits data exports?", "Who gets unpaid parental leave?", and "Which account charges foreign transaction fees?", where it's the original that says "no".
The six models are all open and run locally: all-MiniLM-L6-v2, bge-base-en-v1.5, e5-base-v2, gte-base, mxbai-embed-large-v1 and Qwen3-Embedding-0.6B, each with the query and passage prefixes its model card asks for. I didn't test hosted embedding APIs, which is the main reason to run the check below on whatever you use.
Test 1: questions, the semantic-cache case
For each of the 40 items: how similar is the original question to its rewording, and to its opposite?
| Model | Mean similarity, rewording | Mean similarity, opposite | Opposite closer than rewording | Served at 0.90: rewordings | Served at 0.90: opposites |
|---|---|---|---|---|---|
| all-MiniLM-L6-v2 | 0.74 | 0.96 | 100% | 7.5% | 97.5% |
| bge-base-en-v1.5 | 0.84 | 0.93 | 90% | 15% | 87.5% |
| e5-base-v2 | 0.92 | 0.97 | 92.5% | 70% | 100% |
| gte-base | 0.92 | 0.97 | 90% | 72.5% | 100% |
| mxbai-embed-large-v1 | 0.83 | 0.89 | 77.5% | 12.5% | 37.5% |
| Qwen3-Embedding-0.6B | 0.81 | 0.93 | 90% | 5% | 85% |
The last two columns use 0.90, which is what RedisVL's SemanticCache defaults to (a cosine distance of 0.1). That's RedisVL's threshold applied to these six models, not RedisVL's own default embedding model, which I didn't test. On every model, a cache at that setting answers the opposite question more often than it answers the same question asked differently.
You can't tune your way out of it. Set the threshold low enough to catch 95% of genuine rewordings on each model, and it catches all 40 opposites, on all six. As a separation score, similarity managed an AUROC between 0.01 and 0.25, where 0.5 is chance. A number that low means the score is reliably backwards: higher similarity was evidence that the question meant the opposite.
Test 2: statements, the deduplication case
The same comparison between the fact, its rewording and its contradiction. This is what a deduplication step or a memory store faces when it decides whether a new fact is "the same" as one it already has.
| Model | Contradiction closer than rewording | AUROC, rewording vs. contradiction |
|---|---|---|
| all-MiniLM-L6-v2 | 52.5% | 0.46 |
| bge-base-en-v1.5 | 2.5% | 0.94 |
| e5-base-v2 | 47.5% | 0.57 |
| gte-base | 15% | 0.81 |
| mxbai-embed-large-v1 | 0% | 0.98 |
| Qwen3-Embedding-0.6B | 55% | 0.43 |
This one is mixed, and it's worth saying so. Three models separate statements well, and mxbai separates them almost perfectly. Three don't: MiniLM, e5 and Qwen3 sit near chance, so "X is not covered" is as likely as a rewording to look like a duplicate of "X is covered". The models that cope with statements still failed the question test. Part of that is my data: the question rewordings change more words than the statement rewordings do, and on every model the rewordings score lower for questions. That's also how people actually ask questions, so I don't think it makes the question result less real. The safe conclusion isn't "embeddings can't do negation". It's that it depends on the model and the text shape, and you can't know which case you're in without measuring.
Test 3: retrieval, and what fixes it
The cache and deduplication cases compare two texts directly. Retrieval is subtler, because the query and the chunk aren't phrased alike. So I built an index where every item contributes two chunks about different entities: "The Gold plan covers dental implants" and "The Silver plan does not cover dental implants". Then I asked both questions. "Which plan covers dental implants?" should rank Gold's chunk above Silver's, and "Which plan doesn't cover dental implants?" should do the reverse. All 80 chunks sit in the same index, so the other 39 items act as distractors.
Every dense model answered the positive questions well (90–100%). The negated questions are where they split:
| Model | Dense alone | Dense + BM25 (rank fusion) | Dense top 10, re-ranked: ms-marco-MiniLM-L-6-v2 | Dense top 10, re-ranked: bge-reranker-base |
|---|---|---|---|---|
| all-MiniLM-L6-v2 | 27.5% | 32.5% | 97.5% | 100% |
| bge-base-en-v1.5 | 85% | 70% | 97.5% | 100% |
| e5-base-v2 | 92.5% | 77.5% | 97.5% | 100% |
| gte-base | 70% | 57.5% | 97.5% | 100% |
| mxbai-embed-large-v1 | 77.5% | 65% | 97.5% | 100% |
| Qwen3-Embedding-0.6B | 80% | 72.5% | 97.5% | 100% |
Share of the 40 opposite questions ("doesn't", "excludes", "no", "unpaid", "limits" and so on) where the right chunk ranked above its opposite. A coin flip scores 50%.
The MiniLM row is worth a second look. Counting an item as right only when both its questions came out right, the way NevIR scores it, MiniLM got 17.5%. Guessing at random would get 25%.
The fix everyone recommends made it worse
The standard advice for negation is hybrid retrieval: the "not" is a keyword, so add BM25 and fuse the rankings. I believed that going in, and it's what the outline for this article said.
In this test, BM25 on its own got 77.5% of the negated questions right, which beats two of the dense models and ties a third. Fusing it with them using reciprocal rank fusion (Cormack, Clarke and Büttcher, 2009, k = 60) lowered the score for five of the six. Only MiniLM improved, from 27.5% to 32.5%. The reason is in the wording. The keyword arm only sees the negation when the question and the chunk use the same word. In 25 of the 40 items they did ("excludes" and "excludes", "does not" and "does not"), and BM25 got all 25 right. In the other 15 they didn't ("doesn't" against "not covered", "won't refund" against "non-refundable", "no API access" against "unavailable"), and BM25 got 6 right, which is below a coin flip. There it rewards whichever chunk shares more terms with the question, and that's often the wrong one. Rank fusion gives that vote the same weight as the dense model's.
There's a second trap underneath, and it's a configuration setting. Elasticsearch's english analyzer and its stop token filter both default to the _english_ stop list, which is Lucene's ENGLISH_STOP_WORDS_SET. That list contains "no" and "not". So the keyword that's supposed to rescue you is deleted before it reaches the index. My BM25 is a simple Python one (rank_bm25, lowercase word tokens, no stemming), not Elasticsearch itself, but I gave it the same stop list. With that list applied, it scored exactly 50% on the negated questions, a coin flip, and every hybrid score dropped further. (The standard analyzer keeps stop words by default. Check which one your text fields actually use.)
What worked was the slow, boring stage. A cross-encoder reads the question and the chunk together, so "doesn't" can change its answer in a way that two separately computed vectors can't. Re-ranking each dense model's top ten with ms-marco-MiniLM-L-6-v2, a 22-million-parameter model, took every row to 97.5% on the negated questions. bge-reranker-base took them all to 100%. The dense model you started with stopped mattering here, because the right chunk and its opposite were both in every top ten. The ordering inside it was the problem. That's an easy case: an 80-chunk index where the two rivals share most of their words. In a large index full of near-identical clauses, check that both make it into the candidates you re-rank. This matches NevIR's ranking, and it's why re-ranking gets a whole lesson of its own in SophiArch's Retrieval & RAG Systems course, two lessons after the one on hybrid retrieval: the two stages fix different failures.
The re-rankers aren't perfect, and both made a couple of mistakes in the other direction. bge-reranker-base put "The standard warranty excludes accidental drops" above "The extended warranty covers accidental drops" for "Which warranty covers accidental drops?". ms-marco-MiniLM missed "Which account offers no overdraft?" and "Who refunds return shipping costs?". e5 also missed the overdraft question, and the opposite of the shipping one ("Who doesn't refund return shipping costs?"). That's 2 misses in 80 questions for each re-ranker, against 3 for the best dense model (e5) and 33 for MiniLM. One caution from Frias's audit: swapping in an NLI model as the gate "did no better" there. A re-ranker trained for search relevance and an NLI classifier aren't the same tool, and short policy questions are an easy case. So re-run the triples with the re-ranker in the loop before you rely on it.
Try it on your model before you set a threshold
This takes a few minutes and needs no labelled data. Write 20 question triples from your own domain: a question, a rewording, and its opposite. Then run:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("your-embedding-model") # or call your hosted embedding API
triples = [
("Which plan covers dental implants?",
"What plan pays for dental implants?",
"Which plan doesn't cover dental implants?"),
# ... 19 more from your own domain
]
backwards = 0
for question, reworded, opposite in triples:
q, r, o = model.encode([question, reworded, opposite], normalize_embeddings=True)
sim_r, sim_o = float(q @ r), float(q @ o)
backwards += sim_o > sim_r
print(f"{sim_r:.2f} {sim_o:.2f} {question}")
print(f"opposite closer than rewording: {backwards}/{len(triples)}")
If the opposite wins most rows, and on every model I tried it did, then no similarity threshold on that model can tell a repeat question from its opposite. Use the same triples for statements if you deduplicate or merge facts.
Where each fix goes
- Semantic caches. Don't serve a hit on similarity alone. The cheapest guard is a polarity check: if one question contains a negation word or a word like "exclude" or "without" and the other doesn't, treat it as a miss. A stronger one is to score the (new question, cached question) pair with a cross-encoder before serving. Either way, test the cache with opposite questions, not just rewordings, since rewordings are the only thing most cache evaluations try.
- Retrieval. Re-rank the top k with a cross-encoder. In this test, putting it after any of the six dense models beat every combination without one. If you also run BM25, check the analyzer's stop list for "no" and "not", and measure whether fusion helps on your queries rather than assuming it does.
- Deduplication and memory. Don't merge or drop a fact because it's close to one you already have. A contradiction is often closer than a duplicate. Compare the claims themselves, with a cross-encoder or an LLM check, before deciding they're the same.
What this test doesn't show
The facts are invented, short and in English, and there are 40 of them. Real policy text is longer, and a longer chunk dilutes the negation further, so I'd expect statement-level separation to get worse, not better. The retrieval index has 80 chunks, so both rivals always reached the top ten. In a large index, recall into the re-ranker becomes a separate thing to measure. The BM25 numbers come from a simple Python implementation without stemming, not from an Elasticsearch analyzer, so treat them as a direction, not a figure for your cluster. I only tested open models; hosted embedding APIs may do better or worse, which is the point of running the triples yourself. And the rewordings change more words than the negations do. That's deliberate, because it's the comparison a cache faces in production (people reword freely, and a negation is one word), but it means the results describe how real questions differ, not a controlled single-word edit.
None of it means vector search is broken. It means a similarity score answers "are these about the same thing?" and gets used as if it answered "do these mean the same thing?". For dental implants, those are opposite answers.
References
- Weller, O., Lawrie, D., & Van Durme, B. (2024). NevIR: Negation in Neural Information Retrieval. arXiv:2305.07614 (EACL 2024). Negation-pair benchmark; most retrievers perform at or below random, cross-encoders best.
- Cao, H. (2025). Semantic Adapter for Universal Text Embeddings: Diagnosing and Mitigating Negation Blindness to Enhance Universality. arXiv:2504.00584 (ECAI 2025). Negation awareness in general-purpose text embeddings.
- Yadav, N. (2026). Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge. arXiv:2606.26511 (v1). Cosine similarity separating contradicted facts from duplicates at AUROC 0.59; the "smaller edit" explanation.
- Frias, S. E. (2026). Similarity Gates Approve Reversals: A Validity Audit of Embedding-Cosine Thresholds in Agent Systems. arXiv:2608.10216 (v1). Cosine-threshold gates in agent frameworks (deduplication, semantic caches, drift guards) approving meaning reversals; the NLI drop-in result.
- Cormack, G. V., Clarke, C. L. A., & Büttcher, S. (2009). Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods. SIGIR 2009. The rank-fusion method used for the hybrid rows (k = 60).
- Redis. RedisVL cache API: SemanticCache. Default distance_threshold=0.1 in cosine distance units.
- Elastic. Language analyzers, Stop token filter and Standard analyzer. The english analyzer's _english_ stop list and the standard analyzer's _none_ default.
- Apache Lucene. EnglishAnalyzer.java. ENGLISH_STOP_WORDS_SET, which includes "no" and "not".
- Model cards for the six embedding models and two re-rankers tested: all-MiniLM-L6-v2, bge-base-en-v1.5, e5-base-v2, gte-base, mxbai-embed-large-v1, Qwen3-Embedding-0.6B, ms-marco-MiniLM-L-6-v2, bge-reranker-base.
- SophiArch. Retrieval & RAG Systems. Course covering chunking, hybrid retrieval with rank fusion, query rewriting, cross-encoder re-ranking, retrieval metrics and RAG evaluation.
Mentions
Lucene 10.6.0 の doc-values updates は列ごとの書き直しをどうやめたのか はじめに OpenSearch や Elasticsearch では、検索したいデータを JSON のドキュメント（doc）として index に保存します。 doc は title や price のような field の集まりで、OpenSearch や Elasticsearch はこの field の値を使って検索、ソート、集計をします。 この index を実際にディスクに書き、検索するのが、Java で書かれた検索ライブラリの Apache Lucene です。 Lucene は index を segment という単位のファイル群に分けて書き、一度書いた segment は変更しません。 こうして保存する doc には、一部の field だけが頻繁に変わるものがよくあります。 EC の在庫数や価格、記事の閲覧数、ランキング用の人気スコアなどです。 doc 本体には本文や 512 次元の vector が入っていて、変わるのは 8 バイトの数値 1 つだけ、という形です。 新しい doc は新しい segment に書かれ、既存の segment のファイルはそのまま残ります。 Lucene でこの数値を変える方法は 2 つあります。 1 つは IndexWriter#updateDocument で doc ごと入れ直す方法で、本文も vector も index し直すことになります。 もう 1 つは IndexWriter#updateNumericDocValue で、doc-values の値だけを書き換える方法です。 doc-values は、Lucene が field の値を doc ID 順に並べて列として持つ構造です。 全文検索に使う転置インデックスは「語から doc」を引く向きの構造で、doc-values はその逆の「doc から値」を引く向きです。 ソート、ファセット、集計は、この列を doc ID で引いて値を読みます。 OpenSearch や Elasticsearch の mapping にある doc_values は、この列を作るかどうかの設定で、text 以外の多くの field では default で有効です。 updateNumericDocValue は doc を入れ直さないので、軽い操作に見えます。 ところが Lucene 10.5 までの doc-values update は、別のものを丸ごと書き直していました。 更新した doc の数に関係なく、その field の列（column）を segment 1 つぶん書き直すのです。 2026 年 10 月 29 日にリリースされた Apache Lucene 10.6.0 で、この書き直しをやめる仕組みが入りました。 CHANGES.txt の New Features には次のように書かれています。 GITHUB#16418: Incremental doc-values updates. A set-only doc-values update (NUMERIC or BINARY) is written as a sparse "delta" overlay holding just the updated documents and layered over the base column at read time, instead of rewriting the whole column, turning per-update write amplification from O(column) into O(updated docs). Opt-in; enable with IndexWriterConfig#setMaxDocValuesOverlays(n). 気になるのは、書き込み量がどれだけ減り、代わりに何を払うのか、そして OpenSearch と Elasticsearch にどう関わるのかです。 元になった Pull Request #16418 と、10.5.0 と 10.6.0 のソースを読んで確かめました。 ! 本文のクラス名とメソッド名は Lucene 10.5.0 と 10.6.0 のタグのものです。 issue や PR の状態は 2026 年 11 月 1 日時点で確認しました。 setMaxDocValuesOverlays には @lucene.experimental が付いており、今後挙動が変わる可能性があります。 1 doc を更新すると何が書かれるのか gen ファイルに書かれる列 doc-values の列も segment の一部なので、segment と同じく一度書いたら変更しません。 値を更新するには、別のファイルを書き足す必要があります。 Lucene はこの書き足しを generation（gen）という番号で管理しています。 doc-values を更新すると、その field について新しい gen のファイルを書き、読み取り時には最新の gen を使う仕組みです。 10.5.0 の ReadersAndUpdates#handleDVUpdates では、次の gen 番号を 36 進数の文字列にしてファイル名の suffix にしています。1 final long nextDocValuesGen = info.getNextDocValuesGen(); final String segmentSuffix = Long.toString(nextDocValuesGen, Character.MAX_RADIX); 問題は新しい gen ファイルの中身です。 同じメソッドの中で、MergedDocValues が既存の列の値と更新値を doc ID 順に混ぜ合わせ、その結果を新しい gen として書き出します。 更新されなかった doc の値も含めて、列全体です。 新しい gen ファイルの大きさは列全体（値を持つすべての doc）で決まり、更新した doc が 1 件でも 3 件でも同じです。 PR の説明はこの状態を「Lucene rewrites the whole column for that field, even if you only touched a few docs」と書いています。 segment が大きくなるほど、1 回の更新で書く量も増えます。 soft delete の場合 この書き直しは明示的に doc-values を更新したときだけの話ではありません。 Lucene の soft delete は、削除の印を soft-deletes 用の numeric doc-values field に書くことで実現しています。 つまり soft delete も doc-values update であり、同じ経路で列を書き直します。 soft-deletes の列は印を付けた doc だけを持つので、1 回に書く量は数値 field の列ほど大きくありません。 Elasticsearch は操作の履歴を soft delete した doc として Lucene に残しています。 shard の復旧や CCR（cross-cluster replication）で、この履歴を使うためです。 後で見る Elasticsearch の PR #157749 の説明にも、この仕組みが出てきます。 OpenSearch は、この仕組みを持つ Elasticsearch 7.10 から分岐したプロジェクトです。 そのため、doc-values を明示的に更新していない index でも、update や delete で doc を置き換えるたびにこの経路を通ります。 10.6.0 で入った overlay 読み取り時の重ね方 PR の説明によると、10.6.0 で入った仕組みの考え方は単純です。 値を設定するだけの更新（set-only）なら、変わった doc だけを持つ疎な（sparse）差分を新しい gen として書きます。 PR ではこの差分を delta、読み取り時に重ねる層を overlay と呼んでいます。 先ほどの図と同じ更新でも、書くのは変わった doc の数だけになります。 読み取り時は base の列の上に delta を新しい順に重ね、同じ doc が複数の層にあれば最新の値を使う、という読み方です。 PR の説明はこれで「Updating a field becomes O(changed docs) instead of O(column)」になると書いています。 読み取り側は新しく入った OverlayNumericDocValues と OverlayBinaryDocValues（10.6.0 では OverlaySortedNumericDocValues も）が担当します。 delta は codec に元からある疎な形式で書くので、PR は「No doc-values format change」と書いています。 overlay が溜まったときの fold delta を書き足すだけだと、層がどんどん積み上がり、読み取りが遅くなります。 そこで field ごと、segment ごとに、次の決まりで delta を畳みます（fold）。 flush のたびに、変わった doc だけの delta を 1 つ書く delta の数が上限に達していたら、次の更新を書くときに既存の delta と今回の更新を 1 つにまとめる fold のとき、既存の delta が持つ doc 数の合計が segment の doc 数の 50% 以上なら、base と合わせて新しい密な（dense）base を書く segment の merge で、普通の列に戻る 上限の数は後で見る setMaxDocValuesOverlays で指定します。 最初の実装では、dense な base に戻すのは delta が列全体を覆ったときだけでした。 review で neoremind が bug を見つけます。 一度 dense な base に fold すると、それ以降の fold がすべて列全体の書き直しに戻っていました。 作者は判定を segment に対する割合に変え、閾値を 50% にして直しています。 overlay にならない更新 すべての更新が delta になるわけではありません。 PR #16418 の時点の対象は NUMERIC と BINARY の doc-values でした。 10.6.0 では #16581 で SORTED_NUMERIC も単一の値で更新できるようになりました。 この更新は、列が単一値か複数値かに関係なく同じ delta の経路を通ります（複数値の doc は読み取り時に doc 単位で保たれます）。2 OpenSearch や Elasticsearch の数値 field は SORTED_NUMERIC の列として書かれるので、この拡張も関係します。 どの型でも対象は値を設定するだけの更新に限られ、値を消す（unset する）更新が混ざると、従来どおり列全体を書き直します。 ベンチマークで見る効果と代償 更新の間隔による書き込み量の違い PR #16418 には、作者自身のベンチマークが載っています。 対象は 8 バイトの数値と 512 次元の vector を持つ 5M（500 万）doc です。 4 スレッドがランダムな順序で更新し、NRT reader を 1 秒ごとに refresh します。 比べている方法は 3 つです。 full reindex は updateDocument で doc ごと入れ直す方法、dense は 10.5 までの doc-values update、sparse は今回の overlay です。 作者は、書き込み量が更新の間隔で大きく変わると考え、全速で更新する batch と、毎秒 5,000 件に絞った throttled の 2 通りで測っています。 数値 field の結果です。3 条件 方法 bytes/update 書き込みの倍率 batch full reindex 13,138 1,642× batch dense 80 10× batch sparse 8 1.0× throttled 5k/s full reindex 14,359 1,795× throttled 5k/s dense 7,202 900× throttled 5k/s sparse 18 2.2× 倍率は書いたバイト数を実際に変わった 8 バイトで割った値です。 batch では dense でも 10× で済んでいます。 refresh 1 回のあいだに大量の doc が更新されるので、列全体の書き直し 1 回を多くの doc で分け合えるからです。 throttled にすると、同じ書き直しが数千件ぶんしかカバーしなくなり、900× に跳ね上がります。 sparse は間隔に関係なく変わった doc だけを書くので、どちらでも 1.0〜2.2× に収まります。 throttled の dense と sparse を比べると、書き込み量は約 400 分の 1 です。 throttled の行では、dense と full reindex の差も小さくなっています。 doc ごと入れ直す 14,359 バイトに対して、doc-values update は 7,202 バイトで、半分にしかなっていません。 32 バイトの binary field では、さらに逆転します。 条件 方法 bytes/update throttled 5k/s full reindex 15,756 throttled 5k/s dense 27,734 throttled 5k/s sparse 55 このベンチマークの binary field では、更新が少しずつ流れ続けると、10.5 までの doc-values update が doc ごと入れ直すより多く書いていたことになります。 作者は throttled を「closer to a steady trickle on a live index」と位置づけていて、運用中の index に近いのはこちらの条件です。 soft delete でも同じ傾向が出ていて、throttled では dense の 33 bytes/update が sparse で 6 bytes/update になっています。 読み取りの遅れ 書き込みが減るぶん、読み取りは遅くなります。 読み取りのたびに base と delta を重ね合わせるからです。 同じベンチマークで、500 万 doc の field を頭から読む scan の時間はこう変わりました。 field 条件 dense sparse 数値 batch 35 ms 118 ms 数値 throttled 5k/s 21 ms 103 ms binary 32B batch 29 ms 327 ms binary 32B throttled 5k/s 28 ms 172 ms scan は 3〜11 倍遅くなっています。 作者はソートやファセットのように doc を飛び飛びに引く読み方も測っています。 advanceExact で 10% の doc を引く測定では、overlay が 16 層のとき約 18 ms で、通常の列は約 1 ms でした。4 throttled で約 350 の gen が残った状態では、約 26 ms 対約 7 ms です。 作者はこれを「tens of ns per lookup」と書き、クエリ全体の時間と比べれば小さいとしています。 ただし、どちらの数値も作者が PR の review のために書いたベンチマークのものです。 benchmark のコードは merge 前に外すと書かれていましたが、2026 年 11 月 2 日時点では main のテストソースに、単独で実行する IncrementalDocValuesUpdatesBenchmark として残っています。 luceneutil や nightly benchmark のような第三者の測定は見つかりませんでした。 自分の index で書き込みの減少が読み取りの遅れに見合うかは、更新の間隔と読み取りの重さを比べて決めることになります。 overlay を有効にする設定 overlay は IndexWriterConfig#setMaxDocValuesOverlays(int) で有効にします。 0 は無効で、従来どおり列全体を書き直す設定です。 1 以上の値を渡すと有効になり、その数まで overlay を溜めてから fold します。 10.6.0 の Javadoc はこの値のトレードオフを次のように書いています。5 higher values lower write amplification but keep more overlays (hence more files) live to merge at read time. 10.6.0 の default は 0 で、この機能は opt-in です。 setMaxDocValuesOverlays の Javadoc には「Only takes effect when IndexWriter is first created」ともあり、6動いている IndexWriter に後から設定しても効きません。 ! 10.6.0 のリリースノートの Highlights には「This is on by default」と書かれていますが、10.6.0 のソースの default は 0 です。 backport の PR #16554 も「opt-in on 10.x」と明記しています。 on by default なのは 11.0 系（branch_11x）と main（12.0）で、default は #16648 で 16 から 8 に下がっています。 アップグレード後に古い Lucene で開けるか overlay を有効にしなくても、10.6.0 で変わることが 1 つあります。 segments ファイルの format version が上がりました。7 /** The version that records per-field incremental doc-values overlay generations. */ public static final int VERSION_10_6 = 11; static final int VERSION_CURRENT = VERSION_10_6; 古い Lucene は overlay を読めないので、delta を列全体と取り違えないよう、format version で弾く必要があります。 PR #16554 はこの version を機能を使うかどうかに関係なくすべての commit で書くと説明しています。 So any index written by 10.6+ is rejected by older 10.x readers with IndexFormatTooNewException, whether or not it uses the feature. Lucene は新しいバージョンで書いた index を古いバージョンで読めることを元から約束していません。 そのため規約の上では何も失われていない、というのが PR の立場です。 運用側から見ると、10.6.0 で一度でも commit した index は、10.5 系の Lucene に戻した時点で開けなくなります。 ローリングアップグレードの途中で旧版に戻す手順を用意しているなら、10.6.0 からは使えないと考えたほうが安全です。 review では ChrisHegarty が、commit 時に dense へ fold すれば古いバージョンでも読めるのでは、と提案しています。 作者は、refresh と commit を同じ頻度で行う使い方ではcommit 時に fold する案ではこの機能の効果が消えるとして、採用しませんでした。 同じリリースの .dvp 分離 10.6.0 では doc-values のファイル構成も別の変更で変わっています。 PR #16540（issue #16298）は、疎な field の doc ID 集合（IndexedDISI）を .dvd から新しい .dvp ファイルに分けました。 overlay とは独立した変更ですが、どちらも「doc-values まわりのファイルが変わった」と読めるので、混同しないよう気をつけてください。 Lucene90DocValuesConsumer や Lucene90DocValuesProducer のコンストラクタを直接呼んでいるコードは、こちらの変更に合わせて直す必要があります。 OpenSearch の star-tree がその例です。 OpenSearch と Elasticsearch の対応状況 OpenSearch OpenSearch は PR #23475 で Lucene を 10.5.2 から 10.6.0 に上げ、2026 年 10 月 31 日に main へ merge しました。 Version.V_3_10_0 が LUCENE_10_6_0 を指すようになっています。 PR で直したコードの 1 つは、前の節で触れた .dvp 分離に合わせた star-tree の doc-values の対応です。 OpenSearch は overlay の設定には手を入れていません。 PR の diff にも、GitHub のコード検索で見た OpenSearch のリポジトリにも setMaxDocValuesOverlays は出てきません。 Lucene 10.6.0 は default の 0、つまり overlay が無効のまま取り込まれています。 一方、segments format の変化は、機能の有無に関係なく OpenSearch にも及びます。 PR には書かれていませんが、Lucene 側の仕様から考えると、OpenSearch 3.10 で書いた index は Lucene 10.5 系の OpenSearch では開けなくなるはずです。 Elasticsearch Elasticsearch には、10.6.0 への upgrade PR は見つかりませんでした。 Elasticsearch は lucene_snapshot というブランチで Lucene の開発版に追従しているので、公開リポジトリ上の取り込み方が OpenSearch とは違います。 代わりに、overlay を使う機能の提案があります。 #16418 の作者である jimczi による PR #157749「Add updatable doc-values fields」で、2026 年 11 月 1 日時点で open のままです。 feature flag doc_values_updatable の内側にあり、ラベルは v9.7.0 です。 この PR は mapping に doc_values: { updatable: true } を足し、_update や _bulk の部分更新を doc の入れ直しではなく doc-values の書き換えで行います。 対象は keyword、数値、date の field で、index: false であること、index sort に含まれないこと、sub-field を持たないことが条件です。 PR が「columnar indices only」と書く種類の index で、新しい index version（IndexVersions.DOC_VALUES_UPDATABLE）以降に作ったものに限られます。 PR はねらいを「avoiding the cost of rewriting large immutable fields such as vectors」と書いていて、はじめにで挙げた「vector を持つ doc の数値 1 つだけを変える」場面そのものです。 overlay の上限は index 設定 index.doc_values.max_overlays で決めます。 doc-values の書き換えだけで済ませるには、Elasticsearch 側にも工夫が要ります。 _seq_no は in-place で書き換えられないので、更新の sequence number を _dv_update_seq_no という別の列に記録しています。 また、更新の差分だけを replica に送っても、replica が base の doc を持っていなければ意味がありません。 そのため、base の doc が global checkpoint 以下（すべての in-sync なコピーが持っていることが保証される範囲）にあるときだけ差分を送り、それ以外は従来の reindex に戻しています。 関連する変更と残っている課題 #16418 の後にも、いくつかの変更と議論が続いています。 ファイル数の問題は issue #16644 で dweiss が報告しました。 テストがファイルハンドルの上限に引っかかったというもので、issue に引用された分析は、開いているファイルの数が「segments × 更新した field 数 × (1 + overlay 数) × 2」に比例して増えるという見積もりを書いています（原文は英語）。 作者の jimczi は、増え方は field ごとに overlay の上限で抑えられると答えています。 issue はテスト側で overlay 数をランダムにする #16645 で閉じられ、これとは別に #16648 で default も 16 から 8 に下がりました。 overlay を有効にするなら、OS のファイルハンドル上限も合わせて確認しておくと安全です。 merge 中の更新の扱いも変わりました。 10.5 までは、merge 中に来た doc-values update を ReadersAndUpdates が heap に溜めていました。 #16570 はこれをやめ、merge の完了時に元の segment から読み直すようにしています。 10.x への backport（#16591）では、この挙動は overlay を有効にしたときだけ働きます。 10.6.0 には、SORTED_NUMERIC の更新のほかにも関連する変更が入っています。 #16565 は index sort に使う field への binary update を拒否するようにしました。 PR #16418 の review で後回しにされた項目と、10.6.0 のソースに TODO として残っている項目もあります。 reader を開くときに delta を 1 つにまとめる。shaie の提案に、作者は「it's the top follow-up」と答えています8 読み取りで intoBitSet を使う、dense な base を直接読んで変わった doc だけ差し替える9 field ごとに設定を変えられるようにする。ChrisHegarty の提案で、作者は「Good follow-up」と答えています10 毎回すべての delta を畳み直すのをやめ、size-tiered な fold にする11 fold の閾値 50% を設定で変えられるようにする12 確認できていないこと dev メーリングリスト13に、この機能を扱った専用のスレッドは見つかりませんでした。確認できたのはリリースの告知だけです。 ベンチマークの数値は作者のものだけで、第三者の測定は見つかっていません。 overlay を有効にしたときに、実際の index でファイルハンドルがどれだけ増えるかは測っていません。 Elasticsearch が Lucene 10.6.0 をどのバージョンで取り込むかは確認できていません。 おわりに はじめにの在庫数の例に戻ります。 10.5 までの updateNumericDocValue は在庫数を数件変えただけでも、refresh のたびに segment ぶんの在庫数の列を書き直していました。 10.6.0 で overlay を有効にすれば、書くのは変わった在庫数だけになります。 その代わり、在庫数でソートや集計をするたびに、base に delta を重ねる手間がかかります。 10.6.0 では opt-in なので、何もしなければこの挙動は変わりません。 ただし segments format は必ず上がるので、10.6.0 で一度 commit した index は 10.5 では開けなくなります。 Lucene を直接使っているなら、まず自分の index の更新の間隔と、ソートやファセットの重さを見比べてみてください。 更新が少しずつ続き、読み取りが軽いなら、setMaxDocValuesOverlays に小さな値を渡して試す価値があります。 参考リンク Release 10.6.0 · apache/lucene Lucene Core News CHANGES.txt（10.6.0） Incremental doc-values updates · PR #16418 Backport incremental doc-values updates to 10.x · PR #16554 Incremental doc values increase open file handle count significantly · Issue #16644 Upgrade Lucene to 10.6.0 · OpenSearch PR #23475 Add updatable doc-values fields · Elasticsearch PR #157749 脚注 ReadersAndUpdates.java#L323-L324（10.5.0） ↩︎ OverlaySortedNumericDocValues.java（10.6.0） の Javadoc。CHANGES.txt の GITHUB#16580 は「Single-valued columns reuse the numeric update path」と単一値の列に限った書き方をしていますが、コードは複数値の列にも delta を書きます。 ↩︎ PR #16418 の Numbers 節。この節の表と数値は、特記しない限りここから引いています。 ↩︎ PR #16418 の作者のコメント。「tens of ns per lookup」もこのコメントからの引用です。 ↩︎ IndexWriterConfig.java#L111-L116（10.6.0） ↩︎ IndexWriterConfig.java#L336-L348（10.6.0） ↩︎ SegmentInfos.java#L129-L132（10.6.0） ↩︎ 作者の回答 ↩︎ 作者のコメント ↩︎ ChrisHegarty の提案、作者の回答 ↩︎ ReadersAndUpdates.java#L390-L391（10.6.0） の TODO ↩︎ ReadersAndUpdates.java#L58-L60（10.6.0） の TODO ↩︎ Lucene の開発者向けメーリングリスト dev@lucene.apache.org です。Apache Software Foundation のプロジェクトでは、リリースの投票や設計の議論をこのリストで行います。アーカイブは lists.apache.org で検索できます。確認できた告知は 10.6.0 の ANNOUNCE メール です。 ↩︎
Your Vector Database Thinks "Not Covered" Means "Covered" TL;DR: An embedding's similarity score measures how much two texts share in topic and wording, and a negation is a one-word edit. I wrote 40 invented policy facts and tested six open embedding models. On every model, "Which plan doesn't cover X?" scored closer to "Which plan covers X?" than a plain rewording of the same question did, for 77.5–100% of the items. At a common semantic-cache cut-off (similarity 0.90, RedisVL's default), one model would serve the cached answer to 97.5% of the opposite questions and to 7.5% of the genuine rewordings. In retrieval, models found the "covered" chunk almost every time and the "not covered" chunk between 27.5% and 92.5% of the time. Adding BM25 through rank fusion made five of the six worse. Dropping English stop words, which removes "not", took BM25 to a coin flip. Re-ranking the top ten with a small cross-encoder took every model to 97.5% or better on the negated questions. Run the same test on your own model before you trust a similarity threshold. A benefits assistant had a semantic cache in front of it, so repeated questions skipped the LLM. On Monday someone asked "Which plan covers dental implants?" and got the right answer: the Gold plan. On Tuesday someone else asked "Which plan doesn't cover dental implants?" The cache embedded the new question, found Monday's at similarity 0.97, and returned "The Gold plan." That's the opposite of the right answer, delivered in a fraction of a second with no LLM call to blame. Someone asking "What plan pays for dental implants?", which means exactly the same as Monday's question, would have scored 0.85 and missed the cache. The assistant is invented. The two numbers aren't: they're what all-MiniLM-L6-v2, the most downloaded sentence-similarity model on Hugging Face (about 200 million downloads a month in October 2026), gives for those three questions. The rest of this article is what happened when I ran the same comparison across six models, three tests and 40 facts. What the score is actually measuring Embedding models are trained to put texts that go together (a question and its answer, a title and its abstract, two captions of one picture) close together. Texts that go together share a topic and, usually, a lot of words. So the model learns a geometry where shared topic and shared wording count for most of the distance, and nothing in training forces a single "not" to move a sentence to the other side of the space. From the vector's point of view, "Which plan doesn't cover dental implants?" is the original question with one word added. "What plan pays for dental implants?" swaps two words and adds a third. The negation is the smaller edit, so it lands closer. This isn't new. NevIR (Weller, Lawrie and Van Durme, EACL 2024) built document pairs that differ only by negation and found most retrieval models, including state-of-the-art ones, "performing the same or worse than a random ranking", with cross-encoders doing best. Cao (ECAI 2025) tested general-purpose text embeddings and reported "a significant lack of negation awareness in these models". A June 2026 preprint on agent memory (Yadav, arXiv 2606.26511) reported that cosine similarity separated contradicted facts from duplicates with an AUROC of 0.59, close to chance, and gave the same reason: contradictions came out more similar to the original than duplicates did "because a value-flip is a smaller edit than a genuine rephrasing of the same fact." That result is small (98 labelled pairs, one embedding model), so I wanted to see how far it generalised. An August 2026 audit of similarity thresholds in agent frameworks (Frias, arXiv 2608.10216) is the closest to what follows: a production drift guard "caught 0 of 56 meaning-breaking mutations", including "withhold the study drug" → "administer the study drug", which "came in at cosine 0.9608". Six models, 40 facts, three tests I wrote 40 short policy facts across twelve domains: dental and health plans, home, travel and car insurance, refunds, software tiers and licences, staff benefits, airline fares, warranties, bank accounts, gym memberships and drug formularies. Each fact has three forms: - the fact: "The Gold plan covers dental implants." - a rewording with the same meaning: "Dental implants are paid for under the Gold plan." - a contradiction: "The Gold plan does not cover dental implants." The contradictions use several kinds of negation on purpose: "does not", "isn't", "excludes", "no", "never", "unpaid", "non-refundable". Each fact also has three questions: one asking which entity covers it, a rewording of that question, and the opposite question ("Which plan doesn't cover dental implants?"). Most opposite questions carry a negation word, but a few flip the meaning without one: "Which tier limits data exports?", "Who gets unpaid parental leave?", and "Which account charges foreign transaction fees?", where it's the original that says "no". The six models are all open and run locally: all-MiniLM-L6-v2, bge-base-en-v1.5, e5-base-v2, gte-base, mxbai-embed-large-v1 and Qwen3-Embedding-0.6B, each with the query and passage prefixes its model card asks for. I didn't test hosted embedding APIs, which is the main reason to run the check below on whatever you use. Test 1: questions, the semantic-cache case For each of the 40 items: how similar is the original question to its rewording, and to its opposite? | Model | Mean similarity, rewording | Mean similarity, opposite | Opposite closer than rewording | Served at 0.90: rewordings | Served at 0.90: opposites | |---|---|---|---|---|---| | all-MiniLM-L6-v2 | 0.74 | 0.96 | 100% | 7.5% | 97.5% | | bge-base-en-v1.5 | 0.84 | 0.93 | 90% | 15% | 87.5% | | e5-base-v2 | 0.92 | 0.97 | 92.5% | 70% | 100% | | gte-base | 0.92 | 0.97 | 90% | 72.5% | 100% | | mxbai-embed-large-v1 | 0.83 | 0.89 | 77.5% | 12.5% | 37.5% | | Qwen3-Embedding-0.6B | 0.81 | 0.93 | 90% | 5% | 85% | The last two columns use 0.90, which is what RedisVL's SemanticCache defaults to (a cosine distance of 0.1). That's RedisVL's threshold applied to these six models, not RedisVL's own default embedding model, which I didn't test. On every model, a cache at that setting answers the opposite question more often than it answers the same question asked differently. You can't tune your way out of it. Set the threshold low enough to catch 95% of genuine rewordings on each model, and it catches all 40 opposites, on all six. As a separation score, similarity managed an AUROC between 0.01 and 0.25, where 0.5 is chance. A number that low means the score is reliably backwards: higher similarity was evidence that the question meant the opposite. Test 2: statements, the deduplication case The same comparison between the fact, its rewording and its contradiction. This is what a deduplication step or a memory store faces when it decides whether a new fact is "the same" as one it already has. | Model | Contradiction closer than rewording | AUROC, rewording vs. contradiction | |---|---|---| | all-MiniLM-L6-v2 | 52.5% | 0.46 | | bge-base-en-v1.5 | 2.5% | 0.94 | | e5-base-v2 | 47.5% | 0.57 | | gte-base | 15% | 0.81 | | mxbai-embed-large-v1 | 0% | 0.98 | | Qwen3-Embedding-0.6B | 55% | 0.43 | This one is mixed, and it's worth saying so. Three models separate statements well, and mxbai separates them almost perfectly. Three don't: MiniLM, e5 and Qwen3 sit near chance, so "X is not covered" is as likely as a rewording to look like a duplicate of "X is covered". The models that cope with statements still failed the question test. Part of that is my data: the question rewordings change more words than the statement rewordings do, and on every model the rewordings score lower for questions. That's also how people actually ask questions, so I don't think it makes the question result less real. The safe conclusion isn't "embeddings can't do negation". It's that it depends on the model and the text shape, and you can't know which case you're in without measuring. Test 3: retrieval, and what fixes it The cache and deduplication cases compare two texts directly. Retrieval is subtler, because the query and the chunk aren't phrased alike. So I built an index where every item contributes two chunks about different entities: "The Gold plan covers dental implants" and "The Silver plan does not cover dental implants". Then I asked both questions. "Which plan covers dental implants?" should rank Gold's chunk above Silver's, and "Which plan doesn't cover dental implants?" should do the reverse. All 80 chunks sit in the same index, so the other 39 items act as distractors. Every dense model answered the positive questions well (90–100%). The negated questions are where they split: | Model | Dense alone | Dense + BM25 (rank fusion) | Dense top 10, re-ranked: ms-marco-MiniLM-L-6-v2 | Dense top 10, re-ranked: bge-reranker-base | |---|---|---|---|---| | all-MiniLM-L6-v2 | 27.5% | 32.5% | 97.5% | 100% | | bge-base-en-v1.5 | 85% | 70% | 97.5% | 100% | | e5-base-v2 | 92.5% | 77.5% | 97.5% | 100% | | gte-base | 70% | 57.5% | 97.5% | 100% | | mxbai-embed-large-v1 | 77.5% | 65% | 97.5% | 100% | | Qwen3-Embedding-0.6B | 80% | 72.5% | 97.5% | 100% | Share of the 40 opposite questions ("doesn't", "excludes", "no", "unpaid", "limits" and so on) where the right chunk ranked above its opposite. A coin flip scores 50%. The MiniLM row is worth a second look. Counting an item as right only when both its questions came out right, the way NevIR scores it, MiniLM got 17.5%. Guessing at random would get 25%. The fix everyone recommends made it worse The standard advice for negation is hybrid retrieval: the "not" is a keyword, so add BM25 and fuse the rankings. I believed that going in, and it's what the outline for this article said. In this test, BM25 on its own got 77.5% of the negated questions right, which beats two of the dense models and ties a third. Fusing it with them using reciprocal rank fusion (Cormack, Clarke and Büttcher, 2009, k = 60) lowered the score for five of the six. Only MiniLM improved, from 27.5% to 32.5%. The reason is in the wording. The keyword arm only sees the negation when the question and the chunk use the same word. In 25 of the 40 items they did ("excludes" and "excludes", "does not" and "does not"), and BM25 got all 25 right. In the other 15 they didn't ("doesn't" against "not covered", "won't refund" against "non-refundable", "no API access" against "unavailable"), and BM25 got 6 right, which is below a coin flip. There it rewards whichever chunk shares more terms with the question, and that's often the wrong one. Rank fusion gives that vote the same weight as the dense model's. There's a second trap underneath, and it's a configuration setting. Elasticsearch's english analyzer and its stop token filter both default to the _english_ stop list, which is Lucene's ENGLISH_STOP_WORDS_SET. That list contains "no" and "not". So the keyword that's supposed to rescue you is deleted before it reaches the index. My BM25 is a simple Python one (rank_bm25, lowercase word tokens, no stemming), not Elasticsearch itself, but I gave it the same stop list. With that list applied, it scored exactly 50% on the negated questions, a coin flip, and every hybrid score dropped further. (The standard analyzer keeps stop words by default. Check which one your text fields actually use.) What worked was the slow, boring stage. A cross-encoder reads the question and the chunk together, so "doesn't" can change its answer in a way that two separately computed vectors can't. Re-ranking each dense model's top ten with ms-marco-MiniLM-L-6-v2, a 22-million-parameter model, took every row to 97.5% on the negated questions. bge-reranker-base took them all to 100%. The dense model you started with stopped mattering here, because the right chunk and its opposite were both in every top ten. The ordering inside it was the problem. That's an easy case: an 80-chunk index where the two rivals share most of their words. In a large index full of near-identical clauses, check that both make it into the candidates you re-rank. This matches NevIR's ranking, and it's why re-ranking gets a whole lesson of its own in SophiArch's Retrieval & RAG Systems course, two lessons after the one on hybrid retrieval: the two stages fix different failures. The re-rankers aren't perfect, and both made a couple of mistakes in the other direction. bge-reranker-base put "The standard warranty excludes accidental drops" above "The extended warranty covers accidental drops" for "Which warranty covers accidental drops?". ms-marco-MiniLM missed "Which account offers no overdraft?" and "Who refunds return shipping costs?". e5 also missed the overdraft question, and the opposite of the shipping one ("Who doesn't refund return shipping costs?"). That's 2 misses in 80 questions for each re-ranker, against 3 for the best dense model (e5) and 33 for MiniLM. One caution from Frias's audit: swapping in an NLI model as the gate "did no better" there. A re-ranker trained for search relevance and an NLI classifier aren't the same tool, and short policy questions are an easy case. So re-run the triples with the re-ranker in the loop before you rely on it. Try it on your model before you set a threshold This takes a few minutes and needs no labelled data. Write 20 question triples from your own domain: a question, a rewording, and its opposite. Then run: from sentence_transformers import SentenceTransformer model = SentenceTransformer("your-embedding-model") # or call your hosted embedding API triples = [ ("Which plan covers dental implants?", "What plan pays for dental implants?", "Which plan doesn't cover dental implants?"), # ... 19 more from your own domain ] backwards = 0 for question, reworded, opposite in triples: q, r, o = model.encode([question, reworded, opposite], normalize_embeddings=True) sim_r, sim_o = float(q @ r), float(q @ o) backwards += sim_o > sim_r print(f"{sim_r:.2f} {sim_o:.2f} {question}") print(f"opposite closer than rewording: {backwards}/{len(triples)}") If the opposite wins most rows, and on every model I tried it did, then no similarity threshold on that model can tell a repeat question from its opposite. Use the same triples for statements if you deduplicate or merge facts. Where each fix goes - Semantic caches. Don't serve a hit on similarity alone. The cheapest guard is a polarity check: if one question contains a negation word or a word like "exclude" or "without" and the other doesn't, treat it as a miss. A stronger one is to score the (new question, cached question) pair with a cross-encoder before serving. Either way, test the cache with opposite questions, not just rewordings, since rewordings are the only thing most cache evaluations try. - Retrieval. Re-rank the top k with a cross-encoder. In this test, putting it after any of the six dense models beat every combination without one. If you also run BM25, check the analyzer's stop list for "no" and "not", and measure whether fusion helps on your queries rather than assuming it does. - Deduplication and memory. Don't merge or drop a fact because it's close to one you already have. A contradiction is often closer than a duplicate. Compare the claims themselves, with a cross-encoder or an LLM check, before deciding they're the same. What this test doesn't show The facts are invented, short and in English, and there are 40 of them. Real policy text is longer, and a longer chunk dilutes the negation further, so I'd expect statement-level separation to get worse, not better. The retrieval index has 80 chunks, so both rivals always reached the top ten. In a large index, recall into the re-ranker becomes a separate thing to measure. The BM25 numbers come from a simple Python implementation without stemming, not from an Elasticsearch analyzer, so treat them as a direction, not a figure for your cluster. I only tested open models; hosted embedding APIs may do better or worse, which is the point of running the triples yourself. And the rewordings change more words than the negations do. That's deliberate, because it's the comparison a cache faces in production (people reword freely, and a negation is one word), but it means the results describe how real questions differ, not a controlled single-word edit. None of it means vector search is broken. It means a similarity score answers "are these about the same thing?" and gets used as if it answered "do these mean the same thing?". For dental implants, those are opposite answers. References - Weller, O., Lawrie, D., & Van Durme, B. (2024). NevIR: Negation in Neural Information Retrieval. arXiv:2305.07614 (EACL 2024). Negation-pair benchmark; most retrievers perform at or below random, cross-encoders best. - Cao, H. (2025). Semantic Adapter for Universal Text Embeddings: Diagnosing and Mitigating Negation Blindness to Enhance Universality. arXiv:2504.00584 (ECAI 2025). Negation awareness in general-purpose text embeddings. - Yadav, N. (2026). Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge. arXiv:2606.26511 (v1). Cosine similarity separating contradicted facts from duplicates at AUROC 0.59; the "smaller edit" explanation. - Frias, S. E. (2026). Similarity Gates Approve Reversals: A Validity Audit of Embedding-Cosine Thresholds in Agent Systems. arXiv:2608.10216 (v1). Cosine-threshold gates in agent frameworks (deduplication, semantic caches, drift guards) approving meaning reversals; the NLI drop-in result. - Cormack, G. V., Clarke, C. L. A., & Büttcher, S. (2009). Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods. SIGIR 2009. The rank-fusion method used for the hybrid rows (k = 60). - Redis. RedisVL cache API: SemanticCache. Default distance_threshold=0.1 in cosine distance units. - Elastic. Language analyzers, Stop token filter and Standard analyzer. The english analyzer's _english_ stop list and the standard analyzer's _none_ default. - Apache Lucene. EnglishAnalyzer.java. ENGLISH_STOP_WORDS_SET, which includes "no" and "not". - Model cards for the six embedding models and two re-rankers tested: all-MiniLM-L6-v2, bge-base-en-v1.5, e5-base-v2, gte-base, mxbai-embed-large-v1, Qwen3-Embedding-0.6B, ms-marco-MiniLM-L-6-v2, bge-reranker-base. - SophiArch. Retrieval & RAG Systems. Course covering chunking, hybrid retrieval with rank fusion, query rewriting, cross-encoder re-ranking, retrieval metrics and RAG evaluation.