A retrieval system that worked well at launch will work less well a year later, and nothing in your monitoring will tell you. There is no error, no exception, no failed health check. Searches still return results. The results are just less often the right ones, and the decline is gradual enough that nobody notices until someone complains that the assistant has got worse.
That decay has three separate causes, and only one of them is fixed by reindexing.
What changed in 2026
- Model upgrade cadence accelerated. With new embedding models released frequently, the question of when to migrate — and what it costs — became a recurring operational decision rather than a one-time choice.
- Golden query sets became standard practice. Maintaining a fixed set of queries with known-correct results, scored on a schedule, emerged as the practical way to detect decay.
- Dimension reduction options complicated migration. Models supporting truncated embeddings gave teams a storage-versus-quality dial and another variable to keep consistent across an index.
- Hybrid retrieval gained ground partly as insurance. Keeping a keyword index alongside vectors limited the damage when embedding quality drifted.
The three kinds of drift
| Type |
Cause |
Symptom |
Fix |
| Model drift |
You changed embedding models |
Retrieval breaks badly and immediately |
Full reindex; never mix models |
| Content drift |
Corpus gained new topics and terminology |
New content retrieves poorly |
Incremental indexing; check coverage |
| Query drift |
Users started asking differently |
Existing content stops matching queries |
Query rewriting; re-evaluate chunking |
Model drift is the catastrophic one and also the easiest to avoid, because it only happens if you do it. Vectors produced by different models occupy different spaces, and similarity between them is meaningless. Mixing them in one index does not error — it returns confident nonsense for the affected subset.
Query drift is the subtle one. Your content is fine and your model is fine, but the vocabulary users bring has moved. A product got renamed, an industry adopted new terminology, a feature became known by a nickname. The index has no way to know, and relevance quietly falls.
Detecting it before users do
Build a golden query set. Fifty to a hundred real queries with the documents that should be retrieved for each, labelled once by someone who knows the domain. Run it on a schedule and track recall at your working cutoff.
That single practice catches all three drift types, because all three show up as declining recall on a fixed set. It also gives you a decision criterion for migration: when a new embedding model scores materially better on your golden set, the reindex is justified; when it does not, skip it regardless of benchmark claims.
Refresh the golden set periodically with recent real queries, or it becomes a museum of how people asked questions two years ago — which is itself query drift, applied to your measurement.
Watch coverage as a leading indicator too. If a growing share of queries return nothing above your similarity threshold, that is content drift announcing itself before recall scores fall. The chunking decisions in RAG chunking strategies affect how gracefully this degrades.
Common mistakes
- Mixing embedding models in one index. It fails silently, which makes it the worst kind of failure.
- No golden query set. Without it you are relying on user complaints as your monitoring.
- Reindexing on every model release. Expensive, and often no better. Measure first.
- Ignoring query-side drift. Teams reindex content and wonder why nothing improved.
- Truncating embedding dimensions inconsistently. If you shorten vectors for storage, do it uniformly across the whole index.
FAQ
How often should I reindex?
On model change, always. Otherwise on measurement — when your golden set shows decline you cannot fix by other means.
Can I migrate models incrementally?
Only by running two complete indexes in parallel and cutting over, not by re-embedding documents piecemeal into one index.
Does reindexing cost much?
Embedding a large corpus is a real but one-time cost, and it is a natural candidate for batch inference pricing.
How do I know if a new embedding model is actually better?
Score it on your golden set against the incumbent. Public benchmarks do not reflect your corpus or your queries.
Where to go next
For retrieval architecture, read what is RAG and RAG vector store comparison. For cheaper bulk re-embedding, batch inference cost savings.