{"post":{"slug":"embedding-model-migration-2026","title":"Embedding Model Migration: Re-indexing Everything, Carefully","excerpt":"A better embedding model is available. Switching means re-embedding your entire corpus, because vectors from two models are not comparable in any way.","category":"ai","date":"August 18, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">A</span> new embedding model publishes benchmark results comfortably ahead of the one you deployed eighteen months ago. Switching looks like a configuration change — swap the model name and carry on.\n\nIt is not. Vectors produced by different models occupy entirely different spaces, and distances between them are meaningless. Not approximate, not degraded: meaningless. A search comparing a new-model query vector against old-model document vectors returns results, ranked confidently, with no error anywhere, and they are nonsense.\n\nWhich makes an embedding change a full re-index of your corpus, and that is a project.\n\n## What changed in 2026\n\n- **Model turnover stayed fast.** Meaningfully better embedding models kept arriving, which means most teams face this decision periodically rather than once.\n- **Variable dimensionality spread.** Models supporting truncatable embeddings let you trade accuracy for storage after the fact, which changes the sizing conversation.\n- **Multimodal shifted requirements.** Teams wanting to index images alongside text often need a different model family entirely, forcing a migration for capability rather than quality.\n- **Benchmark scepticism grew.** Enough teams found leaderboard rankings not reproducing on their own corpora that domain benchmarking became the expected first step.\n\n## Benchmark on your corpus, not a leaderboard\n\nPublic benchmarks measure general-purpose performance across broad datasets. Your corpus is not general-purpose, and rankings frequently reorder on domain-specific content — legal text, code, medical notes, and internal jargon all behave differently from web prose.\n\nBefore committing to a migration, build a small evaluation from your own material: a few dozen real queries with known-correct chunks, exactly as described in [retrieval metrics](/blog/retrieval-metrics-explained-2026). Embed a sample of your corpus with the candidate model, run the queries, and compare hit rate against your current model on the same set.\n\nThis takes an afternoon and regularly overturns the assumption that motivated the migration. A model several places higher on a leaderboard can perform worse on a specialised corpus, and finding that out before re-embedding ten million chunks is worth the afternoon.\n\nCompare on cost and latency too. A larger, better model that doubles your indexing bill and adds latency to every query may not be worth a modest accuracy gain.\n\n## Dimensionality has consequences\n\n| Factor | Larger vectors | Smaller vectors |\n| --- | --- | --- |\n| Retrieval accuracy | Generally better | Generally worse |\n| Storage and memory | Proportionally more | Less |\n| Search latency | Slower | Faster |\n| Index build time | Longer | Shorter |\n| Cost at scale | Higher | Lower |\n\nMoving from a smaller to a larger dimensionality multiplies your vector storage directly. On a large corpus that can mean substantially more memory, which for an in-memory index is a real infrastructure change rather than a config tweak.\n\nModels supporting truncation are useful here: they let you keep a shorter prefix of the vector with graceful degradation, so you can tune the accuracy-cost trade after the fact rather than committing up front. Where available, benchmark at several lengths — the shortest acceptable one is frequently much shorter than the default.\n\n## Running the migration\n\n**Dual-write, then cut over** is the standard shape and the one that avoids downtime.\n\n1. **Stand up a second index** with the new model, alongside the existing one.\n2. **Backfill** the corpus into the new index. This is bulk, offline, latency-insensitive work — exactly what [batch APIs](/blog/batch-api-for-llms-2026) are for, at roughly half the cost.\n3. **Dual-write new documents** to both indexes during the backfill, so the new index does not fall behind.\n4. **Verify** by running your evaluation queries against the new index and comparing to the baseline you measured earlier.\n5. **Cut over reads** once you are satisfied. Keep the old index briefly.\n6. **Decommission** the old index after a period with no need to roll back.\n\nThe step people skip is the second index. Migrating in place — deleting and re-embedding as you go — means a period where the index contains vectors from both models, which is the incomparability problem in production. Every query during that window returns partly-garbage results, silently.\n\nRolling back is why the old index stays. If the new model underperforms in ways your evaluation missed, flipping reads back is a configuration change rather than another full re-index.\n\n## Common mistakes\n\n- **Migrating in place.** Mixed vectors, silent nonsense, no error.\n- **Trusting a leaderboard.** Rankings often do not survive a specialised corpus.\n- **Not measuring a baseline first.** Without a before, you cannot tell if the after is better.\n- **Ignoring dimensionality cost.** Larger vectors change your infrastructure bill.\n- **Backfilling synchronously.** Pays full price for work with no latency requirement.\n- **Deleting the old index immediately.** Removes the cheap rollback.\n- **Forgetting the query path.** Queries must be embedded with the same model as the documents — an obvious statement that is a common bug during cutover.\n\n## FAQ\n\n**How do I know a migration is worth it?**\nMeasured improvement on your own evaluation set, weighed against the re-indexing cost and any change in storage and latency. A hit rate improvement of a few points on a large corpus may not justify the project; a substantial one usually does.\n\n**Can I keep both models running permanently?**\nYou can run two separate indexes and query both, but merging their results is awkward since the scores are not comparable. Rank fusion works around it. Generally it is complexity worth avoiding unless the two models serve genuinely different content types.\n\n**What about the chunks themselves?**\nA re-index is a natural moment to revisit chunking, and changing both at once means you cannot attribute the difference. If you want to improve chunking too, do it as a separate measured change — see [RAG chunking strategies](/blog/rag-chunking-strategies-2026).\n\n**Does this apply to reranking models?**\nMuch less painfully. A reranker scores query-document pairs at query time rather than producing stored vectors, so swapping one is a deployment rather than a re-index.\n\n## Where to go next\n\nFor measuring the before and after properly, read [retrieval metrics](/blog/retrieval-metrics-explained-2026). For the cheap way to run the backfill, [batch APIs for LLMs](/blog/batch-api-for-llms-2026), and for the storage layer this all lands in, [best vector databases for RAG](/blog/best-vector-databases-for-rag-2026).","hue":207,"metaTitle":"Embedding Model Migration: Re-index Everything","metaDescription":"Why switching embedding models requires re-indexing your whole corpus, how to run a dual-write migration safely, and how to benchmark on your own queries first.","keywords":["embedding model migration","re-embedding corpus","vector index rebuild","embedding dimensionality","dual write migration","embedding benchmark","vector database migration","rag infrastructure 2026"],"tldr":["**Vectors from different models are incomparable, not just less accurate** — a mixed index returns confident nonsense with no error.","**Dual-write, then cut over** — index into both old and new while serving from the old, then switch once the new one is complete.","**Benchmark on your own queries** — public leaderboard position frequently does not survive contact with a specific corpus.","**Dimensionality changes your storage and cost** — a larger vector means more memory and slower search, sometimes for marginal gains.","**Skip:** migrating for a small benchmark improvement. Re-indexing a large corpus is a real project with a real bill."],"readTime":"5 min read","wordCount":1029},"related":[{"slug":"embedding-drift-explained-2026","title":"Embedding Drift: When Your Vector Index Goes Stale","excerpt":"Retrieval quality degrades quietly as content, language, and models change. Here is what actually drifts, how to detect it, and when a reindex is unavoidable.","category":"ai","date":"August 13, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":221},{"slug":"rag-vector-store-comparison-2026","title":"RAG Vector Store Comparison for 2026","excerpt":"A practical comparison of vector store options for retrieval-augmented generation in 2026 — pgvector, dedicated vector databases, and managed services, and how to pick between them.","category":"ai","date":"July 7, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":310},{"slug":"embeddings-explained-2026","title":"Embeddings explained in 2026: vectors that power AI search","excerpt":"Embeddings turn text, images, and code into vectors that capture meaning, and they are the engine behind semantic search and RAG. Here is what they are, how to use them, and where they go wrong.","category":"ai","date":"June 29, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":280}],"sameCategory":[{"slug":"agent-cost-attribution-2026","title":"Agent Cost Attribution: Finding Where the Tokens Went","excerpt":"A single monthly API bill tells you that agents are expensive. It does not tell you which feature, which customer, or which retry loop is responsible. That gap is fixable.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":220},{"slug":"human-in-the-loop-agents-2026","title":"Human-in-the-Loop Agents: Where to Put the Approval Gate","excerpt":"Approve everything and the agent is a slow autocomplete. Approve nothing and you have handed irreversible actions to a system that can be talked into things.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":197},{"slug":"reasoning-effort-controls-2026","title":"Reasoning Effort Controls: Tuning How Hard a Model Thinks","excerpt":"Fixed thinking budgets gave way to effort levels, and the default is rarely the right setting. Here is how the dial behaves and where each level actually belongs.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":210},{"slug":"retrieval-metrics-explained-2026","title":"Retrieval Metrics: Measuring the Half of RAG That Fails","excerpt":"When a RAG system answers badly, the retrieval usually failed before the model ever spoke. Measuring generation quality alone hides that entirely.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":215}],"newer":{"slug":"elder-care-cost-planning-2026","title":"Elder Care Cost Planning: The Numbers Families Miss","excerpt":"Long-term care is the largest unplanned expense most families face. Here is what each level of care actually costs, what insurance does not cover, and how to plan.","category":"finance","date":"August 18, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":316},"older":{"slug":"espp-discount-explained-2026","title":"ESPP: The Discount Is Free Money, the Tax Is Confusing","excerpt":"Employee stock purchase plans offer a discount and a lookback that can make the effective return substantial. Here is the arithmetic and the holding period trap.","category":"finance","date":"August 18, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":288}}