{"post":{"slug":"query-expansion-rag-2026","title":"Query Expansion: Fixing Retrieval Before It Searches","excerpt":"Users ask short, ambiguous questions that retrieve badly. Query expansion rewrites them first. Here is what works, what adds latency for nothing, and when to skip it.","category":"ai","date":"July 25, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">A</span> user types four words into a search box. Those four words are all the signal your retriever gets, and if none of them appear in the relevant document under those exact terms, retrieval fails before the model ever sees anything. Query expansion addresses this by rewriting or augmenting the query before it hits the index.\n\nIt is one of the more effective retrieval improvements available, and it is also frequently applied where it earns nothing.\n\n## What changed in 2026\n\n- **Multi-query generation became the standard technique.** Producing several rewrites and merging their results proved more robust than producing one improved rewrite.\n- **Conversational resolution got separated out.** Resolving pronouns and follow-up references emerged as a distinct step from expansion, applied before it.\n- **Latency awareness increased.** As expansion added a model call to every search, teams began gating it on query characteristics rather than applying it universally.\n- **Hypothetical document generation stayed niche.** Generating a fake answer and embedding that, covered in [HyDE retrieval explained](/blog/hyde-retrieval-explained-2026), remained useful for specific cases rather than as a default.\n\n## Techniques compared\n\n| Technique | How it works | Cost | Best for |\n| --- | --- | --- | --- |\n| Synonym expansion | Add related terms from a thesaurus or model | Low | Keyword retrieval channels |\n| Query rewriting | Model produces one clearer version | One call | Vague or badly phrased queries |\n| Multi-query generation | Model produces several variants; retrieve each | One call, several searches | The general default |\n| Conversational resolution | Rewrite using dialogue history | One call | Any multi-turn interface |\n| Step-back questioning | Generate a broader question alongside the specific one | One call | Queries needing background context |\n| Hypothetical document | Generate an answer, embed it, search with that | One call | Corpora where questions and documents differ in style |\n\nMulti-query generation is the reliable default because it hedges. Rather than betting on one rewrite being better than the original, it retrieves for several phrasings and merges the results, so a poor rewrite costs little while a good one adds recall. Merging by rank fusion, as described in [hybrid search with BM25](/blog/hybrid-search-bm25-2026), works well here too.\n\n## Conversational resolution first\n\nIn any multi-turn interface, this step matters more than expansion itself. A user asks about a product, then says \"how much does it cost\". That second query contains no product name, so retrieving on it directly returns nothing useful.\n\nResolving it against conversation history — turning it into a self-contained question naming the product — is what makes follow-up questions work at all. Do this before any expansion, and treat it as mandatory rather than optional in a chat interface.\n\nIt is also cheap to get wrong in a subtle way: resolving too aggressively can drag in context from earlier topics the user has moved on from. Resolve against recent turns rather than the whole conversation.\n\n## The latency question\n\nEvery expansion technique adds a model call before retrieval, and that call sits directly in the user's wait. On a search interface where results should appear immediately, that is a real cost.\n\nGate it. Apply expansion to short queries where signal is scarce, and skip it for long, well-specified ones that already retrieve fine. A simple length and specificity heuristic captures most of the benefit without paying the latency on every request.\n\nMeasure the tradeoff rather than assuming. If expansion improves recall marginally and adds meaningful latency, it may not be worth it — which is exactly the kind of question [RAG evaluation metrics](/blog/rag-evaluation-metrics-2026) exists to answer.\n\n## Common mistakes\n\n- **Expanding every query.** Long specific queries do not need it and pay the latency.\n- **Skipping conversational resolution.** Follow-up questions retrieve nothing.\n- **Betting on a single rewrite.** Multi-query hedges against a bad one.\n- **Resolving against the entire conversation.** Drags in abandoned topics.\n- **Not measuring the recall gain.** Expansion that does not improve recall is pure latency.\n\n## FAQ\n\n**Does expansion help hybrid search too?**\nYes, particularly the keyword channel, where synonym and term expansion directly increases matching.\n\n**How many query variants should I generate?**\nThree to five is a common range. Beyond that the marginal recall gain is small relative to the extra searches.\n\n**Can I use a small model for expansion?**\nUsually yes, and you should — it is a simple rewriting task and a cheap fast model keeps the latency cost down.\n\n**Does this replace better chunking?**\nNo. They address different stages and compose. Poor chunking is not fixed by better queries.\n\n## Where to go next\n\nFor the fusion method, read [hybrid search with BM25](/blog/hybrid-search-bm25-2026). For the hypothetical-document variant, [HyDE retrieval explained](/blog/hyde-retrieval-explained-2026), and for measuring the result, [RAG evaluation metrics](/blog/rag-evaluation-metrics-2026).","hue":210,"metaTitle":"Query Expansion for RAG Retrieval in 2026","metaDescription":"How query expansion improves retrieval on short queries, multi-query generation versus rewriting, resolving conversational references, and the latency cost.","keywords":["query expansion rag","query rewriting retrieval","multi query generation","conversational query resolution","retrieval preprocessing","short query problem","search relevance improvement","rag latency tradeoff","retrieval engineering 2026"],"tldr":["**Short queries retrieve poorly because they carry little signal** — expansion adds the terms the user did not type.","**Multi-query generation is the most reliable technique** — several rewrites, retrieve for each, merge results.","**Conversational queries need history resolution first** — pronouns and follow-ups are meaningless standalone.","**Every expansion costs a model call and latency** — it must earn that against the retrieval gain.","**Skip:** expansion for long, well-specified queries. They already carry plenty of signal."],"readTime":"4 min read","wordCount":799},"related":[{"slug":"query-rewriting-for-rag-2026","title":"Query Rewriting for RAG: Fixing the Question First","excerpt":"Users do not phrase questions the way documents phrase answers. Rewriting the query before searching closes that gap and costs one cheap model call.","category":"ai","date":"August 11, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":202},{"slug":"hyde-retrieval-explained-2026","title":"HyDE Retrieval: Searching With an Imagined Answer","excerpt":"HyDE generates a fake answer and searches with that instead of the question. Here is why the trick works, when it beats ordinary retrieval, and its failure mode.","category":"ai","date":"July 25, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":214},{"slug":"agentic-rag-explained-2026","title":"Agentic RAG Explained: Retrieval That Decides","excerpt":"Classic RAG retrieves once and hopes. Agentic RAG lets the model decide what to search, when to search again, and when it has enough. Here is how it works and what it costs.","category":"ai","date":"August 12, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":208}],"sameCategory":[{"slug":"agent-cost-attribution-2026","title":"Agent Cost Attribution: Finding Where the Tokens Went","excerpt":"A single monthly API bill tells you that agents are expensive. It does not tell you which feature, which customer, or which retry loop is responsible. That gap is fixable.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":220},{"slug":"human-in-the-loop-agents-2026","title":"Human-in-the-Loop Agents: Where to Put the Approval Gate","excerpt":"Approve everything and the agent is a slow autocomplete. Approve nothing and you have handed irreversible actions to a system that can be talked into things.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":197},{"slug":"reasoning-effort-controls-2026","title":"Reasoning Effort Controls: Tuning How Hard a Model Thinks","excerpt":"Fixed thinking budgets gave way to effort levels, and the default is rarely the right setting. Here is how the dial behaves and where each level actually belongs.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":210},{"slug":"retrieval-metrics-explained-2026","title":"Retrieval Metrics: Measuring the Half of RAG That Fails","excerpt":"When a RAG system answers badly, the retrieval usually failed before the model ever spoke. Measuring generation quality alone hides that entirely.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":215}],"newer":{"slug":"profiling-flamegraph-guide-2026","title":"Reading Flame Graphs: Finding Where Time Actually Goes","excerpt":"A flame graph shows where a program spends its time in one picture. Here is how to read one, what the axes mean, and the traps that mislead people.","category":"code","date":"July 25, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":140},"older":{"slug":"runbook-writing-guide-2026","title":"Writing Runbooks People Can Follow at 3am","excerpt":"A runbook read by someone tired and stressed needs different writing than documentation. Here is what to include, what to cut, and how to keep them accurate.","category":"code","date":"July 25, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":265}}