A user types four words into a search box. Those four words are all the signal your retriever gets, and if none of them appear in the relevant document under those exact terms, retrieval fails before the model ever sees anything. Query expansion addresses this by rewriting or augmenting the query before it hits the index.
It is one of the more effective retrieval improvements available, and it is also frequently applied where it earns nothing.
What changed in 2026
- Multi-query generation became the standard technique. Producing several rewrites and merging their results proved more robust than producing one improved rewrite.
- Conversational resolution got separated out. Resolving pronouns and follow-up references emerged as a distinct step from expansion, applied before it.
- Latency awareness increased. As expansion added a model call to every search, teams began gating it on query characteristics rather than applying it universally.
- Hypothetical document generation stayed niche. Generating a fake answer and embedding that, covered in HyDE retrieval explained, remained useful for specific cases rather than as a default.
Techniques compared
| Technique |
How it works |
Cost |
Best for |
| Synonym expansion |
Add related terms from a thesaurus or model |
Low |
Keyword retrieval channels |
| Query rewriting |
Model produces one clearer version |
One call |
Vague or badly phrased queries |
| Multi-query generation |
Model produces several variants; retrieve each |
One call, several searches |
The general default |
| Conversational resolution |
Rewrite using dialogue history |
One call |
Any multi-turn interface |
| Step-back questioning |
Generate a broader question alongside the specific one |
One call |
Queries needing background context |
| Hypothetical document |
Generate an answer, embed it, search with that |
One call |
Corpora where questions and documents differ in style |
Multi-query generation is the reliable default because it hedges. Rather than betting on one rewrite being better than the original, it retrieves for several phrasings and merges the results, so a poor rewrite costs little while a good one adds recall. Merging by rank fusion, as described in hybrid search with BM25, works well here too.
Conversational resolution first
In any multi-turn interface, this step matters more than expansion itself. A user asks about a product, then says "how much does it cost". That second query contains no product name, so retrieving on it directly returns nothing useful.
Resolving it against conversation history — turning it into a self-contained question naming the product — is what makes follow-up questions work at all. Do this before any expansion, and treat it as mandatory rather than optional in a chat interface.
It is also cheap to get wrong in a subtle way: resolving too aggressively can drag in context from earlier topics the user has moved on from. Resolve against recent turns rather than the whole conversation.
The latency question
Every expansion technique adds a model call before retrieval, and that call sits directly in the user's wait. On a search interface where results should appear immediately, that is a real cost.
Gate it. Apply expansion to short queries where signal is scarce, and skip it for long, well-specified ones that already retrieve fine. A simple length and specificity heuristic captures most of the benefit without paying the latency on every request.
Measure the tradeoff rather than assuming. If expansion improves recall marginally and adds meaningful latency, it may not be worth it — which is exactly the kind of question RAG evaluation metrics exists to answer.
Common mistakes
- Expanding every query. Long specific queries do not need it and pay the latency.
- Skipping conversational resolution. Follow-up questions retrieve nothing.
- Betting on a single rewrite. Multi-query hedges against a bad one.
- Resolving against the entire conversation. Drags in abandoned topics.
- Not measuring the recall gain. Expansion that does not improve recall is pure latency.
FAQ
Does expansion help hybrid search too?
Yes, particularly the keyword channel, where synonym and term expansion directly increases matching.
How many query variants should I generate?
Three to five is a common range. Beyond that the marginal recall gain is small relative to the extra searches.
Can I use a small model for expansion?
Usually yes, and you should — it is a simple rewriting task and a cheap fast model keeps the latency cost down.
Does this replace better chunking?
No. They address different stages and compose. Poor chunking is not fixed by better queries.
Where to go next
For the fusion method, read hybrid search with BM25. For the hypothetical-document variant, HyDE retrieval explained, and for measuring the result, RAG evaluation metrics.