{"post":{"slug":"rerankers-explained-2026","title":"Rerankers: Fixing the Order of What You Retrieved","excerpt":"Vector search returns roughly relevant results in roughly the right order. A reranker reads each candidate against the query properly and reorders them, which is a different and better operation.","category":"ai","date":"January 2, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">E</span>mbedding search has a structural limitation that has nothing to do with model quality. The query and the documents are encoded *separately* — each becomes a vector without ever seeing the other — and then compared by distance.\n\nThat is what makes it fast: document vectors are computed once at indexing time and reused for every query. It is also what makes it approximate. The encoder had to compress a document into a fixed vector without knowing what would be asked of it.\n\n## What changed in 2026\n\n- **Reranking became a default pipeline stage.** Retrieve-wide-then-rerank settled as standard rather than an advanced addition.\n- **Small rerankers got good.** Compact cross-encoders reached quality that made the latency cost acceptable for interactive use.\n- **Rank fusion became a companion.** Combining several retrieval signals before reranking improved the candidate pool — see [rank fusion](/blog/rank-fusion-explained-2026).\n- **Measurement improved.** Teams started distinguishing recall problems from ordering problems rather than treating \"bad results\" as one issue.\n\n## Bi-encoder versus cross-encoder\n\n| | Embedding search | Reranker |\n| --- | --- | --- |\n| Encoding | Query and document separately | Together, as a pair |\n| When computed | Documents at index time | Every candidate, at query time |\n| Speed | Very fast over millions | Slow per candidate |\n| Quality | Approximate | Substantially better |\n| Scales to | Whole corpus | Tens of candidates |\n\nA reranker takes the query and one document as a single input and produces a relevance score. Because it sees both at once, it can attend to how specific query terms relate to specific document passages — the interaction that separate encoding necessarily discards.\n\nThe cost is that nothing can be precomputed. Every query-document pair is a fresh forward pass, which is why rerankers cannot search a corpus and embeddings can.\n\n## Retrieve wide, rerank narrow\n\nThe pipeline that follows is straightforward and the sizing is where judgement enters.\n\nRetrieve a generous candidate set with cheap vector or hybrid search — several dozen. Rerank those. Keep the top handful for the prompt.\n\nThe wide first stage exists because retrieval optimises recall and reranking optimises precision. You want the correct document *somewhere* in the candidates; the reranker's job is to move it to the top. Retrieving only five candidates leaves the reranker nothing to work with.\n\nThe narrow second stage exists because context is expensive and models attend unevenly across long inputs. Five well-ordered chunks beat twenty poorly-ordered ones.\n\nSizing the first stage is a measurement question: increase candidate count until recall stops improving, then stop. Every additional candidate costs a reranker call and adds latency.\n\n## It cannot fix recall\n\nThe most important limitation, and the most common misapplication.\n\nA reranker reorders what it is given. If the document containing the answer was never retrieved, no amount of reranking produces it. Reranking a candidate set that does not contain the answer produces a confidently-ordered list of wrong documents.\n\nWhich is why diagnosis order matters. Measure hit rate first — did the correct chunk appear in the candidate set at all? If it did not, the problem is chunking, embeddings, or query formulation, and a reranker is the wrong investment. If it appeared at position 30, reranking is exactly the fix. [Retrieval metrics](/blog/retrieval-metrics-explained-2026) covers making that distinction.\n\n## Common mistakes\n\n- **Adding a reranker to fix low recall.** It reorders; it does not retrieve.\n- **Too few candidates.** Nothing to reorder.\n- **Too many candidates.** Latency and cost with diminishing recall gains.\n- **Not measuring before and after.** Reranking should visibly improve ordering metrics; if it does not, something is wrong.\n- **Reranking on the wrong text.** Score the same chunk text you will put in the prompt, not a truncated version.\n- **Ignoring latency.** A reranker call per candidate is real time on an interactive path.\n\n## FAQ\n\n**How much does it improve results?**\nSubstantially where ordering is the problem, and not at all where recall is. That is why measuring which you have first is the whole recommendation.\n\n**Does it replace hybrid search?**\nNo — they compose well. Hybrid retrieval builds a better candidate pool; reranking orders it. Using both is common and sensible.\n\n**How many candidates should I rerank?**\nEnough that recall plateaus, typically some tens. Measure it rather than picking a number.\n\n**Can a general LLM do the reranking?**\nIt can, and it is usually slower and more expensive than a purpose-trained reranker for the same job. A general model is a reasonable way to test whether reranking helps before investing in a dedicated one.\n\n## Where to go next\n\nFor diagnosing whether you have a recall or ordering problem, read [retrieval metrics](/blog/retrieval-metrics-explained-2026). For building a better candidate pool, [rank fusion](/blog/rank-fusion-explained-2026), and for the indexing-side improvement, [contextual retrieval](/blog/contextual-retrieval-explained-2026).","hue":207,"metaTitle":"Rerankers: Fixing the Order of Retrieved Results","metaDescription":"How cross-encoder rerankers improve RAG result ordering, why they cannot fix poor recall, and how to size the retrieve-wide rerank-narrow pipeline.","keywords":["reranker explained","cross encoder reranking","rag result ordering","retrieve and rerank","relevance scoring","bi encoder vs cross encoder","search quality","rag pipeline 2026"],"tldr":["**A reranker scores query and document together** — unlike embeddings, which encode each independently.","**It fixes ordering, not coverage** — if the right document was never retrieved, reranking cannot help.","**Retrieve wide, rerank narrow** — pull 50 candidates, rerank, keep the top 5.","**It costs a model call per candidate** — which caps how many you can afford to rerank.","**Skip:** a reranker when your hit rate is poor. Fix retrieval first; ordering is the second problem."],"readTime":"4 min read","wordCount":800},"related":[{"slug":"cross-encoder-reranking-2026","title":"Cross-Encoder Reranking: The Cheapest Big Quality Win","excerpt":"Reranking a shortlist with a model that reads query and document together is usually the highest-return change in a retrieval pipeline. Here is how to do it.","category":"ai","date":"August 11, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":205},{"slug":"reranking-in-rag-2026","title":"Reranking in RAG explained in 2026: better answers, cheaply","excerpt":"Reranking is a second-stage step that reorders retrieved documents by true relevance before the model reads them. Here is the 2026 guide to how it works, why it lifts RAG quality, and what it costs.","category":"ai","date":"June 30, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":260},{"slug":"retrieval-metrics-explained-2026","title":"Retrieval Metrics: Measuring the Half of RAG That Fails","excerpt":"When a RAG system answers badly, the retrieval usually failed before the model ever spoke. Measuring generation quality alone hides that entirely.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":215}],"sameCategory":[{"slug":"agent-cost-attribution-2026","title":"Agent Cost Attribution: Finding Where the Tokens Went","excerpt":"A single monthly API bill tells you that agents are expensive. It does not tell you which feature, which customer, or which retry loop is responsible. That gap is fixable.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":220},{"slug":"human-in-the-loop-agents-2026","title":"Human-in-the-Loop Agents: Where to Put the Approval Gate","excerpt":"Approve everything and the agent is a slow autocomplete. Approve nothing and you have handed irreversible actions to a system that can be talked into things.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":197},{"slug":"reasoning-effort-controls-2026","title":"Reasoning Effort Controls: Tuning How Hard a Model Thinks","excerpt":"Fixed thinking budgets gave way to effort levels, and the default is rarely the right setting. Here is how the dial behaves and where each level actually belongs.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":210},{"slug":"retrieval-metrics-explained-2026","title":"Retrieval Metrics: Measuring the Half of RAG That Fails","excerpt":"When a RAG system answers badly, the retrieval usually failed before the model ever spoke. Measuring generation quality alone hides that entirely.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":215}],"newer":{"slug":"qlac-explained-2026","title":"QLACs: Buying Insurance Against Living Too Long","excerpt":"A deferred annuity purchased inside a retirement account that starts paying in your eighties. It reduces required distributions now and addresses the risk no portfolio strategy fully solves.","category":"finance","date":"January 2, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":284},"older":{"slug":"savepoints-explained-2026","title":"Savepoints: Rolling Back Part of a Transaction","excerpt":"An error inside a transaction usually poisons the whole thing. A savepoint lets you undo just the failed part and continue, which is the only way to handle expected failures mid-transaction.","category":"code","date":"January 2, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":311}}