Ask a model a hard reasoning question five times at nonzero temperature and you get five reasoning paths. If the model genuinely knows how to solve the problem, most of those paths reach the same correct answer while the mistakes scatter in different directions. Taking the majority answer exploits that asymmetry.
It is one of the simplest accuracy improvements available and one of the easiest to apply where it does nothing.
What changed in 2026
- Reasoning models absorbed much of the benefit. Models that explore multiple paths internally before answering captured a large share of what external sampling provided, at lower cost.
- The plateau got quantified. Measurement confirmed most gains arrive within the first handful of samples, discouraging the large sample counts early enthusiasm suggested.
- Weighted aggregation improved on plain voting. Weighting samples by model confidence or by verifier scores outperformed counting alone.
- It stayed strong for verifiable tasks. For arithmetic, multiple choice, and structured extraction, it remained a reliable accuracy lever.
Where it applies
| Task |
Comparable answers |
Self-consistency value |
| Arithmetic and word problems |
Yes; a number |
High |
| Multiple choice |
Yes; a label |
High |
| Classification |
Yes; a category |
High |
| Structured extraction |
Yes; field by field |
Moderate to high |
| Yes-or-no factual questions |
Yes |
Moderate |
| Code generation |
Only via test results |
Use tests instead |
| Summarization |
No comparable answer |
None |
| Creative writing |
No |
None; harmful |
| Open-ended advice |
No |
None |
The requirement is a comparable answer. Majority voting needs a notion of two answers being the same, and free-form text does not have one — five summaries are five different texts with no majority to take. Attempts to extend the technique to prose by clustering semantically similar outputs exist and are considerably less reliable than the original.
Cost and the plateau
Five samples cost five times one sample. That is the whole economic story, and it means the technique is only worth applying where accuracy on that specific task matters more than the multiplied cost.
Gains plateau fast. The jump from one sample to three is substantial, three to five smaller, and beyond that the curve flattens while cost continues rising linearly. Sampling twenty times is rarely justified by the marginal accuracy.
There is also an interaction with reasoning models worth thinking about. A model with extended reasoning already explores alternatives internally before committing, which delivers part of what self-consistency provides. Running five samples of a high-reasoning model multiplies an already-expensive call, and the combined cost frequently exceeds what the accuracy gain is worth. The reasoning-budget tradeoff is covered in test-time compute explained, and the two should be budgeted together rather than stacked thoughtlessly.
Where a verifier exists — a test suite, a calculator, a schema — use it instead. Checking answers against ground truth beats voting among guesses, which is the argument in the reflection pattern.
Common mistakes
- Applying it to free-form text. No majority exists.
- Large sample counts. Cost grows linearly, accuracy plateaus.
- Stacking it on high reasoning budgets. Multiplies an already expensive call.
- Zero temperature sampling. Identical outputs; there is nothing to vote among.
- Ignoring an available verifier. Checking beats voting whenever checking is possible.
- Not measuring the gain. On easy tasks the majority is unanimous and you paid five times for nothing.
FAQ
What temperature should I sample at?
High enough to produce genuinely different reasoning paths, low enough that samples remain sensible. A moderate setting is the usual starting point; measure on your task.
How many samples?
Start at three, measure, and increase only if the accuracy curve is still climbing. Five is a common practical ceiling.
Can I sample from different models?
Yes, and it adds diversity of failure modes, which frequently helps more than additional samples from one model.
Does it work on classification?
Well, and it is one of the cleanest applications since the answer space is small and comparable.
Where to go next
For verification-based alternatives, read the reflection pattern and chain of verification. For the reasoning-budget interaction, test-time compute explained.