Full fine-tuning a large model means holding the weights, the gradients, and the optimiser state in memory at once. The optimiser state is the part that surprises people — common optimisers keep two additional values per parameter, so the memory requirement lands at several times the model size before you have processed a single token.
That puts full fine-tuning of a large model beyond what most teams can rent. QLoRA changes the arithmetic by freezing and quantising the base model, then training a small number of new parameters alongside it.
What changed in 2026
- Adapter fine-tuning became the default. Full fine-tuning stayed the exception rather than the starting point for most teams customising a model.
- Quantisation formats matured. Four-bit representations designed for frozen weights reduced the quality cost of the compression considerably.
- Serving adapters got easier. Loading several adapters against one base model in a single server became a supported pattern rather than a research demo.
- The "should you fine-tune at all" question got sharper. As retrieval and long context improved, the set of problems where fine-tuning is the right answer narrowed.
Where the memory goes
| Component |
Full fine-tune |
QLoRA |
| Base weights |
Full precision, trainable |
Quantised, frozen |
| Gradients |
Every parameter |
Adapter only |
| Optimiser state |
Two values per parameter |
Adapter only |
| Activations |
Full |
Similar |
The second and third rows are the substance. Freezing the base model means no gradients and no optimiser state for the overwhelming majority of parameters. The adapter is a tiny fraction of the model, so its optimiser state is negligible.
Quantising the frozen weights then shrinks the one remaining large item. Because those weights never update, aggressive quantisation costs less quality than it would during training.
Rank, and what to adapt
A low-rank adapter approximates a weight update using two small matrices. Rank controls their size, and it is the main quality-versus-memory dial.
Low rank learns less and uses less memory. Higher rank has more capacity and eventually approaches full fine-tuning in both quality and cost. Most practical work sits in a modest range, and the useful discipline is to start low and raise it only if the task is underfitting — a rank that is too high mostly wastes memory and can overfit a small dataset.
Which layers to adapt matters as much as rank. Adapting the attention projections captures most of the benefit; extending to the feed-forward layers adds capacity for tasks that need it and costs proportionally more. Starting with attention only and expanding if quality demands it is the sensible order.
When not to reach for it
Fine-tuning teaches a model how to behave, not what is true. That distinction decides whether this is your tool.
Good fits: enforcing a specific output format reliably, adopting a domain's tone and vocabulary, improving performance on a narrow repeated task, and reducing prompt length by baking instructions into the weights.
Poor fits: teaching new facts, keeping up with changing information, and anything where the underlying knowledge updates. Retrieval handles those and fine-tuning does not — a fine-tuned model with outdated facts is harder to fix than a stale index.
The honest first question is whether a better prompt or a retrieval step solves the problem. Both are cheaper, faster to iterate, and easier to reverse.
Common mistakes
- Fine-tuning to inject knowledge. Use retrieval; the model will confabulate around gaps.
- Rank set high by default. Wastes memory and invites overfitting on small datasets.
- Too little data. A few hundred well-chosen examples beats thousands of noisy ones, and both beat a handful.
- No held-out evaluation. Without one you cannot tell learning from memorisation.
- Adapting every layer immediately. Start with attention projections.
- Ignoring the serving side. An adapter needs to be loaded at inference; plan how before you train.
FAQ
How much data do I need?
Fewer examples than people expect, and higher quality than they expect. Consistent, well-formatted examples of the exact behaviour you want work far better than volume. Build the evaluation set first, per golden datasets.
Does quantisation hurt quality?
Some, and less than intuition suggests, because the quantised weights are frozen rather than being updated through a lossy representation. Measure on your task rather than trusting a benchmark — see quantization explained.
Can I merge the adapter into the base model?
Yes, and it removes the inference-time overhead at the cost of losing the ability to swap adapters. Adapter merging covers the trade.
Will this beat prompting?
On a narrow, repeated, format-sensitive task, frequently. On anything requiring current knowledge or broad reasoning, usually not. Establish a prompting baseline before training anything.
Where to go next
For merging or serving what you train, read adapter merging. For the memory techniques that complement it, gradient accumulation and mixed precision training.