Full fine-tuning updates every weight in a model, which requires enough memory to hold the model, its gradients, and its optimizer state — usually several times the model size. LoRA sidesteps that by freezing the base model and training small low-rank matrices alongside it. The adapters capture the adjustment; the original weights never change.
The result is that adapting a large model becomes something you can do on modest hardware, which changed who can do it at all.
What changed in 2026
- Quantized training became routine. Combining LoRA with a quantized base model brought adaptation of substantial models within reach of single consumer accelerators.
- Multi-adapter serving matured. Serving many task-specific adapters over one loaded base model became a standard deployment pattern rather than a research demonstration.
- Data curation displaced data volume. Practitioner consensus settled firmly on small curated datasets outperforming large scraped ones.
- The prompting alternative got stronger. As base models improved at instruction following, the set of problems genuinely requiring fine-tuning narrowed.
When fine-tuning is the right tool
| Goal |
Right approach |
| Consistent output format |
Fine-tuning, or constrained decoding |
| Specific tone or house style |
Fine-tuning |
| Domain vocabulary and phrasing |
Fine-tuning |
| Following a complex multi-step procedure |
Fine-tuning, after prompting fails |
| Knowing your company's current facts |
Retrieval, never fine-tuning |
| Answering from documents that change |
Retrieval |
| Reducing prompt length and cost |
Fine-tuning can help |
| Improving general reasoning |
Neither; use a better model |
The knowledge rows are the important ones. Fine-tuning teaches behavior, not facts, and attempts to inject knowledge produce a model that confidently states outdated information with no way to update it short of retraining. When the underlying information changes, retrieval changes with it. The comparison is laid out in RAG vs fine-tuning.
Doing it well
Start with data. A few hundred examples that genuinely represent the behavior you want will outperform tens of thousands of noisy ones, and the effort is better spent curating than collecting. Every example should be something you would be happy for the model to imitate exactly, because that is what it will do.
Hold out a test set before training and never look at it during iteration. Fine-tuning overfits easily on small datasets, and a model that scores well on data it effectively memorized tells you nothing. Build the evaluation before the training run.
Keep the rank modest. Higher rank means more capacity to fit, which on a small dataset means more capacity to overfit. Start low and increase only if underfitting is demonstrable.
Then compare honestly against the alternatives. Run your evaluation set against the base model with a good prompt, against retrieval, and against your fine-tune. The fine-tune has to win by enough to justify the ongoing cost of maintaining it through base model deprecations — which is a real recurring cost, since an adapter is tied to the base model it was trained against.
Common mistakes
- Fine-tuning to add knowledge. It produces confident staleness. Use retrieval.
- Large scraped datasets. Noise teaches noise.
- No held-out evaluation. You cannot tell overfitting from success.
- Skipping the prompting baseline. Many fine-tunes turn out to be beaten by a better prompt.
- Forgetting base model lifecycle. When the base retires, the adapter must be retrained.
FAQ
How much data do I need?
For style and format tasks, a few hundred high-quality examples is a reasonable starting point. Quality and consistency matter far more than quantity.
Can I fine-tune on consumer hardware?
With quantized base models and LoRA, meaningfully sized models are trainable on a single strong consumer accelerator. Memory requirements are covered in GPU memory estimate for LLMs.
How is LoRA different from other efficient methods?
It is one approach within a broader family covered in PEFT methods explained. Others adjust different parts of the network with different tradeoffs.
Can I combine adapters?
Sometimes, with mixed results. Merging techniques exist and quality varies — see model merging explained.
Where to go next
For the decision itself, read RAG vs fine-tuning. For the wider family of methods, PEFT methods explained, and for building the evaluation first, RAG evaluation metrics.