{"post":{"slug":"qlora-explained-2026","title":"QLoRA Explained: Fine-Tuning a Large Model on One GPU","excerpt":"Full fine-tuning needs more memory than most people can rent. QLoRA quantises the frozen base model and trains a small adapter on top, which brings the requirement down enormously.","category":"ai","date":"January 28, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">F</span>ull fine-tuning a large model means holding the weights, the gradients, and the optimiser state in memory at once. The optimiser state is the part that surprises people — common optimisers keep two additional values per parameter, so the memory requirement lands at several times the model size before you have processed a single token.\n\nThat puts full fine-tuning of a large model beyond what most teams can rent. QLoRA changes the arithmetic by freezing and quantising the base model, then training a small number of new parameters alongside it.\n\n## What changed in 2026\n\n- **Adapter fine-tuning became the default.** Full fine-tuning stayed the exception rather than the starting point for most teams customising a model.\n- **Quantisation formats matured.** Four-bit representations designed for frozen weights reduced the quality cost of the compression considerably.\n- **Serving adapters got easier.** Loading several adapters against one base model in a single server became a supported pattern rather than a research demo.\n- **The \"should you fine-tune at all\" question got sharper.** As retrieval and long context improved, the set of problems where fine-tuning is the right answer narrowed.\n\n## Where the memory goes\n\n| Component | Full fine-tune | QLoRA |\n| --- | --- | --- |\n| Base weights | Full precision, trainable | Quantised, frozen |\n| Gradients | Every parameter | Adapter only |\n| Optimiser state | Two values per parameter | Adapter only |\n| Activations | Full | Similar |\n\nThe second and third rows are the substance. Freezing the base model means no gradients and no optimiser state for the overwhelming majority of parameters. The adapter is a tiny fraction of the model, so its optimiser state is negligible.\n\nQuantising the frozen weights then shrinks the one remaining large item. Because those weights never update, aggressive quantisation costs less quality than it would during training.\n\n## Rank, and what to adapt\n\nA low-rank adapter approximates a weight update using two small matrices. **Rank** controls their size, and it is the main quality-versus-memory dial.\n\nLow rank learns less and uses less memory. Higher rank has more capacity and eventually approaches full fine-tuning in both quality and cost. Most practical work sits in a modest range, and the useful discipline is to start low and raise it only if the task is underfitting — a rank that is too high mostly wastes memory and can overfit a small dataset.\n\nWhich layers to adapt matters as much as rank. Adapting the attention projections captures most of the benefit; extending to the feed-forward layers adds capacity for tasks that need it and costs proportionally more. Starting with attention only and expanding if quality demands it is the sensible order.\n\n## When not to reach for it\n\nFine-tuning teaches a model *how to behave*, not *what is true*. That distinction decides whether this is your tool.\n\nGood fits: enforcing a specific output format reliably, adopting a domain's tone and vocabulary, improving performance on a narrow repeated task, and reducing prompt length by baking instructions into the weights.\n\nPoor fits: teaching new facts, keeping up with changing information, and anything where the underlying knowledge updates. Retrieval handles those and fine-tuning does not — a fine-tuned model with outdated facts is harder to fix than a stale index.\n\nThe honest first question is whether a better prompt or a retrieval step solves the problem. Both are cheaper, faster to iterate, and easier to reverse.\n\n## Common mistakes\n\n- **Fine-tuning to inject knowledge.** Use retrieval; the model will confabulate around gaps.\n- **Rank set high by default.** Wastes memory and invites overfitting on small datasets.\n- **Too little data.** A few hundred well-chosen examples beats thousands of noisy ones, and both beat a handful.\n- **No held-out evaluation.** Without one you cannot tell learning from memorisation.\n- **Adapting every layer immediately.** Start with attention projections.\n- **Ignoring the serving side.** An adapter needs to be loaded at inference; plan how before you train.\n\n## FAQ\n\n**How much data do I need?**\nFewer examples than people expect, and higher quality than they expect. Consistent, well-formatted examples of the exact behaviour you want work far better than volume. Build the evaluation set first, per [golden datasets](/blog/llm-golden-datasets-2026).\n\n**Does quantisation hurt quality?**\nSome, and less than intuition suggests, because the quantised weights are frozen rather than being updated through a lossy representation. Measure on your task rather than trusting a benchmark — see [quantization explained](/blog/quantization-explained-2026).\n\n**Can I merge the adapter into the base model?**\nYes, and it removes the inference-time overhead at the cost of losing the ability to swap adapters. [Adapter merging](/blog/adapter-merging-explained-2026) covers the trade.\n\n**Will this beat prompting?**\nOn a narrow, repeated, format-sensitive task, frequently. On anything requiring current knowledge or broad reasoning, usually not. Establish a prompting baseline before training anything.\n\n## Where to go next\n\nFor merging or serving what you train, read [adapter merging](/blog/adapter-merging-explained-2026). For the memory techniques that complement it, [gradient accumulation](/blog/gradient-accumulation-explained-2026) and [mixed precision training](/blog/mixed-precision-training-2026).","hue":204,"metaTitle":"QLoRA Explained: Fine-Tune a Large Model on One GPU","metaDescription":"How QLoRA combines quantisation with low-rank adapters to cut fine-tuning memory, which layers to target, and when fine-tuning is the wrong tool entirely.","keywords":["qlora explained","lora fine tuning","quantized fine tuning","low rank adaptation","single gpu fine tuning","adapter training","peft memory","llm customization 2026"],"tldr":["**The base model is quantised and frozen** — only a small adapter trains, so optimiser state stays tiny.","**Memory savings come from the optimiser, not just the weights** — full fine-tuning stores gradients and momentum for every parameter.","**Rank is the main quality dial** — higher rank means more capacity to learn and more memory.","**Target the attention projections first** — adapting those gives most of the benefit for the least cost.","**Skip:** QLoRA when a prompt or retrieval solves your problem. Fine-tuning teaches behaviour, not facts."],"readTime":"4 min read","wordCount":828},"related":[{"slug":"lora-fine-tuning-guide-2026","title":"LoRA Fine-Tuning: Adapting a Model Without Retraining It","excerpt":"LoRA trains a small set of extra weights instead of the whole model. Here is when adaptation beats prompting, what data you actually need, and how to avoid overfitting.","category":"ai","date":"June 26, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":213},{"slug":"peft-methods-explained-2026","title":"PEFT Methods Explained: Beyond LoRA","excerpt":"Parameter-efficient fine-tuning is a family, not one technique. Here is how the main approaches differ, what each is good at, and which to reach for first.","category":"ai","date":"June 26, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":216},{"slug":"lora-vs-full-fine-tuning-2026","title":"LoRA vs full fine-tuning in 2026: which one to choose","excerpt":"LoRA trains a small set of adapter weights while full fine-tuning updates the whole model. This 2026 guide compares cost, quality, and portability so you can pick the right approach for your project.","category":"ai","date":"June 29, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":305}],"sameCategory":[{"slug":"agent-cost-attribution-2026","title":"Agent Cost Attribution: Finding Where the Tokens Went","excerpt":"A single monthly API bill tells you that agents are expensive. It does not tell you which feature, which customer, or which retry loop is responsible. That gap is fixable.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":220},{"slug":"human-in-the-loop-agents-2026","title":"Human-in-the-Loop Agents: Where to Put the Approval Gate","excerpt":"Approve everything and the agent is a slow autocomplete. Approve nothing and you have handed irreversible actions to a system that can be talked into things.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":197},{"slug":"reasoning-effort-controls-2026","title":"Reasoning Effort Controls: Tuning How Hard a Model Thinks","excerpt":"Fixed thinking budgets gave way to effort levels, and the default is rarely the right setting. Here is how the dial behaves and where each level actually belongs.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":210},{"slug":"retrieval-metrics-explained-2026","title":"Retrieval Metrics: Measuring the Half of RAG That Fails","excerpt":"When a RAG system answers badly, the retrieval usually failed before the model ever spoke. Measuring generation quality alone hides that entirely.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":215}],"newer":{"slug":"how-to-write-cold-email-2026","title":"How to write a cold email in 2026 that actually gets replies","excerpt":"Cold email in 2026 is harder because everyone is using AI. Here is the four-line template, the personalization that still works, and the deliverability rules.","category":"guides","date":"January 28, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":285},"older":{"slug":"solid-state-batteries-2026-update","title":"Solid-state batteries in 2026: what shipped, what didn't, what's next","excerpt":"Solid-state batteries were promised by every EV maker. Here's what actually shipped in 2026 and what's still in the lab.","category":"tech","date":"January 28, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":245}}