{"post":{"slug":"fine-tune-a-small-model-2026","title":"Fine-tune a small model in 2026: practical guide from data to deploy","excerpt":"Fine-tuning a 3B–8B parameter model in 2026 takes hours, not weeks, and costs tens of dollars — not thousands. Here is the full practical workflow.","category":"code","date":"April 24, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">F</span>ine-tuning a small language model in 2026 is an afternoon project, not a research endeavour. With LoRA adapters, QLoRA quantisation, and mature training libraries, a 7B model can be specialised on a consumer-grade GPU in 2–4 hours for under $20 in cloud compute. Here is the end-to-end workflow.\n\n## What changed in 2026\n\n- **Base models improved dramatically.** Llama 4 Scout (3B) and Phi-4 Mini (3.8B) punch well above their size, meaning you often need fewer fine-tuning examples to reach production quality.\n- **LoRA became the default fine-tuning method.** Full fine-tuning a 7B model requires 80+ GB VRAM; LoRA + 4-bit quantisation fits the same model into 12 GB.\n- **Unsloth and TRL dramatically cut training time.** Both libraries are now the go-to for efficient fine-tuning.\n- **Model merging is practical.** You can merge LoRA adapters from different task specialists into one model without retraining.\n\n## Decide if fine-tuning is the right tool\n\nFine-tuning changes **how** the model behaves (format, style, domain vocabulary). It does not add new knowledge efficiently. If your problem is \"the model gives wrong factual answers,\" fine-tuning will not fix it — RAG will.\n\n| Goal | Better approach |\n| --- | --- |\n| Model always outputs JSON schema | Fine-tune (format) |\n| Model knows your domain vocabulary | Fine-tune (style) |\n| Model answers from your docs correctly | RAG |\n| Model follows complex multi-step instructions | Prompt engineering first |\n| Model classifies into your custom categories | Fine-tune (classification) |\n\n## Step 1: Prepare your data\n\nLoRA fine-tuning needs instruction-response pairs in JSONL format:\n\n```jsonl\n{\"prompt\": \"Classify this support ticket: 'My invoice is wrong'\", \"completion\": \"billing\"}\n{\"prompt\": \"Classify this support ticket: 'App crashes on login'\", \"completion\": \"bug\"}\n```\n\nRules for good training data:\n- **Minimum ~200 examples** for format tasks; **~1,000+** for domain adaptation.\n- **Consistent output format** — every example should show exactly what you want.\n- **20% held-out validation set** — never train and evaluate on the same data.\n- **Deduplication** — near-duplicates inflate training curves without improving generalisation.\n\n## Step 2: Fine-tune with Unsloth + TRL\n\n```python\nfrom unsloth import FastLanguageModel\nfrom trl import SFTTrainer\nfrom transformers import TrainingArguments\nfrom datasets import load_dataset\n\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n    model_name=\"unsloth/llama-4-scout-3b-instruct\",\n    max_seq_length=2048,\n    load_in_4bit=True,\n)\n\nmodel = FastLanguageModel.get_peft_model(\n    model,\n    r=16,           # LoRA rank — higher = more capacity, more VRAM\n    lora_alpha=16,\n    target_modules=[\"q_proj\", \"v_proj\"],\n    lora_dropout=0,\n)\n\ndataset = load_dataset(\"json\", data_files=\"train.jsonl\", split=\"train\")\n\ntrainer = SFTTrainer(\n    model=model,\n    tokenizer=tokenizer,\n    train_dataset=dataset,\n    dataset_text_field=\"text\",\n    max_seq_length=2048,\n    args=TrainingArguments(\n        output_dir=\"./output\",\n        num_train_epochs=3,\n        per_device_train_batch_size=4,\n        learning_rate=2e-4,\n        fp16=True,\n    ),\n)\ntrainer.train()\nmodel.save_pretrained(\"./my-adapter\")\n```\n\nTotal training time for 1,000 examples on an A10G: ~30–90 minutes.\n\n## Step 3: Evaluate properly\n\nRun your held-out validation set through the fine-tuned model and measure task-specific metrics — not perplexity.\n\n```python\ncorrect = 0\nfor item in val_set:\n    pred = model.generate(item[\"prompt\"])\n    if pred.strip() == item[\"expected\"]:\n        correct += 1\naccuracy = correct / len(val_set)\n```\n\nCompare against: (a) the base model without fine-tuning, (b) the base model with a few-shot prompt. Fine-tuning should beat both by a meaningful margin to justify the maintenance cost.\n\n## How to start\n\n1. **Try prompt engineering first.** If 5-shot prompting reaches 85% accuracy, fine-tuning to reach 88% may not be worth the ops cost.\n2. **Pick a model close to your target size** — Llama 4 Scout for resource-constrained, Mistral 7B for balance.\n3. **Start with 200–500 high-quality examples** before scaling.\n4. **Validate on your task metric**, not on generic benchmarks.\n5. **Log all experiments** — learning rate, rank, epochs, dataset size.\n\n## Common mistakes\n\n**Training on the full dataset without a validation split.** You will overfit without knowing it.\n\n**High learning rate.** Fine-tuning usually needs `1e-4` to `3e-4`. Higher rates cause catastrophic forgetting.\n\n**Using LoRA rank too low.** r=4 is too low for complex tasks. Start at r=16 and lower only if you are VRAM-constrained.\n\n**Evaluating with the same examples you trained on.** Contamination produces misleadingly high scores.\n\n**Skipping the base model baseline.** You need to know what the fine-tuned model gains relative to the base.\n\n## What to skip\n\n- **Full fine-tuning** unless you have 4+ A100s and a clear need — LoRA matches or exceeds full fine-tuning on most task-specific benchmarks.\n- **RLHF from scratch** — use DPO (Direct Preference Optimisation) if you need preference alignment; it is far simpler.\n- **Fine-tuning a model larger than you can serve** — a 70B fine-tuned model you cannot deploy serves no one.\n\n## FAQ\n\n**How much data do I need?**\nFor format/style tasks: 200–500 examples. For domain adaptation: 1,000–5,000. For general instruction tuning: 50k+. Quality always beats quantity.\n\n**Can I fine-tune a closed API model?**\nOpenAI and Anthropic both offer fine-tuning endpoints for some models. Costs are higher than self-hosted but require zero infra.\n\n**Will fine-tuning hurt general capabilities?**\nYes, slightly — this is called \"catastrophic forgetting.\" LoRA minimises it; full fine-tuning is worse. Keep the adapter separate and merge only for the specific deployment.\n\n**How long does a LoRA fine-tune take?**\nOn an A10G (24 GB): 1,000 examples, 3 epochs, 7B model = ~45 minutes with Unsloth.\n\n## Where to go next\n\n- [Self-host an LLM in 2026](/blog/self-host-an-llm-2026)\n- [Best AI API providers in 2026](/blog/best-ai-api-providers-2026)\n- [Prompt engineering in 2026](/blog/prompt-engineering-2026)","hue":320,"metaTitle":"Fine-Tune a Small Model in 2026: Step-by-Step Guide","metaDescription":"Practical 2026 guide to fine-tuning a small LLM. Covers LoRA, data prep, training on a single GPU, evaluation, and deployment — from scratch.","keywords":["fine tune small model 2026","lora fine tuning 2026","qlora tutorial 2026","fine tune llama 2026","llm fine tuning guide","small model fine tuning","fine tuning vs rag 2026","huggingface fine tuning 2026","low rank adaptation 2026","custom llm training 2026"],"tldr":["**Start with a base that is already close** — Llama 4 Scout, Mistral 7B, and Phi-4 Mini are strong 2026 starting points.","**LoRA and QLoRA cut VRAM by 60–80%** — you can fine-tune a 7B model on a single A10G.","**Data quality beats data quantity** — 500 clean, representative examples outperform 50k noisy ones.","**Evaluate on held-out task data** — MMLU scores do not predict performance on your domain task.","**Skip:** fine-tuning when prompt engineering or RAG can close the gap — fine-tuning compounds maintenance."],"readTime":"4 min read","wordCount":859},"related":[{"slug":"lora-fine-tuning-guide-2026","title":"LoRA Fine-Tuning: Adapting a Model Without Retraining It","excerpt":"LoRA trains a small set of extra weights instead of the whole model. Here is when adaptation beats prompting, what data you actually need, and how to avoid overfitting.","category":"ai","date":"June 26, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":213},{"slug":"how-to-fine-tune-llm-2026","title":"How to fine-tune an LLM in 2026: when it's worth it (and when it isn't)","excerpt":"When fine-tuning an LLM actually pays off in 2026, the realistic costs, and the alternatives that beat fine-tuning for most use cases.","category":"code","date":"January 16, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":310},{"slug":"how-to-fine-tune-an-llm-2026","title":"How to fine-tune an LLM in 2026: when, how, and why","excerpt":"A grounded guide to fine-tuning a large language model in 2026 — when it beats prompting or RAG, the steps to do it, and the costs and traps to expect.","category":"ai","date":"June 5, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":265}],"sameCategory":[{"slug":"bazel-build-explained-2026","title":"Bazel Explained: What Hermetic Builds Actually Cost","excerpt":"Bazel delivers reproducible, precisely cached builds and demands that you describe everything. Here is what you get, what it costs, and who should say no.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":145},{"slug":"graphql-federation-2026","title":"GraphQL Federation: One Graph, Many Teams","excerpt":"Federation composes independently-owned subgraphs into a single API. Here is what it solves, the operational cost it adds, and when a simpler gateway is enough.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":300},{"slug":"hermetic-builds-explained-2026","title":"Hermetic Builds Explained: Same Inputs, Same Output","excerpt":"A hermetic build depends only on declared inputs. Here is what that property gives you, the common leaks that break it, and how to move toward it incrementally.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":130},{"slug":"monorepo-tooling-compared-2026","title":"Monorepo Tooling Compared: What You Actually Need","excerpt":"Monorepo tools solve task orchestration and caching, not repository layout. Here is what each tier provides and how to tell when you have outgrown the simple option.","category":"code","date":"August 19, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":150}],"newer":{"slug":"claude-code-vs-copilot-2026","title":"Claude Code vs GitHub Copilot in 2026: two very different tools","excerpt":"Claude Code and GitHub Copilot solve different problems and target different workflows. Here is how to choose between them in 2026.","category":"code","date":"April 24, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":308},"older":{"slug":"fractional-shares-explained-2026","title":"Fractional Shares Explained in 2026: Invest Any Dollar Amount","excerpt":"Fractional shares let you buy a piece of any stock or ETF regardless of its price. Here is how they work, where to buy them, and what to watch out for.","category":"finance","date":"April 24, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":300}}