Fine-tuning sat awkwardly between "expensive enterprise tool" and "research curiosity" through most of 2023–2024. By 2026, LoRA and QLoRA dropped costs to where a serious experiment fits in a small team's budget, and managed platforms (Together AI, OpenAI, Predibase) made it possible to run a fine-tune without owning GPUs. This guide is the honest beginner path — what fine-tuning is for, when it beats the alternatives, and how to actually start.
What changed in 2026
- LoRA cost collapsed. A 70B-parameter LoRA fine-tune now runs $150–$500 on Together or Predibase. A 7B fine-tune is $20–$60.
- Frontier models opened. OpenAI, Anthropic, and Google all offer fine-tuning on their hosted frontier models (with limits) and charge inference at parity with the base model.
- Dataset tooling matured. Argilla, Hugging Face datasets, and Distilabel make data curation tractable for non-research teams.
What fine-tuning is actually for
Fine-tuning teaches a model behavior, not facts. Three things it does well:
- Output format. Reliably produce JSON of a specific schema, narrow report templates, or formatted summaries.
- Tone and style. Match a brand voice, technical writing style, or persona consistency that prompt-engineering can't lock in.
- Domain language. Internal vocabulary, jargon, abbreviations the base model handles awkwardly.
Three things it does badly:
- Adding facts. New knowledge belongs in RAG. Fine-tuned facts cannot be cited and degrade unpredictably.
- One-shot personalization. Few-shot prompting is cheaper for narrow per-user customization.
- Edge-case behavior. A small dataset can over-fit and break general capability.
When fine-tuning beats the alternatives
Decision rule:
- Prompt engineering if a 20-line system prompt gets you 90% of the way there.
- RAG if the answer depends on documents that change.
- Fine-tuning if neither of the above produces consistent output format, tone, or specialized behavior — and you have ~200–2,000 high-quality examples.
For most production AI products, the right pattern is fine-tune for shape, RAG for facts.
How to actually start
- Define the gap. Write down what the base model does wrong — bad format, wrong tone, refuses helpful work, gets jargon wrong. Be specific.
- Curate 200–500 examples. Each example is
(input, ideal_output). Quality matters more than quantity. Use the model itself to draft, then edit aggressively.
- Pick a base. Llama 3.3 70B or Llama 4 8B for open models. GPT-5 mini or o4-mini if staying on OpenAI. Pick by inference cost and deployment requirements.
- Run a LoRA on a managed platform. Together, Predibase, OpenAI fine-tuning UI. Expect 30–90 minutes for a small dataset.
- Evaluate on a held-out set. 50 examples you didn't train on. Compare to base model side-by-side.
- Iterate. Most fine-tunes need 2–4 dataset improvements before they ship.
Real-cost examples
| Setup |
Dataset |
Cost |
| Llama 4 8B LoRA on Together |
500 examples |
~$25 |
| Llama 3.3 70B LoRA on Together |
1,000 examples |
~$200 |
| GPT-5-mini fine-tune |
500 examples |
~$30 + 1.5× inference |
| Predibase 8B with custom adapters |
1,000 examples |
~$50 |
Inference costs after fine-tuning are usually 1.5–2× the base model on hosted endpoints. Self-host removes that markup.
Common pitfalls
- Training on model's own output without filtering. Causes mode collapse. Mix with real data.
- Too little data. Below ~100 examples, you're usually better off with prompt engineering.
- Too much data of mixed quality. Below 80% quality threshold across the dataset, you teach the wrong patterns.
- No eval set. "It feels better" is not data. Run side-by-side comparisons.
FAQ
Can I fine-tune for free?
Yes — Hugging Face Spaces with free-tier GPUs handle small experiments. Beyond toy datasets, the managed platforms are cheap enough to skip the headache.
Is fine-tuning still worth it on frontier models?
For behavior, less so — frontier models follow instructions well. For narrow domain language or producing smaller deployment-class models, yes.
Do I need to know PyTorch?
For managed platform fine-tuning, no. Upload a JSONL, click run.
How long does a fine-tuned model last?
Useful for 6–18 months on most workloads. Re-tune when the underlying base model improves substantially.
Where to go next
For related material see RAG vs fine-tuning in 2026, Small language models on the edge in 2026, and Local LLM setup guide in 2026.