Stable Diffusion is still the only mainstream image-generation model you can run entirely on your own hardware, modify freely, and extend with community models — no API key, no usage limits, no content policy fine print. In 2026 the ecosystem matured past the chaotic early days: the tooling is better, the models are sharper, and the workflow is learnable in an afternoon. Here is exactly how to get started and where the real leverage is.
What changed in 2026
- SD 3.5 Medium and Large became the new quality benchmarks, with significantly better text rendering and prompt adherence than SDXL.
- SDXL Turbo and Lightning variants cut generation to 1–4 steps, making iteration nearly instant on a decent GPU.
- ComfyUI became the de facto power interface, overtaking Automatic1111 in active development and community workflows.
- ControlNet 2.0 landed with better depth, pose, and lineart conditioning — essential for production illustration work.
- Civitai and Hugging Face consolidated into the two go-to LoRA and checkpoint repositories; the model zoo is enormous.
Which version to use
| Model |
Best for |
VRAM floor |
| SD 1.5 |
Legacy LoRAs, low-VRAM machines |
~4 GB |
| SDXL 1.0 |
High quality, huge LoRA library |
~8 GB |
| SDXL Turbo |
Fast iteration, drafts |
~6 GB |
| SD 3.5 Medium |
Best quality/VRAM ratio in 2026 |
~8 GB |
| SD 3.5 Large |
Maximum quality, slower |
~16 GB |
For most new users: SD 3.5 Medium if you have 8–12 GB VRAM; SDXL Turbo for rapid experimentation.
How to install
ComfyUI (recommended):
- Clone the repo from GitHub, create a Python 3.10+ venv, and run
pip install -r requirements.txt.
- Download a checkpoint (
.safetensors) from Hugging Face into models/checkpoints/.
- Run
python main.py — the node graph UI opens in your browser at localhost:8188.
- Load a starter workflow JSON from the ComfyUI examples folder to test immediately.
A1111 Forge (easier start):
- Download the one-click installer from the Forge GitHub.
- Drop checkpoints into
models/Stable-diffusion/.
- Launch
webui-user.bat (Windows) or webui.sh (Linux/Mac).
Both run on Windows, Linux, and Mac (Apple Silicon works well via MPS with Metal acceleration).
The prompt formula
Structure prompts as: [Subject] + [Style/Medium] + [Lighting] + [Quality modifiers], then add a negative prompt.
Good positive prompt example:
portrait of a woman, oil painting, rembrandt lighting, intricate detail, 4k
Negative prompt (always include):
blurry, low quality, extra limbs, watermark, ugly, deformed
Key principles:
- Be specific early in the prompt — models weight the beginning heavily.
- Use style references (
in the style of Studio Ghibli, photorealistic DSLR) rather than vague adjectives.
- Clip skip 2 helps with SDXL and avoids over-processing.
- CFG scale 5–7 hits the sweet spot; higher values make images oversaturated and weird.
ControlNet and LoRA
ControlNet lets you pass a reference image (pose skeleton, depth map, edge map) to control composition — critical for consistent characters or product shots. Download the ControlNet extension in Forge or add it as a node in ComfyUI.
LoRA (Low-Rank Adaptation) files (~50–150 MB) inject a style or character into any base model. Stack 2–3 LoRAs with weights of 0.5–0.8 each for blended results. Civitai has tens of thousands; check the trigger words for each one.
How to pick settings
| Setting |
Recommended range |
Notes |
| Steps |
20–30 (full), 4–8 (turbo) |
More steps ≠ always better past ~30 |
| CFG scale |
5–7 |
7+ oversaturates, <4 ignores the prompt |
| Sampler |
DPM++ 2M Karras or Euler a |
Turbo models need specific samplers |
| Resolution |
1024×1024 (SDXL/SD3.5) |
Avoid non-native resolutions |
| Clip skip |
1 for SD1.5, 2 for SDXL |
|
Common mistakes
Running a model at the wrong resolution. SDXL and SD 3.5 were trained at 1024 px. Running them at 512 produces artifacts. Always use native resolution.
Ignoring the negative prompt. Even a basic negative prompt cuts garbage outputs by 30–40%.
Stacking too many LoRAs at full weight. Three LoRAs at 1.0 each fight each other. Drop weights and test incrementally.
Not using img2img for refinement. Generating at 70–80% denoising strength from your draft sharpens details without losing composition.
Skipping model hashes. Civitai sometimes hosts corrupted files — verify .safetensors hash before running.
What to skip
- SD 2.x checkpoints — skip it entirely; SDXL and SD 3.5 are strictly better.
- Automatic1111 (original) — Forge is the maintained fork; use that instead.
- Upscaling to 4× on every image — Real-ESRGAN 2× on selectively good images is faster and saves storage.
FAQ
Do I need a GPU?
For practical use, yes — a CPU-only run takes minutes per image. NVIDIA cards with 8 GB+ VRAM are ideal; Apple M2/M3 chips run SD 3.5 Medium adequately via MPS.
Is Stable Diffusion legal for commercial work?
SD 3.5 uses the Stability AI Community License — free for personal and research; commercial use requires a paid plan above certain revenue thresholds. SDXL uses a similar model. Check the license for your specific checkpoint.
How is it different from Midjourney?
Midjourney is a closed API with no local option; Stable Diffusion runs on your machine, is fully customizable, and has no per-image cost after hardware. Midjourney still has an edge in default aesthetic quality out of the box.
Can I fine-tune my own model?
Yes — Dreambooth and Kohya trainer let you fine-tune with 10–30 reference images. A 3–5 hour training run on an A100 (rented for ~$3–6) produces a personal LoRA.
Where to go next