When a model is named "Llama 70B" or "Mixtral 8x7B," the number refers to its parameter count — the total quantity of learned weights and biases stored inside the network. It is the single most quoted number in AI, used as shorthand for model size, capability, and hardware requirements all at once. It is a useful shorthand, but it collapses several different things into one figure, and reading it correctly requires knowing what it does and does not tell you.
What changed in 2026
- "Active parameters" became a required second number. As mixture-of-experts models became common, reputable model cards now list active parameters alongside total parameters, because the two predict very different things.
- Parameter count stopped being the main marketing lever. Labs shifted emphasis toward benchmark results and cost-per-token, treating parameter count as one input among several rather than the headline.
- Community tools for estimating hardware needs from parameter count matured, making it easier to translate a raw number like "70B" into "you need roughly this much VRAM at this precision."
What the number actually measures
A parameter count is simply the sum of every weight and bias across every layer of the network. It correlates with, but does not equal, three things people often conflate with it: capability, memory footprint, and training cost. Capability depends heavily on training data and architecture, not size alone — see our explainer on what parameters actually are for the mechanics. Memory footprint depends on precision (a parameter stored in 16-bit takes twice the space of one stored in 8-bit). Training cost depends on both parameter count and the number of tokens the model was trained on.
Total vs active parameter count
| Model type |
Total parameters |
Active parameters per token |
What it means for you |
| Dense model |
Equal to active count |
Equal to total count |
Every parameter used on every token |
| Mixture-of-experts model |
Can be much higher |
Only a fraction of the total |
Capacity of a large model, cost closer to a smaller one |
| Quantized model (any type) |
Unchanged |
Unchanged |
Memory footprint shrinks; parameter count does not |
If you see a mixture-of-experts model listed as "8x7B," resist doing the naive multiplication — the router shares some layers across experts, so the true total is usually less than 8 times 7 billion, and only one or two experts activate per token regardless.
Using parameter count to estimate hardware needs
A rough (and conservative) rule: at 16-bit precision, a model needs about 2 gigabytes of memory per billion parameters just to load the weights, before accounting for activations and context. At 8-bit that halves, and at 4-bit it roughly quarters. This is why quantized small models became practical to run locally — see our post on small language models for how far that trend went in 2026. Always verify current figures against the specific model card, since exact memory overhead varies by framework.
FAQ
Does a higher parameter count always mean a smarter model?
No. It sets an upper bound on capacity but says nothing about training quality. Compare benchmark results for your specific task rather than the raw number.
Why do some model names list two numbers, like 8x7B?
That format usually denotes a mixture-of-experts model with 8 expert sub-networks of roughly 7 billion parameters each, though shared layers mean the true total is lower than simple multiplication suggests.
Does quantizing a model change its parameter count?
No. Quantization changes how each parameter is stored (fewer bits per number), not how many parameters exist. A quantized 70B model still has 70 billion parameters.
Is parameter count the best way to compare two different models?
Not on its own. Use it as a rough sizing signal for hardware planning, and use independent benchmarks for actual capability comparisons.
Where to go next