Spec sheets need a single number, and for AI PCs that number became TOPS — trillions of operations per second, delivered by the neural processing unit. It appears on boxes, in comparison tables, and in minimum requirements for platform AI features. It is also, taken alone, one of the least useful numbers on the sheet.
That is not because NPUs do not matter. It is because TOPS measures a ceiling under ideal conditions and says nothing about the conditions you will actually run in.
What changed in 2026
- TOPS became a platform requirement, not just marketing. Operating system AI features started gating on minimum NPU ratings, which gave the number real purchasing consequence for the first time.
- The measurement inconsistency got noticed. Vendors quote TOPS at different numeric precisions, and a figure measured at very low precision is not comparable to one measured at higher precision.
- Local model use grew. More people started running models on their own machines, which exposed that memory, not NPU rating, was the limiting factor.
- Workload split clarified. A rough consensus emerged: NPU for sustained low-power inference, GPU for heavy bursts, CPU for orchestration and small models.
What actually determines local AI performance
| Spec |
Predicts |
How much it matters |
| Unified memory capacity |
Whether a model fits at all |
Decisive; a model that does not fit does not run |
| Memory bandwidth |
Token generation speed |
Very high for language models |
| NPU TOPS |
Sustained low-power throughput |
Moderate; matters for background tasks |
| GPU compute |
Burst performance, image generation |
High for diffusion and vision workloads |
| Sustained thermal headroom |
Whether peak performance lasts |
High in thin laptops |
| Software framework support |
Whether the NPU is used |
Binary; unsupported means unused |
Capacity is the first gate. If a model does not fit in memory, no amount of TOPS helps, and the difference between a 16 GB and a 32 GB machine determines which model sizes are available to you at all.
Bandwidth is the second. Generating text means streaming the model weights through the processor for every token, so bandwidth sets the ceiling on tokens per second. This is the same physics described in HBM4 explained, scaled down to a laptop. It is why machines with high-bandwidth unified memory often outperform machines with higher headline NPU ratings on local language models.
When the NPU actually earns its place
The NPU is not a smaller GPU. It is a low-power engine optimized for running the same modest model continuously without draining the battery or spinning fans. That profile fits a specific set of tasks well: live captioning, background noise suppression, camera effects on a video call, always-on wake detection, and local search indexing.
For those, an NPU is genuinely better than the alternatives, because doing them on the GPU would cost battery life you would notice within an hour. For running a large language model locally, the NPU is frequently not the component doing the work — the GPU or CPU handles it, and memory determines the outcome.
Software support is the wildcard. Many local AI tools have partial or no NPU acceleration, and support varies by framework, model architecture, and operating system version. Check whether the specific software you intend to use targets the NPU before paying for one, and see small language models explained for what actually runs well on-device.
Common mistakes
- Comparing TOPS across vendors without checking precision. Numbers measured at different precisions are not the same unit in any meaningful sense.
- Buying NPU capability for local language models. Prioritize memory capacity and bandwidth instead.
- Ignoring sustained thermals. A thin laptop that hits peak for ninety seconds and then throttles has a peak number that misrepresents its real behavior.
- Assuming platform AI features justify the upgrade. Evaluate whether you would actually use them; many are pleasant and few are decisive.
- Overlooking that most AI still runs in the cloud. On-device handles a specific slice, and the slice is growing, but it is not the whole workload.
FAQ
Is a higher TOPS rating ever the right thing to optimize?
If you specifically need sustained background inference — long transcription sessions, continuous video processing — then yes, it maps to something real.
How much memory do I need to run models locally?
Generally 16 GB is a workable floor for small models, 32 GB opens up considerably more, and 64 GB and above starts making mid-sized models comfortable. Bandwidth matters alongside capacity.
Do NPUs help with image generation?
Some, but diffusion models are usually GPU work. Check the specific tool; support varies widely.
Will my existing laptop get these features?
Platform AI features frequently gate on NPU presence and rating, so older machines without a qualifying NPU generally will not, regardless of how fast the CPU is.
Where to go next
For hardware recommendations, see best AI laptops. For what to run on them, small language models explained, and for the memory physics underneath, HBM4 explained.