You know how we were literally just talking about how Nvidia's GPUs massively outstrip the AI performance of the NPUs in upcoming CPUs (that's enough 'PUs)? Well, whaddya know, Nvidia has spotted that, too.
According to Chinese site Benchlife (via Tom's Hardware), Nvidia has been bigging up the AI processing chops of its GPUs, apparently dismissing the AI-accelerating NPUs in upcoming AMD, Intel, Apple and Qualcomm chips as "basic".
More specifically, Nvidia classed AI performance in the 10 to 45 TOPS range as "Basic AI" while characterising 100 TOPS to 1300+ TOPS as "Premium AI'. The NPUs in AMD and Intel's current chips achieve between 10 and 16 TOPS. Updates later this year will increase that to 45 TOPS.
Of course, it just so happens that Nvidia's GPU fall into the latter 100 TOPS and upwards category. As we pointed out earlier today in comparing upcoming laptop CPUs and APUs which will offer 45 TOPS of performance from their dedicated AI NPUs or Neural Processing Units, even Nvidia's entry level laptop GPU, the RTX 4060, hits a claimed 194 TOPS. The RTX 4090 is good for 686 TOPS. And it's the desktop RTX 4090 that Nvidia is referring to with the 1,300+ TOPS figure.
The key point in all this is that Microsoft's new "AI PC" definition requires 45 TOPS, and not just any old 45 TOPS but 45 TOPS from a dedicated NPU. That means Nvidia's GPUs don't qualify, no matter by how much their TOPS top those NPUs.
Nvidia helpfully provides a list of AI-related advantages its "Premium" GPUs have over those awful "Basic" NPUs. For instance, Nvidia reckons they won't be capable of AI video generation or 3D de-noising, but its GPUs will. Or in gaming the NPUs won't be able to do AI frame generation, ray reconstruction or AI in-game characters. Its GPUs will.
Nvidia also had some benchmarks to bolster its claims, comparing the RTX 4090 mobile GPU, the RTX 4050 mobile GPU and the Apple Macbook Pro M3 Max, which has a dedicated NPU. Applications included Stable Diffusion, Arnold, Blender, Chaos V-ray, Octane, Adobe Premier Pro Enhance Speech, DaVinci Resolve, and ON1 Resize AI. Nvidia's numbers pegged the RTX 4090 at seven times the performance of the Apple NPU in some tests, with the 4050 around twice as fast in many results.
So, the question is whether Nvidia's beef is fair. No doubt part of Microsoft's motivation is to ensure that AI is added to PCs in an efficient and cost effective way. Nvidia's GPUs don't do much for the efficiency of a laptop in day-to-day use and there is no denying they are very, very expensive.
Best gaming PC: The top pre-built machines.
Best gaming laptop: Great devices for mobile gaming.
It certainly makes sense to encourage a power efficient and cost effective NPU to be built into every CPU or APU. If ubiquitous AI is the aim, Nvidia's GPUs price themselves out in terms of both power and cost efficiency.
On the other hand, it is a bit preposterous to have a laptop with a 45 TOPS NPU qualify as an "AI PC" but another laptop with 686 TOPS's worth of AI processing not count.
Meanwhile, it's unclear exactly how and when the various AI platforms are supported. We assume Microsoft's AI Copilot won't currently run on an Nvidia GPU, for instance. But surely it could? And whose fault is that? Nvidia's? Microsoft's? Both?
Anyhoo, you can guarantee this one is going to run and run and run. And run. So, you'd better enjoy it, because the AI grudge match has only just begun.
