This week, I've been busy putting Lenovo's ThinkCentre neo Ultra mini PC to the test. It's a sleek, powerful, and compact desktop that I'll have a full review to check out in the next couple of days. In the meantime, I've run into a snag.
Alongside its desktop-class Intel processor (CPU) and NVIDIA graphics card (GPU), the ThinkCentre neo Ultra has a Neural Processing Unit (NPU). Unlike the AI PCs I've tested in the past — primarily in laptop form — the NPU is not a part of the System-on-Chip (SoC).
The NPU in the ThinkCentre neo Ultra comes from a company called Kinara, which unveiled its second-gen Ara-2 NPU in December 2023. Kinara says it's designed to power "edge servers and laptops with high performance, cost effective, and energy efficient inference to run applications such as video analytics, Large Language Models (LLMs), Latent Diffusion Models (LDMs) and other Generative AI models."
What's different about the Kinara Ara-2 from other NPUs I've tested is that it's built onto its own M.2 board to utilize PCIe lanes. In the neo Ultra, as far as I can tell, it's installed on the PCIe conversion board that the GPU slots into. It has its own heatsink and a tiny powered fan. Neat. It's the first time I've seen this type of NPU, and I was eager to test it out and compare it to other AI PCs we've tested.
Where, oh where, has my NPU gone?
It didn't take long to realize that, aside from the Kinara Ara-2 NPU showing up in Device Manager, it wasn't evident anywhere else. The ThinkCentre neo Ultra product page claims it has "integrated generative AI features and software, boosting productivity and creativity," but, at least in my case, they proved impossible to find.
AI performance is hard to quantify, but I saw no difference in performance results when the NPU was enabled or disabled. Not even Lenovo's onboard Vantage app hardware monitor seemed to know that the PC has a discrete NPU inside.
That's not entirely uncommon, as I've noted that some NPUs, even from AMD and Intel chips, don't show up in Task Manager or benchmarking software until updates come in later. We're still very much watching the infancy of AI PCs, and there are some kinks to work out. Fine, just as long as it's making a difference in the background.
The way I test AI PCs is relatively basic, at least in the view of any AI professionals out there. UL Procyon's AI Computer Vision and Geekbench AI tools are considered mainstream ways of testing NPUs, but both options are new on the scene and have their fair share of drawbacks.
In AI laptops, it’s always interesting to see when and where the NPU kicks in, as it’s designed to provide AI acceleration at a lower power draw than a CPU or GPU. For a system running on battery power, that’s especially important. Less so in a desktop PC like the neo Ultra, especially when it has an RTX 4060 GPU that’s also capable of handling AI work thanks to its specialized Tensor cores.
Disabling the Ara-2 NPU in Device Manager didn't seem to make any difference in the benchmark tests I ran, and its exclusion from any hardware monitors meant I couldn't see when or if it was attempting to step in to help other apps utilize AI tools. I ultimately reached out to Lenovo for support.
Lenovo's answer isn't what I was expecting
I tapped the Lenovo support team to inquire about the discrete NPU's lack of visibility in the ThinkCentre neo Ultra. The answer I received — after drafting this article for publication for the first time — wasn't what I was expecting.
Despite the PC shipping out with the Kinara Ara-2 NPU, I was informed that the discrete NPU will only become accessible in applications after "a future software update." There's no confirmed date for when to expect the update.
When I asked about a quote from the product brief I received with the review unit, which mentions how the "synergy between the GPU and independent NPU elevates the ThinkCentre neo Ultra desktop into a powerhouse for content creation," I was informed that I'd received outdated information.
This sort of miscommunication happens, but it doesn't excuse the fact that nowhere on Lenovo's site does it mention that the NPU will only become usable sometime in the future.
Yes, the world of AI PCs is moving very quickly, and builders want to stay ahead of the trend, but anyone buying the PC now and expecting the NPU they paid $125 extra for to work will be, at the very least, disappointed.
AI PC benchmarking needs to improve
This leads me to my next point, which I also wrote before receiving a response from Lenovo: AI benchmarking tools are inadequate, and the usefulness of an AI PC for the average user is limited. These points are still valid.
The inability to benchmark AI PCs on an even plane, as well as the rush to advertise AI acceleration in new hardware, is a recipe that doesn't benefit consumers. If I hadn't done some extra digging in an attempt to test the NPU's performance, I might have never noticed that it was disabled, assuming instead that the benchmarking software or my own ignorance was getting in the way.
So, how can an AI PC's usefulness be conveyed to the average buyer? One term that is used a lot to advertise "AI power" is Trillion Operations Per Second (TOPS.) TOPS is more like a flashy term used to compare NPUs in the most general sense. An NPU designed for one task won't match the performance of another NPU designed for a different task, even if the TOPS counts are the same. Tests that use OpenVINO, for example, are going to make Intel NPUs look very good compared to their competition.
The Kinara Ara-2 is advertised as having 40 TOPS, which comes close to matching the likes of AMD's Ryzen AI 300 (50 TOPS), Intel's Core Ultra Series 2 (48 TOPS), and Qualcomm's Snapdragon X (45 TOPS) chips. The Ara-2 technically has enough power to run Copilot+ tools in Windows 11, but it doesn’t have access. That could change in the future, as we saw with Intel and AMD, but for now, the NPU in the neo Ultra will be reserved for other tasks (whenever it receives the crucial software update to make it do anything).
The lack of mainstream tests that can accurately compare all NPUs is mostly a byproduct of the speed at which AI models are advancing. Benchmarking software can’t keep up. That's kind of a good thing for all PC builders — not just Lenovo — as they can advertise an AI PC without having a straightforward way to test and compare AI performance. For consumers, however, it's a definite drawback.
Do you really need an AI PC with NPU?
AI PCs are only gaining in popularity, and most new laptops hitting the market have an NPU inside. However, for most people, they won't really make a difference in day-to-day use. The average user will likely see the NPU kick on for things like Windows Studio Effects, but beyond that, you might have a hard time finding a good use for it.
In terms of the ThinkCentre neo Ultra specifically, it's aimed at AI professionals who want a compact PC with a desktop-class CPU and GPU. The NVIDIA RTX 4060 provides about 242 TOPS of power from its Tensor cores, dwarfing the 40 TOPS available in the Ara-2. Again, it's a rough estimation of power, but that's a huge difference.
It's difficult to take advertising speech, prop it up against actual testing, and deliver accurate buying advice for the general PC user when everything is so obfuscated. That gets even tougher when huge companies like Lenovo start advertising and selling AI hardware that can't be used because the software hasn't yet caught up.
This episode has reaffirmed the fact that AI PCs are very much in their infancy stage, as well as my belief that most AI marketing is nothing more than, well, marketing. I hope that Lenovo quickly adds some sort of disclaimer on its website, noting that the NPU isn't functioning without a future software update. It's the only fair action, even if the AI PC world still has that Wild West feeling. Keep an eye out for my full ThinkCentre neo Ultra review in the coming days.
