During Computex 2023, I had a chance to visit AMD's towering offices in Taipei, Taiwan, to see the company's Ryzen AI demo and speak with David McAfee, the Corporate VP and GM of the Client Channel Business. Most of our conversation centered on AMD's efforts in the consumer AI space, but I also squeezed in a few questions about AMD's take on hybrid CPUs. McAfee told me AMD has a different vision of hybrid processors than Intel that would avoid the complexity that forced Intel to remove AVX-512 support from its chips.
I interviewed AMD CTO Mark Papermaster two weeks ago in Antwerp, Belgium. He told me that we would "see high-performance cores mixed with power-efficient cores mixed with acceleration" in future AMD client [consumer] processors, signaling that, like Intel before it, AMD would adopt a hybrid CPU execution core design in the future. That wasn't too surprising -- we saw the first signs of two different CPU core types in AMD's software manuals months ago. Besides, AMD is already laying the foundation with its coming EPYC Bergamo chips with dense Zen 4c cores akin to efficiency cores.
AMD's current Ryzen 7040 laptop chips already feature a hybrid design, but not with two different types of CPU cores. Instead, the Ryzen 7040 has just one type of CPU core paired with an in-built AI accelerator engine that operates independently of the CPU and GPU cores. This engine provides advantages for certain types of AI inference workloads, but the CPU and GPU cores are better for other types of inference. So, the trick is to direct the different AI workloads to the correct type of cores to extract the best performance and power efficiency.
Throwing separate performance and efficiency CPU cores into that mix would introduce yet another compute option for AI inference workloads, and I asked McAfee if, conceptually, it would be feasible that efficiency cores would be better for AI than a dedicated piece of silicon (the AI engine). McAfee explained that the AI engines' strict focus on AI-specific operations would give it an efficiency advantage over any general-purpose CPU compute -- even an efficiency core.
Then we shifted to discussing Intel's hybrid chips, which have two types of cores, each with its own unique microarchitecture. That's created interesting problems: Intel's performance cores support AVX-512, but the smaller efficiency cores do not. That led Intel to disable AVX-512 support entirely (forcibly in the end), thus de-featuring its own chip and wasting precious die area.
I asked McAfee how AMD felt about that approach to hybrid designs.
"What I will say is this, I think the way that we think about it, the approach of two very different performance and efficiency cores with very different ISA support and IPC and capability is not necessarily the right approach," McAfee responded. "I think it invites far more complexity around what can execute where, and as we've looked at different options for core design, that's not the approach that we're taking.
"I think as we roll more of this out over time, what you'll see from us is an approach that takes into consideration the advantages that different core targeting can provide, but doing it in a way that's much more, from an application perspective, much more homogeneous."
We already know that AMD's Zen 4C efficiency cores, which it will use in the upcoming Bergamo server chips, will support the same instructions, like AVX-512, as the full-featured performance cores. However, they'll have a cut-down cache hierarchy to reduce die area consumption. The goal of both core types having the same IPC with the performance and efficiency cores is important. In contrast, Intel's efficiency cores have lower IPC than its performance cores (that could result in tradeoffs in its other e-core aspirations, like Sierra Forest).
"ISA, first of all, keeping that consistent to where a workload can operate on any core, has dramatic advantages," McAfee said. "And even when you look at a Ryzen desktop CPU today, the way that the Windows scheduler is plumbed, the ability to identify cores that are faster, slower, etc., and steer threads to different cores depending on the ranking or capability within a CPU; That's a well-established technique that we've used for quite some time. This then leads to, in our opinion, using a mechanism where the capability of the cores is more consistent.
"This is a far more tried and true way to look at bringing multiple different core targeting types into a design. I think the Intel approach invites a lot of complexity into the way that it operates. And I think our analysis has been that. I don't think you'll see us go down that path in the same way they have, if and when it comes to a Ryzen processor." McAfee concluded.
Unlike Papermaster, McAfee was noncommital on if or when hybrid would come to Ryzen, and we don't know where AMD would first introduce a hybrid architecture with Ryzen, be it with a monolithic APU or one of its chiplet-based models. However, it is clear that AMD envisions a hybrid future that would avoid the tradeoffs we've seen with Intel's design decisions behind the Alder and Raptor Lake processors.
Some of AMD's own decisions might be informed by analyzing Intel's missteps, or it may have just been the common sense of IP reuse with the existing core architecture -- it's a far lighter lift to tweak a microarchitecture than embarking upon a clean-sheet design. In either case, the ability to preserve support for AVX-512 would likely give AMD the performance advantage in vectorized workloads, provided Intel doesn't follow suit.
Conversely, one could argue that Intel's approach of having a separate microarchitecture tuned for lower-power operation is a better approach, albeit if it were paired with uniform ISA support across both types of cores. If Intel has corrected its ISA mismatch with Meteor Lake and maintained support for AVX-512 across both core types, it could also prove to be a potent combo.
In either case, it's clear that while AMD would be second to market with a hybrid design, it will take a much different approach. Only time will tell how the two techniques stack up in the benchmarks.
