The MI300X is AMD's latest and greatest AI GPU flagship, designed to compete with the Nvidia H100 — the upcoming MI325X will take on the H200, with MI350 and MI400 gunning for the Blackwell B200. Chips and Cheese tested AMD's monster GPU in a variety of low-level and AI benchmarks, and found that it often vastly outperforms Nvidia's H100.
However, before we get started, there are some caveats worth mentioning. Chips and Cheese's article does not mention what level of tuning was done on the various test systems, and software can have a major impact on performance — Nvidia says it doubled the inference performance of the H100 via software updates since launch, for example. The site also had contact with AMD but apparently not with Nvidia, so there could be some inherent bias or missed settings that could affect results. Chips and Cheese is also comparing the MI300X primarily to the PCIe version of the H100 in its low-level testing, which is the weakest version of the H100 with the lowest specs.
Caveats and disclaimers aside, Chips and Cheese's low-level benchmarks reveal that the MI300X, built on AMD's bleeding edge CDNA 3 architecture, is a good design from a hardware perspective. The chip's caching performance looks downright impressive, thanks to its combination of four caches in total, including a 32KB L1 cache, 16KB scalar cache, 4MB L2 cache, and a massive 256MB Infinity Cache (which serves as an L3 cache). CDNA 3 is the first architecture to inherit Infinity Cache, which first debuted on RDNA 2 (AMD's 2nd generation gaming graphics architecture driving the RX 6000 series).
