AMD kicked off Computex with a few pretty significant announcements: its new Instinct MI325X accelerators (pictured), which will be available in the fourth quarter of 2024; the upcoming Instinct MI350-series powered by the CDNA4 architecture that will launch next year; and the all-new CDNA 'Next'-based Instinct MI400-series products set to come out in 2026.
Perhaps the most significant announcement is that AMD's updated product roadmap commits to an annual release schedule, ensuring continuous increases in AI and HPC performance with enhanced instruction sets and higher memory capacity and bandwidth.
The AMD Instinct MI325X, set for release in Q4 2024, will feature up to 288GB of HBM3E memory with 6 TB/s of memory bandwidth. According to AMD, the MI325X will deliver 1.3x better inference performance and token generation compared to Nvidia's H100. One must remember that AMD's Instinct MI325X will be competing against Nvidia's H200 or even B100/B200 accelerators.
Following this, the MI350 series, built on the AMD CDNA 4 architecture, is expected in 2025. This series promises a 35-fold increase in AI inference performance over the current MI300 series. The Instinct MI350 series will use a 3nm-class process technology and support new data formats — FP4 and FP6 — and instructions to boost AI performance and efficiency.
The AMD ROCm 6 software stack plays a crucial role in maximizing the performance of the MI300X accelerators, AMD said. The company's benchmarks showed that systems using eight MI300X accelerators outperformed Nvidia's H100 1.3 times in Meta Llama-3 70B model inference and token generation. Single MI300X accelerators also demonstrated superior performance in Mistral-7B model tasks, beating its competitor 1.2 times, based on AMD's tests.
The adoption of AMD's Instinct MI200 and MI300-series products by cloud service providers and system integrators is also accelerating. Microsoft Azure uses these accelerators for OpenAI services, Dell T integrates them into PowerEdge enterprise AI machines, and Lenovo and HPE utilize them for their servers.
"The AMD Instinct MI300X accelerators continue their strong adoption from numerous partners and customers, including Microsoft Azure, Meta, Dell Technologies, HPE, Lenovo, and others, a direct result of the AMD Instinct MI300X accelerator exceptional performance and value proposition," said Brad McCredie, corporate vice president, Data Center Accelerated Compute, AMD. "With our updated annual cadence of products, we are relentless in our pace of innovation, providing the leadership capabilities and performance in the AI industry, and our customers expect to drive the next evolution of data center AI training and inference."
