The AI revolution is upon us, but unlike past shifts in computing, most of us interact with the most advanced versions of AI models in the cloud. Leading services like ChatGPT, Claude, and Gemini all remain cloud-based. For reasons including privacy, research, and control, however, locally run AI models are still of interest, and it's important to be able to reliably and neutrally measure the AI performance of client systems with GPUs and NPUs on board.
Client AI remains a highly fluid space as hardware and software vendors work to define the types of workloads best suited for local execution and the best compute resources upon which to execute them. To help navigate this rapidly changing environment, the MLCommons consortium and its MLPerf Client working group maintain a client benchmark that's developed in collaboration with major hardware and software vendors.
