It’s becoming increasingly difficult to make today’s artificial intelligence (AI) systems work at the scale required to keep advancing. They require enormous amounts of memory to ensure all their processing chips can quickly share all the data they generate in order to work as a unit.
The chips that have mostly been powering the deep-learning boom for the past decade are called graphics processing units (GPUs). They were originally designed for gaming, not for AI models where each step in their thinking process must take place in well under a millisecond.
Each chip contains only a modest amount of memory, so the large language models (LLMs) that underpin our AI systems must be partitioned across many GPUs connected by high-speed networks. LLMs work by training an AI on huge amounts of text, and every part of them involves moving data between chips – a process that is not only slow and energy-intensive but also requires ever more chips as models get bigger.
