Elon Musk has announced that xAI's Grok 3 large language model (LLM) has been pretrained, and took 10X more compute power than Grok 2. He did not reveal many details, but based on timing, the Grok 3 LLM was pre-trained on the Colossus supercluster, which contains some 100,000 Nvidia H100 GPUs.
"Grok 3 is coming soon," Elon Musk wrote in an X post. "Pretraining is now complete with 10X more compute than Grok 2."
