Earlier this month, Andrej Karpathy, a well-known AI researcher who was one of the founding employees of OpenAI and later headed up AI for Tesla, went viral on X. This alone isn’t so unusual. Karpathy—who now works as an independent AI researcher and is also the founder of Eureka Labs, which says it is creating a new kind of school for the AI era—has 1.9 million followers on X and his reputation is such that almost anything he says about AI is treated as either gospel or prophecy.
But this post was about an experiment he’d run where put an AI coding agent to work running a series of experiments to figure out how to improve the training of a small language model. He let the AI agent run continuously for two days, during which time it conducted 700 different experiments. Over the course of those experiments, it discovered 20 optimizations that improved the training time.
