The release of Chinese AI startup DeepSeek's latest AI model disrupted the tech sector and caused $1 trillion in stock market losses on Monday. Nvidia, the world's leading graphics processing unit (GPU) manufacturer, lost $593 billion in market capitalization. American export controls on advanced semiconductors and manufacturing equipment, which were designed to hamstring Chinese AI companies like DeepSeek, incentivized the firm to forgo expensive hardware, resulting in a much more cost-effective AI model than its American counterparts.
DeepSeek released its R1 model last week, which performs on par with a similar model developed by OpenAI. R1 reportedly only cost $5.6 million to develop, which was made possible by using a cluster of memory-constrained Nvidia H800s instead of H100s, hundreds of thousands of which are used by American AI firms. (Export controls banned the sale of H100s to Chinese firms in September 2022 and H800s in 2023, which DeepSeek acquired before the ban took effect.)
To get around memory constraints, DeepSeek "programmed 20 of the 132 processing units on each H800 specifically to manage cross-chip communications [by modifying] a low-level instruction set for Nvidia GPUs," writes technology reporter Ben Thompson. The firm also employed a mixture of expert model and other software optimizations to reduce training and inference costs, explains Morgan Brown, Dropbox's vice president of product and growth for AI products. The optimization of hardware and software allowed the company to bring model training costs down from $100 million to $5 million, 100,000 to 2,000 GPUs, and reduce API costs by 95 percent, according to Brown.
