Prices on Nvidia H100 processors for AI and HPC applications, as well as servers based on these GPUs, set records last year as supply was very constrained. But as supply improves — and demand softens as the market preps for the imminent arrival of the H200 GPU and servers — prices on H100 are coming down. In fact, even black-market prices of the H100 in mainland China are dropping, reports TrendForce citing Economic Daily News.
After the U.S. government restricted sales of Nvidia's A100/A800 and H100/H800 (along with other high-performance GPUs), sales of these GPUs essentially moved underground. Despite the ban, the H100 remained available through covert channels, including purchasing agents and shell companies that imported the processors and even servers from elsewhere into mainland China. In particular, it's reportedly fairly easy to get an H100 in areas like the Huaqiangbei electronics market in northern Shenzhen, though at a premium price.
At one point, H100-based servers were being quoted at over ¥3 million (more than $420,000), which was about 50% higher than Nvidia's official pricing of $280,000 to $300,000. This price inflation allowed middlemen to profit generously, even after accounting for additional costs like logistics and tariffs.
However, the landscape has shifted dramatically with Nvidia's upcoming release of its H200 141GB HBM3E processor that has more memory, and its memory is considerably faster. That means it will offer tangible performance advantages over its predecessor, particularly when it comes to training large language models and other AI workloads. Anticipating slowing demand for H100, scalpers have begun to rapidly unload their stocks, the report says. This in turn led to a swift price correction in the market, with the cost of H100-equipped servers plummeting to about ¥2.7 to ¥2.8 million in mainland China — and even lower in Hong Kong, to around ¥2.6 million, according to the report.
Although Nvidia's H200 is poised to be a significantly more powerful product than H100, the latter remains a viable piece of hardware, particularly for deep learning inference workloads. As a result, it's likely that Nvidia will keep selling its first generation Hopper-based product for a while. There's additional good news in that TSMC now has more CoWoS capacity and can presumably produce enough H100 and H200 processors to meet the anticipated demand. As a result, pricing of these products will likely not be as high as what we've seen previously.
Of course, neither the H100 nor the newer H200 are supposed to be exported to China. After reports of the H100's ready availability in China, Nvidia reaffirmed its adherence to U.S. export regulations, emphasizing that the sale and distribution of its products through illicit channels do not imply any breach of export controls by Nvidia or its direct partners. Moreover, Nvidia asserts that these transactions represent a negligible fraction of its global sales. But as usual, where there's a will there's a way.
