Tiny corp., a startup that manages the neural network framework tinygrad, has unveiled a new AI system that uses six AMD Radeon RX 7900 XTX graphics cards. The tinybox, priced at $15,000, is aimed at democratizing PetaFLOPS-class performance for artificial intelligence.
The tinybox system is a compact powerhouse, packing six of AMD's fastest gaming GPUs into a 12U rack case. The GPUs are connected using 'full fabric' PCIe 4.0 x16 links to ensure maximum bandwidth. Tiny corp. says it chose the Radeon RX 7900 XTX over Nvidia's GeForce RTX 4090 because the GPUs fully support the peer-to-peer interconnections necessary for LLMs.
Tom’s Hardware reports that “the tinybox can offer up to 738 FP16 TFLOPS (0.738 FP16 TFLOPS) of performance along with 96GB of GDDR6 memory that delivers 21TB/s of aggregated memory bandwidth. To put the numbers into context, one tinybox offers 37% of Nvidia H100 compute performance (FP16) but slightly more memory (96GB instead of 80GB) and considerably higher peak memory bandwidth (21TB/s instead of 3.35TB/s).”
Shipping soon
The tinybox includes five 1TB Western Digital SN850X SSDs (4 in RAID, 1 for boot) and also features an empty 16x OCP 3.0 slot for networking. It will ship with Ubuntu 22.04 and the tinygrad framework, although tinycorp is happy for buyers to run other machine learning frameworks such as PyTorch and JAX on AMD hardware. As the company puts it, “your hardware is your hardware… you are welcome to buy it for any purpose.”
Not everyone is raving at the idea of the tinybox though. As a Tom’s Hardware reader commented, “oh cool, yet another startup that will cause gaming GPU prices to spike.”
The BOM cost of the tinybox machine is reportedly about $10,000, but the company will sell it for $15,000, which is considerably lower than Nvidia's H100 which can sell for over $40,000. To date, the company says it has received 583 preorders and plans to begin shipping in April. It’s possible to preorder the tinybox for $100.
The company says of its creation's future, “We will push the limits of what's possible on this box. The hardware is solid, but the software is a huge journey. I'm excited to be on that journey together with all of you soon to be tinybox owners. To the metal. We will squeeze every last FLOP out of this thing”.
