External GPU (eGPU) support on Apple Silicon Macs and MacBooks has been a persistent pain point for AI/ML developers. Through what some may consider to be black magic, Tiny Corp has managed to get an AMD eGPU working in Tiny Grad over USB3, a standard that inherently lacks PCIe capabilities. As they're using libusb, this functionality extends to Windows, Linux, and even macOS, including devices with Apple Silicon.
Traditionally, GPUs are connected through PCIe slots or the Thunderbolt/USB4 interfaces, which offer PCI Express tunneling support. As such, external GPU solutions rely on the aforementioned interfaces, which limits their support for older systems and laptops. Unlike Intel-based Macs/MacBooks, Apple Silicon based devices do not support external GPUs, mainly due to the lack of driver support and architectural differences. So, despite their efficiency compared to traditional x86-based systems, users have reported challenges in AI workloads, especially when it comes to prompt processing.
