ZLUDA, an open-source CUDA translation layer, has lived two quite vivid lives with Intel and then AMD GPUs. It was nearly killed in August when AMD asked to take down the code developed using its funds. However, as its developer, Andrzej Janik, secured funding from a mysterious sponsor, ZLUDA now has a third life. This time around, the focus of ZLUDA will be to run AI/ML software designed for CUDA GPUs on processors from other vendors using a translation layer, reports Phoronix.
ZLUDA was originally designed to run creative professional CUDA-based applications on Intel and then AMD GPUs, while the upcoming iteration of ZLUDA shifts focus to accommodate AI and machine learning workloads. Also, the emphasis is now not just on Intel or AMD. Instead, it offers multiple GPU vendor support, making ZLUDA applicable across different GPU architectures. Nonetheless, for the time being, most development efforts are concentrated on AMD GPUs, particularly RDNA1 and newer architectures. Support is being built around AMD’s ROCm 6.1+ compute stack, laying the foundation for broader, multi-architecture compatibility in the future.
Andrzej Janik is currently working to make AI/ML frameworks like PyTorch, TensorFlow, and Llama.cpp function seamlessly using CUDA on non-Nvidia GPUs using his translation layer, according to Phoronix, who spoke to the developer. Janik predicts it will take about a year to develop the new ZLUDA code to a stable state where it can effectively handle AI/ML workloads across multiple GPUs. Contributions from the open-source community will be welcomed as the project evolves. So, ZLUDA will remain open source, or at least it looks so today.
Although ZLUDA now has a financial backer, the sponsor has chosen to remain anonymous for now. We can only speculate who the sponsor is because they need to run AI workloads at scale and opted for multi-GPU vendor support. Also, we presume it is big enough not to be afraid of getting into a conflict over running CUDA software through a translation layer, which Nvidia does not endorse these days. Yet, the developer says that this ‘stealth’ sponsor is expected to be revealed later, providing more insight into the direction and future support of ZLUDA.
