Neural rendering—the use of AI models to create pixels—is already a familiar concept in real-time graphics. When you’re using DLSS upscaling or frame generation, for example, many of the pixels you see are already generated rather than natively shaded, and those extra pixels and frames come at a surprisingly low computational cost via the matrix math acceleration of Nvidia’s Tensor Cores.
Given that almost magical cost-to-benefit ratio, research is unsurprisingly well under way to expand generative AI techniques beyond relatively transparent applications like upscaling and frame generation to replace portions of the traditional graphics pipeline as we know it today—and possibly even in its entirety.
