If AI is the future of gaming, perhaps it makes sense to begin with Doom. The classic game has been ported for all kinds of things it wasn't originally meant for. Honouring that tradition is GameNGen, developed by researchers at Google and Tel Aviv University.
GameNGen is a neural model-based game engine that allows real-time interaction with complex gaming environments without the use of a traditional game engine. The researchers think could transform how video games are made, and how they're played, and Doom is just the beginning.
As demonstrated in the real-time recordings above, GameNGen can simulate the classic Doom at over 20 frames per second with a visual quality that resembles the original. The team trained a reinforcement learning agent to play the game to provide the data for a generative AI model. They repurposed Stable Diffusion v1.4, an open-source generative AI diffusion model, and conditioned it on a sequence of previous actions and observations.
The data was used to train the diffusion model to predict the next frame, allowing it to simulate complex state updates like health and ammo, attacks and interactions with the environment. The model employs conditioning augmentations to mitigate sampling divergence and achieve stable auto-regressive generation through long sequences (see the paper).
The researchers Dani Valevski, Yaniv Leviathan, Moab Arar and Shlomi Fruchter plan to refine GameNGen's capabilities, increasing its memory and improving its handling of complex environments. While the proof of concept has only been demonstrated on Doom so far, the team thinks the tech could have much broader applications for interactive software as well as other games. The suggestion is that we could see games that are generated by AI instead of being coded manually, allowing design via text description or example images.
Generative AI could also be used to create potentially endless expanding worlds via procedural generation and more immersive games that respond dynamically to player actions. Generative AI isn't only being used to create playable games. AI robot boxing is also now a thing and AI live stream software is going viral due to its ability to create very realistic live deepfakes.
