Artificial intelligence image generation platform MidJourney is working on new 3D, video and real-time creation models that would allow it to simulate the entire world.
Details came out as part of the recent “office hours” on Discord where company leadership answer questions from users about the future of the platform.
Rumors have circulated for some time that MidJourney was expanding beyond images, but this seems to suggest the new moves will be tightly focused on creating a world model.
They have been adding and upgrading other features such as image description and consistent characters while building new models.
The exact nature of this is unclear but the company suggests it would be a virtual environment where people could build video games or shoot movies — an “open world sandbox”.
Moving beyond images with AI
MidJourney was founded in 2022 by David Holz, a self-proclaimed serial entrepreneur.
The first open beta was released in July that year and rapidly iteratted and upgraded over the following year, with its most powerful v6 model releasing in December last year. In the same office hours Holtz said v7 would be a bigger jump than from 5 to 6.
MidJourney has remained stubbornly inside Discord as the way to interact with and make images using its model, although some high intensity users can use use the web interface.
Work has already started on training the seventh version of the MidJourney image model and Holtz told users on Discord that 3D generation may arrive before video.
The value of moving into 3D worlds
To be clear we're not working on a headset right now (we're working on hardware related to data capture), but we did build and open source an AR headset >6 years ago at Leap Motion (images and URL below) Blogpost: https://t.co/8mX5QTddL9Open source website:… pic.twitter.com/Z7Bz2WMy0JMarch 13, 2024
It makes some sense dimensionality is part of the focus. One of Holtz former startups was Leap Motion, a company that built sensors for tracking hand and finger motions in VR.
"We're really trying to get to the world simulation,” he said on Discord. “We're building 3D midjourney, video MJ, & real-time MJ where things move really really fast.”
He added that if you put this all together you get world simulation. The goal, he said is to build each of those three elements independently then work on ways to bring them together.
“It will be more of a sandbox,” he explained. Adding that “people will make video games in it, people will shoot movies in it, but the goal is to build the open world sandbox."
So the metaverse is back?
On the surface this sounds like some modern interpretation of the metaverse. The idea of a fully interconnected series of virtual environments where people can carry out every day tasks, earn money and have fun.
However, it seems more like a new way to create content of any type, including generating entire virtual environments where future Sci-Fi movies could be filmed without having to use VFX tools to create every element at geat cost.
The sandbox vision from MidJourney's Holtz bears more resemblance to some of the ideas being put forward by Roblox for its developer studio.
The company recently revealed new texture and avatar generators and during a conversation with Head of Studio Stefano Corazza I was told there are plans down the line to have AI generate entire environments from a text prompt.
