LTX Studio is a new generative AI video platform that allows creators to make an entire production from a single prompt. During a demo of the tool I saw a full production appear in seconds with clips, sounds and characters.
Generative AI video has quickly gone from barely viewable short clips to near production-ready in under a year. We’ve got shots so close to reality its hard to tell fact from fiction, but for the most part they are still short three or four second shots with no sound.
While there is a multitude of generation tools, and new models on the horizon from OpenAI such as Sora that will transform the market, LTX Studio’s developers are focusing on production — the whole picture not just one clip.
Built by Lightricks, the team behind the Facetune and Photoleap apps, it utilizes a number of private, in-house and open source generative AI models to make AI video production easier and from what I've seen it is a game changing moment for AI.
How does LTX Studio work?
LTX Studio is a generative content platform. It allows you to create short video clips, music, sound effects and even dialogue from a text prompt. What makes it stand out is the fact you can do all of that from within a single interface.
The team utilize a range of generative AI tools and models then integrate them across the interface to make the creation of multi-shot AI video easier.
Starting with a prompt, such as "make a drama about a couple stranded on a beach after a plane crash", it generates character profiles complete with photo, style, name and voice.
Once it has the profiles it then creates a series of scenes, each scene contains a series of 3-4 second shots made from generative videos. You can then customize any component.
How well does LTX Studio work?
@DONDADAproducer A young man entering a sound-healing chamber. Surrounded by scientist in labcoats, anticipating the maiden power-up, eventual results. pic.twitter.com/OHnpVeYEbjFebruary 28, 2024
As part of my job I create AI content on a regular basis, this includes making music videos, commercials and trailers to demonstrate how different models and tools work. This usually involves post-production in traditional apps like Adobe Premiere.
During a demo of LTX Studio with Lightricks CEO Zeev Farbman he took the basic idea behind one of my music videos and in a few words had dozens of shots spread over a handful of scenes and custom characters ready to go.
It wasn’t perfect, the characters needed some work but the fact it got from a basic idea to a full storyboard with generated content in seconds was astounding.
The fact it got from a basic idea to a full storyboard with generated content in seconds was astounding.
What makes it even more exciting is I could customize the characters once and it changed them in every shot.
I could also change the style for any one shot or for the entire production — for example going from photorealism to anime.
This meant there was no need to go back and regenerate new individual clips, load them back into Premiere Pro and start again as is the case at the moment.
Farbman told me there was a big market for content creation at scale, but that one of the more surprising was from directors looking to create rapid storyboarding of ideas that could be used in pitch meetings with investors or studios.
When will LTX Studio be available?
@calliped a wizard stealing a pink lemon from a tree on his broom, gets caught goes to jail. pic.twitter.com/W2qLMWstX6February 28, 2024
The Lightricks team are still developing LTX Studio and finalizing some of the features, including an impressive sounding ability to control the camera position and movement within a shot.
This feature effectively converts the still image into a 3D scene that you can move through to change the angle and motion of the camera. You will be able to do this to any shot in a scene.
They are also exploring the potential for lip synching but admitted “none are particularly good yet”. For now dialogue is more like a voice over than you’d see in a movie or TV show.
The waiting list is already open for people wanting to get in early and the first public access is expected sometime this month. From what I’ve seen it is already a useful storyboarding tool even without the generative video component.
Why does this matter?
@xwitai a rookie cyberpunk bounty hunter finds himself at the center of a city gang war after saving a mob bosses son from a rival attack. Gotta learn the rules fast if he wants to survive life of on this side of the law. A crime thriller packed with hits, jealousy, and power. pic.twitter.com/HY1GoVlnXvFebruary 28, 2024
This isn’t the only generative AI studio product I’ve had pitched to me recently. Some start with a story, others with an image — this is more a pure generative product starting with a simple prompt but then giving very fine-tuned granular control over every element.
Studio products are the next logical step for generative AI. Being able to make a fun video clip is great, but being able to make multiple clips, add music and sound effect and dialogue is a game changing moment — and gets us closer to “Netflix, make a movie about cats on the moon”.
