Meta announced a new AI-powered video and audio generator today (October 4) to rival the likes of OpenAI's Sora and ElevenLabs' AI voice tools. Called Movie Gen, it can create realistic-looking video and audio clips based on user prompts as well as edit existing footage or images with text.
While there's no timeline yet on when Movie Gen will be available to the public, samples of its creations shared by Meta showed videos of thunder cracking overhead in the desert, animals swimming, and a child running along the beach with a kite generated by text prompts.
The software suite can also generate audio like ambient noise, sound effects, and background music that syncs to the imagery, Meta said in a blog post. Movie Gen can create 1080p videos up to 16 seconds long in different aspect ratios, while audio can be up to 45 seconds long, Meta said.
In addition to generating new clips, Movie Gen can be used to edit existing video footage as well. In one such video, Meta had the tool replace a floating lantern with a string of bubbles. In another, it inserted pom-poms into the hands of a man running in the desert, while a third replaced his clothes with an inflatable dinosaur costume, showing how further tweaks can be made via text prompts. Meta also demonstrated how it can generate a video of a person using just a still image and a text prompt.
Meta Movie Gen is on the scene! Our breakthrough generative AI research for media enables:-turning text into video -creation of personalized video-precision video editing -audio creationAnd while it’s just research today, we can’t wait to see all the ways people enhance… pic.twitter.com/I4Bq9if3eKOctober 4, 2024
Chris Cox, chief product officer at Meta Platforms, said a full release is still a ways away.
"We aren’t ready to release this as a product anytime soon — it’s still expensive and generation time is too long — but we wanted to share where we are since the results are getting quite impressive," he wrote on Threads.
The announcement makes Meta the latest major tech company pushing full speed ahead into the AI space. In the past few months, both Google and Adobe have unveiled AI video generators to compete with third-generation AI video tools from leading media generation startups.
In its announcement, Meta shared data showing blind tests that indicate Movie Gen performs favorably compared with offerings from startups like Runway, OpenAI, ElevenLabs and Kling. So while we'll have to wait and see whether Movie Gen ranks among our list of the best AI video generators, the early results are promising.
