Pika Labs has added the ability to create sound effects from a text prompt for its generative artificial intelligence videos.
Now you can make bacon sizzle, lions roar or add footsteps to the video of someone walking down the street.
This is the second audio feature added to the Pika platform, joining a lip sync tool that lets you give characters in AI videos — or action figures — a voice.
Sounds effects are currently only available to subscribers of the Pro plan or above but will likely be rolled out more widely in the coming weeks.
How do Pika Labs sound effects sound?
The sounds are impressive, generated from a simple text prompt they generally mirror what the user requests and, when paired with a video, bring that clip to life.
Creating sound effects from text is a rapidly emerging field, with other tools like MyEdit, AudioCraft and ElevenLabs also offering similar functionality.
I haven't had a chance to try it myself yet but the example videos from Pika Labs show bacon sizzling in a ban with realistic crackling sounds and a frog.
I can't confirm if it's an accurate croak for the species as I have very limited frog knowledge but it also includes the sound of distant flies buzzing.
Another video provides sound to fireworks, although this isn't nearly as good as the frog or the bacon as the timing doesn't match the motion in the video.
What comes next for AI video?
Crispy bacon and AI sound effects - what else could we ask for on a Sunday morning? pic.twitter.com/OcFzGbZM75March 10, 2024
While adding sound to video is impressive, it is somewhat of a blunt instrument, dumping a related noise on to the video.
What the next big development needs is image to sound where the AI analyzes the picture, uses that to create the sound effect and applies it automatically.
This could then allow for the sounds to be created alongside the video clip all from a single prompt. For example "bacon sizzling in a pan" would create a video of sizzling bacon with associated sounds with no extra steps.
A move to platform-based creation
So far AI video has been a largely isolated tool. You make a short clip and if you want to use it then you export it to another app, where you import sounds, effects and other features needed to create a compelling video.
What we're starting to see from Pika Labs, with Runway and its own voice AI tools, from LTX Studio and independent platforms like FinalFrame.ai is a one-stop-shop.
They are providing a place to go with an idea and come out with a production, or at the very least be able to pair the sound an audio even if you have to edit everything together in something like Adobe Premiere or Final Cut Pro.
We are close to being able to generate fully formed AI films. Not just the effects but an entire production with characters and sound. The big challenge is, and will continue to be, consistency.
