AI video generation isn’t a new thing, but with the arrival of OpenAI’s Sora text-to-video tool, it’s never been easier to make your own fake news.
The photorealistic capabilities of Sora have taken many of us by surprise. While we’ve seen AI-generated video clips before, there’s a degree of accuracy and realism in these incredible Sora video clips that makes me… a little nervous, to say the least. It’s undoubtedly very impressive, but it’s not a good sign that my first reaction to that video of playing puppies was immediate concern.
Our lovely editor-in-chief Lance Ulanoff penned an article earlier this year discussing how AI will make it impossible to determine truth from fiction in 2024, but he was mostly talking about image-generation software at the time. Soon, anyone will be able to get their hands on a simple and easy-to-use tool for producing entire video clips. Combined with the existing power of voice deepfake artificial intelligence (AI) software, the potential for politically-motivated video impersonation is more prevalent than ever.
'Fake news!' cried the AI-generated Trump avatar
Now, I don’t want to simply sit here and fearmonger endlessly about the dangers of AI. Sora isn’t widely available just yet (it’s currently invite-only), and I genuinely do believe that AI has plenty of use cases that could improve human lives; implementations in medical and scientific professions can potentially serve to eliminate some of the busywork faced by doctors and researchers, making it easier to cut through the chaff and get to the important stuff.
Unfortunately, just as with Adobe Photoshop before it, Sora and other generative AI tools will be used for nefarious purposes. Trying to deny this is trying to deny human nature. We already saw Joe Biden’s voice hijacked for robocall scams – how long will it be before ersatz videos of political figures start to flood social media?
Sora, much like OpenAI’s flagship AI product ChatGPT, probably won’t be the tool used to produce this fakery. Sora and ChatGPT both have a large number of safety ‘guardrails’ in place to prevent them from being used to produce content that goes against OpenAI’s user guidelines. For example, prompts that request explicit sexual content or the likeness of others will be rejected. OpenAI, in its defense, does state that it intends to continue ‘engaging policymakers, educators and artists around the world to understand their concerns’.
However, there are ways to circumvent these guardrails – I tested this myself, and the results were occasionally hilarious – and OpenAI’s transparent approach to AI development means that Sora imitators will surely pop up everywhere in the not-too-distant future. These knock-offs (much like chatbots based on ChatGPT) won’t necessarily have the same safety and security features.
Robocop, meet Robocriminal
AI tools are already being used for a lot of dodgy stuff online. Some of it is relatively harmless; if you want to have a steamy R-rated conversation with an AI pretending to be your favorite anime character, I’m not here to judge you (well, maybe I am a little, but at least it’s not a crime). Elsewhere, though, bots are being used to scam vulnerable internet users, spread misinformation online, and scrape social media platforms for people’s personal data.
The power of something like Sora could make this even worse, allowing for even more sophisticated fakery. It’s not just about what the AI can make, remember – it’s what a talented video editor can do with the raw footage provided by a tool like Sora. A bit of tweaking here, a filter there, and suddenly we could have grainy phone-camera footage of a prominent politician beating up a homeless person in an alley. Don’t even get me started on how high-quality AI video generation is practically guaranteed to disproportionately affect women thanks to the recent online trend of AI-powered ‘revenge porn’.
The worst part? It’s only going to get harder to determine the fakes from reality. Despite what some AI proponents might tell you, there’s currently no reliable way to definitively confirm whether footage is AI-generated.
Software for this does exist, but doesn’t have a great track record. When Scribblr tested out several AI detection tools, it found that the paid software with the highest success rate (Winston AI) was only correct 84% of the time. The most accurate free AI detector (Sapling) only offered 68% accuracy. As time goes on, this software might improve – but the incredibly fast development of generative AI could outpace it, and there’s always the risk of false positives.
Sure, many AI-produced videos and images can be readily identified as such by a seasoned internet user, but the average voter isn’t so eagle-eyed, and the telltale signs - usually weird morphing around human digits and limbs, or unrealistic camera movements - are only going to fade as the technology improves. Sora represents an enormous leap forward, and I’m frankly a bit concerned about what the next big jump will look like.
The era of disinformation
When we discuss AI deepfakes and scams, we’re often doing so on quite a macro scale: AI influencing upcoming elections, an AI-generated CFO stealing 25 million dollars, and AI art winning a photography contest are all prime examples. But while the idea of secret AI senators and chief executives does worry me, it’s on the small scale where lives will truly be ruined.
If you’ve ever sent a nude photo, then congrats – your jealous ex can now skim your social media for more material and turn that into a full-blown sex tape. Accused of a crime, but you’ve got a video recording that exonerates you? Tough, the court’s AI-detection software returned a false positive, and now you’re being hit with an extra felony count for producing fake evidence. Individuals stand to lose the most in the face of emergent new technologies like Sora – I don’t care about big corporations losing money because of a hallucinating chatbot.
We’ve been living in a time where the sum total of human knowledge is almost entirely accessible to us from the little rectangles in our pockets, but AI threatens to poison the well. It’s nothing new – this isn’t the first threat to facts that the internet has faced, and it won’t be the last, but it very well could be the most devastating to date.
Calling it quits
Of course, you can say ‘same sh*t, different day’ about all this – and you wouldn’t be wrong. Scams and disinformation aren’t new, and the targets of technologically-augmented deception haven’t changed: it’s mostly the very young and the very old, those who haven’t learned enough about tech yet or aren’t able to keep up with its relentless march.
I hate this argument, though. It’s defeatist, and it fails to acknowledge the sheer power and scalability that AI tools can put in the hands of scammers. Snail mail frauds have been around for decades, but let’s be honest: that takes a lot more time and effort than ordering a bot to write and send thirty thousand phishing emails.
Before I wrap this up, I do want to make one thing clear, because my social inboxes invariably get clogged up with angry AI lovers whenever I write an article like this: I’m not blaming AI for this. I’m not even blaming the people who make it. OpenAI seems to be taking a more cautious and transparent approach to deep learning technology than I’d expect from many of the world’s biggest corporations. What I want is for people to be fully aware of the dangers, because text-to-video AI is simply the latest trick in the bad actors’ playbook.
If you’d like to try out Sora for yourself (hopefully for more wholesome purposes), you can make an OpenAI account today by following this link – but bear in mind that unless you have an invite, the software isn’t available just yet. OpenAI is treading carefully this time around, with the first wave of testers comprised mainly of ‘red teamers’ who are stress-testing the tool to eliminate bugs and vulnerabilities. There isn’t an official release date just yet, but it likely won’t be far off if OpenAI’s previous releases are anything to go on.
