An Arizona woman has recounted the story of how she was embroiled in a kidnapping scam in which AI was used to spoof the voice of her daughter.
Jennifer DeStefano described the ordeal during a Senate Judiciary Committee hearing in the US.
On January 20, DeStefano received a call claiming her 15-year-old daughter Briana had been kidnapped. A $1 million (£790,000) ransom was demanded for her safe return.
“Mom, these bad men have me, help me, help me!” DeStefano heard her daughter say, “sobbing and crying,” before the line was taken up by a “threatening and vulgar man”.
“Listen here, I have your daughter. If you call anybody, if you call the police, I’m going to pump her stomach so full of drugs and have my way with her. I’m going to drop her in Mexico and you’ll never see your daughter again,” the man reportedly said.
DeStefano says the kidnapper demanded $1 million, before dropping that demand down to $50,000 (£39,500) in cash.
After the police were called by another local mother, DeStefano was informed this was a known scam, but she was still convinced the situation was real.
“[I was told] 911 is very familiar with an AI scam where they can use someone’s voice,” DeStefano said. “But I didn’t process that. It wasn’t just her voice, it was her cries, it was her sobs… that are unique to her.”
The supposed kidnappers wanted her to get $50,000 in cash and ride in a car, with a bag over her head, to deliver the ransom.
However, the scam was revealed when DeStefano’s husband was able to check on their daughter Briana in person, as the pair were on a ski trip at the time.
How prevalent are AI scams?
In a follow-up statement, MIT professor Aleksander Madry said “the newest wave of generative AI is poised to fundamentally transform our collective sense-making… AI enables the creation of content that is not only extremely realistic but also persuasive, even though it may be false.”
He says such deceptive content is “frighteningly easy to deploy at scale” thanks to the latest AI technologies.
The Senate Judiciary Committee hearing’s goal was to detail some of the dangerous knock-on effects of the profliferation of AI technology, with a view to arguing for swift regulation.
“Generated images can also twist public understanding of political figures and events. Videos and images have already been digitally altered to compromise public officials. Fake content is now cheaper, easier, and more convincing because of the growth of AI tools,” said Alexandra Givens, CEO of the Center for Democracy & Technology, during the hearing.
On June 5, the FBI released a public service announcement warning US citizens about the prevalence of scams using deep-fake pornographic videos and images to extort people.
“The FBI urges the public to exercise caution when posting or direct messaging personal photos, videos, and identifying information on social media, dating apps, and other online sites. Although seemingly innocuous when posted or shared, the images and videos can provide malicious actors an abundant supply of content to exploit for criminal activity,” the statement reads.
These scams and malicious uses of AI are by no means limited to the US. Research on AI voice scams conducted by security software company McAfee was published in May. Its report estimates one in 12 people in the UK have already experienced an AI voice scam.
This was based on a survey of more than 7,000 people across the world, including 1,009 in the UK.
